Jul 07 23:13:02.485836 ip-10-0-134-140 systemd[1]: Starting Kubernetes Kubelet... Jul 07 23:13:02.904188 ip-10-0-134-140 kubenswrapper[2570]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 07 23:13:02.904188 ip-10-0-134-140 kubenswrapper[2570]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 07 23:13:02.904188 ip-10-0-134-140 kubenswrapper[2570]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 07 23:13:02.904188 ip-10-0-134-140 kubenswrapper[2570]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 07 23:13:02.904188 ip-10-0-134-140 kubenswrapper[2570]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 07 23:13:02.904868 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:02.904242 2570 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 07 23:13:02.912208 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.912185 2570 feature_gate.go:328] unrecognized feature gate: Example Jul 07 23:13:02.912208 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.912204 2570 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 07 23:13:02.912208 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.912208 2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 07 23:13:02.912208 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.912211 2570 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 07 23:13:02.912208 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.912215 2570 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 07 23:13:02.912208 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.912218 2570 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 07 23:13:02.912446 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.912221 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 07 23:13:02.912446 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.912224 2570 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 07 23:13:02.912446 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.912227 2570 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 07 23:13:02.912446 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.912230 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 07 23:13:02.912446 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.912233 2570 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 07 23:13:02.912446 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.912236 2570 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 07 23:13:02.912446 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.912239 2570 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 07 23:13:02.912446 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.912241 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 07 23:13:02.912446 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.912244 2570 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 07 23:13:02.912446 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.912247 2570 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 07 23:13:02.912446 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.912250 2570 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 07 23:13:02.912446 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.912252 2570 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 07 23:13:02.912446 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.912255 2570 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 07 23:13:02.912446 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.912257 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 07 23:13:02.912446 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.912260 2570 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 07 23:13:02.912446 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.912263 2570 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 07 23:13:02.912446 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.912266 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 07 23:13:02.912446 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.912268 2570 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 07 23:13:02.912446 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.912271 2570 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 07 23:13:02.912936 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.912273 2570 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 07 23:13:02.912936 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.912276 2570 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 07 23:13:02.912936 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.912280 2570 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 07 23:13:02.912936 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.912283 2570 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 07 23:13:02.912936 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.912285 2570 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 07 23:13:02.912936 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.912288 2570 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 07 23:13:02.912936 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.912290 2570 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 07 23:13:02.912936 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.912293 2570 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 07 23:13:02.912936 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.912295 2570 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 07 23:13:02.912936 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.912298 2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 07 23:13:02.912936 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.912300 2570 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 07 23:13:02.912936 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.912305 2570 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 07 23:13:02.912936 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.912308 2570 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 07 23:13:02.912936 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.912312 2570 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 07 23:13:02.912936 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.912316 2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 07 23:13:02.912936 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.912318 2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 07 23:13:02.912936 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.912321 2570 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 07 23:13:02.912936 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.912324 2570 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 07 23:13:02.912936 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.912329 2570 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 07 23:13:02.913409 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.912331 2570 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 07 23:13:02.913409 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.912334 2570 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 07 23:13:02.913409 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.912337 2570 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 07 23:13:02.913409 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.912340 2570 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 07 23:13:02.913409 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.912343 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 07 23:13:02.913409 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.912346 2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 07 23:13:02.913409 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.912348 2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 07 23:13:02.913409 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.912351 2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 07 23:13:02.913409 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.912354 2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 07 23:13:02.913409 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.912357 2570 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 07 23:13:02.913409 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.912359 2570 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 07 23:13:02.913409 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.912362 2570 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 07 23:13:02.913409 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.912365 2570 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 07 23:13:02.913409 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.912367 2570 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 07 23:13:02.913409 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.912370 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 07 23:13:02.913409 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.912373 2570 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 07 23:13:02.913409 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.912375 2570 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 07 23:13:02.913409 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.912378 2570 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 07 23:13:02.913409 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.912380 2570 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 07 23:13:02.913892 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.912383 2570 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 07 23:13:02.913892 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.912385 2570 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 07 23:13:02.913892 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.912387 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 07 23:13:02.913892 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.912392 2570 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 07 23:13:02.913892 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.912395 2570 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 07 23:13:02.913892 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.912398 2570 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 07 23:13:02.913892 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.912401 2570 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 07 23:13:02.913892 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.912404 2570 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 07 23:13:02.913892 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.912407 2570 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 07 23:13:02.913892 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.912410 2570 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 07 23:13:02.913892 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.912415 2570 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 07 23:13:02.913892 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.912419 2570 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 07 23:13:02.913892 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.912422 2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 07 23:13:02.913892 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.912424 2570 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 07 23:13:02.913892 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.912427 2570 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 07 23:13:02.913892 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.912430 2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 07 23:13:02.913892 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.912432 2570 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 07 23:13:02.913892 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.912435 2570 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 07 23:13:02.913892 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.912438 2570 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 07 23:13:02.914373 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.912440 2570 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 07 23:13:02.914373 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.912444 2570 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 07 23:13:02.914373 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.912447 2570 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 07 23:13:02.914373 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.912450 2570 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 07 23:13:02.914373 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.912453 2570 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 07 23:13:02.914373 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.912455 2570 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 07 23:13:02.914373 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.912458 2570 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 07 23:13:02.914373 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.912460 2570 feature_gate.go:328] unrecognized feature gate: Example2 Jul 07 23:13:02.914373 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.912479 2570 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 07 23:13:02.914373 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.912481 2570 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 07 23:13:02.914373 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.912484 2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 07 23:13:02.914373 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.912487 2570 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 07 23:13:02.914373 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.912489 2570 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 07 23:13:02.914373 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.912492 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 07 23:13:02.914373 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.912494 2570 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 07 23:13:02.914373 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.912497 2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 07 23:13:02.914373 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.912500 2570 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 07 23:13:02.914373 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.913206 2570 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 07 23:13:02.914373 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.913212 2570 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 07 23:13:02.914373 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.913216 2570 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 07 23:13:02.914910 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.913221 2570 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 07 23:13:02.914910 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.913224 2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 07 23:13:02.914910 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.913227 2570 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 07 23:13:02.914910 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.913231 2570 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 07 23:13:02.914910 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.913234 2570 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 07 23:13:02.914910 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.913236 2570 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 07 23:13:02.914910 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.913239 2570 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 07 23:13:02.914910 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.913244 2570 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 07 23:13:02.914910 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.913247 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 07 23:13:02.914910 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.913251 2570 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 07 23:13:02.914910 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.913254 2570 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 07 23:13:02.914910 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.913257 2570 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 07 23:13:02.914910 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.913259 2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 07 23:13:02.914910 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.913262 2570 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 07 23:13:02.914910 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.913265 2570 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 07 23:13:02.914910 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.913268 2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 07 23:13:02.914910 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.913270 2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 07 23:13:02.914910 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.913273 2570 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 07 23:13:02.914910 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.913276 2570 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 07 23:13:02.915366 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.913278 2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 07 23:13:02.915366 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.913281 2570 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 07 23:13:02.915366 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.913283 2570 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 07 23:13:02.915366 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.913286 2570 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 07 23:13:02.915366 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.913288 2570 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 07 23:13:02.915366 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.913291 2570 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 07 23:13:02.915366 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.913294 2570 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 07 23:13:02.915366 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.913296 2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 07 23:13:02.915366 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.913298 2570 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 07 23:13:02.915366 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.913301 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 07 23:13:02.915366 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.913304 2570 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 07 23:13:02.915366 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.913306 2570 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 07 23:13:02.915366 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.913310 2570 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 07 23:13:02.915366 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.913316 2570 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 07 23:13:02.915366 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.913319 2570 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 07 23:13:02.915366 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.913321 2570 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 07 23:13:02.915366 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.913324 2570 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 07 23:13:02.915366 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.913327 2570 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 07 23:13:02.915366 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.913329 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 07 23:13:02.915366 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.913332 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 07 23:13:02.915905 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.913335 2570 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 07 23:13:02.915905 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.913337 2570 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 07 23:13:02.915905 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.913340 2570 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 07 23:13:02.915905 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.913342 2570 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 07 23:13:02.915905 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.913345 2570 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 07 23:13:02.915905 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.913347 2570 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 07 23:13:02.915905 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.913350 2570 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 07 23:13:02.915905 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.913352 2570 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 07 23:13:02.915905 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.913355 2570 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 07 23:13:02.915905 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.913358 2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 07 23:13:02.915905 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.913360 2570 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 07 23:13:02.915905 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.913364 2570 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 07 23:13:02.915905 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.913367 2570 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 07 23:13:02.915905 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.913369 2570 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 07 23:13:02.915905 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.913371 2570 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 07 23:13:02.915905 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.913374 2570 feature_gate.go:328] unrecognized feature gate: Example2 Jul 07 23:13:02.915905 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.913377 2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 07 23:13:02.915905 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.913379 2570 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 07 23:13:02.915905 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.913382 2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 07 23:13:02.915905 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.913384 2570 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 07 23:13:02.916384 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.913387 2570 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 07 23:13:02.916384 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.913389 2570 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 07 23:13:02.916384 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.913392 2570 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 07 23:13:02.916384 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.913394 2570 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 07 23:13:02.916384 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.913397 2570 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 07 23:13:02.916384 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.913401 2570 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 07 23:13:02.916384 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.913404 2570 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 07 23:13:02.916384 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.913406 2570 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 07 23:13:02.916384 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.913409 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 07 23:13:02.916384 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.913411 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 07 23:13:02.916384 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.913414 2570 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 07 23:13:02.916384 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.913417 2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 07 23:13:02.916384 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.913419 2570 feature_gate.go:328] unrecognized feature gate: Example Jul 07 23:13:02.916384 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.913422 2570 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 07 23:13:02.916384 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.913425 2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 07 23:13:02.916384 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.913428 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 07 23:13:02.916384 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.913430 2570 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 07 23:13:02.916384 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.913432 2570 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 07 23:13:02.916384 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.913435 2570 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 07 23:13:02.916384 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.913437 2570 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 07 23:13:02.916889 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.913440 2570 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 07 23:13:02.916889 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.913443 2570 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 07 23:13:02.916889 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.913446 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 07 23:13:02.916889 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.913448 2570 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 07 23:13:02.916889 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.913451 2570 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 07 23:13:02.916889 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.913453 2570 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 07 23:13:02.916889 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.913456 2570 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 07 23:13:02.916889 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.913458 2570 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 07 23:13:02.916889 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.913461 2570 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 07 23:13:02.916889 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.913479 2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 07 23:13:02.916889 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.913483 2570 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 07 23:13:02.916889 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.913487 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 07 23:13:02.916889 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.913490 2570 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 07 23:13:02.916889 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.913505 2570 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 07 23:13:02.916889 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.913508 2570 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 07 23:13:02.916889 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.913510 2570 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 07 23:13:02.916889 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.913513 2570 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 07 23:13:02.916889 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.913896 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 07 23:13:02.916889 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.913901 2570 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 07 23:13:02.917347 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.913905 2570 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 07 23:13:02.917347 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.913910 2570 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 07 23:13:02.917347 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.913914 2570 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 07 23:13:02.917347 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.913917 2570 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 07 23:13:02.917347 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.913920 2570 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 07 23:13:02.917347 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.913923 2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 07 23:13:02.917347 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.913926 2570 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 07 23:13:02.917347 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.913928 2570 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 07 23:13:02.917347 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.913931 2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 07 23:13:02.917347 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.913933 2570 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 07 23:13:02.917347 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.913936 2570 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 07 23:13:02.917347 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.913938 2570 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 07 23:13:02.917347 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.913941 2570 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 07 23:13:02.917347 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.913943 2570 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 07 23:13:02.917347 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.913947 2570 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 07 23:13:02.917347 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.913950 2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 07 23:13:02.917347 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.913952 2570 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 07 23:13:02.917347 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.913955 2570 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 07 23:13:02.917347 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.913957 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 07 23:13:02.917867 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.913961 2570 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 07 23:13:02.917867 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.913965 2570 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 07 23:13:02.917867 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.913967 2570 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 07 23:13:02.917867 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.913971 2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 07 23:13:02.917867 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.913973 2570 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 07 23:13:02.917867 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.913976 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 07 23:13:02.917867 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.913978 2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 07 23:13:02.917867 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.913981 2570 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 07 23:13:02.917867 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.913983 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 07 23:13:02.917867 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.913986 2570 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 07 23:13:02.917867 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.913988 2570 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 07 23:13:02.917867 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.913992 2570 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 07 23:13:02.917867 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.913995 2570 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 07 23:13:02.917867 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.913997 2570 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 07 23:13:02.917867 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.914000 2570 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 07 23:13:02.917867 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.914002 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 07 23:13:02.917867 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.914005 2570 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 07 23:13:02.917867 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.914007 2570 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 07 23:13:02.917867 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.914010 2570 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 07 23:13:02.917867 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.914013 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 07 23:13:02.918372 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.914015 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 07 23:13:02.918372 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.914018 2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 07 23:13:02.918372 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.914020 2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 07 23:13:02.918372 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.914023 2570 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 07 23:13:02.918372 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.914025 2570 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 07 23:13:02.918372 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.914027 2570 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 07 23:13:02.918372 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.914030 2570 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 07 23:13:02.918372 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.914034 2570 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 07 23:13:02.918372 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.914037 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 07 23:13:02.918372 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.914040 2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 07 23:13:02.918372 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.914042 2570 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 07 23:13:02.918372 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.914045 2570 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 07 23:13:02.918372 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.914047 2570 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 07 23:13:02.918372 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.914050 2570 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 07 23:13:02.918372 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.914052 2570 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 07 23:13:02.918372 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.914055 2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 07 23:13:02.918372 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.914057 2570 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 07 23:13:02.918372 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.914060 2570 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 07 23:13:02.918372 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.914062 2570 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 07 23:13:02.918372 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.914065 2570 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 07 23:13:02.918862 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.914067 2570 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 07 23:13:02.918862 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.914070 2570 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 07 23:13:02.918862 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.914072 2570 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 07 23:13:02.918862 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.914076 2570 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 07 23:13:02.918862 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.914079 2570 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 07 23:13:02.918862 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.914081 2570 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 07 23:13:02.918862 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.914084 2570 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 07 23:13:02.918862 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.914087 2570 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 07 23:13:02.918862 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.914089 2570 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 07 23:13:02.918862 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.914092 2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 07 23:13:02.918862 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.914094 2570 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 07 23:13:02.918862 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.914097 2570 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 07 23:13:02.918862 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.914099 2570 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 07 23:13:02.918862 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.914102 2570 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 07 23:13:02.918862 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.914104 2570 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 07 23:13:02.918862 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.914107 2570 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 07 23:13:02.918862 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.914109 2570 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 07 23:13:02.918862 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.914112 2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 07 23:13:02.918862 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.914114 2570 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 07 23:13:02.919320 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.914118 2570 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 07 23:13:02.919320 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.914120 2570 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 07 23:13:02.919320 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.914124 2570 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 07 23:13:02.919320 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.914126 2570 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 07 23:13:02.919320 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.914129 2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 07 23:13:02.919320 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.914132 2570 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 07 23:13:02.919320 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.914134 2570 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 07 23:13:02.919320 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.914137 2570 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 07 23:13:02.919320 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.914139 2570 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 07 23:13:02.919320 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.914142 2570 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 07 23:13:02.919320 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.914144 2570 feature_gate.go:328] unrecognized feature gate: Example Jul 07 23:13:02.919320 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.914148 2570 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 07 23:13:02.919320 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.914150 2570 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 07 23:13:02.919320 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.914153 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 07 23:13:02.919320 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.914155 2570 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 07 23:13:02.919320 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.914158 2570 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 07 23:13:02.919320 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.914162 2570 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 07 23:13:02.919320 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.914164 2570 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 07 23:13:02.919320 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.914167 2570 feature_gate.go:328] unrecognized feature gate: Example2 Jul 07 23:13:02.919320 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:02.914239 2570 flags.go:64] FLAG: --address="0.0.0.0" Jul 07 23:13:02.919320 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:02.914246 2570 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 07 23:13:02.919837 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:02.914252 2570 flags.go:64] FLAG: --anonymous-auth="true" Jul 07 23:13:02.919837 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:02.914257 2570 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 07 23:13:02.919837 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:02.914263 2570 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 07 23:13:02.919837 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:02.914268 2570 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 07 23:13:02.919837 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:02.914275 2570 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 07 23:13:02.919837 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:02.914281 2570 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 07 23:13:02.919837 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:02.914286 2570 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 07 23:13:02.919837 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:02.914289 2570 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 07 23:13:02.919837 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:02.914293 2570 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 07 23:13:02.919837 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:02.914296 2570 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 07 23:13:02.919837 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:02.914299 2570 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 07 23:13:02.919837 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:02.914304 2570 flags.go:64] FLAG: --cgroup-root="" Jul 07 23:13:02.919837 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:02.914308 2570 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 07 23:13:02.919837 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:02.914311 2570 flags.go:64] FLAG: --client-ca-file="" Jul 07 23:13:02.919837 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:02.914315 2570 flags.go:64] FLAG: --cloud-provider="external" Jul 07 23:13:02.919837 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:02.914318 2570 flags.go:64] FLAG: --cluster-dns="[]" Jul 07 23:13:02.919837 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:02.914323 2570 flags.go:64] FLAG: --cluster-domain="" Jul 07 23:13:02.919837 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:02.914326 2570 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 07 23:13:02.919837 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:02.914329 2570 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 07 23:13:02.919837 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:02.914332 2570 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 07 23:13:02.919837 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:02.914335 2570 flags.go:64] FLAG: --container-log-max-files="5" Jul 07 23:13:02.919837 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:02.914339 2570 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 07 23:13:02.919837 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:02.914342 2570 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 07 23:13:02.920387 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:02.914345 2570 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 07 23:13:02.920387 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:02.914348 2570 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 07 23:13:02.920387 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:02.914351 2570 flags.go:64] FLAG: --contention-profiling="false" Jul 07 23:13:02.920387 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:02.914354 2570 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 07 23:13:02.920387 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:02.914358 2570 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 07 23:13:02.920387 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:02.914361 2570 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 07 23:13:02.920387 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:02.914364 2570 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 07 23:13:02.920387 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:02.914368 2570 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 07 23:13:02.920387 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:02.914371 2570 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 07 23:13:02.920387 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:02.914374 2570 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 07 23:13:02.920387 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:02.914377 2570 flags.go:64] FLAG: --enable-load-reader="false" Jul 07 23:13:02.920387 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:02.914380 2570 flags.go:64] FLAG: --enable-server="true" Jul 07 23:13:02.920387 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:02.914383 2570 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 07 23:13:02.920387 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:02.914387 2570 flags.go:64] FLAG: --event-burst="100" Jul 07 23:13:02.920387 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:02.914390 2570 flags.go:64] FLAG: --event-qps="50" Jul 07 23:13:02.920387 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:02.914393 2570 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 07 23:13:02.920387 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:02.914396 2570 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 07 23:13:02.920387 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:02.914399 2570 flags.go:64] FLAG: --eviction-hard="" Jul 07 23:13:02.920387 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:02.914403 2570 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 07 23:13:02.920387 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:02.914405 2570 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 07 23:13:02.920387 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:02.914409 2570 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 07 23:13:02.920387 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:02.914413 2570 flags.go:64] FLAG: --eviction-soft="" Jul 07 23:13:02.920387 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:02.914416 2570 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 07 23:13:02.920387 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:02.914418 2570 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 07 23:13:02.920387 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:02.914422 2570 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 07 23:13:02.921064 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:02.914425 2570 flags.go:64] FLAG: --experimental-mounter-path="" Jul 07 23:13:02.921064 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:02.914428 2570 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 07 23:13:02.921064 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:02.914431 2570 flags.go:64] FLAG: --fail-swap-on="true" Jul 07 23:13:02.921064 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:02.914434 2570 flags.go:64] FLAG: --feature-gates="" Jul 07 23:13:02.921064 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:02.914438 2570 flags.go:64] FLAG: --file-check-frequency="20s" Jul 07 23:13:02.921064 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:02.914441 2570 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 07 23:13:02.921064 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:02.914443 2570 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 07 23:13:02.921064 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:02.914447 2570 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 07 23:13:02.921064 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:02.914450 2570 flags.go:64] FLAG: --healthz-port="10248" Jul 07 23:13:02.921064 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:02.914453 2570 flags.go:64] FLAG: --help="false" Jul 07 23:13:02.921064 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:02.914455 2570 flags.go:64] FLAG: --hostname-override="ip-10-0-134-140.ec2.internal" Jul 07 23:13:02.921064 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:02.914459 2570 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 07 23:13:02.921064 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:02.914475 2570 flags.go:64] FLAG: --http-check-frequency="20s" Jul 07 23:13:02.921064 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:02.914478 2570 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 07 23:13:02.921064 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:02.914482 2570 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 07 23:13:02.921064 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:02.914485 2570 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 07 23:13:02.921064 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:02.914488 2570 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 07 23:13:02.921064 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:02.914490 2570 flags.go:64] FLAG: --image-service-endpoint="" Jul 07 23:13:02.921064 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:02.914493 2570 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 07 23:13:02.921064 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:02.914496 2570 flags.go:64] FLAG: --kube-api-burst="100" Jul 07 23:13:02.921064 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:02.914499 2570 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 07 23:13:02.921064 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:02.914502 2570 flags.go:64] FLAG: --kube-api-qps="50" Jul 07 23:13:02.921064 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:02.914505 2570 flags.go:64] FLAG: --kube-reserved="" Jul 07 23:13:02.921064 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:02.914508 2570 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 07 23:13:02.921684 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:02.914510 2570 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 07 23:13:02.921684 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:02.914513 2570 flags.go:64] FLAG: --kubelet-cgroups="" Jul 07 23:13:02.921684 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:02.914516 2570 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 07 23:13:02.921684 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:02.914520 2570 flags.go:64] FLAG: --lock-file="" Jul 07 23:13:02.921684 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:02.914523 2570 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 07 23:13:02.921684 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:02.914526 2570 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 07 23:13:02.921684 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:02.914529 2570 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 07 23:13:02.921684 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:02.914534 2570 flags.go:64] FLAG: --log-json-split-stream="false" Jul 07 23:13:02.921684 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:02.914537 2570 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 07 23:13:02.921684 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:02.914540 2570 flags.go:64] FLAG: --log-text-split-stream="false" Jul 07 23:13:02.921684 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:02.914543 2570 flags.go:64] FLAG: --logging-format="text" Jul 07 23:13:02.921684 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:02.914546 2570 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 07 23:13:02.921684 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:02.914549 2570 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 07 23:13:02.921684 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:02.914552 2570 flags.go:64] FLAG: --manifest-url="" Jul 07 23:13:02.921684 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:02.914554 2570 flags.go:64] FLAG: --manifest-url-header="" Jul 07 23:13:02.921684 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:02.914562 2570 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 07 23:13:02.921684 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:02.914564 2570 flags.go:64] FLAG: --max-open-files="1000000" Jul 07 23:13:02.921684 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:02.914569 2570 flags.go:64] FLAG: --max-pods="110" Jul 07 23:13:02.921684 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:02.914572 2570 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 07 23:13:02.921684 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:02.914575 2570 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 07 23:13:02.921684 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:02.914578 2570 flags.go:64] FLAG: --memory-manager-policy="None" Jul 07 23:13:02.921684 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:02.914581 2570 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 07 23:13:02.921684 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:02.914584 2570 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 07 23:13:02.921684 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:02.914587 2570 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 07 23:13:02.921684 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:02.914590 2570 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 07 23:13:02.922275 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:02.914597 2570 flags.go:64] FLAG: --node-status-max-images="50" Jul 07 23:13:02.922275 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:02.914600 2570 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 07 23:13:02.922275 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:02.914603 2570 flags.go:64] FLAG: --oom-score-adj="-999" Jul 07 23:13:02.922275 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:02.914606 2570 flags.go:64] FLAG: --pod-cidr="" Jul 07 23:13:02.922275 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:02.914609 2570 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ce93f6b4197b9c62a26a3131f8057ffe9c9379c5079a4f62e5919b75151394e5" Jul 07 23:13:02.922275 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:02.914614 2570 flags.go:64] FLAG: --pod-manifest-path="" Jul 07 23:13:02.922275 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:02.914617 2570 flags.go:64] FLAG: --pod-max-pids="-1" Jul 07 23:13:02.922275 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:02.914620 2570 flags.go:64] FLAG: --pods-per-core="0" Jul 07 23:13:02.922275 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:02.914623 2570 flags.go:64] FLAG: --port="10250" Jul 07 23:13:02.922275 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:02.914626 2570 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 07 23:13:02.922275 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:02.914631 2570 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-01c1fdd6871ebaf4e" Jul 07 23:13:02.922275 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:02.914634 2570 flags.go:64] FLAG: --qos-reserved="" Jul 07 23:13:02.922275 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:02.914637 2570 flags.go:64] FLAG: --read-only-port="10255" Jul 07 23:13:02.922275 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:02.914640 2570 flags.go:64] FLAG: --register-node="true" Jul 07 23:13:02.922275 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:02.914642 2570 flags.go:64] FLAG: --register-with-taints="" Jul 07 23:13:02.922275 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:02.914646 2570 flags.go:64] FLAG: --registry-burst="10" Jul 07 23:13:02.922275 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:02.914649 2570 flags.go:64] FLAG: --registry-qps="5" Jul 07 23:13:02.922275 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:02.914652 2570 flags.go:64] FLAG: --reserved-cpus="" Jul 07 23:13:02.922275 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:02.914655 2570 flags.go:64] FLAG: --reserved-memory="" Jul 07 23:13:02.922275 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:02.914658 2570 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 07 23:13:02.922275 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:02.914661 2570 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 07 23:13:02.922275 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:02.914664 2570 flags.go:64] FLAG: --rotate-certificates="false" Jul 07 23:13:02.922275 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:02.914667 2570 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 07 23:13:02.922275 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:02.914670 2570 flags.go:64] FLAG: --runonce="false" Jul 07 23:13:02.922275 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:02.914672 2570 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 07 23:13:02.922897 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:02.914675 2570 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 07 23:13:02.922897 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:02.914678 2570 flags.go:64] FLAG: --seccomp-default="false" Jul 07 23:13:02.922897 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:02.914681 2570 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 07 23:13:02.922897 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:02.914684 2570 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 07 23:13:02.922897 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:02.914687 2570 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 07 23:13:02.922897 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:02.914690 2570 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 07 23:13:02.922897 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:02.914692 2570 flags.go:64] FLAG: --storage-driver-password="root" Jul 07 23:13:02.922897 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:02.914695 2570 flags.go:64] FLAG: --storage-driver-secure="false" Jul 07 23:13:02.922897 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:02.914698 2570 flags.go:64] FLAG: --storage-driver-table="stats" Jul 07 23:13:02.922897 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:02.914701 2570 flags.go:64] FLAG: --storage-driver-user="root" Jul 07 23:13:02.922897 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:02.914704 2570 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 07 23:13:02.922897 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:02.914707 2570 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 07 23:13:02.922897 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:02.914710 2570 flags.go:64] FLAG: --system-cgroups="" Jul 07 23:13:02.922897 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:02.914712 2570 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 07 23:13:02.922897 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:02.914718 2570 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 07 23:13:02.922897 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:02.914720 2570 flags.go:64] FLAG: --tls-cert-file="" Jul 07 23:13:02.922897 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:02.914723 2570 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 07 23:13:02.922897 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:02.914728 2570 flags.go:64] FLAG: --tls-min-version="" Jul 07 23:13:02.922897 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:02.914731 2570 flags.go:64] FLAG: --tls-private-key-file="" Jul 07 23:13:02.922897 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:02.914734 2570 flags.go:64] FLAG: --topology-manager-policy="none" Jul 07 23:13:02.922897 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:02.914736 2570 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 07 23:13:02.922897 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:02.914739 2570 flags.go:64] FLAG: --topology-manager-scope="container" Jul 07 23:13:02.922897 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:02.914742 2570 flags.go:64] FLAG: --v="2" Jul 07 23:13:02.922897 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:02.914746 2570 flags.go:64] FLAG: --version="false" Jul 07 23:13:02.922897 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:02.914751 2570 flags.go:64] FLAG: --vmodule="" Jul 07 23:13:02.923640 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:02.914754 2570 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 07 23:13:02.923640 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:02.914757 2570 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 07 23:13:02.923640 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.914869 2570 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 07 23:13:02.923640 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.914873 2570 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 07 23:13:02.923640 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.914876 2570 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 07 23:13:02.923640 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.914880 2570 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 07 23:13:02.923640 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.914884 2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 07 23:13:02.923640 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.914888 2570 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 07 23:13:02.923640 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.914892 2570 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 07 23:13:02.923640 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.914895 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 07 23:13:02.923640 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.914897 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 07 23:13:02.923640 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.914900 2570 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 07 23:13:02.923640 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.914903 2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 07 23:13:02.923640 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.914906 2570 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 07 23:13:02.923640 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.914909 2570 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 07 23:13:02.923640 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.914911 2570 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 07 23:13:02.923640 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.914915 2570 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 07 23:13:02.923640 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.914917 2570 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 07 23:13:02.923640 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.914920 2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 07 23:13:02.924096 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.914923 2570 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 07 23:13:02.924096 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.914926 2570 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 07 23:13:02.924096 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.914929 2570 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 07 23:13:02.924096 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.914931 2570 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 07 23:13:02.924096 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.914934 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 07 23:13:02.924096 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.914938 2570 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 07 23:13:02.924096 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.914941 2570 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 07 23:13:02.924096 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.914944 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 07 23:13:02.924096 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.914946 2570 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 07 23:13:02.924096 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.914949 2570 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 07 23:13:02.924096 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.914953 2570 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 07 23:13:02.924096 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.914956 2570 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 07 23:13:02.924096 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.914959 2570 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 07 23:13:02.924096 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.914963 2570 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 07 23:13:02.924096 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.914966 2570 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 07 23:13:02.924096 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.914968 2570 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 07 23:13:02.924096 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.914971 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 07 23:13:02.924096 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.914974 2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 07 23:13:02.924096 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.914977 2570 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 07 23:13:02.924618 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.914979 2570 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 07 23:13:02.924618 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.914982 2570 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 07 23:13:02.924618 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.914985 2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 07 23:13:02.924618 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.914988 2570 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 07 23:13:02.924618 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.914990 2570 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 07 23:13:02.924618 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.914993 2570 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 07 23:13:02.924618 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.914996 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 07 23:13:02.924618 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.914998 2570 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 07 23:13:02.924618 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.915001 2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 07 23:13:02.924618 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.915003 2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 07 23:13:02.924618 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.915006 2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 07 23:13:02.924618 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.915008 2570 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 07 23:13:02.924618 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.915011 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 07 23:13:02.924618 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.915013 2570 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 07 23:13:02.924618 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.915016 2570 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 07 23:13:02.924618 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.915018 2570 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 07 23:13:02.924618 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.915021 2570 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 07 23:13:02.924618 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.915024 2570 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 07 23:13:02.924618 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.915028 2570 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 07 23:13:02.924618 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.915030 2570 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 07 23:13:02.925115 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.915033 2570 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 07 23:13:02.925115 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.915035 2570 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 07 23:13:02.925115 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.915038 2570 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 07 23:13:02.925115 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.915040 2570 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 07 23:13:02.925115 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.915043 2570 feature_gate.go:328] unrecognized feature gate: Example2 Jul 07 23:13:02.925115 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.915045 2570 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 07 23:13:02.925115 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.915048 2570 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 07 23:13:02.925115 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.915051 2570 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 07 23:13:02.925115 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.915054 2570 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 07 23:13:02.925115 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.915056 2570 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 07 23:13:02.925115 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.915059 2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 07 23:13:02.925115 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.915061 2570 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 07 23:13:02.925115 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.915064 2570 feature_gate.go:328] unrecognized feature gate: Example Jul 07 23:13:02.925115 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.915066 2570 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 07 23:13:02.925115 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.915070 2570 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 07 23:13:02.925115 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.915072 2570 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 07 23:13:02.925115 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.915075 2570 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 07 23:13:02.925115 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.915078 2570 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 07 23:13:02.925115 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.915080 2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 07 23:13:02.925115 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.915083 2570 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 07 23:13:02.925623 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.915085 2570 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 07 23:13:02.925623 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.915088 2570 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 07 23:13:02.925623 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.915090 2570 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 07 23:13:02.925623 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.915093 2570 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 07 23:13:02.925623 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.915096 2570 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 07 23:13:02.925623 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.915099 2570 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 07 23:13:02.925623 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.915101 2570 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 07 23:13:02.925623 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.915103 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 07 23:13:02.925623 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.915106 2570 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 07 23:13:02.925623 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.915109 2570 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 07 23:13:02.925623 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.915113 2570 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 07 23:13:02.925623 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.915116 2570 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 07 23:13:02.925623 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.915118 2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 07 23:13:02.925623 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.915121 2570 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 07 23:13:02.925623 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.915123 2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 07 23:13:02.925623 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.915126 2570 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 07 23:13:02.925623 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.915128 2570 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 07 23:13:02.925623 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.915131 2570 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 07 23:13:02.925623 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.915133 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 07 23:13:02.925623 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.915136 2570 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 07 23:13:02.926106 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.915139 2570 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 07 23:13:02.926106 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.915141 2570 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 07 23:13:02.926106 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.915144 2570 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 07 23:13:02.926106 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:02.915766 2570 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 07 23:13:02.926106 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:02.922068 2570 server.go:529] "Kubelet version" kubeletVersion="v1.34.8" Jul 07 23:13:02.926106 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:02.922082 2570 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 07 23:13:02.926106 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:02.922877 2570 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 07 23:13:02.926106 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:02.922884 2570 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 07 23:13:02.926106 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.922944 2570 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 07 23:13:02.926106 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.922949 2570 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 07 23:13:02.926106 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.922952 2570 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 07 23:13:02.926106 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.922955 2570 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 07 23:13:02.926106 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.922958 2570 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 07 23:13:02.926106 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.922962 2570 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 07 23:13:02.926106 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.922965 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 07 23:13:02.926518 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.922968 2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 07 23:13:02.926518 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.922971 2570 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 07 23:13:02.926518 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.922974 2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 07 23:13:02.926518 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.922978 2570 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 07 23:13:02.926518 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.922982 2570 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 07 23:13:02.926518 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.922985 2570 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 07 23:13:02.926518 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.922988 2570 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 07 23:13:02.926518 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.922991 2570 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 07 23:13:02.926518 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.922994 2570 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 07 23:13:02.926518 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.922997 2570 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 07 23:13:02.926518 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.923000 2570 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 07 23:13:02.926518 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.923003 2570 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 07 23:13:02.926518 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.923006 2570 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 07 23:13:02.926518 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.923009 2570 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 07 23:13:02.926518 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.923012 2570 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 07 23:13:02.926518 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.923014 2570 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 07 23:13:02.926518 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.923017 2570 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 07 23:13:02.926518 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.923020 2570 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 07 23:13:02.926518 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.923023 2570 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 07 23:13:02.926518 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.923026 2570 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 07 23:13:02.927008 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.923028 2570 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 07 23:13:02.927008 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.923031 2570 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 07 23:13:02.927008 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.923036 2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 07 23:13:02.927008 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.923038 2570 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 07 23:13:02.927008 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.923041 2570 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 07 23:13:02.927008 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.923044 2570 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 07 23:13:02.927008 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.923047 2570 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 07 23:13:02.927008 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.923049 2570 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 07 23:13:02.927008 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.923052 2570 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 07 23:13:02.927008 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.923054 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 07 23:13:02.927008 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.923057 2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 07 23:13:02.927008 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.923059 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 07 23:13:02.927008 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.923062 2570 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 07 23:13:02.927008 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.923065 2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 07 23:13:02.927008 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.923067 2570 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 07 23:13:02.927008 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.923070 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 07 23:13:02.927008 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.923072 2570 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 07 23:13:02.927008 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.923075 2570 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 07 23:13:02.927008 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.923078 2570 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 07 23:13:02.927488 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.923080 2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 07 23:13:02.927488 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.923083 2570 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 07 23:13:02.927488 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.923086 2570 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 07 23:13:02.927488 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.923088 2570 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 07 23:13:02.927488 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.923091 2570 feature_gate.go:328] unrecognized feature gate: Example Jul 07 23:13:02.927488 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.923095 2570 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 07 23:13:02.927488 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.923099 2570 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 07 23:13:02.927488 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.923101 2570 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 07 23:13:02.927488 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.923104 2570 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 07 23:13:02.927488 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.923107 2570 feature_gate.go:328] unrecognized feature gate: Example2 Jul 07 23:13:02.927488 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.923110 2570 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 07 23:13:02.927488 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.923113 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 07 23:13:02.927488 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.923115 2570 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 07 23:13:02.927488 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.923118 2570 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 07 23:13:02.927488 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.923121 2570 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 07 23:13:02.927488 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.923125 2570 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 07 23:13:02.927488 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.923128 2570 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 07 23:13:02.927488 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.923131 2570 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 07 23:13:02.927488 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.923134 2570 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 07 23:13:02.927488 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.923137 2570 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 07 23:13:02.927960 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.923139 2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 07 23:13:02.927960 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.923143 2570 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 07 23:13:02.927960 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.923147 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 07 23:13:02.927960 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.923150 2570 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 07 23:13:02.927960 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.923152 2570 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 07 23:13:02.927960 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.923155 2570 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 07 23:13:02.927960 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.923158 2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 07 23:13:02.927960 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.923161 2570 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 07 23:13:02.927960 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.923163 2570 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 07 23:13:02.927960 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.923166 2570 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 07 23:13:02.927960 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.923168 2570 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 07 23:13:02.927960 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.923171 2570 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 07 23:13:02.927960 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.923174 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 07 23:13:02.927960 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.923176 2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 07 23:13:02.927960 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.923179 2570 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 07 23:13:02.927960 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.923181 2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 07 23:13:02.927960 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.923184 2570 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 07 23:13:02.927960 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.923186 2570 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 07 23:13:02.927960 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.923189 2570 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 07 23:13:02.927960 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.923191 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 07 23:13:02.928458 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.923194 2570 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 07 23:13:02.928458 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.923196 2570 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 07 23:13:02.928458 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.923199 2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 07 23:13:02.928458 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.923201 2570 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 07 23:13:02.928458 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.923204 2570 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 07 23:13:02.928458 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.923206 2570 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 07 23:13:02.928458 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.923209 2570 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 07 23:13:02.928458 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.923212 2570 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 07 23:13:02.928458 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.923215 2570 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 07 23:13:02.928458 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.923218 2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 07 23:13:02.928458 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.923221 2570 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 07 23:13:02.928458 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.923223 2570 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 07 23:13:02.928458 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.923226 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 07 23:13:02.928458 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:02.923231 2570 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 07 23:13:02.928458 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.923341 2570 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 07 23:13:02.928863 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.923347 2570 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 07 23:13:02.928863 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.923350 2570 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 07 23:13:02.928863 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.923353 2570 feature_gate.go:328] unrecognized feature gate: Example Jul 07 23:13:02.928863 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.923356 2570 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 07 23:13:02.928863 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.923358 2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 07 23:13:02.928863 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.923361 2570 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 07 23:13:02.928863 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.923363 2570 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 07 23:13:02.928863 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.923366 2570 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 07 23:13:02.928863 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.923369 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 07 23:13:02.928863 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.923371 2570 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 07 23:13:02.928863 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.923374 2570 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 07 23:13:02.928863 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.923377 2570 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 07 23:13:02.928863 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.923379 2570 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 07 23:13:02.928863 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.923382 2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 07 23:13:02.928863 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.923384 2570 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 07 23:13:02.928863 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.923387 2570 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 07 23:13:02.928863 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.923389 2570 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 07 23:13:02.928863 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.923392 2570 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 07 23:13:02.928863 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.923395 2570 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 07 23:13:02.928863 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.923397 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 07 23:13:02.929348 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.923400 2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 07 23:13:02.929348 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.923403 2570 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 07 23:13:02.929348 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.923405 2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 07 23:13:02.929348 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.923408 2570 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 07 23:13:02.929348 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.923411 2570 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 07 23:13:02.929348 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.923414 2570 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 07 23:13:02.929348 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.923416 2570 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 07 23:13:02.929348 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.923419 2570 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 07 23:13:02.929348 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.923422 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 07 23:13:02.929348 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.923425 2570 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 07 23:13:02.929348 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.923427 2570 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 07 23:13:02.929348 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.923430 2570 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 07 23:13:02.929348 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.923432 2570 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 07 23:13:02.929348 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.923435 2570 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 07 23:13:02.929348 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.923438 2570 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 07 23:13:02.929348 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.923440 2570 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 07 23:13:02.929348 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.923442 2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 07 23:13:02.929348 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.923445 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 07 23:13:02.929348 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.923447 2570 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 07 23:13:02.929834 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.923450 2570 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 07 23:13:02.929834 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.923453 2570 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 07 23:13:02.929834 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.923455 2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 07 23:13:02.929834 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.923458 2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 07 23:13:02.929834 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.923476 2570 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 07 23:13:02.929834 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.923482 2570 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 07 23:13:02.929834 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.923486 2570 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 07 23:13:02.929834 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.923489 2570 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 07 23:13:02.929834 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.923491 2570 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 07 23:13:02.929834 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.923494 2570 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 07 23:13:02.929834 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.923497 2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 07 23:13:02.929834 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.923501 2570 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 07 23:13:02.929834 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.923504 2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 07 23:13:02.929834 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.923507 2570 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 07 23:13:02.929834 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.923509 2570 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 07 23:13:02.929834 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.923512 2570 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 07 23:13:02.929834 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.923516 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 07 23:13:02.929834 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.923518 2570 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 07 23:13:02.929834 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.923521 2570 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 07 23:13:02.930294 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.923524 2570 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 07 23:13:02.930294 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.923526 2570 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 07 23:13:02.930294 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.923530 2570 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 07 23:13:02.930294 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.923533 2570 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 07 23:13:02.930294 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.923535 2570 feature_gate.go:328] unrecognized feature gate: Example2 Jul 07 23:13:02.930294 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.923538 2570 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 07 23:13:02.930294 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.923540 2570 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 07 23:13:02.930294 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.923543 2570 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 07 23:13:02.930294 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.923545 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 07 23:13:02.930294 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.923548 2570 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 07 23:13:02.930294 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.923550 2570 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 07 23:13:02.930294 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.923552 2570 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 07 23:13:02.930294 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.923555 2570 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 07 23:13:02.930294 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.923557 2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 07 23:13:02.930294 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.923560 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 07 23:13:02.930294 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.923562 2570 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 07 23:13:02.930294 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.923565 2570 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 07 23:13:02.930294 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.923567 2570 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 07 23:13:02.930294 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.923570 2570 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 07 23:13:02.930767 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.923574 2570 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 07 23:13:02.930767 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.923577 2570 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 07 23:13:02.930767 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.923580 2570 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 07 23:13:02.930767 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.923582 2570 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 07 23:13:02.930767 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.923585 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 07 23:13:02.930767 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.923588 2570 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 07 23:13:02.930767 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.923590 2570 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 07 23:13:02.930767 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.923593 2570 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 07 23:13:02.930767 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.923595 2570 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 07 23:13:02.930767 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.923598 2570 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 07 23:13:02.930767 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.923601 2570 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 07 23:13:02.930767 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.923603 2570 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 07 23:13:02.930767 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.923606 2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 07 23:13:02.930767 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.923609 2570 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 07 23:13:02.930767 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.923612 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 07 23:13:02.930767 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.923614 2570 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 07 23:13:02.930767 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.923617 2570 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 07 23:13:02.930767 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.923620 2570 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 07 23:13:02.930767 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.923622 2570 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 07 23:13:02.930767 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.923625 2570 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 07 23:13:02.931242 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:02.923627 2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 07 23:13:02.931242 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:02.923632 2570 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 07 23:13:02.931242 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:02.923736 2570 server.go:962] "Client rotation is on, will bootstrap in background" Jul 07 23:13:02.931242 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:02.925654 2570 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 07 23:13:02.931242 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:02.926988 2570 server.go:1019] "Starting client certificate rotation" Jul 07 23:13:02.931242 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:02.927082 2570 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 07 23:13:02.931242 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:02.927116 2570 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 07 23:13:02.931242 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:02.929381 2570 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 07 23:13:02.931242 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:02.931189 2570 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 07 23:13:02.940271 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:02.940252 2570 log.go:25] "Validated CRI v1 runtime API" Jul 07 23:13:02.942200 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:02.942180 2570 log.go:25] "Validated CRI v1 image API" Jul 07 23:13:02.945226 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:02.945212 2570 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 07 23:13:02.947621 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:02.947599 2570 fs.go:135] Filesystem UUIDs: map[197a88ea-9081-4474-84b0-f55762fd1a60:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 c13a2a09-2008-445e-ad57-c9f9f12de7c6:/dev/nvme0n1p4] Jul 07 23:13:02.947712 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:02.947619 2570 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 07 23:13:02.952760 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:02.952649 2570 manager.go:216] Machine: {Timestamp:2026-07-07 23:13:02.951294444 +0000 UTC m=+0.348295969 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3100740 MemoryCapacity:33164496896 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec27be115f68951524889751e7dee854 SystemUUID:ec27be11-5f68-9515-2488-9751e7dee854 BootID:19f8d037-adbf-4076-8d5d-48d87a4027b7 Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:b2:ad:e0:2c:81 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:b2:ad:e0:2c:81 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:02:5c:e4:38:7d:0e Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164496896 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 07 23:13:02.952760 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:02.952748 2570 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 07 23:13:02.952922 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:02.952855 2570 manager.go:232] Version: {KernelVersion:5.14.0-570.125.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260630-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 07 23:13:02.955636 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:02.955578 2570 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 07 23:13:02.955788 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:02.955613 2570 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-134-140.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 07 23:13:02.956294 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:02.956284 2570 topology_manager.go:138] "Creating topology manager with none policy" Jul 07 23:13:02.956325 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:02.956297 2570 container_manager_linux.go:309] "Creating device plugin manager" Jul 07 23:13:02.956325 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:02.956304 2570 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 07 23:13:02.956811 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:02.956801 2570 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 07 23:13:02.957748 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:02.957733 2570 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 07 23:13:02.958027 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:02.958016 2570 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 07 23:13:02.961310 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:02.961298 2570 state_mem.go:36] "Initialized new in-memory state store" Jul 07 23:13:02.961422 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:02.961413 2570 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 07 23:13:02.961457 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:02.961443 2570 kubelet.go:486] "Attempting to sync node with API server" Jul 07 23:13:02.961457 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:02.961455 2570 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 07 23:13:02.961565 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:02.961481 2570 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 07 23:13:02.961565 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:02.961501 2570 kubelet.go:398] "Adding apiserver pod source" Jul 07 23:13:02.961565 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:02.961514 2570 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 07 23:13:02.961940 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:02.961929 2570 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 07 23:13:02.961985 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:02.961954 2570 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 07 23:13:02.964099 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:02.964082 2570 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.8-3.rhaos4.21.git91b4b0b.el9" apiVersion="v1" Jul 07 23:13:02.966044 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:02.966031 2570 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 07 23:13:02.966094 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:02.966061 2570 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 07 23:13:02.967055 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:02.967035 2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 07 23:13:02.967055 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:02.967052 2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 07 23:13:02.967172 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:02.967060 2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 07 23:13:02.967172 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:02.967068 2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 07 23:13:02.967172 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:02.967078 2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 07 23:13:02.967172 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:02.967084 2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 07 23:13:02.967172 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:02.967092 2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 07 23:13:02.967172 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:02.967106 2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 07 23:13:02.967172 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:02.967115 2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 07 23:13:02.967172 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:02.967120 2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 07 23:13:02.967172 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:02.967129 2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 07 23:13:02.967172 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:02.967135 2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 07 23:13:02.968424 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:02.968413 2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 07 23:13:02.968424 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:02.968423 2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 07 23:13:02.971953 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:02.971936 2570 server.go:1268] "Started kubelet" Jul 07 23:13:02.972046 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:02.971988 2570 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 07 23:13:02.972532 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:02.972515 2570 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-134-140.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 07 23:13:02.972667 ip-10-0-134-140 kubenswrapper[2570]: E0707 23:13:02.972648 2570 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 07 23:13:02.972705 ip-10-0-134-140 systemd[1]: Started Kubernetes Kubelet. Jul 07 23:13:02.972889 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:02.972776 2570 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 07 23:13:02.972889 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:02.972863 2570 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 07 23:13:02.973133 ip-10-0-134-140 kubenswrapper[2570]: E0707 23:13:02.973110 2570 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-134-140.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 07 23:13:02.977174 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:02.977072 2570 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 07 23:13:02.978460 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:02.978442 2570 server.go:310] "Adding debug handlers to kubelet server" Jul 07 23:13:02.979647 ip-10-0-134-140 kubenswrapper[2570]: E0707 23:13:02.979267 2570 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 07 23:13:02.981562 ip-10-0-134-140 kubenswrapper[2570]: E0707 23:13:02.980654 2570 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-134-140.ec2.internal.18c024dcc00519c6 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-134-140.ec2.internal,UID:ip-10-0-134-140.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-134-140.ec2.internal,},FirstTimestamp:2026-07-07 23:13:02.971681222 +0000 UTC m=+0.368682724,LastTimestamp:2026-07-07 23:13:02.971681222 +0000 UTC m=+0.368682724,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-134-140.ec2.internal,}" Jul 07 23:13:02.984793 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:02.984776 2570 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 07 23:13:02.984893 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:02.984780 2570 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 07 23:13:02.984893 ip-10-0-134-140 kubenswrapper[2570]: E0707 23:13:02.984882 2570 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-140.ec2.internal\" not found" Jul 07 23:13:02.985372 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:02.985356 2570 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 07 23:13:02.985372 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:02.985358 2570 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 07 23:13:02.985521 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:02.985383 2570 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 07 23:13:02.985574 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:02.985524 2570 reconstruct.go:97] "Volume reconstruction finished" Jul 07 23:13:02.985574 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:02.985533 2570 reconciler.go:29] "Reconciler: start to sync state" Jul 07 23:13:02.985865 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:02.985849 2570 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 07 23:13:02.985865 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:02.985863 2570 factory.go:55] Registering systemd factory Jul 07 23:13:02.985865 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:02.985869 2570 factory.go:223] Registration of the systemd container factory successfully Jul 07 23:13:02.986045 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:02.986033 2570 factory.go:153] Registering CRI-O factory Jul 07 23:13:02.986077 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:02.986050 2570 factory.go:223] Registration of the crio container factory successfully Jul 07 23:13:02.986077 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:02.986071 2570 factory.go:103] Registering Raw factory Jul 07 23:13:02.986146 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:02.986082 2570 manager.go:1196] Started watching for new ooms in manager Jul 07 23:13:02.986513 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:02.986500 2570 manager.go:349] Starting recovery of all containers Jul 07 23:13:02.992900 ip-10-0-134-140 kubenswrapper[2570]: E0707 23:13:02.992870 2570 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-134-140.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 07 23:13:02.993022 ip-10-0-134-140 kubenswrapper[2570]: E0707 23:13:02.993001 2570 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 07 23:13:02.996841 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:02.996821 2570 manager.go:354] Recovery completed Jul 07 23:13:02.998596 ip-10-0-134-140 kubenswrapper[2570]: E0707 23:13:02.998577 2570 watcher.go:152] Failed to watch directory "/sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service": inotify_add_watch /sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service: no such file or directory Jul 07 23:13:03.001862 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:03.001849 2570 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 23:13:03.003628 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:03.003611 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-140.ec2.internal" event="NodeHasSufficientMemory" Jul 07 23:13:03.003710 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:03.003646 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-140.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 23:13:03.003710 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:03.003660 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-140.ec2.internal" event="NodeHasSufficientPID" Jul 07 23:13:03.004303 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:03.004288 2570 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 07 23:13:03.004303 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:03.004303 2570 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 07 23:13:03.004422 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:03.004318 2570 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 23:13:03.004422 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:03.004321 2570 state_mem.go:36] "Initialized new in-memory state store" Jul 07 23:13:03.005149 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:03.005133 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-140.ec2.internal" event="NodeHasSufficientMemory" Jul 07 23:13:03.005202 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:03.005162 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-140.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 23:13:03.005202 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:03.005181 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-140.ec2.internal" event="NodeHasSufficientPID" Jul 07 23:13:03.006255 ip-10-0-134-140 kubenswrapper[2570]: E0707 23:13:03.006155 2570 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-134-140.ec2.internal.18c024dcc1ec9673 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-134-140.ec2.internal,UID:ip-10-0-134-140.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-134-140.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-134-140.ec2.internal,},FirstTimestamp:2026-07-07 23:13:03.003629171 +0000 UTC m=+0.400630675,LastTimestamp:2026-07-07 23:13:03.003629171 +0000 UTC m=+0.400630675,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-134-140.ec2.internal,}" Jul 07 23:13:03.010191 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:03.010175 2570 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-49bw8" Jul 07 23:13:03.015879 ip-10-0-134-140 kubenswrapper[2570]: E0707 23:13:03.015796 2570 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-134-140.ec2.internal.18c024dcc1ecf2c1 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-134-140.ec2.internal,UID:ip-10-0-134-140.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-134-140.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-134-140.ec2.internal,},FirstTimestamp:2026-07-07 23:13:03.003652801 +0000 UTC m=+0.400654305,LastTimestamp:2026-07-07 23:13:03.003652801 +0000 UTC m=+0.400654305,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-134-140.ec2.internal,}" Jul 07 23:13:03.018850 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:03.018834 2570 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-49bw8" Jul 07 23:13:03.085944 ip-10-0-134-140 kubenswrapper[2570]: E0707 23:13:03.085920 2570 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-140.ec2.internal\" not found" Jul 07 23:13:03.157758 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:03.089052 2570 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 07 23:13:03.157758 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:03.090203 2570 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 07 23:13:03.157758 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:03.090241 2570 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 07 23:13:03.157758 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:03.090275 2570 kubelet.go:2448] "Starting kubelet main sync loop" Jul 07 23:13:03.157758 ip-10-0-134-140 kubenswrapper[2570]: E0707 23:13:03.090313 2570 kubelet.go:2472] "Skipping pod synchronization" err="[container runtime status check may not have completed yet, PLEG is not healthy: pleg has yet to be successful]" Jul 07 23:13:03.157758 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:03.092725 2570 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 07 23:13:03.157758 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:03.157360 2570 policy_none.go:49] "None policy: Start" Jul 07 23:13:03.157758 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:03.157379 2570 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 07 23:13:03.157758 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:03.157392 2570 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 07 23:13:03.158477 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:03.158453 2570 policy_none.go:47] "Start" Jul 07 23:13:03.186273 ip-10-0-134-140 kubenswrapper[2570]: E0707 23:13:03.186250 2570 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-140.ec2.internal\" not found" Jul 07 23:13:03.191399 ip-10-0-134-140 kubenswrapper[2570]: E0707 23:13:03.191384 2570 kubelet.go:2472] "Skipping pod synchronization" err="container runtime status check may not have completed yet" Jul 07 23:13:03.191963 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:03.191953 2570 manager.go:337] "Starting Device Plugin manager" Jul 07 23:13:03.192018 ip-10-0-134-140 kubenswrapper[2570]: E0707 23:13:03.191981 2570 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 07 23:13:03.192018 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:03.191990 2570 server.go:87] "Starting device plugin registration server" Jul 07 23:13:03.192230 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:03.192219 2570 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 07 23:13:03.192286 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:03.192232 2570 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 07 23:13:03.192360 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:03.192344 2570 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 07 23:13:03.192443 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:03.192426 2570 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 07 23:13:03.192443 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:03.192442 2570 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 07 23:13:03.192980 ip-10-0-134-140 kubenswrapper[2570]: E0707 23:13:03.192953 2570 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 07 23:13:03.193057 ip-10-0-134-140 kubenswrapper[2570]: E0707 23:13:03.193001 2570 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-134-140.ec2.internal\" not found" Jul 07 23:13:03.200127 ip-10-0-134-140 kubenswrapper[2570]: E0707 23:13:03.200110 2570 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-134-140.ec2.internal\" not found" node="ip-10-0-134-140.ec2.internal" Jul 07 23:13:03.292613 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:03.292528 2570 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 23:13:03.294092 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:03.294076 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-140.ec2.internal" event="NodeHasSufficientMemory" Jul 07 23:13:03.294209 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:03.294112 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-140.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 23:13:03.294209 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:03.294126 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-140.ec2.internal" event="NodeHasSufficientPID" Jul 07 23:13:03.294209 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:03.294155 2570 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-134-140.ec2.internal" Jul 07 23:13:03.303587 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:03.303569 2570 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-134-140.ec2.internal" Jul 07 23:13:03.303638 ip-10-0-134-140 kubenswrapper[2570]: E0707 23:13:03.303590 2570 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-134-140.ec2.internal\": node \"ip-10-0-134-140.ec2.internal\" not found" Jul 07 23:13:03.328736 ip-10-0-134-140 kubenswrapper[2570]: E0707 23:13:03.328576 2570 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-140.ec2.internal\" not found" Jul 07 23:13:03.392054 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:03.392009 2570 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["kube-system/kube-apiserver-proxy-ip-10-0-134-140.ec2.internal","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-140.ec2.internal"] Jul 07 23:13:03.392127 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:03.392091 2570 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 23:13:03.393055 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:03.393040 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-140.ec2.internal" event="NodeHasSufficientMemory" Jul 07 23:13:03.393120 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:03.393067 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-140.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 23:13:03.393120 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:03.393080 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-140.ec2.internal" event="NodeHasSufficientPID" Jul 07 23:13:03.394404 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:03.394392 2570 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 23:13:03.394539 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:03.394525 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-140.ec2.internal" Jul 07 23:13:03.394585 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:03.394555 2570 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 23:13:03.395141 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:03.395127 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-140.ec2.internal" event="NodeHasSufficientMemory" Jul 07 23:13:03.395265 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:03.395151 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-140.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 23:13:03.395265 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:03.395160 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-140.ec2.internal" event="NodeHasSufficientPID" Jul 07 23:13:03.395265 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:03.395129 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-140.ec2.internal" event="NodeHasSufficientMemory" Jul 07 23:13:03.395265 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:03.395222 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-140.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 23:13:03.395265 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:03.395231 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-140.ec2.internal" event="NodeHasSufficientPID" Jul 07 23:13:03.396317 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:03.396303 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-140.ec2.internal" Jul 07 23:13:03.396386 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:03.396326 2570 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 23:13:03.396966 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:03.396950 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-140.ec2.internal" event="NodeHasSufficientMemory" Jul 07 23:13:03.397030 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:03.396974 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-140.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 23:13:03.397030 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:03.396984 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-140.ec2.internal" event="NodeHasSufficientPID" Jul 07 23:13:03.424073 ip-10-0-134-140 kubenswrapper[2570]: E0707 23:13:03.424054 2570 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-134-140.ec2.internal\" not found" node="ip-10-0-134-140.ec2.internal" Jul 07 23:13:03.428540 ip-10-0-134-140 kubenswrapper[2570]: E0707 23:13:03.428526 2570 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-134-140.ec2.internal\" not found" node="ip-10-0-134-140.ec2.internal" Jul 07 23:13:03.429539 ip-10-0-134-140 kubenswrapper[2570]: E0707 23:13:03.429525 2570 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-140.ec2.internal\" not found" Jul 07 23:13:03.487370 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:03.487348 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/7257f873d4401d71530f0192e065f67c-config\") pod \"kube-apiserver-proxy-ip-10-0-134-140.ec2.internal\" (UID: \"7257f873d4401d71530f0192e065f67c\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-134-140.ec2.internal" Jul 07 23:13:03.487448 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:03.487373 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/3bc0442220b7598e60bd507f09022656-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-134-140.ec2.internal\" (UID: \"3bc0442220b7598e60bd507f09022656\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-140.ec2.internal" Jul 07 23:13:03.487448 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:03.487392 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3bc0442220b7598e60bd507f09022656-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-134-140.ec2.internal\" (UID: \"3bc0442220b7598e60bd507f09022656\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-140.ec2.internal" Jul 07 23:13:03.530345 ip-10-0-134-140 kubenswrapper[2570]: E0707 23:13:03.530320 2570 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-140.ec2.internal\" not found" Jul 07 23:13:03.587762 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:03.587688 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/7257f873d4401d71530f0192e065f67c-config\") pod \"kube-apiserver-proxy-ip-10-0-134-140.ec2.internal\" (UID: \"7257f873d4401d71530f0192e065f67c\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-134-140.ec2.internal" Jul 07 23:13:03.587762 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:03.587725 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/3bc0442220b7598e60bd507f09022656-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-134-140.ec2.internal\" (UID: \"3bc0442220b7598e60bd507f09022656\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-140.ec2.internal" Jul 07 23:13:03.587762 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:03.587748 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3bc0442220b7598e60bd507f09022656-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-134-140.ec2.internal\" (UID: \"3bc0442220b7598e60bd507f09022656\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-140.ec2.internal" Jul 07 23:13:03.587944 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:03.587821 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/7257f873d4401d71530f0192e065f67c-config\") pod \"kube-apiserver-proxy-ip-10-0-134-140.ec2.internal\" (UID: \"7257f873d4401d71530f0192e065f67c\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-134-140.ec2.internal" Jul 07 23:13:03.587944 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:03.587862 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/3bc0442220b7598e60bd507f09022656-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-134-140.ec2.internal\" (UID: \"3bc0442220b7598e60bd507f09022656\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-140.ec2.internal" Jul 07 23:13:03.587944 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:03.587896 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3bc0442220b7598e60bd507f09022656-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-134-140.ec2.internal\" (UID: \"3bc0442220b7598e60bd507f09022656\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-140.ec2.internal" Jul 07 23:13:03.630837 ip-10-0-134-140 kubenswrapper[2570]: E0707 23:13:03.630806 2570 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-140.ec2.internal\" not found" Jul 07 23:13:03.727842 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:03.727795 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-140.ec2.internal" Jul 07 23:13:03.730920 ip-10-0-134-140 kubenswrapper[2570]: E0707 23:13:03.730898 2570 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-140.ec2.internal\" not found" Jul 07 23:13:03.732253 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:03.732181 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-140.ec2.internal" Jul 07 23:13:03.834235 ip-10-0-134-140 kubenswrapper[2570]: E0707 23:13:03.834195 2570 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-140.ec2.internal\" not found" Jul 07 23:13:03.928897 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:03.928818 2570 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 07 23:13:03.929322 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:03.928992 2570 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 07 23:13:03.934403 ip-10-0-134-140 kubenswrapper[2570]: E0707 23:13:03.934384 2570 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-140.ec2.internal\" not found" Jul 07 23:13:04.020850 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:04.020783 2570 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-06 23:08:03 +0000 UTC" deadline="2028-03-06 23:50:13.562977405 +0000 UTC" Jul 07 23:13:04.020850 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:04.020840 2570 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14592h37m9.54214074s" Jul 07 23:13:04.034981 ip-10-0-134-140 kubenswrapper[2570]: E0707 23:13:04.034955 2570 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-140.ec2.internal\" not found" Jul 07 23:13:04.090612 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:04.090580 2570 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 23:13:04.091798 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:04.091772 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-140.ec2.internal" event="NodeHasSufficientMemory" Jul 07 23:13:04.091926 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:04.091810 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-140.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 23:13:04.091926 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:04.091824 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-140.ec2.internal" event="NodeHasSufficientPID" Jul 07 23:13:04.105374 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:04.105350 2570 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 07 23:13:04.135368 ip-10-0-134-140 kubenswrapper[2570]: E0707 23:13:04.135338 2570 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-140.ec2.internal\" not found" Jul 07 23:13:04.142604 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:04.142588 2570 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 07 23:13:04.187167 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:04.187101 2570 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-140.ec2.internal" Jul 07 23:13:04.198277 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:04.198254 2570 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 07 23:13:04.199371 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:04.199358 2570 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-140.ec2.internal" Jul 07 23:13:04.215042 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:04.215022 2570 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 07 23:13:04.365170 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:04.365137 2570 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 07 23:13:04.402573 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:04.402538 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7257f873d4401d71530f0192e065f67c.slice/crio-5980ec3f4b1f3566143e3312279e7d6303c1808039a4cf6f473e3b3134789dbf WatchSource:0}: Error finding container 5980ec3f4b1f3566143e3312279e7d6303c1808039a4cf6f473e3b3134789dbf: Status 404 returned error can't find the container with id 5980ec3f4b1f3566143e3312279e7d6303c1808039a4cf6f473e3b3134789dbf Jul 07 23:13:04.409857 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:04.409838 2570 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 07 23:13:04.412964 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:04.412938 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3bc0442220b7598e60bd507f09022656.slice/crio-53184eb9d44e4748d0356c1099991b970204325fe477e296a757322f655ff634 WatchSource:0}: Error finding container 53184eb9d44e4748d0356c1099991b970204325fe477e296a757322f655ff634: Status 404 returned error can't find the container with id 53184eb9d44e4748d0356c1099991b970204325fe477e296a757322f655ff634 Jul 07 23:13:04.735249 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:04.735226 2570 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 07 23:13:04.964248 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:04.964207 2570 apiserver.go:52] "Watching apiserver" Jul 07 23:13:04.973970 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:04.973927 2570 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 07 23:13:04.976186 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:04.976159 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-node-tuning-operator/tuned-cm9wz","openshift-multus/multus-additional-cni-plugins-vztph","openshift-multus/network-metrics-daemon-ndz2q","openshift-network-diagnostics/network-check-target-p27rw","openshift-network-operator/iptables-alerter-6mdgj","kube-system/global-pull-secret-syncer-84hdc","kube-system/kube-apiserver-proxy-ip-10-0-134-140.ec2.internal","openshift-dns/node-resolver-qgcpr","openshift-image-registry/node-ca-7z6hc","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-140.ec2.internal","openshift-multus/multus-4xzds","openshift-ovn-kubernetes/ovnkube-node-w9gmc","kube-system/konnectivity-agent-wjn84","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ltc6z"] Jul 07 23:13:04.979151 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:04.978823 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-wjn84" Jul 07 23:13:04.981334 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:04.981293 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-vztph" Jul 07 23:13:04.982066 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:04.982048 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 07 23:13:04.982147 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:04.982107 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 07 23:13:04.982276 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:04.982261 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-jcsgn\"" Jul 07 23:13:04.984335 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:04.984040 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ndz2q" Jul 07 23:13:04.984335 ip-10-0-134-140 kubenswrapper[2570]: E0707 23:13:04.984125 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ndz2q" podUID="14406dc0-620c-4615-b98a-b7205dc72966" Jul 07 23:13:04.986622 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:04.986333 2570 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 07 23:13:04.986930 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:04.986905 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-p27rw" Jul 07 23:13:04.987015 ip-10-0-134-140 kubenswrapper[2570]: E0707 23:13:04.986984 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-p27rw" podUID="f47fd470-2c7a-4ea8-bf73-e3f94320f117" Jul 07 23:13:04.987599 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:04.987409 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 07 23:13:04.988039 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:04.988021 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 07 23:13:04.988195 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:04.988148 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-lbzf7\"" Jul 07 23:13:04.988707 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:04.988684 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 07 23:13:04.988817 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:04.988726 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 07 23:13:04.989140 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:04.988913 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 07 23:13:04.990646 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:04.990623 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-6mdgj" Jul 07 23:13:04.993053 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:04.993033 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 07 23:13:04.993631 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:04.993434 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 07 23:13:04.993873 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:04.993855 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 07 23:13:04.994115 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:04.994098 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-78k5x\"" Jul 07 23:13:04.994904 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:04.994881 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-84hdc" Jul 07 23:13:04.994997 ip-10-0-134-140 kubenswrapper[2570]: E0707 23:13:04.994965 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-84hdc" podUID="1496ff7f-366b-4cd2-8396-2cdbfed841fa" Jul 07 23:13:04.997203 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:04.997178 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/0bb01852-570a-4456-9aca-52c99f9f83e5-konnectivity-ca\") pod \"konnectivity-agent-wjn84\" (UID: \"0bb01852-570a-4456-9aca-52c99f9f83e5\") " pod="kube-system/konnectivity-agent-wjn84" Jul 07 23:13:04.997285 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:04.997219 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/573e90ac-1d7d-4584-b231-6635ac4e12e2-system-cni-dir\") pod \"multus-additional-cni-plugins-vztph\" (UID: \"573e90ac-1d7d-4584-b231-6635ac4e12e2\") " pod="openshift-multus/multus-additional-cni-plugins-vztph" Jul 07 23:13:04.997285 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:04.997244 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/fc88a6f6-fb63-4fcd-b58f-980e71c1b5d0-iptables-alerter-script\") pod \"iptables-alerter-6mdgj\" (UID: \"fc88a6f6-fb63-4fcd-b58f-980e71c1b5d0\") " pod="openshift-network-operator/iptables-alerter-6mdgj" Jul 07 23:13:04.997285 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:04.997268 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/573e90ac-1d7d-4584-b231-6635ac4e12e2-cni-binary-copy\") pod \"multus-additional-cni-plugins-vztph\" (UID: \"573e90ac-1d7d-4584-b231-6635ac4e12e2\") " pod="openshift-multus/multus-additional-cni-plugins-vztph" Jul 07 23:13:04.997285 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:04.997274 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-cm9wz" Jul 07 23:13:04.997493 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:04.997295 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/573e90ac-1d7d-4584-b231-6635ac4e12e2-tuning-conf-dir\") pod \"multus-additional-cni-plugins-vztph\" (UID: \"573e90ac-1d7d-4584-b231-6635ac4e12e2\") " pod="openshift-multus/multus-additional-cni-plugins-vztph" Jul 07 23:13:04.997493 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:04.997320 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/573e90ac-1d7d-4584-b231-6635ac4e12e2-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-vztph\" (UID: \"573e90ac-1d7d-4584-b231-6635ac4e12e2\") " pod="openshift-multus/multus-additional-cni-plugins-vztph" Jul 07 23:13:04.997493 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:04.997348 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bbqzk\" (UniqueName: \"kubernetes.io/projected/573e90ac-1d7d-4584-b231-6635ac4e12e2-kube-api-access-bbqzk\") pod \"multus-additional-cni-plugins-vztph\" (UID: \"573e90ac-1d7d-4584-b231-6635ac4e12e2\") " pod="openshift-multus/multus-additional-cni-plugins-vztph" Jul 07 23:13:04.997493 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:04.997372 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/14406dc0-620c-4615-b98a-b7205dc72966-metrics-certs\") pod \"network-metrics-daemon-ndz2q\" (UID: \"14406dc0-620c-4615-b98a-b7205dc72966\") " pod="openshift-multus/network-metrics-daemon-ndz2q" Jul 07 23:13:04.997493 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:04.997400 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2bvrg\" (UniqueName: \"kubernetes.io/projected/14406dc0-620c-4615-b98a-b7205dc72966-kube-api-access-2bvrg\") pod \"network-metrics-daemon-ndz2q\" (UID: \"14406dc0-620c-4615-b98a-b7205dc72966\") " pod="openshift-multus/network-metrics-daemon-ndz2q" Jul 07 23:13:04.997493 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:04.997427 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/fc88a6f6-fb63-4fcd-b58f-980e71c1b5d0-host-slash\") pod \"iptables-alerter-6mdgj\" (UID: \"fc88a6f6-fb63-4fcd-b58f-980e71c1b5d0\") " pod="openshift-network-operator/iptables-alerter-6mdgj" Jul 07 23:13:04.997493 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:04.997453 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/573e90ac-1d7d-4584-b231-6635ac4e12e2-cnibin\") pod \"multus-additional-cni-plugins-vztph\" (UID: \"573e90ac-1d7d-4584-b231-6635ac4e12e2\") " pod="openshift-multus/multus-additional-cni-plugins-vztph" Jul 07 23:13:04.997493 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:04.997479 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/573e90ac-1d7d-4584-b231-6635ac4e12e2-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-vztph\" (UID: \"573e90ac-1d7d-4584-b231-6635ac4e12e2\") " pod="openshift-multus/multus-additional-cni-plugins-vztph" Jul 07 23:13:04.997864 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:04.997505 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/573e90ac-1d7d-4584-b231-6635ac4e12e2-os-release\") pod \"multus-additional-cni-plugins-vztph\" (UID: \"573e90ac-1d7d-4584-b231-6635ac4e12e2\") " pod="openshift-multus/multus-additional-cni-plugins-vztph" Jul 07 23:13:04.997864 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:04.997538 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bkbfq\" (UniqueName: \"kubernetes.io/projected/f47fd470-2c7a-4ea8-bf73-e3f94320f117-kube-api-access-bkbfq\") pod \"network-check-target-p27rw\" (UID: \"f47fd470-2c7a-4ea8-bf73-e3f94320f117\") " pod="openshift-network-diagnostics/network-check-target-p27rw" Jul 07 23:13:04.997864 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:04.997580 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mqwp7\" (UniqueName: \"kubernetes.io/projected/fc88a6f6-fb63-4fcd-b58f-980e71c1b5d0-kube-api-access-mqwp7\") pod \"iptables-alerter-6mdgj\" (UID: \"fc88a6f6-fb63-4fcd-b58f-980e71c1b5d0\") " pod="openshift-network-operator/iptables-alerter-6mdgj" Jul 07 23:13:04.997864 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:04.997714 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/0bb01852-570a-4456-9aca-52c99f9f83e5-agent-certs\") pod \"konnectivity-agent-wjn84\" (UID: \"0bb01852-570a-4456-9aca-52c99f9f83e5\") " pod="kube-system/konnectivity-agent-wjn84" Jul 07 23:13:05.000451 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:05.000431 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-qgcpr" Jul 07 23:13:05.003868 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:05.003847 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-7z6hc" Jul 07 23:13:05.005979 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:05.005959 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 07 23:13:05.006198 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:05.006180 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ltc6z" Jul 07 23:13:05.006287 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:05.006219 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 07 23:13:05.006500 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:05.006480 2570 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 07 23:13:05.007041 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:05.006837 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-hp7s5\"" Jul 07 23:13:05.007127 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:05.007063 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 07 23:13:05.007480 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:05.007458 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 07 23:13:05.007480 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:05.007522 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-r6xxm\"" Jul 07 23:13:05.008106 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:05.008088 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-lw7gj\"" Jul 07 23:13:05.008176 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:05.008157 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 07 23:13:05.008241 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:05.008179 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 07 23:13:05.009323 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:05.009299 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-4xzds" Jul 07 23:13:05.010094 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:05.010075 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 07 23:13:05.011637 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:05.011600 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-w9gmc" Jul 07 23:13:05.012205 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:05.012185 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 07 23:13:05.012466 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:05.012442 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 07 23:13:05.012689 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:05.012671 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 07 23:13:05.012954 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:05.012933 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-vm876\"" Jul 07 23:13:05.013378 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:05.013359 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 07 23:13:05.013700 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:05.013681 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-qhzgl\"" Jul 07 23:13:05.015435 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:05.015413 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 07 23:13:05.015705 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:05.015688 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-fkh87\"" Jul 07 23:13:05.015878 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:05.015863 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 07 23:13:05.018628 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:05.018240 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 07 23:13:05.018995 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:05.018863 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 07 23:13:05.020279 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:05.020259 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 07 23:13:05.020279 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:05.020279 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 07 23:13:05.028285 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:05.028000 2570 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-xtbpl" Jul 07 23:13:05.042654 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:05.042632 2570 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-xtbpl" Jul 07 23:13:05.086482 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:05.086441 2570 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 07 23:13:05.098074 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:05.098046 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a8270634-fda9-4520-85cb-0ab49e4ce06a-etc-openvswitch\") pod \"ovnkube-node-w9gmc\" (UID: \"a8270634-fda9-4520-85cb-0ab49e4ce06a\") " pod="openshift-ovn-kubernetes/ovnkube-node-w9gmc" Jul 07 23:13:05.098208 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:05.098089 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f2ec914a-0546-4c91-ac24-1fa24cde33f6-etc-kubernetes\") pod \"tuned-cm9wz\" (UID: \"f2ec914a-0546-4c91-ac24-1fa24cde33f6\") " pod="openshift-cluster-node-tuning-operator/tuned-cm9wz" Jul 07 23:13:05.098208 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:05.098151 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/dd19d3d5-47a8-4052-97ea-f4e5fbbb88cc-socket-dir\") pod \"aws-ebs-csi-driver-node-ltc6z\" (UID: \"dd19d3d5-47a8-4052-97ea-f4e5fbbb88cc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ltc6z" Jul 07 23:13:05.098208 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:05.098172 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/0bb01852-570a-4456-9aca-52c99f9f83e5-agent-certs\") pod \"konnectivity-agent-wjn84\" (UID: \"0bb01852-570a-4456-9aca-52c99f9f83e5\") " pod="kube-system/konnectivity-agent-wjn84" Jul 07 23:13:05.098208 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:05.098189 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/fc88a6f6-fb63-4fcd-b58f-980e71c1b5d0-iptables-alerter-script\") pod \"iptables-alerter-6mdgj\" (UID: \"fc88a6f6-fb63-4fcd-b58f-980e71c1b5d0\") " pod="openshift-network-operator/iptables-alerter-6mdgj" Jul 07 23:13:05.098208 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:05.098206 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/a8270634-fda9-4520-85cb-0ab49e4ce06a-host-kubelet\") pod \"ovnkube-node-w9gmc\" (UID: \"a8270634-fda9-4520-85cb-0ab49e4ce06a\") " pod="openshift-ovn-kubernetes/ovnkube-node-w9gmc" Jul 07 23:13:05.098476 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:05.098221 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/a8270634-fda9-4520-85cb-0ab49e4ce06a-run-systemd\") pod \"ovnkube-node-w9gmc\" (UID: \"a8270634-fda9-4520-85cb-0ab49e4ce06a\") " pod="openshift-ovn-kubernetes/ovnkube-node-w9gmc" Jul 07 23:13:05.098476 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:05.098267 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/dd19d3d5-47a8-4052-97ea-f4e5fbbb88cc-registration-dir\") pod \"aws-ebs-csi-driver-node-ltc6z\" (UID: \"dd19d3d5-47a8-4052-97ea-f4e5fbbb88cc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ltc6z" Jul 07 23:13:05.098476 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:05.098287 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/573e90ac-1d7d-4584-b231-6635ac4e12e2-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-vztph\" (UID: \"573e90ac-1d7d-4584-b231-6635ac4e12e2\") " pod="openshift-multus/multus-additional-cni-plugins-vztph" Jul 07 23:13:05.098476 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:05.098303 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/14406dc0-620c-4615-b98a-b7205dc72966-metrics-certs\") pod \"network-metrics-daemon-ndz2q\" (UID: \"14406dc0-620c-4615-b98a-b7205dc72966\") " pod="openshift-multus/network-metrics-daemon-ndz2q" Jul 07 23:13:05.098476 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:05.098319 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/48b46894-bcf9-4176-82a7-bab0ee169c67-host-run-multus-certs\") pod \"multus-4xzds\" (UID: \"48b46894-bcf9-4176-82a7-bab0ee169c67\") " pod="openshift-multus/multus-4xzds" Jul 07 23:13:05.098476 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:05.098336 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a8270634-fda9-4520-85cb-0ab49e4ce06a-host-slash\") pod \"ovnkube-node-w9gmc\" (UID: \"a8270634-fda9-4520-85cb-0ab49e4ce06a\") " pod="openshift-ovn-kubernetes/ovnkube-node-w9gmc" Jul 07 23:13:05.098476 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:05.098351 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a8270634-fda9-4520-85cb-0ab49e4ce06a-host-run-ovn-kubernetes\") pod \"ovnkube-node-w9gmc\" (UID: \"a8270634-fda9-4520-85cb-0ab49e4ce06a\") " pod="openshift-ovn-kubernetes/ovnkube-node-w9gmc" Jul 07 23:13:05.098476 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:05.098368 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f2ec914a-0546-4c91-ac24-1fa24cde33f6-var-lib-kubelet\") pod \"tuned-cm9wz\" (UID: \"f2ec914a-0546-4c91-ac24-1fa24cde33f6\") " pod="openshift-cluster-node-tuning-operator/tuned-cm9wz" Jul 07 23:13:05.098476 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:05.098404 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/48b46894-bcf9-4176-82a7-bab0ee169c67-cnibin\") pod \"multus-4xzds\" (UID: \"48b46894-bcf9-4176-82a7-bab0ee169c67\") " pod="openshift-multus/multus-4xzds" Jul 07 23:13:05.098476 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:05.098431 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bkbfq\" (UniqueName: \"kubernetes.io/projected/f47fd470-2c7a-4ea8-bf73-e3f94320f117-kube-api-access-bkbfq\") pod \"network-check-target-p27rw\" (UID: \"f47fd470-2c7a-4ea8-bf73-e3f94320f117\") " pod="openshift-network-diagnostics/network-check-target-p27rw" Jul 07 23:13:05.098476 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:05.098460 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lc4nw\" (UniqueName: \"kubernetes.io/projected/df04fe1d-eb1a-4711-80ea-72d68116bbca-kube-api-access-lc4nw\") pod \"node-ca-7z6hc\" (UID: \"df04fe1d-eb1a-4711-80ea-72d68116bbca\") " pod="openshift-image-registry/node-ca-7z6hc" Jul 07 23:13:05.099075 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:05.098485 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/48b46894-bcf9-4176-82a7-bab0ee169c67-multus-conf-dir\") pod \"multus-4xzds\" (UID: \"48b46894-bcf9-4176-82a7-bab0ee169c67\") " pod="openshift-multus/multus-4xzds" Jul 07 23:13:05.099075 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:05.098528 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/48b46894-bcf9-4176-82a7-bab0ee169c67-multus-daemon-config\") pod \"multus-4xzds\" (UID: \"48b46894-bcf9-4176-82a7-bab0ee169c67\") " pod="openshift-multus/multus-4xzds" Jul 07 23:13:05.099075 ip-10-0-134-140 kubenswrapper[2570]: E0707 23:13:05.098536 2570 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 23:13:05.099075 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:05.098557 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/f2ec914a-0546-4c91-ac24-1fa24cde33f6-etc-sysctl-conf\") pod \"tuned-cm9wz\" (UID: \"f2ec914a-0546-4c91-ac24-1fa24cde33f6\") " pod="openshift-cluster-node-tuning-operator/tuned-cm9wz" Jul 07 23:13:05.099075 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:05.098591 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/0bb01852-570a-4456-9aca-52c99f9f83e5-konnectivity-ca\") pod \"konnectivity-agent-wjn84\" (UID: \"0bb01852-570a-4456-9aca-52c99f9f83e5\") " pod="kube-system/konnectivity-agent-wjn84" Jul 07 23:13:05.099075 ip-10-0-134-140 kubenswrapper[2570]: E0707 23:13:05.098622 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/14406dc0-620c-4615-b98a-b7205dc72966-metrics-certs podName:14406dc0-620c-4615-b98a-b7205dc72966 nodeName:}" failed. No retries permitted until 2026-07-07 23:13:05.598598482 +0000 UTC m=+2.995599996 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/14406dc0-620c-4615-b98a-b7205dc72966-metrics-certs") pod "network-metrics-daemon-ndz2q" (UID: "14406dc0-620c-4615-b98a-b7205dc72966") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 23:13:05.099075 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:05.098653 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a8270634-fda9-4520-85cb-0ab49e4ce06a-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-w9gmc\" (UID: \"a8270634-fda9-4520-85cb-0ab49e4ce06a\") " pod="openshift-ovn-kubernetes/ovnkube-node-w9gmc" Jul 07 23:13:05.099075 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:05.098683 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/a8270634-fda9-4520-85cb-0ab49e4ce06a-ovnkube-script-lib\") pod \"ovnkube-node-w9gmc\" (UID: \"a8270634-fda9-4520-85cb-0ab49e4ce06a\") " pod="openshift-ovn-kubernetes/ovnkube-node-w9gmc" Jul 07 23:13:05.099075 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:05.098715 2570 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 07 23:13:05.099075 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:05.098744 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/48b46894-bcf9-4176-82a7-bab0ee169c67-host-var-lib-kubelet\") pod \"multus-4xzds\" (UID: \"48b46894-bcf9-4176-82a7-bab0ee169c67\") " pod="openshift-multus/multus-4xzds" Jul 07 23:13:05.099075 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:05.098774 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/48b46894-bcf9-4176-82a7-bab0ee169c67-hostroot\") pod \"multus-4xzds\" (UID: \"48b46894-bcf9-4176-82a7-bab0ee169c67\") " pod="openshift-multus/multus-4xzds" Jul 07 23:13:05.099075 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:05.098800 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/48b46894-bcf9-4176-82a7-bab0ee169c67-etc-kubernetes\") pod \"multus-4xzds\" (UID: \"48b46894-bcf9-4176-82a7-bab0ee169c67\") " pod="openshift-multus/multus-4xzds" Jul 07 23:13:05.099075 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:05.098825 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/f2ec914a-0546-4c91-ac24-1fa24cde33f6-etc-sysctl-d\") pod \"tuned-cm9wz\" (UID: \"f2ec914a-0546-4c91-ac24-1fa24cde33f6\") " pod="openshift-cluster-node-tuning-operator/tuned-cm9wz" Jul 07 23:13:05.099075 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:05.098851 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c02c0747-b1ae-452c-81aa-5101d7125fc9-tmp-dir\") pod \"node-resolver-qgcpr\" (UID: \"c02c0747-b1ae-452c-81aa-5101d7125fc9\") " pod="openshift-dns/node-resolver-qgcpr" Jul 07 23:13:05.099075 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:05.098882 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/573e90ac-1d7d-4584-b231-6635ac4e12e2-cni-binary-copy\") pod \"multus-additional-cni-plugins-vztph\" (UID: \"573e90ac-1d7d-4584-b231-6635ac4e12e2\") " pod="openshift-multus/multus-additional-cni-plugins-vztph" Jul 07 23:13:05.099075 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:05.098914 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bbqzk\" (UniqueName: \"kubernetes.io/projected/573e90ac-1d7d-4584-b231-6635ac4e12e2-kube-api-access-bbqzk\") pod \"multus-additional-cni-plugins-vztph\" (UID: \"573e90ac-1d7d-4584-b231-6635ac4e12e2\") " pod="openshift-multus/multus-additional-cni-plugins-vztph" Jul 07 23:13:05.099075 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:05.098942 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2bvrg\" (UniqueName: \"kubernetes.io/projected/14406dc0-620c-4615-b98a-b7205dc72966-kube-api-access-2bvrg\") pod \"network-metrics-daemon-ndz2q\" (UID: \"14406dc0-620c-4615-b98a-b7205dc72966\") " pod="openshift-multus/network-metrics-daemon-ndz2q" Jul 07 23:13:05.099867 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:05.098968 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/fc88a6f6-fb63-4fcd-b58f-980e71c1b5d0-host-slash\") pod \"iptables-alerter-6mdgj\" (UID: \"fc88a6f6-fb63-4fcd-b58f-980e71c1b5d0\") " pod="openshift-network-operator/iptables-alerter-6mdgj" Jul 07 23:13:05.099867 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:05.098996 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/1496ff7f-366b-4cd2-8396-2cdbfed841fa-kubelet-config\") pod \"global-pull-secret-syncer-84hdc\" (UID: \"1496ff7f-366b-4cd2-8396-2cdbfed841fa\") " pod="kube-system/global-pull-secret-syncer-84hdc" Jul 07 23:13:05.099867 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:05.099022 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/f2ec914a-0546-4c91-ac24-1fa24cde33f6-etc-modprobe-d\") pod \"tuned-cm9wz\" (UID: \"f2ec914a-0546-4c91-ac24-1fa24cde33f6\") " pod="openshift-cluster-node-tuning-operator/tuned-cm9wz" Jul 07 23:13:05.099867 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:05.099053 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/573e90ac-1d7d-4584-b231-6635ac4e12e2-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-vztph\" (UID: \"573e90ac-1d7d-4584-b231-6635ac4e12e2\") " pod="openshift-multus/multus-additional-cni-plugins-vztph" Jul 07 23:13:05.099867 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:05.099079 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/573e90ac-1d7d-4584-b231-6635ac4e12e2-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-vztph\" (UID: \"573e90ac-1d7d-4584-b231-6635ac4e12e2\") " pod="openshift-multus/multus-additional-cni-plugins-vztph" Jul 07 23:13:05.099867 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:05.099082 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/a8270634-fda9-4520-85cb-0ab49e4ce06a-run-ovn\") pod \"ovnkube-node-w9gmc\" (UID: \"a8270634-fda9-4520-85cb-0ab49e4ce06a\") " pod="openshift-ovn-kubernetes/ovnkube-node-w9gmc" Jul 07 23:13:05.099867 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:05.099132 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/a8270634-fda9-4520-85cb-0ab49e4ce06a-node-log\") pod \"ovnkube-node-w9gmc\" (UID: \"a8270634-fda9-4520-85cb-0ab49e4ce06a\") " pod="openshift-ovn-kubernetes/ovnkube-node-w9gmc" Jul 07 23:13:05.099867 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:05.099147 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/0bb01852-570a-4456-9aca-52c99f9f83e5-konnectivity-ca\") pod \"konnectivity-agent-wjn84\" (UID: \"0bb01852-570a-4456-9aca-52c99f9f83e5\") " pod="kube-system/konnectivity-agent-wjn84" Jul 07 23:13:05.099867 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:05.099160 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a8270634-fda9-4520-85cb-0ab49e4ce06a-host-cni-bin\") pod \"ovnkube-node-w9gmc\" (UID: \"a8270634-fda9-4520-85cb-0ab49e4ce06a\") " pod="openshift-ovn-kubernetes/ovnkube-node-w9gmc" Jul 07 23:13:05.099867 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:05.099185 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/a8270634-fda9-4520-85cb-0ab49e4ce06a-host-cni-netd\") pod \"ovnkube-node-w9gmc\" (UID: \"a8270634-fda9-4520-85cb-0ab49e4ce06a\") " pod="openshift-ovn-kubernetes/ovnkube-node-w9gmc" Jul 07 23:13:05.099867 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:05.099231 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/f2ec914a-0546-4c91-ac24-1fa24cde33f6-etc-systemd\") pod \"tuned-cm9wz\" (UID: \"f2ec914a-0546-4c91-ac24-1fa24cde33f6\") " pod="openshift-cluster-node-tuning-operator/tuned-cm9wz" Jul 07 23:13:05.099867 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:05.099307 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f2ec914a-0546-4c91-ac24-1fa24cde33f6-tmp\") pod \"tuned-cm9wz\" (UID: \"f2ec914a-0546-4c91-ac24-1fa24cde33f6\") " pod="openshift-cluster-node-tuning-operator/tuned-cm9wz" Jul 07 23:13:05.099867 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:05.099356 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/573e90ac-1d7d-4584-b231-6635ac4e12e2-os-release\") pod \"multus-additional-cni-plugins-vztph\" (UID: \"573e90ac-1d7d-4584-b231-6635ac4e12e2\") " pod="openshift-multus/multus-additional-cni-plugins-vztph" Jul 07 23:13:05.099867 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:05.099387 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/c02c0747-b1ae-452c-81aa-5101d7125fc9-hosts-file\") pod \"node-resolver-qgcpr\" (UID: \"c02c0747-b1ae-452c-81aa-5101d7125fc9\") " pod="openshift-dns/node-resolver-qgcpr" Jul 07 23:13:05.099867 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:05.099435 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/f2ec914a-0546-4c91-ac24-1fa24cde33f6-etc-sysconfig\") pod \"tuned-cm9wz\" (UID: \"f2ec914a-0546-4c91-ac24-1fa24cde33f6\") " pod="openshift-cluster-node-tuning-operator/tuned-cm9wz" Jul 07 23:13:05.099867 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:05.099473 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/48b46894-bcf9-4176-82a7-bab0ee169c67-cni-binary-copy\") pod \"multus-4xzds\" (UID: \"48b46894-bcf9-4176-82a7-bab0ee169c67\") " pod="openshift-multus/multus-4xzds" Jul 07 23:13:05.099867 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:05.099510 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/48b46894-bcf9-4176-82a7-bab0ee169c67-host-var-lib-cni-bin\") pod \"multus-4xzds\" (UID: \"48b46894-bcf9-4176-82a7-bab0ee169c67\") " pod="openshift-multus/multus-4xzds" Jul 07 23:13:05.100750 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:05.099537 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/573e90ac-1d7d-4584-b231-6635ac4e12e2-os-release\") pod \"multus-additional-cni-plugins-vztph\" (UID: \"573e90ac-1d7d-4584-b231-6635ac4e12e2\") " pod="openshift-multus/multus-additional-cni-plugins-vztph" Jul 07 23:13:05.100750 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:05.099601 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7dp8c\" (UniqueName: \"kubernetes.io/projected/48b46894-bcf9-4176-82a7-bab0ee169c67-kube-api-access-7dp8c\") pod \"multus-4xzds\" (UID: \"48b46894-bcf9-4176-82a7-bab0ee169c67\") " pod="openshift-multus/multus-4xzds" Jul 07 23:13:05.100750 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:05.099630 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1496ff7f-366b-4cd2-8396-2cdbfed841fa-original-pull-secret\") pod \"global-pull-secret-syncer-84hdc\" (UID: \"1496ff7f-366b-4cd2-8396-2cdbfed841fa\") " pod="kube-system/global-pull-secret-syncer-84hdc" Jul 07 23:13:05.100750 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:05.099633 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/fc88a6f6-fb63-4fcd-b58f-980e71c1b5d0-host-slash\") pod \"iptables-alerter-6mdgj\" (UID: \"fc88a6f6-fb63-4fcd-b58f-980e71c1b5d0\") " pod="openshift-network-operator/iptables-alerter-6mdgj" Jul 07 23:13:05.100750 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:05.099675 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9c9vq\" (UniqueName: \"kubernetes.io/projected/c02c0747-b1ae-452c-81aa-5101d7125fc9-kube-api-access-9c9vq\") pod \"node-resolver-qgcpr\" (UID: \"c02c0747-b1ae-452c-81aa-5101d7125fc9\") " pod="openshift-dns/node-resolver-qgcpr" Jul 07 23:13:05.100750 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:05.099704 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/a8270634-fda9-4520-85cb-0ab49e4ce06a-env-overrides\") pod \"ovnkube-node-w9gmc\" (UID: \"a8270634-fda9-4520-85cb-0ab49e4ce06a\") " pod="openshift-ovn-kubernetes/ovnkube-node-w9gmc" Jul 07 23:13:05.100750 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:05.099731 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/fc88a6f6-fb63-4fcd-b58f-980e71c1b5d0-iptables-alerter-script\") pod \"iptables-alerter-6mdgj\" (UID: \"fc88a6f6-fb63-4fcd-b58f-980e71c1b5d0\") " pod="openshift-network-operator/iptables-alerter-6mdgj" Jul 07 23:13:05.100750 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:05.099740 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9cc2p\" (UniqueName: \"kubernetes.io/projected/a8270634-fda9-4520-85cb-0ab49e4ce06a-kube-api-access-9cc2p\") pod \"ovnkube-node-w9gmc\" (UID: \"a8270634-fda9-4520-85cb-0ab49e4ce06a\") " pod="openshift-ovn-kubernetes/ovnkube-node-w9gmc" Jul 07 23:13:05.100750 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:05.099777 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/dd19d3d5-47a8-4052-97ea-f4e5fbbb88cc-kubelet-dir\") pod \"aws-ebs-csi-driver-node-ltc6z\" (UID: \"dd19d3d5-47a8-4052-97ea-f4e5fbbb88cc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ltc6z" Jul 07 23:13:05.100750 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:05.099841 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-km9rw\" (UniqueName: \"kubernetes.io/projected/dd19d3d5-47a8-4052-97ea-f4e5fbbb88cc-kube-api-access-km9rw\") pod \"aws-ebs-csi-driver-node-ltc6z\" (UID: \"dd19d3d5-47a8-4052-97ea-f4e5fbbb88cc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ltc6z" Jul 07 23:13:05.100750 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:05.099879 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/48b46894-bcf9-4176-82a7-bab0ee169c67-multus-cni-dir\") pod \"multus-4xzds\" (UID: \"48b46894-bcf9-4176-82a7-bab0ee169c67\") " pod="openshift-multus/multus-4xzds" Jul 07 23:13:05.100750 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:05.099922 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/48b46894-bcf9-4176-82a7-bab0ee169c67-multus-socket-dir-parent\") pod \"multus-4xzds\" (UID: \"48b46894-bcf9-4176-82a7-bab0ee169c67\") " pod="openshift-multus/multus-4xzds" Jul 07 23:13:05.100750 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:05.099947 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/df04fe1d-eb1a-4711-80ea-72d68116bbca-serviceca\") pod \"node-ca-7z6hc\" (UID: \"df04fe1d-eb1a-4711-80ea-72d68116bbca\") " pod="openshift-image-registry/node-ca-7z6hc" Jul 07 23:13:05.100750 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:05.100001 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/573e90ac-1d7d-4584-b231-6635ac4e12e2-tuning-conf-dir\") pod \"multus-additional-cni-plugins-vztph\" (UID: \"573e90ac-1d7d-4584-b231-6635ac4e12e2\") " pod="openshift-multus/multus-additional-cni-plugins-vztph" Jul 07 23:13:05.100750 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:05.100029 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/a8270634-fda9-4520-85cb-0ab49e4ce06a-systemd-units\") pod \"ovnkube-node-w9gmc\" (UID: \"a8270634-fda9-4520-85cb-0ab49e4ce06a\") " pod="openshift-ovn-kubernetes/ovnkube-node-w9gmc" Jul 07 23:13:05.100750 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:05.100053 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a8270634-fda9-4520-85cb-0ab49e4ce06a-var-lib-openvswitch\") pod \"ovnkube-node-w9gmc\" (UID: \"a8270634-fda9-4520-85cb-0ab49e4ce06a\") " pod="openshift-ovn-kubernetes/ovnkube-node-w9gmc" Jul 07 23:13:05.101537 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:05.100078 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/a8270634-fda9-4520-85cb-0ab49e4ce06a-log-socket\") pod \"ovnkube-node-w9gmc\" (UID: \"a8270634-fda9-4520-85cb-0ab49e4ce06a\") " pod="openshift-ovn-kubernetes/ovnkube-node-w9gmc" Jul 07 23:13:05.101537 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:05.100102 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/48b46894-bcf9-4176-82a7-bab0ee169c67-os-release\") pod \"multus-4xzds\" (UID: \"48b46894-bcf9-4176-82a7-bab0ee169c67\") " pod="openshift-multus/multus-4xzds" Jul 07 23:13:05.101537 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:05.100125 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/a8270634-fda9-4520-85cb-0ab49e4ce06a-ovnkube-config\") pod \"ovnkube-node-w9gmc\" (UID: \"a8270634-fda9-4520-85cb-0ab49e4ce06a\") " pod="openshift-ovn-kubernetes/ovnkube-node-w9gmc" Jul 07 23:13:05.101537 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:05.100150 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/48b46894-bcf9-4176-82a7-bab0ee169c67-host-var-lib-cni-multus\") pod \"multus-4xzds\" (UID: \"48b46894-bcf9-4176-82a7-bab0ee169c67\") " pod="openshift-multus/multus-4xzds" Jul 07 23:13:05.101537 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:05.100161 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/573e90ac-1d7d-4584-b231-6635ac4e12e2-cni-binary-copy\") pod \"multus-additional-cni-plugins-vztph\" (UID: \"573e90ac-1d7d-4584-b231-6635ac4e12e2\") " pod="openshift-multus/multus-additional-cni-plugins-vztph" Jul 07 23:13:05.101537 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:05.100173 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a8270634-fda9-4520-85cb-0ab49e4ce06a-run-openvswitch\") pod \"ovnkube-node-w9gmc\" (UID: \"a8270634-fda9-4520-85cb-0ab49e4ce06a\") " pod="openshift-ovn-kubernetes/ovnkube-node-w9gmc" Jul 07 23:13:05.101537 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:05.100149 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/573e90ac-1d7d-4584-b231-6635ac4e12e2-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-vztph\" (UID: \"573e90ac-1d7d-4584-b231-6635ac4e12e2\") " pod="openshift-multus/multus-additional-cni-plugins-vztph" Jul 07 23:13:05.101537 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:05.100197 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f2ec914a-0546-4c91-ac24-1fa24cde33f6-sys\") pod \"tuned-cm9wz\" (UID: \"f2ec914a-0546-4c91-ac24-1fa24cde33f6\") " pod="openshift-cluster-node-tuning-operator/tuned-cm9wz" Jul 07 23:13:05.101537 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:05.100198 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/573e90ac-1d7d-4584-b231-6635ac4e12e2-tuning-conf-dir\") pod \"multus-additional-cni-plugins-vztph\" (UID: \"573e90ac-1d7d-4584-b231-6635ac4e12e2\") " pod="openshift-multus/multus-additional-cni-plugins-vztph" Jul 07 23:13:05.101537 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:05.100220 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f2ec914a-0546-4c91-ac24-1fa24cde33f6-lib-modules\") pod \"tuned-cm9wz\" (UID: \"f2ec914a-0546-4c91-ac24-1fa24cde33f6\") " pod="openshift-cluster-node-tuning-operator/tuned-cm9wz" Jul 07 23:13:05.101537 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:05.100263 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/f2ec914a-0546-4c91-ac24-1fa24cde33f6-etc-tuned\") pod \"tuned-cm9wz\" (UID: \"f2ec914a-0546-4c91-ac24-1fa24cde33f6\") " pod="openshift-cluster-node-tuning-operator/tuned-cm9wz" Jul 07 23:13:05.101537 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:05.100303 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/dd19d3d5-47a8-4052-97ea-f4e5fbbb88cc-sys-fs\") pod \"aws-ebs-csi-driver-node-ltc6z\" (UID: \"dd19d3d5-47a8-4052-97ea-f4e5fbbb88cc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ltc6z" Jul 07 23:13:05.101537 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:05.100343 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/48b46894-bcf9-4176-82a7-bab0ee169c67-system-cni-dir\") pod \"multus-4xzds\" (UID: \"48b46894-bcf9-4176-82a7-bab0ee169c67\") " pod="openshift-multus/multus-4xzds" Jul 07 23:13:05.101537 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:05.100368 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/f2ec914a-0546-4c91-ac24-1fa24cde33f6-run\") pod \"tuned-cm9wz\" (UID: \"f2ec914a-0546-4c91-ac24-1fa24cde33f6\") " pod="openshift-cluster-node-tuning-operator/tuned-cm9wz" Jul 07 23:13:05.101537 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:05.100392 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/df04fe1d-eb1a-4711-80ea-72d68116bbca-host\") pod \"node-ca-7z6hc\" (UID: \"df04fe1d-eb1a-4711-80ea-72d68116bbca\") " pod="openshift-image-registry/node-ca-7z6hc" Jul 07 23:13:05.101537 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:05.100419 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/573e90ac-1d7d-4584-b231-6635ac4e12e2-system-cni-dir\") pod \"multus-additional-cni-plugins-vztph\" (UID: \"573e90ac-1d7d-4584-b231-6635ac4e12e2\") " pod="openshift-multus/multus-additional-cni-plugins-vztph" Jul 07 23:13:05.101537 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:05.100444 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a8270634-fda9-4520-85cb-0ab49e4ce06a-host-run-netns\") pod \"ovnkube-node-w9gmc\" (UID: \"a8270634-fda9-4520-85cb-0ab49e4ce06a\") " pod="openshift-ovn-kubernetes/ovnkube-node-w9gmc" Jul 07 23:13:05.102406 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:05.100463 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/f2ec914a-0546-4c91-ac24-1fa24cde33f6-host\") pod \"tuned-cm9wz\" (UID: \"f2ec914a-0546-4c91-ac24-1fa24cde33f6\") " pod="openshift-cluster-node-tuning-operator/tuned-cm9wz" Jul 07 23:13:05.102406 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:05.100502 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7f8v8\" (UniqueName: \"kubernetes.io/projected/f2ec914a-0546-4c91-ac24-1fa24cde33f6-kube-api-access-7f8v8\") pod \"tuned-cm9wz\" (UID: \"f2ec914a-0546-4c91-ac24-1fa24cde33f6\") " pod="openshift-cluster-node-tuning-operator/tuned-cm9wz" Jul 07 23:13:05.102406 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:05.100523 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/dd19d3d5-47a8-4052-97ea-f4e5fbbb88cc-device-dir\") pod \"aws-ebs-csi-driver-node-ltc6z\" (UID: \"dd19d3d5-47a8-4052-97ea-f4e5fbbb88cc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ltc6z" Jul 07 23:13:05.102406 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:05.100558 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/a8270634-fda9-4520-85cb-0ab49e4ce06a-ovn-node-metrics-cert\") pod \"ovnkube-node-w9gmc\" (UID: \"a8270634-fda9-4520-85cb-0ab49e4ce06a\") " pod="openshift-ovn-kubernetes/ovnkube-node-w9gmc" Jul 07 23:13:05.102406 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:05.100560 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/573e90ac-1d7d-4584-b231-6635ac4e12e2-system-cni-dir\") pod \"multus-additional-cni-plugins-vztph\" (UID: \"573e90ac-1d7d-4584-b231-6635ac4e12e2\") " pod="openshift-multus/multus-additional-cni-plugins-vztph" Jul 07 23:13:05.102406 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:05.100579 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/dd19d3d5-47a8-4052-97ea-f4e5fbbb88cc-etc-selinux\") pod \"aws-ebs-csi-driver-node-ltc6z\" (UID: \"dd19d3d5-47a8-4052-97ea-f4e5fbbb88cc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ltc6z" Jul 07 23:13:05.102406 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:05.100602 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/573e90ac-1d7d-4584-b231-6635ac4e12e2-cnibin\") pod \"multus-additional-cni-plugins-vztph\" (UID: \"573e90ac-1d7d-4584-b231-6635ac4e12e2\") " pod="openshift-multus/multus-additional-cni-plugins-vztph" Jul 07 23:13:05.102406 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:05.100623 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/48b46894-bcf9-4176-82a7-bab0ee169c67-host-run-k8s-cni-cncf-io\") pod \"multus-4xzds\" (UID: \"48b46894-bcf9-4176-82a7-bab0ee169c67\") " pod="openshift-multus/multus-4xzds" Jul 07 23:13:05.102406 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:05.100643 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/48b46894-bcf9-4176-82a7-bab0ee169c67-host-run-netns\") pod \"multus-4xzds\" (UID: \"48b46894-bcf9-4176-82a7-bab0ee169c67\") " pod="openshift-multus/multus-4xzds" Jul 07 23:13:05.102406 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:05.100641 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-140.ec2.internal" event={"ID":"3bc0442220b7598e60bd507f09022656","Type":"ContainerStarted","Data":"53184eb9d44e4748d0356c1099991b970204325fe477e296a757322f655ff634"} Jul 07 23:13:05.102406 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:05.100678 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/573e90ac-1d7d-4584-b231-6635ac4e12e2-cnibin\") pod \"multus-additional-cni-plugins-vztph\" (UID: \"573e90ac-1d7d-4584-b231-6635ac4e12e2\") " pod="openshift-multus/multus-additional-cni-plugins-vztph" Jul 07 23:13:05.102406 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:05.100668 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mqwp7\" (UniqueName: \"kubernetes.io/projected/fc88a6f6-fb63-4fcd-b58f-980e71c1b5d0-kube-api-access-mqwp7\") pod \"iptables-alerter-6mdgj\" (UID: \"fc88a6f6-fb63-4fcd-b58f-980e71c1b5d0\") " pod="openshift-network-operator/iptables-alerter-6mdgj" Jul 07 23:13:05.102406 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:05.100730 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/1496ff7f-366b-4cd2-8396-2cdbfed841fa-dbus\") pod \"global-pull-secret-syncer-84hdc\" (UID: \"1496ff7f-366b-4cd2-8396-2cdbfed841fa\") " pod="kube-system/global-pull-secret-syncer-84hdc" Jul 07 23:13:05.103888 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:05.103856 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-140.ec2.internal" event={"ID":"7257f873d4401d71530f0192e065f67c","Type":"ContainerStarted","Data":"5980ec3f4b1f3566143e3312279e7d6303c1808039a4cf6f473e3b3134789dbf"} Jul 07 23:13:05.107292 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:05.107265 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/0bb01852-570a-4456-9aca-52c99f9f83e5-agent-certs\") pod \"konnectivity-agent-wjn84\" (UID: \"0bb01852-570a-4456-9aca-52c99f9f83e5\") " pod="kube-system/konnectivity-agent-wjn84" Jul 07 23:13:05.113558 ip-10-0-134-140 kubenswrapper[2570]: E0707 23:13:05.113524 2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 07 23:13:05.113558 ip-10-0-134-140 kubenswrapper[2570]: E0707 23:13:05.113555 2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 07 23:13:05.113680 ip-10-0-134-140 kubenswrapper[2570]: E0707 23:13:05.113580 2570 projected.go:196] Error preparing data for projected volume kube-api-access-bkbfq for pod openshift-network-diagnostics/network-check-target-p27rw: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 23:13:05.113680 ip-10-0-134-140 kubenswrapper[2570]: E0707 23:13:05.113667 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f47fd470-2c7a-4ea8-bf73-e3f94320f117-kube-api-access-bkbfq podName:f47fd470-2c7a-4ea8-bf73-e3f94320f117 nodeName:}" failed. No retries permitted until 2026-07-07 23:13:05.613625509 +0000 UTC m=+3.010627038 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-bkbfq" (UniqueName: "kubernetes.io/projected/f47fd470-2c7a-4ea8-bf73-e3f94320f117-kube-api-access-bkbfq") pod "network-check-target-p27rw" (UID: "f47fd470-2c7a-4ea8-bf73-e3f94320f117") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 23:13:05.118275 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:05.118226 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bbqzk\" (UniqueName: \"kubernetes.io/projected/573e90ac-1d7d-4584-b231-6635ac4e12e2-kube-api-access-bbqzk\") pod \"multus-additional-cni-plugins-vztph\" (UID: \"573e90ac-1d7d-4584-b231-6635ac4e12e2\") " pod="openshift-multus/multus-additional-cni-plugins-vztph" Jul 07 23:13:05.118960 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:05.118936 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2bvrg\" (UniqueName: \"kubernetes.io/projected/14406dc0-620c-4615-b98a-b7205dc72966-kube-api-access-2bvrg\") pod \"network-metrics-daemon-ndz2q\" (UID: \"14406dc0-620c-4615-b98a-b7205dc72966\") " pod="openshift-multus/network-metrics-daemon-ndz2q" Jul 07 23:13:05.120595 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:05.120563 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mqwp7\" (UniqueName: \"kubernetes.io/projected/fc88a6f6-fb63-4fcd-b58f-980e71c1b5d0-kube-api-access-mqwp7\") pod \"iptables-alerter-6mdgj\" (UID: \"fc88a6f6-fb63-4fcd-b58f-980e71c1b5d0\") " pod="openshift-network-operator/iptables-alerter-6mdgj" Jul 07 23:13:05.201830 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:05.201791 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/df04fe1d-eb1a-4711-80ea-72d68116bbca-serviceca\") pod \"node-ca-7z6hc\" (UID: \"df04fe1d-eb1a-4711-80ea-72d68116bbca\") " pod="openshift-image-registry/node-ca-7z6hc" Jul 07 23:13:05.201830 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:05.201836 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/a8270634-fda9-4520-85cb-0ab49e4ce06a-systemd-units\") pod \"ovnkube-node-w9gmc\" (UID: \"a8270634-fda9-4520-85cb-0ab49e4ce06a\") " pod="openshift-ovn-kubernetes/ovnkube-node-w9gmc" Jul 07 23:13:05.202079 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:05.201865 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a8270634-fda9-4520-85cb-0ab49e4ce06a-var-lib-openvswitch\") pod \"ovnkube-node-w9gmc\" (UID: \"a8270634-fda9-4520-85cb-0ab49e4ce06a\") " pod="openshift-ovn-kubernetes/ovnkube-node-w9gmc" Jul 07 23:13:05.202079 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:05.201929 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/a8270634-fda9-4520-85cb-0ab49e4ce06a-systemd-units\") pod \"ovnkube-node-w9gmc\" (UID: \"a8270634-fda9-4520-85cb-0ab49e4ce06a\") " pod="openshift-ovn-kubernetes/ovnkube-node-w9gmc" Jul 07 23:13:05.202079 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:05.202043 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/a8270634-fda9-4520-85cb-0ab49e4ce06a-log-socket\") pod \"ovnkube-node-w9gmc\" (UID: \"a8270634-fda9-4520-85cb-0ab49e4ce06a\") " pod="openshift-ovn-kubernetes/ovnkube-node-w9gmc" Jul 07 23:13:05.202248 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:05.202083 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/48b46894-bcf9-4176-82a7-bab0ee169c67-os-release\") pod \"multus-4xzds\" (UID: \"48b46894-bcf9-4176-82a7-bab0ee169c67\") " pod="openshift-multus/multus-4xzds" Jul 07 23:13:05.202248 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:05.202108 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/a8270634-fda9-4520-85cb-0ab49e4ce06a-ovnkube-config\") pod \"ovnkube-node-w9gmc\" (UID: \"a8270634-fda9-4520-85cb-0ab49e4ce06a\") " pod="openshift-ovn-kubernetes/ovnkube-node-w9gmc" Jul 07 23:13:05.202248 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:05.202134 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/48b46894-bcf9-4176-82a7-bab0ee169c67-host-var-lib-cni-multus\") pod \"multus-4xzds\" (UID: \"48b46894-bcf9-4176-82a7-bab0ee169c67\") " pod="openshift-multus/multus-4xzds" Jul 07 23:13:05.202248 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:05.202159 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a8270634-fda9-4520-85cb-0ab49e4ce06a-run-openvswitch\") pod \"ovnkube-node-w9gmc\" (UID: \"a8270634-fda9-4520-85cb-0ab49e4ce06a\") " pod="openshift-ovn-kubernetes/ovnkube-node-w9gmc" Jul 07 23:13:05.202248 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:05.202196 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f2ec914a-0546-4c91-ac24-1fa24cde33f6-sys\") pod \"tuned-cm9wz\" (UID: \"f2ec914a-0546-4c91-ac24-1fa24cde33f6\") " pod="openshift-cluster-node-tuning-operator/tuned-cm9wz" Jul 07 23:13:05.202248 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:05.202220 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f2ec914a-0546-4c91-ac24-1fa24cde33f6-lib-modules\") pod \"tuned-cm9wz\" (UID: \"f2ec914a-0546-4c91-ac24-1fa24cde33f6\") " pod="openshift-cluster-node-tuning-operator/tuned-cm9wz" Jul 07 23:13:05.202571 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:05.202268 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/f2ec914a-0546-4c91-ac24-1fa24cde33f6-etc-tuned\") pod \"tuned-cm9wz\" (UID: \"f2ec914a-0546-4c91-ac24-1fa24cde33f6\") " pod="openshift-cluster-node-tuning-operator/tuned-cm9wz" Jul 07 23:13:05.202571 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:05.202294 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/dd19d3d5-47a8-4052-97ea-f4e5fbbb88cc-sys-fs\") pod \"aws-ebs-csi-driver-node-ltc6z\" (UID: \"dd19d3d5-47a8-4052-97ea-f4e5fbbb88cc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ltc6z" Jul 07 23:13:05.202571 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:05.202321 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/48b46894-bcf9-4176-82a7-bab0ee169c67-system-cni-dir\") pod \"multus-4xzds\" (UID: \"48b46894-bcf9-4176-82a7-bab0ee169c67\") " pod="openshift-multus/multus-4xzds" Jul 07 23:13:05.202571 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:05.202349 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/f2ec914a-0546-4c91-ac24-1fa24cde33f6-run\") pod \"tuned-cm9wz\" (UID: \"f2ec914a-0546-4c91-ac24-1fa24cde33f6\") " pod="openshift-cluster-node-tuning-operator/tuned-cm9wz" Jul 07 23:13:05.202571 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:05.202372 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/df04fe1d-eb1a-4711-80ea-72d68116bbca-host\") pod \"node-ca-7z6hc\" (UID: \"df04fe1d-eb1a-4711-80ea-72d68116bbca\") " pod="openshift-image-registry/node-ca-7z6hc" Jul 07 23:13:05.202571 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:05.202400 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a8270634-fda9-4520-85cb-0ab49e4ce06a-host-run-netns\") pod \"ovnkube-node-w9gmc\" (UID: \"a8270634-fda9-4520-85cb-0ab49e4ce06a\") " pod="openshift-ovn-kubernetes/ovnkube-node-w9gmc" Jul 07 23:13:05.202571 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:05.202404 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/df04fe1d-eb1a-4711-80ea-72d68116bbca-serviceca\") pod \"node-ca-7z6hc\" (UID: \"df04fe1d-eb1a-4711-80ea-72d68116bbca\") " pod="openshift-image-registry/node-ca-7z6hc" Jul 07 23:13:05.202571 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:05.202425 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/f2ec914a-0546-4c91-ac24-1fa24cde33f6-host\") pod \"tuned-cm9wz\" (UID: \"f2ec914a-0546-4c91-ac24-1fa24cde33f6\") " pod="openshift-cluster-node-tuning-operator/tuned-cm9wz" Jul 07 23:13:05.202571 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:05.202517 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7f8v8\" (UniqueName: \"kubernetes.io/projected/f2ec914a-0546-4c91-ac24-1fa24cde33f6-kube-api-access-7f8v8\") pod \"tuned-cm9wz\" (UID: \"f2ec914a-0546-4c91-ac24-1fa24cde33f6\") " pod="openshift-cluster-node-tuning-operator/tuned-cm9wz" Jul 07 23:13:05.202571 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:05.202545 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/48b46894-bcf9-4176-82a7-bab0ee169c67-os-release\") pod \"multus-4xzds\" (UID: \"48b46894-bcf9-4176-82a7-bab0ee169c67\") " pod="openshift-multus/multus-4xzds" Jul 07 23:13:05.202571 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:05.202569 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/dd19d3d5-47a8-4052-97ea-f4e5fbbb88cc-device-dir\") pod \"aws-ebs-csi-driver-node-ltc6z\" (UID: \"dd19d3d5-47a8-4052-97ea-f4e5fbbb88cc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ltc6z" Jul 07 23:13:05.203102 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:05.202607 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/a8270634-fda9-4520-85cb-0ab49e4ce06a-ovn-node-metrics-cert\") pod \"ovnkube-node-w9gmc\" (UID: \"a8270634-fda9-4520-85cb-0ab49e4ce06a\") " pod="openshift-ovn-kubernetes/ovnkube-node-w9gmc" Jul 07 23:13:05.203102 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:05.202610 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/a8270634-fda9-4520-85cb-0ab49e4ce06a-log-socket\") pod \"ovnkube-node-w9gmc\" (UID: \"a8270634-fda9-4520-85cb-0ab49e4ce06a\") " pod="openshift-ovn-kubernetes/ovnkube-node-w9gmc" Jul 07 23:13:05.203102 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:05.202640 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/dd19d3d5-47a8-4052-97ea-f4e5fbbb88cc-etc-selinux\") pod \"aws-ebs-csi-driver-node-ltc6z\" (UID: \"dd19d3d5-47a8-4052-97ea-f4e5fbbb88cc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ltc6z" Jul 07 23:13:05.203102 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:05.202723 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/48b46894-bcf9-4176-82a7-bab0ee169c67-host-run-k8s-cni-cncf-io\") pod \"multus-4xzds\" (UID: \"48b46894-bcf9-4176-82a7-bab0ee169c67\") " pod="openshift-multus/multus-4xzds" Jul 07 23:13:05.203102 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:05.202722 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a8270634-fda9-4520-85cb-0ab49e4ce06a-run-openvswitch\") pod \"ovnkube-node-w9gmc\" (UID: \"a8270634-fda9-4520-85cb-0ab49e4ce06a\") " pod="openshift-ovn-kubernetes/ovnkube-node-w9gmc" Jul 07 23:13:05.203102 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:05.202751 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/48b46894-bcf9-4176-82a7-bab0ee169c67-host-run-netns\") pod \"multus-4xzds\" (UID: \"48b46894-bcf9-4176-82a7-bab0ee169c67\") " pod="openshift-multus/multus-4xzds" Jul 07 23:13:05.203102 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:05.202805 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/dd19d3d5-47a8-4052-97ea-f4e5fbbb88cc-sys-fs\") pod \"aws-ebs-csi-driver-node-ltc6z\" (UID: \"dd19d3d5-47a8-4052-97ea-f4e5fbbb88cc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ltc6z" Jul 07 23:13:05.203102 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:05.202819 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/dd19d3d5-47a8-4052-97ea-f4e5fbbb88cc-device-dir\") pod \"aws-ebs-csi-driver-node-ltc6z\" (UID: \"dd19d3d5-47a8-4052-97ea-f4e5fbbb88cc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ltc6z" Jul 07 23:13:05.203102 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:05.202852 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/f2ec914a-0546-4c91-ac24-1fa24cde33f6-host\") pod \"tuned-cm9wz\" (UID: \"f2ec914a-0546-4c91-ac24-1fa24cde33f6\") " pod="openshift-cluster-node-tuning-operator/tuned-cm9wz" Jul 07 23:13:05.203102 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:05.202892 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a8270634-fda9-4520-85cb-0ab49e4ce06a-var-lib-openvswitch\") pod \"ovnkube-node-w9gmc\" (UID: \"a8270634-fda9-4520-85cb-0ab49e4ce06a\") " pod="openshift-ovn-kubernetes/ovnkube-node-w9gmc" Jul 07 23:13:05.203102 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:05.202909 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/f2ec914a-0546-4c91-ac24-1fa24cde33f6-run\") pod \"tuned-cm9wz\" (UID: \"f2ec914a-0546-4c91-ac24-1fa24cde33f6\") " pod="openshift-cluster-node-tuning-operator/tuned-cm9wz" Jul 07 23:13:05.203102 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:05.202921 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/dd19d3d5-47a8-4052-97ea-f4e5fbbb88cc-etc-selinux\") pod \"aws-ebs-csi-driver-node-ltc6z\" (UID: \"dd19d3d5-47a8-4052-97ea-f4e5fbbb88cc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ltc6z" Jul 07 23:13:05.203102 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:05.202943 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/df04fe1d-eb1a-4711-80ea-72d68116bbca-host\") pod \"node-ca-7z6hc\" (UID: \"df04fe1d-eb1a-4711-80ea-72d68116bbca\") " pod="openshift-image-registry/node-ca-7z6hc" Jul 07 23:13:05.203102 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:05.202952 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/48b46894-bcf9-4176-82a7-bab0ee169c67-host-run-k8s-cni-cncf-io\") pod \"multus-4xzds\" (UID: \"48b46894-bcf9-4176-82a7-bab0ee169c67\") " pod="openshift-multus/multus-4xzds" Jul 07 23:13:05.203102 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:05.202969 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/48b46894-bcf9-4176-82a7-bab0ee169c67-system-cni-dir\") pod \"multus-4xzds\" (UID: \"48b46894-bcf9-4176-82a7-bab0ee169c67\") " pod="openshift-multus/multus-4xzds" Jul 07 23:13:05.203102 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:05.202994 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/48b46894-bcf9-4176-82a7-bab0ee169c67-host-var-lib-cni-multus\") pod \"multus-4xzds\" (UID: \"48b46894-bcf9-4176-82a7-bab0ee169c67\") " pod="openshift-multus/multus-4xzds" Jul 07 23:13:05.203102 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:05.203034 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a8270634-fda9-4520-85cb-0ab49e4ce06a-host-run-netns\") pod \"ovnkube-node-w9gmc\" (UID: \"a8270634-fda9-4520-85cb-0ab49e4ce06a\") " pod="openshift-ovn-kubernetes/ovnkube-node-w9gmc" Jul 07 23:13:05.203102 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:05.203071 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/1496ff7f-366b-4cd2-8396-2cdbfed841fa-dbus\") pod \"global-pull-secret-syncer-84hdc\" (UID: \"1496ff7f-366b-4cd2-8396-2cdbfed841fa\") " pod="kube-system/global-pull-secret-syncer-84hdc" Jul 07 23:13:05.204062 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:05.203072 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/48b46894-bcf9-4176-82a7-bab0ee169c67-host-run-netns\") pod \"multus-4xzds\" (UID: \"48b46894-bcf9-4176-82a7-bab0ee169c67\") " pod="openshift-multus/multus-4xzds" Jul 07 23:13:05.204062 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:05.203103 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a8270634-fda9-4520-85cb-0ab49e4ce06a-etc-openvswitch\") pod \"ovnkube-node-w9gmc\" (UID: \"a8270634-fda9-4520-85cb-0ab49e4ce06a\") " pod="openshift-ovn-kubernetes/ovnkube-node-w9gmc" Jul 07 23:13:05.204062 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:05.203119 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f2ec914a-0546-4c91-ac24-1fa24cde33f6-sys\") pod \"tuned-cm9wz\" (UID: \"f2ec914a-0546-4c91-ac24-1fa24cde33f6\") " pod="openshift-cluster-node-tuning-operator/tuned-cm9wz" Jul 07 23:13:05.204062 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:05.203230 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/1496ff7f-366b-4cd2-8396-2cdbfed841fa-dbus\") pod \"global-pull-secret-syncer-84hdc\" (UID: \"1496ff7f-366b-4cd2-8396-2cdbfed841fa\") " pod="kube-system/global-pull-secret-syncer-84hdc" Jul 07 23:13:05.204062 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:05.203234 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f2ec914a-0546-4c91-ac24-1fa24cde33f6-lib-modules\") pod \"tuned-cm9wz\" (UID: \"f2ec914a-0546-4c91-ac24-1fa24cde33f6\") " pod="openshift-cluster-node-tuning-operator/tuned-cm9wz" Jul 07 23:13:05.204062 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:05.203280 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a8270634-fda9-4520-85cb-0ab49e4ce06a-etc-openvswitch\") pod \"ovnkube-node-w9gmc\" (UID: \"a8270634-fda9-4520-85cb-0ab49e4ce06a\") " pod="openshift-ovn-kubernetes/ovnkube-node-w9gmc" Jul 07 23:13:05.204062 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:05.203310 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f2ec914a-0546-4c91-ac24-1fa24cde33f6-etc-kubernetes\") pod \"tuned-cm9wz\" (UID: \"f2ec914a-0546-4c91-ac24-1fa24cde33f6\") " pod="openshift-cluster-node-tuning-operator/tuned-cm9wz" Jul 07 23:13:05.204062 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:05.203354 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/dd19d3d5-47a8-4052-97ea-f4e5fbbb88cc-socket-dir\") pod \"aws-ebs-csi-driver-node-ltc6z\" (UID: \"dd19d3d5-47a8-4052-97ea-f4e5fbbb88cc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ltc6z" Jul 07 23:13:05.204062 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:05.203417 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f2ec914a-0546-4c91-ac24-1fa24cde33f6-etc-kubernetes\") pod \"tuned-cm9wz\" (UID: \"f2ec914a-0546-4c91-ac24-1fa24cde33f6\") " pod="openshift-cluster-node-tuning-operator/tuned-cm9wz" Jul 07 23:13:05.204062 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:05.203496 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/dd19d3d5-47a8-4052-97ea-f4e5fbbb88cc-socket-dir\") pod \"aws-ebs-csi-driver-node-ltc6z\" (UID: \"dd19d3d5-47a8-4052-97ea-f4e5fbbb88cc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ltc6z" Jul 07 23:13:05.204062 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:05.203500 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/a8270634-fda9-4520-85cb-0ab49e4ce06a-host-kubelet\") pod \"ovnkube-node-w9gmc\" (UID: \"a8270634-fda9-4520-85cb-0ab49e4ce06a\") " pod="openshift-ovn-kubernetes/ovnkube-node-w9gmc" Jul 07 23:13:05.204062 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:05.203551 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/a8270634-fda9-4520-85cb-0ab49e4ce06a-run-systemd\") pod \"ovnkube-node-w9gmc\" (UID: \"a8270634-fda9-4520-85cb-0ab49e4ce06a\") " pod="openshift-ovn-kubernetes/ovnkube-node-w9gmc" Jul 07 23:13:05.204062 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:05.203581 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/dd19d3d5-47a8-4052-97ea-f4e5fbbb88cc-registration-dir\") pod \"aws-ebs-csi-driver-node-ltc6z\" (UID: \"dd19d3d5-47a8-4052-97ea-f4e5fbbb88cc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ltc6z" Jul 07 23:13:05.204062 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:05.203595 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/a8270634-fda9-4520-85cb-0ab49e4ce06a-run-systemd\") pod \"ovnkube-node-w9gmc\" (UID: \"a8270634-fda9-4520-85cb-0ab49e4ce06a\") " pod="openshift-ovn-kubernetes/ovnkube-node-w9gmc" Jul 07 23:13:05.204062 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:05.203554 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/a8270634-fda9-4520-85cb-0ab49e4ce06a-host-kubelet\") pod \"ovnkube-node-w9gmc\" (UID: \"a8270634-fda9-4520-85cb-0ab49e4ce06a\") " pod="openshift-ovn-kubernetes/ovnkube-node-w9gmc" Jul 07 23:13:05.204062 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:05.203629 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/dd19d3d5-47a8-4052-97ea-f4e5fbbb88cc-registration-dir\") pod \"aws-ebs-csi-driver-node-ltc6z\" (UID: \"dd19d3d5-47a8-4052-97ea-f4e5fbbb88cc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ltc6z" Jul 07 23:13:05.204062 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:05.203654 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/48b46894-bcf9-4176-82a7-bab0ee169c67-host-run-multus-certs\") pod \"multus-4xzds\" (UID: \"48b46894-bcf9-4176-82a7-bab0ee169c67\") " pod="openshift-multus/multus-4xzds" Jul 07 23:13:05.204062 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:05.203693 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a8270634-fda9-4520-85cb-0ab49e4ce06a-host-slash\") pod \"ovnkube-node-w9gmc\" (UID: \"a8270634-fda9-4520-85cb-0ab49e4ce06a\") " pod="openshift-ovn-kubernetes/ovnkube-node-w9gmc" Jul 07 23:13:05.204987 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:05.203725 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a8270634-fda9-4520-85cb-0ab49e4ce06a-host-run-ovn-kubernetes\") pod \"ovnkube-node-w9gmc\" (UID: \"a8270634-fda9-4520-85cb-0ab49e4ce06a\") " pod="openshift-ovn-kubernetes/ovnkube-node-w9gmc" Jul 07 23:13:05.204987 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:05.203756 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f2ec914a-0546-4c91-ac24-1fa24cde33f6-var-lib-kubelet\") pod \"tuned-cm9wz\" (UID: \"f2ec914a-0546-4c91-ac24-1fa24cde33f6\") " pod="openshift-cluster-node-tuning-operator/tuned-cm9wz" Jul 07 23:13:05.204987 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:05.203776 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a8270634-fda9-4520-85cb-0ab49e4ce06a-host-slash\") pod \"ovnkube-node-w9gmc\" (UID: \"a8270634-fda9-4520-85cb-0ab49e4ce06a\") " pod="openshift-ovn-kubernetes/ovnkube-node-w9gmc" Jul 07 23:13:05.204987 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:05.203793 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/48b46894-bcf9-4176-82a7-bab0ee169c67-cnibin\") pod \"multus-4xzds\" (UID: \"48b46894-bcf9-4176-82a7-bab0ee169c67\") " pod="openshift-multus/multus-4xzds" Jul 07 23:13:05.204987 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:05.203780 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/48b46894-bcf9-4176-82a7-bab0ee169c67-host-run-multus-certs\") pod \"multus-4xzds\" (UID: \"48b46894-bcf9-4176-82a7-bab0ee169c67\") " pod="openshift-multus/multus-4xzds" Jul 07 23:13:05.204987 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:05.203816 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a8270634-fda9-4520-85cb-0ab49e4ce06a-host-run-ovn-kubernetes\") pod \"ovnkube-node-w9gmc\" (UID: \"a8270634-fda9-4520-85cb-0ab49e4ce06a\") " pod="openshift-ovn-kubernetes/ovnkube-node-w9gmc" Jul 07 23:13:05.204987 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:05.203836 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lc4nw\" (UniqueName: \"kubernetes.io/projected/df04fe1d-eb1a-4711-80ea-72d68116bbca-kube-api-access-lc4nw\") pod \"node-ca-7z6hc\" (UID: \"df04fe1d-eb1a-4711-80ea-72d68116bbca\") " pod="openshift-image-registry/node-ca-7z6hc" Jul 07 23:13:05.204987 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:05.203825 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/a8270634-fda9-4520-85cb-0ab49e4ce06a-ovnkube-config\") pod \"ovnkube-node-w9gmc\" (UID: \"a8270634-fda9-4520-85cb-0ab49e4ce06a\") " pod="openshift-ovn-kubernetes/ovnkube-node-w9gmc" Jul 07 23:13:05.204987 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:05.203867 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/48b46894-bcf9-4176-82a7-bab0ee169c67-multus-conf-dir\") pod \"multus-4xzds\" (UID: \"48b46894-bcf9-4176-82a7-bab0ee169c67\") " pod="openshift-multus/multus-4xzds" Jul 07 23:13:05.204987 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:05.203871 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f2ec914a-0546-4c91-ac24-1fa24cde33f6-var-lib-kubelet\") pod \"tuned-cm9wz\" (UID: \"f2ec914a-0546-4c91-ac24-1fa24cde33f6\") " pod="openshift-cluster-node-tuning-operator/tuned-cm9wz" Jul 07 23:13:05.204987 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:05.203888 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/48b46894-bcf9-4176-82a7-bab0ee169c67-cnibin\") pod \"multus-4xzds\" (UID: \"48b46894-bcf9-4176-82a7-bab0ee169c67\") " pod="openshift-multus/multus-4xzds" Jul 07 23:13:05.204987 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:05.203903 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/48b46894-bcf9-4176-82a7-bab0ee169c67-multus-daemon-config\") pod \"multus-4xzds\" (UID: \"48b46894-bcf9-4176-82a7-bab0ee169c67\") " pod="openshift-multus/multus-4xzds" Jul 07 23:13:05.204987 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:05.203907 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/48b46894-bcf9-4176-82a7-bab0ee169c67-multus-conf-dir\") pod \"multus-4xzds\" (UID: \"48b46894-bcf9-4176-82a7-bab0ee169c67\") " pod="openshift-multus/multus-4xzds" Jul 07 23:13:05.204987 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:05.203932 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/f2ec914a-0546-4c91-ac24-1fa24cde33f6-etc-sysctl-conf\") pod \"tuned-cm9wz\" (UID: \"f2ec914a-0546-4c91-ac24-1fa24cde33f6\") " pod="openshift-cluster-node-tuning-operator/tuned-cm9wz" Jul 07 23:13:05.204987 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:05.203965 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a8270634-fda9-4520-85cb-0ab49e4ce06a-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-w9gmc\" (UID: \"a8270634-fda9-4520-85cb-0ab49e4ce06a\") " pod="openshift-ovn-kubernetes/ovnkube-node-w9gmc" Jul 07 23:13:05.204987 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:05.203992 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/a8270634-fda9-4520-85cb-0ab49e4ce06a-ovnkube-script-lib\") pod \"ovnkube-node-w9gmc\" (UID: \"a8270634-fda9-4520-85cb-0ab49e4ce06a\") " pod="openshift-ovn-kubernetes/ovnkube-node-w9gmc" Jul 07 23:13:05.204987 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:05.204014 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/48b46894-bcf9-4176-82a7-bab0ee169c67-host-var-lib-kubelet\") pod \"multus-4xzds\" (UID: \"48b46894-bcf9-4176-82a7-bab0ee169c67\") " pod="openshift-multus/multus-4xzds" Jul 07 23:13:05.204987 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:05.204051 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/48b46894-bcf9-4176-82a7-bab0ee169c67-hostroot\") pod \"multus-4xzds\" (UID: \"48b46894-bcf9-4176-82a7-bab0ee169c67\") " pod="openshift-multus/multus-4xzds" Jul 07 23:13:05.205970 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:05.204105 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/f2ec914a-0546-4c91-ac24-1fa24cde33f6-etc-sysctl-conf\") pod \"tuned-cm9wz\" (UID: \"f2ec914a-0546-4c91-ac24-1fa24cde33f6\") " pod="openshift-cluster-node-tuning-operator/tuned-cm9wz" Jul 07 23:13:05.205970 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:05.204172 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/48b46894-bcf9-4176-82a7-bab0ee169c67-host-var-lib-kubelet\") pod \"multus-4xzds\" (UID: \"48b46894-bcf9-4176-82a7-bab0ee169c67\") " pod="openshift-multus/multus-4xzds" Jul 07 23:13:05.205970 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:05.204217 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a8270634-fda9-4520-85cb-0ab49e4ce06a-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-w9gmc\" (UID: \"a8270634-fda9-4520-85cb-0ab49e4ce06a\") " pod="openshift-ovn-kubernetes/ovnkube-node-w9gmc" Jul 07 23:13:05.205970 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:05.204252 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/48b46894-bcf9-4176-82a7-bab0ee169c67-etc-kubernetes\") pod \"multus-4xzds\" (UID: \"48b46894-bcf9-4176-82a7-bab0ee169c67\") " pod="openshift-multus/multus-4xzds" Jul 07 23:13:05.205970 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:05.204278 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/f2ec914a-0546-4c91-ac24-1fa24cde33f6-etc-sysctl-d\") pod \"tuned-cm9wz\" (UID: \"f2ec914a-0546-4c91-ac24-1fa24cde33f6\") " pod="openshift-cluster-node-tuning-operator/tuned-cm9wz" Jul 07 23:13:05.205970 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:05.204303 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c02c0747-b1ae-452c-81aa-5101d7125fc9-tmp-dir\") pod \"node-resolver-qgcpr\" (UID: \"c02c0747-b1ae-452c-81aa-5101d7125fc9\") " pod="openshift-dns/node-resolver-qgcpr" Jul 07 23:13:05.205970 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:05.204334 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/1496ff7f-366b-4cd2-8396-2cdbfed841fa-kubelet-config\") pod \"global-pull-secret-syncer-84hdc\" (UID: \"1496ff7f-366b-4cd2-8396-2cdbfed841fa\") " pod="kube-system/global-pull-secret-syncer-84hdc" Jul 07 23:13:05.205970 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:05.204366 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/f2ec914a-0546-4c91-ac24-1fa24cde33f6-etc-modprobe-d\") pod \"tuned-cm9wz\" (UID: \"f2ec914a-0546-4c91-ac24-1fa24cde33f6\") " pod="openshift-cluster-node-tuning-operator/tuned-cm9wz" Jul 07 23:13:05.205970 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:05.204393 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/a8270634-fda9-4520-85cb-0ab49e4ce06a-run-ovn\") pod \"ovnkube-node-w9gmc\" (UID: \"a8270634-fda9-4520-85cb-0ab49e4ce06a\") " pod="openshift-ovn-kubernetes/ovnkube-node-w9gmc" Jul 07 23:13:05.205970 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:05.204420 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/a8270634-fda9-4520-85cb-0ab49e4ce06a-node-log\") pod \"ovnkube-node-w9gmc\" (UID: \"a8270634-fda9-4520-85cb-0ab49e4ce06a\") " pod="openshift-ovn-kubernetes/ovnkube-node-w9gmc" Jul 07 23:13:05.205970 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:05.204445 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a8270634-fda9-4520-85cb-0ab49e4ce06a-host-cni-bin\") pod \"ovnkube-node-w9gmc\" (UID: \"a8270634-fda9-4520-85cb-0ab49e4ce06a\") " pod="openshift-ovn-kubernetes/ovnkube-node-w9gmc" Jul 07 23:13:05.205970 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:05.204472 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/a8270634-fda9-4520-85cb-0ab49e4ce06a-host-cni-netd\") pod \"ovnkube-node-w9gmc\" (UID: \"a8270634-fda9-4520-85cb-0ab49e4ce06a\") " pod="openshift-ovn-kubernetes/ovnkube-node-w9gmc" Jul 07 23:13:05.205970 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:05.204497 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/f2ec914a-0546-4c91-ac24-1fa24cde33f6-etc-systemd\") pod \"tuned-cm9wz\" (UID: \"f2ec914a-0546-4c91-ac24-1fa24cde33f6\") " pod="openshift-cluster-node-tuning-operator/tuned-cm9wz" Jul 07 23:13:05.205970 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:05.204524 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/48b46894-bcf9-4176-82a7-bab0ee169c67-multus-daemon-config\") pod \"multus-4xzds\" (UID: \"48b46894-bcf9-4176-82a7-bab0ee169c67\") " pod="openshift-multus/multus-4xzds" Jul 07 23:13:05.205970 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:05.204545 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f2ec914a-0546-4c91-ac24-1fa24cde33f6-tmp\") pod \"tuned-cm9wz\" (UID: \"f2ec914a-0546-4c91-ac24-1fa24cde33f6\") " pod="openshift-cluster-node-tuning-operator/tuned-cm9wz" Jul 07 23:13:05.205970 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:05.204574 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/c02c0747-b1ae-452c-81aa-5101d7125fc9-hosts-file\") pod \"node-resolver-qgcpr\" (UID: \"c02c0747-b1ae-452c-81aa-5101d7125fc9\") " pod="openshift-dns/node-resolver-qgcpr" Jul 07 23:13:05.205970 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:05.204583 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/48b46894-bcf9-4176-82a7-bab0ee169c67-hostroot\") pod \"multus-4xzds\" (UID: \"48b46894-bcf9-4176-82a7-bab0ee169c67\") " pod="openshift-multus/multus-4xzds" Jul 07 23:13:05.205970 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:05.204598 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/f2ec914a-0546-4c91-ac24-1fa24cde33f6-etc-sysconfig\") pod \"tuned-cm9wz\" (UID: \"f2ec914a-0546-4c91-ac24-1fa24cde33f6\") " pod="openshift-cluster-node-tuning-operator/tuned-cm9wz" Jul 07 23:13:05.206880 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:05.204625 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/48b46894-bcf9-4176-82a7-bab0ee169c67-cni-binary-copy\") pod \"multus-4xzds\" (UID: \"48b46894-bcf9-4176-82a7-bab0ee169c67\") " pod="openshift-multus/multus-4xzds" Jul 07 23:13:05.206880 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:05.204628 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/a8270634-fda9-4520-85cb-0ab49e4ce06a-ovnkube-script-lib\") pod \"ovnkube-node-w9gmc\" (UID: \"a8270634-fda9-4520-85cb-0ab49e4ce06a\") " pod="openshift-ovn-kubernetes/ovnkube-node-w9gmc" Jul 07 23:13:05.206880 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:05.204637 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/f2ec914a-0546-4c91-ac24-1fa24cde33f6-etc-modprobe-d\") pod \"tuned-cm9wz\" (UID: \"f2ec914a-0546-4c91-ac24-1fa24cde33f6\") " pod="openshift-cluster-node-tuning-operator/tuned-cm9wz" Jul 07 23:13:05.206880 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:05.204650 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/48b46894-bcf9-4176-82a7-bab0ee169c67-host-var-lib-cni-bin\") pod \"multus-4xzds\" (UID: \"48b46894-bcf9-4176-82a7-bab0ee169c67\") " pod="openshift-multus/multus-4xzds" Jul 07 23:13:05.206880 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:05.204685 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7dp8c\" (UniqueName: \"kubernetes.io/projected/48b46894-bcf9-4176-82a7-bab0ee169c67-kube-api-access-7dp8c\") pod \"multus-4xzds\" (UID: \"48b46894-bcf9-4176-82a7-bab0ee169c67\") " pod="openshift-multus/multus-4xzds" Jul 07 23:13:05.206880 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:05.204699 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/a8270634-fda9-4520-85cb-0ab49e4ce06a-run-ovn\") pod \"ovnkube-node-w9gmc\" (UID: \"a8270634-fda9-4520-85cb-0ab49e4ce06a\") " pod="openshift-ovn-kubernetes/ovnkube-node-w9gmc" Jul 07 23:13:05.206880 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:05.204705 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/f2ec914a-0546-4c91-ac24-1fa24cde33f6-etc-sysctl-d\") pod \"tuned-cm9wz\" (UID: \"f2ec914a-0546-4c91-ac24-1fa24cde33f6\") " pod="openshift-cluster-node-tuning-operator/tuned-cm9wz" Jul 07 23:13:05.206880 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:05.204713 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1496ff7f-366b-4cd2-8396-2cdbfed841fa-original-pull-secret\") pod \"global-pull-secret-syncer-84hdc\" (UID: \"1496ff7f-366b-4cd2-8396-2cdbfed841fa\") " pod="kube-system/global-pull-secret-syncer-84hdc" Jul 07 23:13:05.206880 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:05.204743 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9c9vq\" (UniqueName: \"kubernetes.io/projected/c02c0747-b1ae-452c-81aa-5101d7125fc9-kube-api-access-9c9vq\") pod \"node-resolver-qgcpr\" (UID: \"c02c0747-b1ae-452c-81aa-5101d7125fc9\") " pod="openshift-dns/node-resolver-qgcpr" Jul 07 23:13:05.206880 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:05.204752 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/48b46894-bcf9-4176-82a7-bab0ee169c67-etc-kubernetes\") pod \"multus-4xzds\" (UID: \"48b46894-bcf9-4176-82a7-bab0ee169c67\") " pod="openshift-multus/multus-4xzds" Jul 07 23:13:05.206880 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:05.204770 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/a8270634-fda9-4520-85cb-0ab49e4ce06a-env-overrides\") pod \"ovnkube-node-w9gmc\" (UID: \"a8270634-fda9-4520-85cb-0ab49e4ce06a\") " pod="openshift-ovn-kubernetes/ovnkube-node-w9gmc" Jul 07 23:13:05.206880 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:05.204776 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a8270634-fda9-4520-85cb-0ab49e4ce06a-host-cni-bin\") pod \"ovnkube-node-w9gmc\" (UID: \"a8270634-fda9-4520-85cb-0ab49e4ce06a\") " pod="openshift-ovn-kubernetes/ovnkube-node-w9gmc" Jul 07 23:13:05.206880 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:05.204799 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9cc2p\" (UniqueName: \"kubernetes.io/projected/a8270634-fda9-4520-85cb-0ab49e4ce06a-kube-api-access-9cc2p\") pod \"ovnkube-node-w9gmc\" (UID: \"a8270634-fda9-4520-85cb-0ab49e4ce06a\") " pod="openshift-ovn-kubernetes/ovnkube-node-w9gmc" Jul 07 23:13:05.206880 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:05.204842 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/dd19d3d5-47a8-4052-97ea-f4e5fbbb88cc-kubelet-dir\") pod \"aws-ebs-csi-driver-node-ltc6z\" (UID: \"dd19d3d5-47a8-4052-97ea-f4e5fbbb88cc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ltc6z" Jul 07 23:13:05.206880 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:05.204864 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/c02c0747-b1ae-452c-81aa-5101d7125fc9-hosts-file\") pod \"node-resolver-qgcpr\" (UID: \"c02c0747-b1ae-452c-81aa-5101d7125fc9\") " pod="openshift-dns/node-resolver-qgcpr" Jul 07 23:13:05.206880 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:05.204895 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c02c0747-b1ae-452c-81aa-5101d7125fc9-tmp-dir\") pod \"node-resolver-qgcpr\" (UID: \"c02c0747-b1ae-452c-81aa-5101d7125fc9\") " pod="openshift-dns/node-resolver-qgcpr" Jul 07 23:13:05.206880 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:05.204895 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-km9rw\" (UniqueName: \"kubernetes.io/projected/dd19d3d5-47a8-4052-97ea-f4e5fbbb88cc-kube-api-access-km9rw\") pod \"aws-ebs-csi-driver-node-ltc6z\" (UID: \"dd19d3d5-47a8-4052-97ea-f4e5fbbb88cc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ltc6z" Jul 07 23:13:05.206880 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:05.204925 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/f2ec914a-0546-4c91-ac24-1fa24cde33f6-etc-sysconfig\") pod \"tuned-cm9wz\" (UID: \"f2ec914a-0546-4c91-ac24-1fa24cde33f6\") " pod="openshift-cluster-node-tuning-operator/tuned-cm9wz" Jul 07 23:13:05.207794 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:05.204931 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/48b46894-bcf9-4176-82a7-bab0ee169c67-multus-cni-dir\") pod \"multus-4xzds\" (UID: \"48b46894-bcf9-4176-82a7-bab0ee169c67\") " pod="openshift-multus/multus-4xzds" Jul 07 23:13:05.207794 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:05.204984 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/1496ff7f-366b-4cd2-8396-2cdbfed841fa-kubelet-config\") pod \"global-pull-secret-syncer-84hdc\" (UID: \"1496ff7f-366b-4cd2-8396-2cdbfed841fa\") " pod="kube-system/global-pull-secret-syncer-84hdc" Jul 07 23:13:05.207794 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:05.204988 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/48b46894-bcf9-4176-82a7-bab0ee169c67-multus-socket-dir-parent\") pod \"multus-4xzds\" (UID: \"48b46894-bcf9-4176-82a7-bab0ee169c67\") " pod="openshift-multus/multus-4xzds" Jul 07 23:13:05.207794 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:05.205030 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/48b46894-bcf9-4176-82a7-bab0ee169c67-multus-socket-dir-parent\") pod \"multus-4xzds\" (UID: \"48b46894-bcf9-4176-82a7-bab0ee169c67\") " pod="openshift-multus/multus-4xzds" Jul 07 23:13:05.207794 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:05.205069 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/a8270634-fda9-4520-85cb-0ab49e4ce06a-host-cni-netd\") pod \"ovnkube-node-w9gmc\" (UID: \"a8270634-fda9-4520-85cb-0ab49e4ce06a\") " pod="openshift-ovn-kubernetes/ovnkube-node-w9gmc" Jul 07 23:13:05.207794 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:05.205348 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/48b46894-bcf9-4176-82a7-bab0ee169c67-multus-cni-dir\") pod \"multus-4xzds\" (UID: \"48b46894-bcf9-4176-82a7-bab0ee169c67\") " pod="openshift-multus/multus-4xzds" Jul 07 23:13:05.207794 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:05.205417 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/48b46894-bcf9-4176-82a7-bab0ee169c67-cni-binary-copy\") pod \"multus-4xzds\" (UID: \"48b46894-bcf9-4176-82a7-bab0ee169c67\") " pod="openshift-multus/multus-4xzds" Jul 07 23:13:05.207794 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:05.205437 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/dd19d3d5-47a8-4052-97ea-f4e5fbbb88cc-kubelet-dir\") pod \"aws-ebs-csi-driver-node-ltc6z\" (UID: \"dd19d3d5-47a8-4052-97ea-f4e5fbbb88cc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ltc6z" Jul 07 23:13:05.207794 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:05.205448 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/a8270634-fda9-4520-85cb-0ab49e4ce06a-env-overrides\") pod \"ovnkube-node-w9gmc\" (UID: \"a8270634-fda9-4520-85cb-0ab49e4ce06a\") " pod="openshift-ovn-kubernetes/ovnkube-node-w9gmc" Jul 07 23:13:05.207794 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:05.204744 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/a8270634-fda9-4520-85cb-0ab49e4ce06a-node-log\") pod \"ovnkube-node-w9gmc\" (UID: \"a8270634-fda9-4520-85cb-0ab49e4ce06a\") " pod="openshift-ovn-kubernetes/ovnkube-node-w9gmc" Jul 07 23:13:05.207794 ip-10-0-134-140 kubenswrapper[2570]: E0707 23:13:05.205469 2570 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 07 23:13:05.207794 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:05.205480 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/48b46894-bcf9-4176-82a7-bab0ee169c67-host-var-lib-cni-bin\") pod \"multus-4xzds\" (UID: \"48b46894-bcf9-4176-82a7-bab0ee169c67\") " pod="openshift-multus/multus-4xzds" Jul 07 23:13:05.207794 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:05.205487 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/f2ec914a-0546-4c91-ac24-1fa24cde33f6-etc-systemd\") pod \"tuned-cm9wz\" (UID: \"f2ec914a-0546-4c91-ac24-1fa24cde33f6\") " pod="openshift-cluster-node-tuning-operator/tuned-cm9wz" Jul 07 23:13:05.207794 ip-10-0-134-140 kubenswrapper[2570]: E0707 23:13:05.205539 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1496ff7f-366b-4cd2-8396-2cdbfed841fa-original-pull-secret podName:1496ff7f-366b-4cd2-8396-2cdbfed841fa nodeName:}" failed. No retries permitted until 2026-07-07 23:13:05.705518325 +0000 UTC m=+3.102519822 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/1496ff7f-366b-4cd2-8396-2cdbfed841fa-original-pull-secret") pod "global-pull-secret-syncer-84hdc" (UID: "1496ff7f-366b-4cd2-8396-2cdbfed841fa") : object "kube-system"/"original-pull-secret" not registered Jul 07 23:13:05.207794 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:05.205928 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/f2ec914a-0546-4c91-ac24-1fa24cde33f6-etc-tuned\") pod \"tuned-cm9wz\" (UID: \"f2ec914a-0546-4c91-ac24-1fa24cde33f6\") " pod="openshift-cluster-node-tuning-operator/tuned-cm9wz" Jul 07 23:13:05.207794 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:05.206158 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/a8270634-fda9-4520-85cb-0ab49e4ce06a-ovn-node-metrics-cert\") pod \"ovnkube-node-w9gmc\" (UID: \"a8270634-fda9-4520-85cb-0ab49e4ce06a\") " pod="openshift-ovn-kubernetes/ovnkube-node-w9gmc" Jul 07 23:13:05.209305 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:05.209247 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f2ec914a-0546-4c91-ac24-1fa24cde33f6-tmp\") pod \"tuned-cm9wz\" (UID: \"f2ec914a-0546-4c91-ac24-1fa24cde33f6\") " pod="openshift-cluster-node-tuning-operator/tuned-cm9wz" Jul 07 23:13:05.213514 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:05.213383 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lc4nw\" (UniqueName: \"kubernetes.io/projected/df04fe1d-eb1a-4711-80ea-72d68116bbca-kube-api-access-lc4nw\") pod \"node-ca-7z6hc\" (UID: \"df04fe1d-eb1a-4711-80ea-72d68116bbca\") " pod="openshift-image-registry/node-ca-7z6hc" Jul 07 23:13:05.214248 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:05.214082 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7f8v8\" (UniqueName: \"kubernetes.io/projected/f2ec914a-0546-4c91-ac24-1fa24cde33f6-kube-api-access-7f8v8\") pod \"tuned-cm9wz\" (UID: \"f2ec914a-0546-4c91-ac24-1fa24cde33f6\") " pod="openshift-cluster-node-tuning-operator/tuned-cm9wz" Jul 07 23:13:05.216084 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:05.216054 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-km9rw\" (UniqueName: \"kubernetes.io/projected/dd19d3d5-47a8-4052-97ea-f4e5fbbb88cc-kube-api-access-km9rw\") pod \"aws-ebs-csi-driver-node-ltc6z\" (UID: \"dd19d3d5-47a8-4052-97ea-f4e5fbbb88cc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ltc6z" Jul 07 23:13:05.218366 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:05.218337 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9cc2p\" (UniqueName: \"kubernetes.io/projected/a8270634-fda9-4520-85cb-0ab49e4ce06a-kube-api-access-9cc2p\") pod \"ovnkube-node-w9gmc\" (UID: \"a8270634-fda9-4520-85cb-0ab49e4ce06a\") " pod="openshift-ovn-kubernetes/ovnkube-node-w9gmc" Jul 07 23:13:05.228305 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:05.228277 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7dp8c\" (UniqueName: \"kubernetes.io/projected/48b46894-bcf9-4176-82a7-bab0ee169c67-kube-api-access-7dp8c\") pod \"multus-4xzds\" (UID: \"48b46894-bcf9-4176-82a7-bab0ee169c67\") " pod="openshift-multus/multus-4xzds" Jul 07 23:13:05.228902 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:05.228879 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9c9vq\" (UniqueName: \"kubernetes.io/projected/c02c0747-b1ae-452c-81aa-5101d7125fc9-kube-api-access-9c9vq\") pod \"node-resolver-qgcpr\" (UID: \"c02c0747-b1ae-452c-81aa-5101d7125fc9\") " pod="openshift-dns/node-resolver-qgcpr" Jul 07 23:13:05.293901 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:05.293811 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-wjn84" Jul 07 23:13:05.303060 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:05.302862 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-vztph" Jul 07 23:13:05.316009 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:05.315520 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-6mdgj" Jul 07 23:13:05.323238 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:05.323216 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-cm9wz" Jul 07 23:13:05.328402 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:05.328377 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfc88a6f6_fb63_4fcd_b58f_980e71c1b5d0.slice/crio-d16647fd64ccf3343db26a8b293e7d6145168afd0a827c2b36f438e721d40d92 WatchSource:0}: Error finding container d16647fd64ccf3343db26a8b293e7d6145168afd0a827c2b36f438e721d40d92: Status 404 returned error can't find the container with id d16647fd64ccf3343db26a8b293e7d6145168afd0a827c2b36f438e721d40d92 Jul 07 23:13:05.333974 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:05.333806 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-qgcpr" Jul 07 23:13:05.336621 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:05.336599 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf2ec914a_0546_4c91_ac24_1fa24cde33f6.slice/crio-e478f838dca33411887c962286ef7c08cceafce514fe8803f5a38fa2982b06ff WatchSource:0}: Error finding container e478f838dca33411887c962286ef7c08cceafce514fe8803f5a38fa2982b06ff: Status 404 returned error can't find the container with id e478f838dca33411887c962286ef7c08cceafce514fe8803f5a38fa2982b06ff Jul 07 23:13:05.344111 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:05.344086 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-7z6hc" Jul 07 23:13:05.345515 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:05.345483 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc02c0747_b1ae_452c_81aa_5101d7125fc9.slice/crio-c035465e68c5773e0c78d90395f406c940181f4bbc77d2975e84bc59a05f0df2 WatchSource:0}: Error finding container c035465e68c5773e0c78d90395f406c940181f4bbc77d2975e84bc59a05f0df2: Status 404 returned error can't find the container with id c035465e68c5773e0c78d90395f406c940181f4bbc77d2975e84bc59a05f0df2 Jul 07 23:13:05.352977 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:05.352950 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ltc6z" Jul 07 23:13:05.355696 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:05.355672 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddf04fe1d_eb1a_4711_80ea_72d68116bbca.slice/crio-c9026ab3468cc049cfcd067df447975f654faebec17ddc21f00bc3d4ca4ccc94 WatchSource:0}: Error finding container c9026ab3468cc049cfcd067df447975f654faebec17ddc21f00bc3d4ca4ccc94: Status 404 returned error can't find the container with id c9026ab3468cc049cfcd067df447975f654faebec17ddc21f00bc3d4ca4ccc94 Jul 07 23:13:05.360259 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:05.360238 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-4xzds" Jul 07 23:13:05.364745 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:05.364710 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddd19d3d5_47a8_4052_97ea_f4e5fbbb88cc.slice/crio-1a145e8f23e1f595b0bf52fe985f73d2c15493f36354742058d13935cb3dab13 WatchSource:0}: Error finding container 1a145e8f23e1f595b0bf52fe985f73d2c15493f36354742058d13935cb3dab13: Status 404 returned error can't find the container with id 1a145e8f23e1f595b0bf52fe985f73d2c15493f36354742058d13935cb3dab13 Jul 07 23:13:05.367820 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:05.367800 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-w9gmc" Jul 07 23:13:05.372596 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:05.372569 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod48b46894_bcf9_4176_82a7_bab0ee169c67.slice/crio-d368166e9212e8e2b725b10e6494418282400a1e55debfd95f978e9a01298475 WatchSource:0}: Error finding container d368166e9212e8e2b725b10e6494418282400a1e55debfd95f978e9a01298475: Status 404 returned error can't find the container with id d368166e9212e8e2b725b10e6494418282400a1e55debfd95f978e9a01298475 Jul 07 23:13:05.383004 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:05.382979 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda8270634_fda9_4520_85cb_0ab49e4ce06a.slice/crio-c9727f5e57524c814552f4d64c324c2daee4eb3a366e92052cd100b82f3861db WatchSource:0}: Error finding container c9727f5e57524c814552f4d64c324c2daee4eb3a366e92052cd100b82f3861db: Status 404 returned error can't find the container with id c9727f5e57524c814552f4d64c324c2daee4eb3a366e92052cd100b82f3861db Jul 07 23:13:05.609254 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:05.609163 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/14406dc0-620c-4615-b98a-b7205dc72966-metrics-certs\") pod \"network-metrics-daemon-ndz2q\" (UID: \"14406dc0-620c-4615-b98a-b7205dc72966\") " pod="openshift-multus/network-metrics-daemon-ndz2q" Jul 07 23:13:05.609408 ip-10-0-134-140 kubenswrapper[2570]: E0707 23:13:05.609301 2570 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 23:13:05.609408 ip-10-0-134-140 kubenswrapper[2570]: E0707 23:13:05.609363 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/14406dc0-620c-4615-b98a-b7205dc72966-metrics-certs podName:14406dc0-620c-4615-b98a-b7205dc72966 nodeName:}" failed. No retries permitted until 2026-07-07 23:13:06.609349063 +0000 UTC m=+4.006350625 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/14406dc0-620c-4615-b98a-b7205dc72966-metrics-certs") pod "network-metrics-daemon-ndz2q" (UID: "14406dc0-620c-4615-b98a-b7205dc72966") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 23:13:05.709920 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:05.709888 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bkbfq\" (UniqueName: \"kubernetes.io/projected/f47fd470-2c7a-4ea8-bf73-e3f94320f117-kube-api-access-bkbfq\") pod \"network-check-target-p27rw\" (UID: \"f47fd470-2c7a-4ea8-bf73-e3f94320f117\") " pod="openshift-network-diagnostics/network-check-target-p27rw" Jul 07 23:13:05.710101 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:05.709941 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1496ff7f-366b-4cd2-8396-2cdbfed841fa-original-pull-secret\") pod \"global-pull-secret-syncer-84hdc\" (UID: \"1496ff7f-366b-4cd2-8396-2cdbfed841fa\") " pod="kube-system/global-pull-secret-syncer-84hdc" Jul 07 23:13:05.710101 ip-10-0-134-140 kubenswrapper[2570]: E0707 23:13:05.710053 2570 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 07 23:13:05.710101 ip-10-0-134-140 kubenswrapper[2570]: E0707 23:13:05.710069 2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 07 23:13:05.710101 ip-10-0-134-140 kubenswrapper[2570]: E0707 23:13:05.710097 2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 07 23:13:05.710259 ip-10-0-134-140 kubenswrapper[2570]: E0707 23:13:05.710110 2570 projected.go:196] Error preparing data for projected volume kube-api-access-bkbfq for pod openshift-network-diagnostics/network-check-target-p27rw: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 23:13:05.710259 ip-10-0-134-140 kubenswrapper[2570]: E0707 23:13:05.710126 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1496ff7f-366b-4cd2-8396-2cdbfed841fa-original-pull-secret podName:1496ff7f-366b-4cd2-8396-2cdbfed841fa nodeName:}" failed. No retries permitted until 2026-07-07 23:13:06.71010568 +0000 UTC m=+4.107107173 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/1496ff7f-366b-4cd2-8396-2cdbfed841fa-original-pull-secret") pod "global-pull-secret-syncer-84hdc" (UID: "1496ff7f-366b-4cd2-8396-2cdbfed841fa") : object "kube-system"/"original-pull-secret" not registered Jul 07 23:13:05.710259 ip-10-0-134-140 kubenswrapper[2570]: E0707 23:13:05.710174 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f47fd470-2c7a-4ea8-bf73-e3f94320f117-kube-api-access-bkbfq podName:f47fd470-2c7a-4ea8-bf73-e3f94320f117 nodeName:}" failed. No retries permitted until 2026-07-07 23:13:06.710161669 +0000 UTC m=+4.107163159 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-bkbfq" (UniqueName: "kubernetes.io/projected/f47fd470-2c7a-4ea8-bf73-e3f94320f117-kube-api-access-bkbfq") pod "network-check-target-p27rw" (UID: "f47fd470-2c7a-4ea8-bf73-e3f94320f117") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 23:13:06.046737 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:06.046653 2570 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-06 23:08:05 +0000 UTC" deadline="2028-03-09 06:20:40.870032814 +0000 UTC" Jul 07 23:13:06.046737 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:06.046688 2570 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14647h7m34.823347901s" Jul 07 23:13:06.106688 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:06.106647 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-w9gmc" event={"ID":"a8270634-fda9-4520-85cb-0ab49e4ce06a","Type":"ContainerStarted","Data":"c9727f5e57524c814552f4d64c324c2daee4eb3a366e92052cd100b82f3861db"} Jul 07 23:13:06.107820 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:06.107795 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-7z6hc" event={"ID":"df04fe1d-eb1a-4711-80ea-72d68116bbca","Type":"ContainerStarted","Data":"c9026ab3468cc049cfcd067df447975f654faebec17ddc21f00bc3d4ca4ccc94"} Jul 07 23:13:06.108977 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:06.108923 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-6mdgj" event={"ID":"fc88a6f6-fb63-4fcd-b58f-980e71c1b5d0","Type":"ContainerStarted","Data":"d16647fd64ccf3343db26a8b293e7d6145168afd0a827c2b36f438e721d40d92"} Jul 07 23:13:06.110128 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:06.110097 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-4xzds" event={"ID":"48b46894-bcf9-4176-82a7-bab0ee169c67","Type":"ContainerStarted","Data":"d368166e9212e8e2b725b10e6494418282400a1e55debfd95f978e9a01298475"} Jul 07 23:13:06.111266 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:06.111242 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ltc6z" event={"ID":"dd19d3d5-47a8-4052-97ea-f4e5fbbb88cc","Type":"ContainerStarted","Data":"1a145e8f23e1f595b0bf52fe985f73d2c15493f36354742058d13935cb3dab13"} Jul 07 23:13:06.112379 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:06.112355 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-qgcpr" event={"ID":"c02c0747-b1ae-452c-81aa-5101d7125fc9","Type":"ContainerStarted","Data":"c035465e68c5773e0c78d90395f406c940181f4bbc77d2975e84bc59a05f0df2"} Jul 07 23:13:06.113585 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:06.113552 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-cm9wz" event={"ID":"f2ec914a-0546-4c91-ac24-1fa24cde33f6","Type":"ContainerStarted","Data":"e478f838dca33411887c962286ef7c08cceafce514fe8803f5a38fa2982b06ff"} Jul 07 23:13:06.114758 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:06.114722 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vztph" event={"ID":"573e90ac-1d7d-4584-b231-6635ac4e12e2","Type":"ContainerStarted","Data":"aef025c07c8c457cd540fead2a41cbc0be994b009b8d8ddd17ede8e047342bc5"} Jul 07 23:13:06.115799 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:06.115778 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-wjn84" event={"ID":"0bb01852-570a-4456-9aca-52c99f9f83e5","Type":"ContainerStarted","Data":"bf10993f0199021fe8253d90fdd55494a9ee88c28786d908a0e7fedfb149f513"} Jul 07 23:13:06.617233 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:06.617192 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/14406dc0-620c-4615-b98a-b7205dc72966-metrics-certs\") pod \"network-metrics-daemon-ndz2q\" (UID: \"14406dc0-620c-4615-b98a-b7205dc72966\") " pod="openshift-multus/network-metrics-daemon-ndz2q" Jul 07 23:13:06.617443 ip-10-0-134-140 kubenswrapper[2570]: E0707 23:13:06.617409 2570 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 23:13:06.617530 ip-10-0-134-140 kubenswrapper[2570]: E0707 23:13:06.617497 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/14406dc0-620c-4615-b98a-b7205dc72966-metrics-certs podName:14406dc0-620c-4615-b98a-b7205dc72966 nodeName:}" failed. No retries permitted until 2026-07-07 23:13:08.617457696 +0000 UTC m=+6.014459191 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/14406dc0-620c-4615-b98a-b7205dc72966-metrics-certs") pod "network-metrics-daemon-ndz2q" (UID: "14406dc0-620c-4615-b98a-b7205dc72966") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 23:13:06.717846 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:06.717803 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bkbfq\" (UniqueName: \"kubernetes.io/projected/f47fd470-2c7a-4ea8-bf73-e3f94320f117-kube-api-access-bkbfq\") pod \"network-check-target-p27rw\" (UID: \"f47fd470-2c7a-4ea8-bf73-e3f94320f117\") " pod="openshift-network-diagnostics/network-check-target-p27rw" Jul 07 23:13:06.718027 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:06.717885 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1496ff7f-366b-4cd2-8396-2cdbfed841fa-original-pull-secret\") pod \"global-pull-secret-syncer-84hdc\" (UID: \"1496ff7f-366b-4cd2-8396-2cdbfed841fa\") " pod="kube-system/global-pull-secret-syncer-84hdc" Jul 07 23:13:06.718027 ip-10-0-134-140 kubenswrapper[2570]: E0707 23:13:06.718020 2570 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 07 23:13:06.718136 ip-10-0-134-140 kubenswrapper[2570]: E0707 23:13:06.718086 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1496ff7f-366b-4cd2-8396-2cdbfed841fa-original-pull-secret podName:1496ff7f-366b-4cd2-8396-2cdbfed841fa nodeName:}" failed. No retries permitted until 2026-07-07 23:13:08.718065563 +0000 UTC m=+6.115067075 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/1496ff7f-366b-4cd2-8396-2cdbfed841fa-original-pull-secret") pod "global-pull-secret-syncer-84hdc" (UID: "1496ff7f-366b-4cd2-8396-2cdbfed841fa") : object "kube-system"/"original-pull-secret" not registered Jul 07 23:13:06.718696 ip-10-0-134-140 kubenswrapper[2570]: E0707 23:13:06.718674 2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 07 23:13:06.718819 ip-10-0-134-140 kubenswrapper[2570]: E0707 23:13:06.718702 2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 07 23:13:06.718819 ip-10-0-134-140 kubenswrapper[2570]: E0707 23:13:06.718715 2570 projected.go:196] Error preparing data for projected volume kube-api-access-bkbfq for pod openshift-network-diagnostics/network-check-target-p27rw: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 23:13:06.718819 ip-10-0-134-140 kubenswrapper[2570]: E0707 23:13:06.718765 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f47fd470-2c7a-4ea8-bf73-e3f94320f117-kube-api-access-bkbfq podName:f47fd470-2c7a-4ea8-bf73-e3f94320f117 nodeName:}" failed. No retries permitted until 2026-07-07 23:13:08.718750256 +0000 UTC m=+6.115751746 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-bkbfq" (UniqueName: "kubernetes.io/projected/f47fd470-2c7a-4ea8-bf73-e3f94320f117-kube-api-access-bkbfq") pod "network-check-target-p27rw" (UID: "f47fd470-2c7a-4ea8-bf73-e3f94320f117") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 23:13:07.047125 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:07.047086 2570 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-06 23:08:05 +0000 UTC" deadline="2028-01-04 11:21:44.624731372 +0000 UTC" Jul 07 23:13:07.047125 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:07.047124 2570 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13092h8m37.577612673s" Jul 07 23:13:07.093097 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:07.093069 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ndz2q" Jul 07 23:13:07.093535 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:07.093513 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-p27rw" Jul 07 23:13:07.093693 ip-10-0-134-140 kubenswrapper[2570]: E0707 23:13:07.093668 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-p27rw" podUID="f47fd470-2c7a-4ea8-bf73-e3f94320f117" Jul 07 23:13:07.093769 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:07.093683 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-84hdc" Jul 07 23:13:07.093900 ip-10-0-134-140 kubenswrapper[2570]: E0707 23:13:07.093851 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-84hdc" podUID="1496ff7f-366b-4cd2-8396-2cdbfed841fa" Jul 07 23:13:07.094697 ip-10-0-134-140 kubenswrapper[2570]: E0707 23:13:07.094631 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ndz2q" podUID="14406dc0-620c-4615-b98a-b7205dc72966" Jul 07 23:13:07.139712 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:07.139673 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-140.ec2.internal" event={"ID":"7257f873d4401d71530f0192e065f67c","Type":"ContainerStarted","Data":"628de0478fee7a45ac2154f0f3369052b0c45dccaadb7621f21a854b74149c3c"} Jul 07 23:13:08.158334 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:08.158092 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-140.ec2.internal" event={"ID":"3bc0442220b7598e60bd507f09022656","Type":"ContainerStarted","Data":"700411bcc2e2e6772f36e143dbbe8ee9afa218b891ef895fa65b10dc831a488a"} Jul 07 23:13:08.174168 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:08.174106 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-140.ec2.internal" podStartSLOduration=4.174084617 podStartE2EDuration="4.174084617s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 23:13:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 23:13:07.158621562 +0000 UTC m=+4.555623095" watchObservedRunningTime="2026-07-07 23:13:08.174084617 +0000 UTC m=+5.571086130" Jul 07 23:13:08.634900 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:08.634862 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/14406dc0-620c-4615-b98a-b7205dc72966-metrics-certs\") pod \"network-metrics-daemon-ndz2q\" (UID: \"14406dc0-620c-4615-b98a-b7205dc72966\") " pod="openshift-multus/network-metrics-daemon-ndz2q" Jul 07 23:13:08.635126 ip-10-0-134-140 kubenswrapper[2570]: E0707 23:13:08.635108 2570 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 23:13:08.635198 ip-10-0-134-140 kubenswrapper[2570]: E0707 23:13:08.635179 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/14406dc0-620c-4615-b98a-b7205dc72966-metrics-certs podName:14406dc0-620c-4615-b98a-b7205dc72966 nodeName:}" failed. No retries permitted until 2026-07-07 23:13:12.635159326 +0000 UTC m=+10.032160822 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/14406dc0-620c-4615-b98a-b7205dc72966-metrics-certs") pod "network-metrics-daemon-ndz2q" (UID: "14406dc0-620c-4615-b98a-b7205dc72966") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 23:13:08.737063 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:08.736242 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bkbfq\" (UniqueName: \"kubernetes.io/projected/f47fd470-2c7a-4ea8-bf73-e3f94320f117-kube-api-access-bkbfq\") pod \"network-check-target-p27rw\" (UID: \"f47fd470-2c7a-4ea8-bf73-e3f94320f117\") " pod="openshift-network-diagnostics/network-check-target-p27rw" Jul 07 23:13:08.737063 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:08.736308 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1496ff7f-366b-4cd2-8396-2cdbfed841fa-original-pull-secret\") pod \"global-pull-secret-syncer-84hdc\" (UID: \"1496ff7f-366b-4cd2-8396-2cdbfed841fa\") " pod="kube-system/global-pull-secret-syncer-84hdc" Jul 07 23:13:08.737063 ip-10-0-134-140 kubenswrapper[2570]: E0707 23:13:08.736447 2570 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 07 23:13:08.737063 ip-10-0-134-140 kubenswrapper[2570]: E0707 23:13:08.736526 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1496ff7f-366b-4cd2-8396-2cdbfed841fa-original-pull-secret podName:1496ff7f-366b-4cd2-8396-2cdbfed841fa nodeName:}" failed. No retries permitted until 2026-07-07 23:13:12.736506407 +0000 UTC m=+10.133507923 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/1496ff7f-366b-4cd2-8396-2cdbfed841fa-original-pull-secret") pod "global-pull-secret-syncer-84hdc" (UID: "1496ff7f-366b-4cd2-8396-2cdbfed841fa") : object "kube-system"/"original-pull-secret" not registered Jul 07 23:13:08.737063 ip-10-0-134-140 kubenswrapper[2570]: E0707 23:13:08.736948 2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 07 23:13:08.737063 ip-10-0-134-140 kubenswrapper[2570]: E0707 23:13:08.736969 2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 07 23:13:08.737063 ip-10-0-134-140 kubenswrapper[2570]: E0707 23:13:08.736982 2570 projected.go:196] Error preparing data for projected volume kube-api-access-bkbfq for pod openshift-network-diagnostics/network-check-target-p27rw: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 23:13:08.737063 ip-10-0-134-140 kubenswrapper[2570]: E0707 23:13:08.737030 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f47fd470-2c7a-4ea8-bf73-e3f94320f117-kube-api-access-bkbfq podName:f47fd470-2c7a-4ea8-bf73-e3f94320f117 nodeName:}" failed. No retries permitted until 2026-07-07 23:13:12.737013827 +0000 UTC m=+10.134015323 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-bkbfq" (UniqueName: "kubernetes.io/projected/f47fd470-2c7a-4ea8-bf73-e3f94320f117-kube-api-access-bkbfq") pod "network-check-target-p27rw" (UID: "f47fd470-2c7a-4ea8-bf73-e3f94320f117") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 23:13:09.093370 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:09.093290 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-p27rw" Jul 07 23:13:09.093545 ip-10-0-134-140 kubenswrapper[2570]: E0707 23:13:09.093422 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-p27rw" podUID="f47fd470-2c7a-4ea8-bf73-e3f94320f117" Jul 07 23:13:09.093652 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:09.093602 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-84hdc" Jul 07 23:13:09.093652 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:09.093623 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ndz2q" Jul 07 23:13:09.093759 ip-10-0-134-140 kubenswrapper[2570]: E0707 23:13:09.093696 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-84hdc" podUID="1496ff7f-366b-4cd2-8396-2cdbfed841fa" Jul 07 23:13:09.093851 ip-10-0-134-140 kubenswrapper[2570]: E0707 23:13:09.093805 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ndz2q" podUID="14406dc0-620c-4615-b98a-b7205dc72966" Jul 07 23:13:11.092257 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:11.091559 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-84hdc" Jul 07 23:13:11.092257 ip-10-0-134-140 kubenswrapper[2570]: E0707 23:13:11.091690 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-84hdc" podUID="1496ff7f-366b-4cd2-8396-2cdbfed841fa" Jul 07 23:13:11.092257 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:11.092103 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ndz2q" Jul 07 23:13:11.092257 ip-10-0-134-140 kubenswrapper[2570]: E0707 23:13:11.092207 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ndz2q" podUID="14406dc0-620c-4615-b98a-b7205dc72966" Jul 07 23:13:11.093205 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:11.093071 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-p27rw" Jul 07 23:13:11.093205 ip-10-0-134-140 kubenswrapper[2570]: E0707 23:13:11.093168 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-p27rw" podUID="f47fd470-2c7a-4ea8-bf73-e3f94320f117" Jul 07 23:13:11.168453 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:11.167677 2570 generic.go:356] "Generic (PLEG): container finished" podID="3bc0442220b7598e60bd507f09022656" containerID="700411bcc2e2e6772f36e143dbbe8ee9afa218b891ef895fa65b10dc831a488a" exitCode=0 Jul 07 23:13:11.168453 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:11.167727 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-140.ec2.internal" event={"ID":"3bc0442220b7598e60bd507f09022656","Type":"ContainerDied","Data":"700411bcc2e2e6772f36e143dbbe8ee9afa218b891ef895fa65b10dc831a488a"} Jul 07 23:13:12.667732 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:12.667696 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/14406dc0-620c-4615-b98a-b7205dc72966-metrics-certs\") pod \"network-metrics-daemon-ndz2q\" (UID: \"14406dc0-620c-4615-b98a-b7205dc72966\") " pod="openshift-multus/network-metrics-daemon-ndz2q" Jul 07 23:13:12.668278 ip-10-0-134-140 kubenswrapper[2570]: E0707 23:13:12.667858 2570 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 23:13:12.668278 ip-10-0-134-140 kubenswrapper[2570]: E0707 23:13:12.667924 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/14406dc0-620c-4615-b98a-b7205dc72966-metrics-certs podName:14406dc0-620c-4615-b98a-b7205dc72966 nodeName:}" failed. No retries permitted until 2026-07-07 23:13:20.667905948 +0000 UTC m=+18.064907453 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/14406dc0-620c-4615-b98a-b7205dc72966-metrics-certs") pod "network-metrics-daemon-ndz2q" (UID: "14406dc0-620c-4615-b98a-b7205dc72966") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 23:13:12.769654 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:12.768856 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bkbfq\" (UniqueName: \"kubernetes.io/projected/f47fd470-2c7a-4ea8-bf73-e3f94320f117-kube-api-access-bkbfq\") pod \"network-check-target-p27rw\" (UID: \"f47fd470-2c7a-4ea8-bf73-e3f94320f117\") " pod="openshift-network-diagnostics/network-check-target-p27rw" Jul 07 23:13:12.769654 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:12.768919 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1496ff7f-366b-4cd2-8396-2cdbfed841fa-original-pull-secret\") pod \"global-pull-secret-syncer-84hdc\" (UID: \"1496ff7f-366b-4cd2-8396-2cdbfed841fa\") " pod="kube-system/global-pull-secret-syncer-84hdc" Jul 07 23:13:12.769654 ip-10-0-134-140 kubenswrapper[2570]: E0707 23:13:12.769062 2570 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 07 23:13:12.769654 ip-10-0-134-140 kubenswrapper[2570]: E0707 23:13:12.769120 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1496ff7f-366b-4cd2-8396-2cdbfed841fa-original-pull-secret podName:1496ff7f-366b-4cd2-8396-2cdbfed841fa nodeName:}" failed. No retries permitted until 2026-07-07 23:13:20.76910155 +0000 UTC m=+18.166103043 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/1496ff7f-366b-4cd2-8396-2cdbfed841fa-original-pull-secret") pod "global-pull-secret-syncer-84hdc" (UID: "1496ff7f-366b-4cd2-8396-2cdbfed841fa") : object "kube-system"/"original-pull-secret" not registered Jul 07 23:13:12.769654 ip-10-0-134-140 kubenswrapper[2570]: E0707 23:13:12.769542 2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 07 23:13:12.769654 ip-10-0-134-140 kubenswrapper[2570]: E0707 23:13:12.769563 2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 07 23:13:12.769654 ip-10-0-134-140 kubenswrapper[2570]: E0707 23:13:12.769575 2570 projected.go:196] Error preparing data for projected volume kube-api-access-bkbfq for pod openshift-network-diagnostics/network-check-target-p27rw: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 23:13:12.769654 ip-10-0-134-140 kubenswrapper[2570]: E0707 23:13:12.769623 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f47fd470-2c7a-4ea8-bf73-e3f94320f117-kube-api-access-bkbfq podName:f47fd470-2c7a-4ea8-bf73-e3f94320f117 nodeName:}" failed. No retries permitted until 2026-07-07 23:13:20.769607297 +0000 UTC m=+18.166608790 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-bkbfq" (UniqueName: "kubernetes.io/projected/f47fd470-2c7a-4ea8-bf73-e3f94320f117-kube-api-access-bkbfq") pod "network-check-target-p27rw" (UID: "f47fd470-2c7a-4ea8-bf73-e3f94320f117") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 23:13:13.092982 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:13.092256 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-p27rw" Jul 07 23:13:13.092982 ip-10-0-134-140 kubenswrapper[2570]: E0707 23:13:13.092368 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-p27rw" podUID="f47fd470-2c7a-4ea8-bf73-e3f94320f117" Jul 07 23:13:13.092982 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:13.092789 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-84hdc" Jul 07 23:13:13.092982 ip-10-0-134-140 kubenswrapper[2570]: E0707 23:13:13.092885 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-84hdc" podUID="1496ff7f-366b-4cd2-8396-2cdbfed841fa" Jul 07 23:13:13.093499 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:13.093349 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ndz2q" Jul 07 23:13:13.093499 ip-10-0-134-140 kubenswrapper[2570]: E0707 23:13:13.093456 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ndz2q" podUID="14406dc0-620c-4615-b98a-b7205dc72966" Jul 07 23:13:15.091556 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:15.091525 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ndz2q" Jul 07 23:13:15.091556 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:15.091558 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-p27rw" Jul 07 23:13:15.091994 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:15.091523 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-84hdc" Jul 07 23:13:15.091994 ip-10-0-134-140 kubenswrapper[2570]: E0707 23:13:15.091669 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ndz2q" podUID="14406dc0-620c-4615-b98a-b7205dc72966" Jul 07 23:13:15.091994 ip-10-0-134-140 kubenswrapper[2570]: E0707 23:13:15.091789 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-84hdc" podUID="1496ff7f-366b-4cd2-8396-2cdbfed841fa" Jul 07 23:13:15.091994 ip-10-0-134-140 kubenswrapper[2570]: E0707 23:13:15.091866 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-p27rw" podUID="f47fd470-2c7a-4ea8-bf73-e3f94320f117" Jul 07 23:13:17.091297 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:17.091257 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-84hdc" Jul 07 23:13:17.091797 ip-10-0-134-140 kubenswrapper[2570]: E0707 23:13:17.091392 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-84hdc" podUID="1496ff7f-366b-4cd2-8396-2cdbfed841fa" Jul 07 23:13:17.091797 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:17.091271 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-p27rw" Jul 07 23:13:17.091797 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:17.091263 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ndz2q" Jul 07 23:13:17.091797 ip-10-0-134-140 kubenswrapper[2570]: E0707 23:13:17.091496 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-p27rw" podUID="f47fd470-2c7a-4ea8-bf73-e3f94320f117" Jul 07 23:13:17.091797 ip-10-0-134-140 kubenswrapper[2570]: E0707 23:13:17.091624 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ndz2q" podUID="14406dc0-620c-4615-b98a-b7205dc72966" Jul 07 23:13:19.090955 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:19.090920 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ndz2q" Jul 07 23:13:19.090955 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:19.090938 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-84hdc" Jul 07 23:13:19.091455 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:19.090932 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-p27rw" Jul 07 23:13:19.091455 ip-10-0-134-140 kubenswrapper[2570]: E0707 23:13:19.091056 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ndz2q" podUID="14406dc0-620c-4615-b98a-b7205dc72966" Jul 07 23:13:19.091455 ip-10-0-134-140 kubenswrapper[2570]: E0707 23:13:19.091113 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-p27rw" podUID="f47fd470-2c7a-4ea8-bf73-e3f94320f117" Jul 07 23:13:19.091455 ip-10-0-134-140 kubenswrapper[2570]: E0707 23:13:19.091178 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-84hdc" podUID="1496ff7f-366b-4cd2-8396-2cdbfed841fa" Jul 07 23:13:20.730714 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:20.730680 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/14406dc0-620c-4615-b98a-b7205dc72966-metrics-certs\") pod \"network-metrics-daemon-ndz2q\" (UID: \"14406dc0-620c-4615-b98a-b7205dc72966\") " pod="openshift-multus/network-metrics-daemon-ndz2q" Jul 07 23:13:20.731262 ip-10-0-134-140 kubenswrapper[2570]: E0707 23:13:20.731227 2570 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 23:13:20.731337 ip-10-0-134-140 kubenswrapper[2570]: E0707 23:13:20.731326 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/14406dc0-620c-4615-b98a-b7205dc72966-metrics-certs podName:14406dc0-620c-4615-b98a-b7205dc72966 nodeName:}" failed. No retries permitted until 2026-07-07 23:13:36.731304094 +0000 UTC m=+34.128305601 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/14406dc0-620c-4615-b98a-b7205dc72966-metrics-certs") pod "network-metrics-daemon-ndz2q" (UID: "14406dc0-620c-4615-b98a-b7205dc72966") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 23:13:20.831912 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:20.831875 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1496ff7f-366b-4cd2-8396-2cdbfed841fa-original-pull-secret\") pod \"global-pull-secret-syncer-84hdc\" (UID: \"1496ff7f-366b-4cd2-8396-2cdbfed841fa\") " pod="kube-system/global-pull-secret-syncer-84hdc" Jul 07 23:13:20.832069 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:20.831953 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bkbfq\" (UniqueName: \"kubernetes.io/projected/f47fd470-2c7a-4ea8-bf73-e3f94320f117-kube-api-access-bkbfq\") pod \"network-check-target-p27rw\" (UID: \"f47fd470-2c7a-4ea8-bf73-e3f94320f117\") " pod="openshift-network-diagnostics/network-check-target-p27rw" Jul 07 23:13:20.832069 ip-10-0-134-140 kubenswrapper[2570]: E0707 23:13:20.832041 2570 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 07 23:13:20.832069 ip-10-0-134-140 kubenswrapper[2570]: E0707 23:13:20.832062 2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 07 23:13:20.832211 ip-10-0-134-140 kubenswrapper[2570]: E0707 23:13:20.832076 2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 07 23:13:20.832211 ip-10-0-134-140 kubenswrapper[2570]: E0707 23:13:20.832085 2570 projected.go:196] Error preparing data for projected volume kube-api-access-bkbfq for pod openshift-network-diagnostics/network-check-target-p27rw: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 23:13:20.832211 ip-10-0-134-140 kubenswrapper[2570]: E0707 23:13:20.832111 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1496ff7f-366b-4cd2-8396-2cdbfed841fa-original-pull-secret podName:1496ff7f-366b-4cd2-8396-2cdbfed841fa nodeName:}" failed. No retries permitted until 2026-07-07 23:13:36.83208892 +0000 UTC m=+34.229090410 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/1496ff7f-366b-4cd2-8396-2cdbfed841fa-original-pull-secret") pod "global-pull-secret-syncer-84hdc" (UID: "1496ff7f-366b-4cd2-8396-2cdbfed841fa") : object "kube-system"/"original-pull-secret" not registered Jul 07 23:13:20.832211 ip-10-0-134-140 kubenswrapper[2570]: E0707 23:13:20.832143 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f47fd470-2c7a-4ea8-bf73-e3f94320f117-kube-api-access-bkbfq podName:f47fd470-2c7a-4ea8-bf73-e3f94320f117 nodeName:}" failed. No retries permitted until 2026-07-07 23:13:36.832132297 +0000 UTC m=+34.229133790 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-bkbfq" (UniqueName: "kubernetes.io/projected/f47fd470-2c7a-4ea8-bf73-e3f94320f117-kube-api-access-bkbfq") pod "network-check-target-p27rw" (UID: "f47fd470-2c7a-4ea8-bf73-e3f94320f117") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 23:13:21.091028 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:21.090909 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-p27rw" Jul 07 23:13:21.091028 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:21.090916 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-84hdc" Jul 07 23:13:21.091256 ip-10-0-134-140 kubenswrapper[2570]: E0707 23:13:21.091033 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-p27rw" podUID="f47fd470-2c7a-4ea8-bf73-e3f94320f117" Jul 07 23:13:21.091256 ip-10-0-134-140 kubenswrapper[2570]: E0707 23:13:21.091078 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-84hdc" podUID="1496ff7f-366b-4cd2-8396-2cdbfed841fa" Jul 07 23:13:21.091256 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:21.090924 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ndz2q" Jul 07 23:13:21.091256 ip-10-0-134-140 kubenswrapper[2570]: E0707 23:13:21.091178 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ndz2q" podUID="14406dc0-620c-4615-b98a-b7205dc72966" Jul 07 23:13:23.092157 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:23.092119 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-p27rw" Jul 07 23:13:23.092650 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:23.092238 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ndz2q" Jul 07 23:13:23.092650 ip-10-0-134-140 kubenswrapper[2570]: E0707 23:13:23.092276 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-p27rw" podUID="f47fd470-2c7a-4ea8-bf73-e3f94320f117" Jul 07 23:13:23.092650 ip-10-0-134-140 kubenswrapper[2570]: E0707 23:13:23.092323 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ndz2q" podUID="14406dc0-620c-4615-b98a-b7205dc72966" Jul 07 23:13:23.092650 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:23.092348 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-84hdc" Jul 07 23:13:23.092650 ip-10-0-134-140 kubenswrapper[2570]: E0707 23:13:23.092424 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-84hdc" podUID="1496ff7f-366b-4cd2-8396-2cdbfed841fa" Jul 07 23:13:25.091104 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:25.090657 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-84hdc" Jul 07 23:13:25.091856 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:25.090657 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ndz2q" Jul 07 23:13:25.091856 ip-10-0-134-140 kubenswrapper[2570]: E0707 23:13:25.091165 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-84hdc" podUID="1496ff7f-366b-4cd2-8396-2cdbfed841fa" Jul 07 23:13:25.091856 ip-10-0-134-140 kubenswrapper[2570]: E0707 23:13:25.091262 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ndz2q" podUID="14406dc0-620c-4615-b98a-b7205dc72966" Jul 07 23:13:25.091856 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:25.090797 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-p27rw" Jul 07 23:13:25.091856 ip-10-0-134-140 kubenswrapper[2570]: E0707 23:13:25.091369 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-p27rw" podUID="f47fd470-2c7a-4ea8-bf73-e3f94320f117" Jul 07 23:13:25.197662 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:25.197625 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-4xzds" event={"ID":"48b46894-bcf9-4176-82a7-bab0ee169c67","Type":"ContainerStarted","Data":"d658cb42b3f56479e04654682e8014305090ab9ab55bb4e7a947a36704fe25ef"} Jul 07 23:13:25.199151 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:25.199124 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ltc6z" event={"ID":"dd19d3d5-47a8-4052-97ea-f4e5fbbb88cc","Type":"ContainerStarted","Data":"b17d934251fedd14946f071575c9ed2737324c9bb1a491d6da2d01876dbd306d"} Jul 07 23:13:25.200425 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:25.200393 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-qgcpr" event={"ID":"c02c0747-b1ae-452c-81aa-5101d7125fc9","Type":"ContainerStarted","Data":"f65e1f1a7d28d2f3cf621d64e015aa3d29da4c1d6897f5636e6fef79d546e6be"} Jul 07 23:13:25.201826 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:25.201797 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-cm9wz" event={"ID":"f2ec914a-0546-4c91-ac24-1fa24cde33f6","Type":"ContainerStarted","Data":"f779e5c267ba917819730c22c96da0ba6be74fed1da3dc7f10ef9ebb6995ec0b"} Jul 07 23:13:25.203196 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:25.203172 2570 generic.go:356] "Generic (PLEG): container finished" podID="573e90ac-1d7d-4584-b231-6635ac4e12e2" containerID="4e4a270335864ca3090ca29917b5562ffad750aaa18ae44af76bdec408f13e75" exitCode=0 Jul 07 23:13:25.203313 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:25.203254 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vztph" event={"ID":"573e90ac-1d7d-4584-b231-6635ac4e12e2","Type":"ContainerDied","Data":"4e4a270335864ca3090ca29917b5562ffad750aaa18ae44af76bdec408f13e75"} Jul 07 23:13:25.208373 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:25.208337 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-wjn84" event={"ID":"0bb01852-570a-4456-9aca-52c99f9f83e5","Type":"ContainerStarted","Data":"e7469aae092bd1128b4cd9ec40a2d1711e19e13d49e41d25a220da3c2cc40045"} Jul 07 23:13:25.210315 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:25.210289 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-140.ec2.internal" event={"ID":"3bc0442220b7598e60bd507f09022656","Type":"ContainerStarted","Data":"ea2c6f7085aba8ce3f403b0ac595b44e2fefabbe1b89abdcf2cf48af31c90c34"} Jul 07 23:13:25.213940 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:25.213921 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-w9gmc" event={"ID":"a8270634-fda9-4520-85cb-0ab49e4ce06a","Type":"ContainerStarted","Data":"2b9493032746f1cc2eb51561dc2e469eb33df3740aa9fb423dd040a1dbd38e20"} Jul 07 23:13:25.214039 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:25.213989 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-w9gmc" event={"ID":"a8270634-fda9-4520-85cb-0ab49e4ce06a","Type":"ContainerStarted","Data":"7f22df0420825a32153c28fa687a6599d8443a30e3003abca19876bb398569c6"} Jul 07 23:13:25.214106 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:25.214041 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-w9gmc" event={"ID":"a8270634-fda9-4520-85cb-0ab49e4ce06a","Type":"ContainerStarted","Data":"d4687aa9397037c2164598d3df493379037c801adb7d55102c1793d91dfbfe55"} Jul 07 23:13:25.214106 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:25.214088 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-w9gmc" event={"ID":"a8270634-fda9-4520-85cb-0ab49e4ce06a","Type":"ContainerStarted","Data":"ac4f43ca89566a69f7065894e6e511f9e0b021163fb5ef094de0ed3d4fc079e7"} Jul 07 23:13:25.214209 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:25.214136 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-w9gmc" event={"ID":"a8270634-fda9-4520-85cb-0ab49e4ce06a","Type":"ContainerStarted","Data":"b9fecfb021aca28663cdf22312afb523ada0aacaf18378ea7f8c62edd171d0cd"} Jul 07 23:13:25.214209 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:25.214185 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-w9gmc" event={"ID":"a8270634-fda9-4520-85cb-0ab49e4ce06a","Type":"ContainerStarted","Data":"dc5508f428c3898c17952819f3ae272a0eb732d65a376301d8948d24e7854f5a"} Jul 07 23:13:25.215269 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:25.215252 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-7z6hc" event={"ID":"df04fe1d-eb1a-4711-80ea-72d68116bbca","Type":"ContainerStarted","Data":"9c185ac00aae6ea2a2825c6e96efb4cbd20f6d895cae63c74f48d6ecb7804009"} Jul 07 23:13:25.220944 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:25.220909 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-4xzds" podStartSLOduration=3.141251445 podStartE2EDuration="22.220898313s" totalImagesPullingTime="19.079646868s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 23:13:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 23:13:25.220742527 +0000 UTC m=+22.617744059" watchObservedRunningTime="2026-07-07 23:13:25.220898313 +0000 UTC m=+22.617899825" Jul 07 23:13:25.274582 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:25.274515 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-qgcpr" podStartSLOduration=3.220381569 podStartE2EDuration="22.274493712s" totalImagesPullingTime="19.054112143s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 23:13:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 23:13:25.274100474 +0000 UTC m=+22.671101986" watchObservedRunningTime="2026-07-07 23:13:25.274493712 +0000 UTC m=+22.671495221" Jul 07 23:13:25.339730 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:25.339682 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-140.ec2.internal" podStartSLOduration=21.339663369 podStartE2EDuration="21.339663369s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 23:13:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 23:13:25.3090923 +0000 UTC m=+22.706093841" watchObservedRunningTime="2026-07-07 23:13:25.339663369 +0000 UTC m=+22.736664859" Jul 07 23:13:25.339984 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:25.339960 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-wjn84" podStartSLOduration=11.365615875 podStartE2EDuration="22.339950383s" totalImagesPullingTime="10.974334508s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 23:13:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 23:13:25.339734007 +0000 UTC m=+22.736735518" watchObservedRunningTime="2026-07-07 23:13:25.339950383 +0000 UTC m=+22.736951899" Jul 07 23:13:25.384060 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:25.383982 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-cm9wz" podStartSLOduration=3.318924457 podStartE2EDuration="22.383968101s" totalImagesPullingTime="19.065043644s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 23:13:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 23:13:25.364974886 +0000 UTC m=+22.761976398" watchObservedRunningTime="2026-07-07 23:13:25.383968101 +0000 UTC m=+22.780969612" Jul 07 23:13:25.384300 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:25.384278 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-7z6hc" podStartSLOduration=3.342370008 podStartE2EDuration="22.384273828s" totalImagesPullingTime="19.04190382s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 23:13:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 23:13:25.384095022 +0000 UTC m=+22.781096526" watchObservedRunningTime="2026-07-07 23:13:25.384273828 +0000 UTC m=+22.781275340" Jul 07 23:13:25.437477 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:25.437442 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-wjn84" Jul 07 23:13:25.438028 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:25.438011 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-wjn84" Jul 07 23:13:25.836248 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:25.836225 2570 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 07 23:13:26.208112 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:26.207997 2570 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-07T23:13:25.836244006Z","UUID":"80881ca6-8544-4869-82b1-ef8a000fa005","Handler":null,"Name":"","Endpoint":""} Jul 07 23:13:26.209662 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:26.209638 2570 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 07 23:13:26.209804 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:26.209679 2570 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 07 23:13:26.219612 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:26.219581 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-6mdgj" event={"ID":"fc88a6f6-fb63-4fcd-b58f-980e71c1b5d0","Type":"ContainerStarted","Data":"40d23b4e000e2384deb1c3a590376e92d2440be5b8518e5aabd212ca8c3a3f86"} Jul 07 23:13:26.221835 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:26.221810 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ltc6z" event={"ID":"dd19d3d5-47a8-4052-97ea-f4e5fbbb88cc","Type":"ContainerStarted","Data":"5b83539a49bf5e809e5fd95a1a2dfaa8da5000c19b0d4d00201e63f2bd8f464c"} Jul 07 23:13:26.222672 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:26.222630 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-wjn84" Jul 07 23:13:26.223069 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:26.223051 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-wjn84" Jul 07 23:13:26.238503 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:26.238444 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-6mdgj" podStartSLOduration=4.167632367 podStartE2EDuration="23.238431269s" totalImagesPullingTime="19.070798902s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 23:13:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 23:13:26.237517653 +0000 UTC m=+23.634519194" watchObservedRunningTime="2026-07-07 23:13:26.238431269 +0000 UTC m=+23.635432780" Jul 07 23:13:27.094780 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:27.094741 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ndz2q" Jul 07 23:13:27.095002 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:27.094741 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-84hdc" Jul 07 23:13:27.095002 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:27.094879 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-p27rw" Jul 07 23:13:27.095002 ip-10-0-134-140 kubenswrapper[2570]: E0707 23:13:27.094878 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ndz2q" podUID="14406dc0-620c-4615-b98a-b7205dc72966" Jul 07 23:13:27.095002 ip-10-0-134-140 kubenswrapper[2570]: E0707 23:13:27.094956 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-84hdc" podUID="1496ff7f-366b-4cd2-8396-2cdbfed841fa" Jul 07 23:13:27.095238 ip-10-0-134-140 kubenswrapper[2570]: E0707 23:13:27.095008 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-p27rw" podUID="f47fd470-2c7a-4ea8-bf73-e3f94320f117" Jul 07 23:13:27.226542 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:27.226507 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ltc6z" event={"ID":"dd19d3d5-47a8-4052-97ea-f4e5fbbb88cc","Type":"ContainerStarted","Data":"4c1978118c71e8f42aa28d72eaefba052f9c10b8345e629515acf50637a53233"} Jul 07 23:13:28.232583 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:28.232376 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-w9gmc" event={"ID":"a8270634-fda9-4520-85cb-0ab49e4ce06a","Type":"ContainerStarted","Data":"18cb60bf85e8e18dbc5e5bc2b35640aba858d8fcde269f10601f94a38af42a01"} Jul 07 23:13:29.094275 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:29.094246 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-84hdc" Jul 07 23:13:29.094275 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:29.094257 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-p27rw" Jul 07 23:13:29.094523 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:29.094257 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ndz2q" Jul 07 23:13:29.094523 ip-10-0-134-140 kubenswrapper[2570]: E0707 23:13:29.094355 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-84hdc" podUID="1496ff7f-366b-4cd2-8396-2cdbfed841fa" Jul 07 23:13:29.094523 ip-10-0-134-140 kubenswrapper[2570]: E0707 23:13:29.094437 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-p27rw" podUID="f47fd470-2c7a-4ea8-bf73-e3f94320f117" Jul 07 23:13:29.094690 ip-10-0-134-140 kubenswrapper[2570]: E0707 23:13:29.094533 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ndz2q" podUID="14406dc0-620c-4615-b98a-b7205dc72966" Jul 07 23:13:30.242875 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:30.242629 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-w9gmc" event={"ID":"a8270634-fda9-4520-85cb-0ab49e4ce06a","Type":"ContainerStarted","Data":"05c3e18ce87c65b7e58be6c45e56aea9cce946206f8681865c2dfe3a430c71a2"} Jul 07 23:13:30.243777 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:30.242962 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-w9gmc" Jul 07 23:13:30.243777 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:30.243032 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-w9gmc" Jul 07 23:13:30.244459 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:30.244434 2570 generic.go:356] "Generic (PLEG): container finished" podID="573e90ac-1d7d-4584-b231-6635ac4e12e2" containerID="d0444f0d57730b213a339925f9423fddf1e24c5f3a65f1839c11b83f6b33615b" exitCode=0 Jul 07 23:13:30.244559 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:30.244488 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vztph" event={"ID":"573e90ac-1d7d-4584-b231-6635ac4e12e2","Type":"ContainerDied","Data":"d0444f0d57730b213a339925f9423fddf1e24c5f3a65f1839c11b83f6b33615b"} Jul 07 23:13:30.258268 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:30.258241 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-w9gmc" Jul 07 23:13:30.277856 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:30.277821 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-w9gmc" podStartSLOduration=8.185551539 podStartE2EDuration="27.277809996s" totalImagesPullingTime="19.092258457s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 23:13:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 23:13:30.276358653 +0000 UTC m=+27.673360164" watchObservedRunningTime="2026-07-07 23:13:30.277809996 +0000 UTC m=+27.674811507" Jul 07 23:13:30.278063 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:30.278038 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ltc6z" podStartSLOduration=14.249960205 podStartE2EDuration="27.278031634s" totalImagesPullingTime="13.028071429s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 23:13:03 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 23:13:27.255544158 +0000 UTC m=+24.652545689" watchObservedRunningTime="2026-07-07 23:13:30.278031634 +0000 UTC m=+27.675033155" Jul 07 23:13:31.093596 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:31.093564 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-p27rw" Jul 07 23:13:31.093891 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:31.093564 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-84hdc" Jul 07 23:13:31.093891 ip-10-0-134-140 kubenswrapper[2570]: E0707 23:13:31.093678 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-p27rw" podUID="f47fd470-2c7a-4ea8-bf73-e3f94320f117" Jul 07 23:13:31.093891 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:31.093564 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ndz2q" Jul 07 23:13:31.093891 ip-10-0-134-140 kubenswrapper[2570]: E0707 23:13:31.093729 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-84hdc" podUID="1496ff7f-366b-4cd2-8396-2cdbfed841fa" Jul 07 23:13:31.093891 ip-10-0-134-140 kubenswrapper[2570]: E0707 23:13:31.093806 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ndz2q" podUID="14406dc0-620c-4615-b98a-b7205dc72966" Jul 07 23:13:31.248188 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:31.248159 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-w9gmc" Jul 07 23:13:31.264620 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:31.264599 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-w9gmc" Jul 07 23:13:31.781523 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:31.781494 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-ndz2q"] Jul 07 23:13:31.781675 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:31.781642 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ndz2q" Jul 07 23:13:31.781769 ip-10-0-134-140 kubenswrapper[2570]: E0707 23:13:31.781749 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ndz2q" podUID="14406dc0-620c-4615-b98a-b7205dc72966" Jul 07 23:13:31.786290 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:31.786265 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-84hdc"] Jul 07 23:13:31.786420 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:31.786361 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-84hdc" Jul 07 23:13:31.786460 ip-10-0-134-140 kubenswrapper[2570]: E0707 23:13:31.786446 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-84hdc" podUID="1496ff7f-366b-4cd2-8396-2cdbfed841fa" Jul 07 23:13:31.791066 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:31.791043 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-p27rw"] Jul 07 23:13:31.791186 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:31.791143 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-p27rw" Jul 07 23:13:31.791242 ip-10-0-134-140 kubenswrapper[2570]: E0707 23:13:31.791222 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-p27rw" podUID="f47fd470-2c7a-4ea8-bf73-e3f94320f117" Jul 07 23:13:32.251653 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:32.251618 2570 generic.go:356] "Generic (PLEG): container finished" podID="573e90ac-1d7d-4584-b231-6635ac4e12e2" containerID="c62561b322da8a481d53ad6c646c0213b5457116c980a6471b8db54e2f3b2ad9" exitCode=0 Jul 07 23:13:32.252025 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:32.251705 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vztph" event={"ID":"573e90ac-1d7d-4584-b231-6635ac4e12e2","Type":"ContainerDied","Data":"c62561b322da8a481d53ad6c646c0213b5457116c980a6471b8db54e2f3b2ad9"} Jul 07 23:13:34.091373 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:34.091288 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-84hdc" Jul 07 23:13:34.091810 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:34.091288 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ndz2q" Jul 07 23:13:34.091810 ip-10-0-134-140 kubenswrapper[2570]: E0707 23:13:34.091408 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-84hdc" podUID="1496ff7f-366b-4cd2-8396-2cdbfed841fa" Jul 07 23:13:34.091810 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:34.091289 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-p27rw" Jul 07 23:13:34.091810 ip-10-0-134-140 kubenswrapper[2570]: E0707 23:13:34.091493 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ndz2q" podUID="14406dc0-620c-4615-b98a-b7205dc72966" Jul 07 23:13:34.091810 ip-10-0-134-140 kubenswrapper[2570]: E0707 23:13:34.091549 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-p27rw" podUID="f47fd470-2c7a-4ea8-bf73-e3f94320f117" Jul 07 23:13:34.258006 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:34.257977 2570 generic.go:356] "Generic (PLEG): container finished" podID="573e90ac-1d7d-4584-b231-6635ac4e12e2" containerID="dc2d25b04adc6cf5e8097242d2e7d9f911f72dae9d1b7db6f39b068fa2caf2ca" exitCode=0 Jul 07 23:13:34.258155 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:34.258030 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vztph" event={"ID":"573e90ac-1d7d-4584-b231-6635ac4e12e2","Type":"ContainerDied","Data":"dc2d25b04adc6cf5e8097242d2e7d9f911f72dae9d1b7db6f39b068fa2caf2ca"} Jul 07 23:13:36.090979 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:36.090796 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-p27rw" Jul 07 23:13:36.091420 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:36.090796 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-84hdc" Jul 07 23:13:36.091420 ip-10-0-134-140 kubenswrapper[2570]: E0707 23:13:36.091071 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-p27rw" podUID="f47fd470-2c7a-4ea8-bf73-e3f94320f117" Jul 07 23:13:36.091420 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:36.090796 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ndz2q" Jul 07 23:13:36.091420 ip-10-0-134-140 kubenswrapper[2570]: E0707 23:13:36.091160 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-84hdc" podUID="1496ff7f-366b-4cd2-8396-2cdbfed841fa" Jul 07 23:13:36.091420 ip-10-0-134-140 kubenswrapper[2570]: E0707 23:13:36.091229 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ndz2q" podUID="14406dc0-620c-4615-b98a-b7205dc72966" Jul 07 23:13:36.751337 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:36.751308 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/14406dc0-620c-4615-b98a-b7205dc72966-metrics-certs\") pod \"network-metrics-daemon-ndz2q\" (UID: \"14406dc0-620c-4615-b98a-b7205dc72966\") " pod="openshift-multus/network-metrics-daemon-ndz2q" Jul 07 23:13:36.751509 ip-10-0-134-140 kubenswrapper[2570]: E0707 23:13:36.751418 2570 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 23:13:36.751509 ip-10-0-134-140 kubenswrapper[2570]: E0707 23:13:36.751484 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/14406dc0-620c-4615-b98a-b7205dc72966-metrics-certs podName:14406dc0-620c-4615-b98a-b7205dc72966 nodeName:}" failed. No retries permitted until 2026-07-07 23:14:08.751453291 +0000 UTC m=+66.148454787 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/14406dc0-620c-4615-b98a-b7205dc72966-metrics-certs") pod "network-metrics-daemon-ndz2q" (UID: "14406dc0-620c-4615-b98a-b7205dc72966") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 23:13:36.852003 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:36.851963 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bkbfq\" (UniqueName: \"kubernetes.io/projected/f47fd470-2c7a-4ea8-bf73-e3f94320f117-kube-api-access-bkbfq\") pod \"network-check-target-p27rw\" (UID: \"f47fd470-2c7a-4ea8-bf73-e3f94320f117\") " pod="openshift-network-diagnostics/network-check-target-p27rw" Jul 07 23:13:36.852180 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:36.852032 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1496ff7f-366b-4cd2-8396-2cdbfed841fa-original-pull-secret\") pod \"global-pull-secret-syncer-84hdc\" (UID: \"1496ff7f-366b-4cd2-8396-2cdbfed841fa\") " pod="kube-system/global-pull-secret-syncer-84hdc" Jul 07 23:13:36.852180 ip-10-0-134-140 kubenswrapper[2570]: E0707 23:13:36.852120 2570 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 07 23:13:36.852180 ip-10-0-134-140 kubenswrapper[2570]: E0707 23:13:36.852123 2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 07 23:13:36.852180 ip-10-0-134-140 kubenswrapper[2570]: E0707 23:13:36.852147 2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 07 23:13:36.852180 ip-10-0-134-140 kubenswrapper[2570]: E0707 23:13:36.852160 2570 projected.go:196] Error preparing data for projected volume kube-api-access-bkbfq for pod openshift-network-diagnostics/network-check-target-p27rw: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 23:13:36.852433 ip-10-0-134-140 kubenswrapper[2570]: E0707 23:13:36.852185 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1496ff7f-366b-4cd2-8396-2cdbfed841fa-original-pull-secret podName:1496ff7f-366b-4cd2-8396-2cdbfed841fa nodeName:}" failed. No retries permitted until 2026-07-07 23:14:08.852166472 +0000 UTC m=+66.249167982 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/1496ff7f-366b-4cd2-8396-2cdbfed841fa-original-pull-secret") pod "global-pull-secret-syncer-84hdc" (UID: "1496ff7f-366b-4cd2-8396-2cdbfed841fa") : object "kube-system"/"original-pull-secret" not registered Jul 07 23:13:36.852433 ip-10-0-134-140 kubenswrapper[2570]: E0707 23:13:36.852209 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f47fd470-2c7a-4ea8-bf73-e3f94320f117-kube-api-access-bkbfq podName:f47fd470-2c7a-4ea8-bf73-e3f94320f117 nodeName:}" failed. No retries permitted until 2026-07-07 23:14:08.85219351 +0000 UTC m=+66.249195006 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-bkbfq" (UniqueName: "kubernetes.io/projected/f47fd470-2c7a-4ea8-bf73-e3f94320f117-kube-api-access-bkbfq") pod "network-check-target-p27rw" (UID: "f47fd470-2c7a-4ea8-bf73-e3f94320f117") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 23:13:37.358818 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:37.358780 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-140.ec2.internal" event="NodeReady" Jul 07 23:13:37.359262 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:37.358933 2570 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 07 23:13:37.395099 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:37.395067 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-5668757489-j9qbl"] Jul 07 23:13:37.397802 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:37.397778 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5668757489-j9qbl" Jul 07 23:13:37.399927 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:37.399902 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 07 23:13:37.401102 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:37.400651 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 07 23:13:37.401102 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:37.400894 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 07 23:13:37.401947 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:37.401774 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-wgd2s\"" Jul 07 23:13:37.408325 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:37.408306 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 07 23:13:37.410196 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:37.410177 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-4th55"] Jul 07 23:13:37.412231 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:37.412211 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-4th55" Jul 07 23:13:37.413552 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:37.413363 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-5668757489-j9qbl"] Jul 07 23:13:37.414461 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:37.414440 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 07 23:13:37.414661 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:37.414645 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-94qrf\"" Jul 07 23:13:37.415184 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:37.414987 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 07 23:13:37.417428 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:37.417406 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-gbf74"] Jul 07 23:13:37.419565 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:37.419344 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-gbf74" Jul 07 23:13:37.421805 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:37.421666 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 07 23:13:37.421805 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:37.421721 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 07 23:13:37.421805 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:37.421735 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-mjlml\"" Jul 07 23:13:37.421984 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:37.421966 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 07 23:13:37.423816 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:37.423797 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-4th55"] Jul 07 23:13:37.430313 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:37.430294 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-gbf74"] Jul 07 23:13:37.459052 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:37.459020 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:13:37.556240 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:37.556212 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/143dd7b8-6ed7-489a-893a-e288926cc1f7-cert\") pod \"ingress-canary-gbf74\" (UID: \"143dd7b8-6ed7-489a-893a-e288926cc1f7\") " pod="openshift-ingress-canary/ingress-canary-gbf74" Jul 07 23:13:37.556427 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:37.556246 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/bd5aafbd-ed3c-46fa-bc87-661f74ba728a-config-volume\") pod \"dns-default-4th55\" (UID: \"bd5aafbd-ed3c-46fa-bc87-661f74ba728a\") " pod="openshift-dns/dns-default-4th55" Jul 07 23:13:37.556427 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:37.556271 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qk6v5\" (UniqueName: \"kubernetes.io/projected/bd5aafbd-ed3c-46fa-bc87-661f74ba728a-kube-api-access-qk6v5\") pod \"dns-default-4th55\" (UID: \"bd5aafbd-ed3c-46fa-bc87-661f74ba728a\") " pod="openshift-dns/dns-default-4th55" Jul 07 23:13:37.556427 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:37.556288 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/cf5100ad-da85-4da6-b099-d10259a611d2-registry-tls\") pod \"image-registry-5668757489-j9qbl\" (UID: \"cf5100ad-da85-4da6-b099-d10259a611d2\") " pod="openshift-image-registry/image-registry-5668757489-j9qbl" Jul 07 23:13:37.556427 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:37.556310 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xvdfj\" (UniqueName: \"kubernetes.io/projected/cf5100ad-da85-4da6-b099-d10259a611d2-kube-api-access-xvdfj\") pod \"image-registry-5668757489-j9qbl\" (UID: \"cf5100ad-da85-4da6-b099-d10259a611d2\") " pod="openshift-image-registry/image-registry-5668757489-j9qbl" Jul 07 23:13:37.556427 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:37.556381 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/cf5100ad-da85-4da6-b099-d10259a611d2-registry-certificates\") pod \"image-registry-5668757489-j9qbl\" (UID: \"cf5100ad-da85-4da6-b099-d10259a611d2\") " pod="openshift-image-registry/image-registry-5668757489-j9qbl" Jul 07 23:13:37.556427 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:37.556413 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/cf5100ad-da85-4da6-b099-d10259a611d2-installation-pull-secrets\") pod \"image-registry-5668757489-j9qbl\" (UID: \"cf5100ad-da85-4da6-b099-d10259a611d2\") " pod="openshift-image-registry/image-registry-5668757489-j9qbl" Jul 07 23:13:37.556735 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:37.556450 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bd5aafbd-ed3c-46fa-bc87-661f74ba728a-tmp-dir\") pod \"dns-default-4th55\" (UID: \"bd5aafbd-ed3c-46fa-bc87-661f74ba728a\") " pod="openshift-dns/dns-default-4th55" Jul 07 23:13:37.556735 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:37.556494 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/cf5100ad-da85-4da6-b099-d10259a611d2-image-registry-private-configuration\") pod \"image-registry-5668757489-j9qbl\" (UID: \"cf5100ad-da85-4da6-b099-d10259a611d2\") " pod="openshift-image-registry/image-registry-5668757489-j9qbl" Jul 07 23:13:37.556735 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:37.556547 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/bd5aafbd-ed3c-46fa-bc87-661f74ba728a-metrics-tls\") pod \"dns-default-4th55\" (UID: \"bd5aafbd-ed3c-46fa-bc87-661f74ba728a\") " pod="openshift-dns/dns-default-4th55" Jul 07 23:13:37.556735 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:37.556569 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/cf5100ad-da85-4da6-b099-d10259a611d2-ca-trust-extracted\") pod \"image-registry-5668757489-j9qbl\" (UID: \"cf5100ad-da85-4da6-b099-d10259a611d2\") " pod="openshift-image-registry/image-registry-5668757489-j9qbl" Jul 07 23:13:37.556735 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:37.556592 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/cf5100ad-da85-4da6-b099-d10259a611d2-trusted-ca\") pod \"image-registry-5668757489-j9qbl\" (UID: \"cf5100ad-da85-4da6-b099-d10259a611d2\") " pod="openshift-image-registry/image-registry-5668757489-j9qbl" Jul 07 23:13:37.556735 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:37.556663 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sgwb9\" (UniqueName: \"kubernetes.io/projected/143dd7b8-6ed7-489a-893a-e288926cc1f7-kube-api-access-sgwb9\") pod \"ingress-canary-gbf74\" (UID: \"143dd7b8-6ed7-489a-893a-e288926cc1f7\") " pod="openshift-ingress-canary/ingress-canary-gbf74" Jul 07 23:13:37.556735 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:37.556689 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/cf5100ad-da85-4da6-b099-d10259a611d2-bound-sa-token\") pod \"image-registry-5668757489-j9qbl\" (UID: \"cf5100ad-da85-4da6-b099-d10259a611d2\") " pod="openshift-image-registry/image-registry-5668757489-j9qbl" Jul 07 23:13:37.564991 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:37.564964 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:13:37.657237 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:37.657141 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/143dd7b8-6ed7-489a-893a-e288926cc1f7-cert\") pod \"ingress-canary-gbf74\" (UID: \"143dd7b8-6ed7-489a-893a-e288926cc1f7\") " pod="openshift-ingress-canary/ingress-canary-gbf74" Jul 07 23:13:37.657237 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:37.657191 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/bd5aafbd-ed3c-46fa-bc87-661f74ba728a-config-volume\") pod \"dns-default-4th55\" (UID: \"bd5aafbd-ed3c-46fa-bc87-661f74ba728a\") " pod="openshift-dns/dns-default-4th55" Jul 07 23:13:37.657237 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:37.657225 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qk6v5\" (UniqueName: \"kubernetes.io/projected/bd5aafbd-ed3c-46fa-bc87-661f74ba728a-kube-api-access-qk6v5\") pod \"dns-default-4th55\" (UID: \"bd5aafbd-ed3c-46fa-bc87-661f74ba728a\") " pod="openshift-dns/dns-default-4th55" Jul 07 23:13:37.657509 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:37.657248 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/cf5100ad-da85-4da6-b099-d10259a611d2-registry-tls\") pod \"image-registry-5668757489-j9qbl\" (UID: \"cf5100ad-da85-4da6-b099-d10259a611d2\") " pod="openshift-image-registry/image-registry-5668757489-j9qbl" Jul 07 23:13:37.657509 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:37.657369 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xvdfj\" (UniqueName: \"kubernetes.io/projected/cf5100ad-da85-4da6-b099-d10259a611d2-kube-api-access-xvdfj\") pod \"image-registry-5668757489-j9qbl\" (UID: \"cf5100ad-da85-4da6-b099-d10259a611d2\") " pod="openshift-image-registry/image-registry-5668757489-j9qbl" Jul 07 23:13:37.657509 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:37.657416 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/cf5100ad-da85-4da6-b099-d10259a611d2-registry-certificates\") pod \"image-registry-5668757489-j9qbl\" (UID: \"cf5100ad-da85-4da6-b099-d10259a611d2\") " pod="openshift-image-registry/image-registry-5668757489-j9qbl" Jul 07 23:13:37.657509 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:37.657447 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/cf5100ad-da85-4da6-b099-d10259a611d2-installation-pull-secrets\") pod \"image-registry-5668757489-j9qbl\" (UID: \"cf5100ad-da85-4da6-b099-d10259a611d2\") " pod="openshift-image-registry/image-registry-5668757489-j9qbl" Jul 07 23:13:37.657509 ip-10-0-134-140 kubenswrapper[2570]: E0707 23:13:37.657492 2570 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 07 23:13:37.657509 ip-10-0-134-140 kubenswrapper[2570]: E0707 23:13:37.657513 2570 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5668757489-j9qbl: secret "image-registry-tls" not found Jul 07 23:13:37.657803 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:37.657495 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bd5aafbd-ed3c-46fa-bc87-661f74ba728a-tmp-dir\") pod \"dns-default-4th55\" (UID: \"bd5aafbd-ed3c-46fa-bc87-661f74ba728a\") " pod="openshift-dns/dns-default-4th55" Jul 07 23:13:37.657803 ip-10-0-134-140 kubenswrapper[2570]: E0707 23:13:37.657575 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/cf5100ad-da85-4da6-b099-d10259a611d2-registry-tls podName:cf5100ad-da85-4da6-b099-d10259a611d2 nodeName:}" failed. No retries permitted until 2026-07-07 23:13:38.157554422 +0000 UTC m=+35.554555940 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/cf5100ad-da85-4da6-b099-d10259a611d2-registry-tls") pod "image-registry-5668757489-j9qbl" (UID: "cf5100ad-da85-4da6-b099-d10259a611d2") : secret "image-registry-tls" not found Jul 07 23:13:37.657803 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:37.657607 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/cf5100ad-da85-4da6-b099-d10259a611d2-image-registry-private-configuration\") pod \"image-registry-5668757489-j9qbl\" (UID: \"cf5100ad-da85-4da6-b099-d10259a611d2\") " pod="openshift-image-registry/image-registry-5668757489-j9qbl" Jul 07 23:13:37.657803 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:37.657655 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/bd5aafbd-ed3c-46fa-bc87-661f74ba728a-metrics-tls\") pod \"dns-default-4th55\" (UID: \"bd5aafbd-ed3c-46fa-bc87-661f74ba728a\") " pod="openshift-dns/dns-default-4th55" Jul 07 23:13:37.657803 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:37.657680 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/cf5100ad-da85-4da6-b099-d10259a611d2-ca-trust-extracted\") pod \"image-registry-5668757489-j9qbl\" (UID: \"cf5100ad-da85-4da6-b099-d10259a611d2\") " pod="openshift-image-registry/image-registry-5668757489-j9qbl" Jul 07 23:13:37.657803 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:37.657708 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/cf5100ad-da85-4da6-b099-d10259a611d2-trusted-ca\") pod \"image-registry-5668757489-j9qbl\" (UID: \"cf5100ad-da85-4da6-b099-d10259a611d2\") " pod="openshift-image-registry/image-registry-5668757489-j9qbl" Jul 07 23:13:37.657803 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:37.657767 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bd5aafbd-ed3c-46fa-bc87-661f74ba728a-tmp-dir\") pod \"dns-default-4th55\" (UID: \"bd5aafbd-ed3c-46fa-bc87-661f74ba728a\") " pod="openshift-dns/dns-default-4th55" Jul 07 23:13:37.657803 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:37.657782 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sgwb9\" (UniqueName: \"kubernetes.io/projected/143dd7b8-6ed7-489a-893a-e288926cc1f7-kube-api-access-sgwb9\") pod \"ingress-canary-gbf74\" (UID: \"143dd7b8-6ed7-489a-893a-e288926cc1f7\") " pod="openshift-ingress-canary/ingress-canary-gbf74" Jul 07 23:13:37.657803 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:37.657806 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/cf5100ad-da85-4da6-b099-d10259a611d2-bound-sa-token\") pod \"image-registry-5668757489-j9qbl\" (UID: \"cf5100ad-da85-4da6-b099-d10259a611d2\") " pod="openshift-image-registry/image-registry-5668757489-j9qbl" Jul 07 23:13:37.658288 ip-10-0-134-140 kubenswrapper[2570]: E0707 23:13:37.657853 2570 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 07 23:13:37.658288 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:37.657888 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/bd5aafbd-ed3c-46fa-bc87-661f74ba728a-config-volume\") pod \"dns-default-4th55\" (UID: \"bd5aafbd-ed3c-46fa-bc87-661f74ba728a\") " pod="openshift-dns/dns-default-4th55" Jul 07 23:13:37.658288 ip-10-0-134-140 kubenswrapper[2570]: E0707 23:13:37.657948 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bd5aafbd-ed3c-46fa-bc87-661f74ba728a-metrics-tls podName:bd5aafbd-ed3c-46fa-bc87-661f74ba728a nodeName:}" failed. No retries permitted until 2026-07-07 23:13:38.157929742 +0000 UTC m=+35.554931249 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/bd5aafbd-ed3c-46fa-bc87-661f74ba728a-metrics-tls") pod "dns-default-4th55" (UID: "bd5aafbd-ed3c-46fa-bc87-661f74ba728a") : secret "dns-default-metrics-tls" not found Jul 07 23:13:37.658288 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:37.658125 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/cf5100ad-da85-4da6-b099-d10259a611d2-registry-certificates\") pod \"image-registry-5668757489-j9qbl\" (UID: \"cf5100ad-da85-4da6-b099-d10259a611d2\") " pod="openshift-image-registry/image-registry-5668757489-j9qbl" Jul 07 23:13:37.658515 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:37.658423 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/cf5100ad-da85-4da6-b099-d10259a611d2-ca-trust-extracted\") pod \"image-registry-5668757489-j9qbl\" (UID: \"cf5100ad-da85-4da6-b099-d10259a611d2\") " pod="openshift-image-registry/image-registry-5668757489-j9qbl" Jul 07 23:13:37.658819 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:37.658724 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/cf5100ad-da85-4da6-b099-d10259a611d2-trusted-ca\") pod \"image-registry-5668757489-j9qbl\" (UID: \"cf5100ad-da85-4da6-b099-d10259a611d2\") " pod="openshift-image-registry/image-registry-5668757489-j9qbl" Jul 07 23:13:37.661924 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:37.661905 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/cf5100ad-da85-4da6-b099-d10259a611d2-installation-pull-secrets\") pod \"image-registry-5668757489-j9qbl\" (UID: \"cf5100ad-da85-4da6-b099-d10259a611d2\") " pod="openshift-image-registry/image-registry-5668757489-j9qbl" Jul 07 23:13:37.662043 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:37.661934 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/cf5100ad-da85-4da6-b099-d10259a611d2-image-registry-private-configuration\") pod \"image-registry-5668757489-j9qbl\" (UID: \"cf5100ad-da85-4da6-b099-d10259a611d2\") " pod="openshift-image-registry/image-registry-5668757489-j9qbl" Jul 07 23:13:37.662043 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:37.661967 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/143dd7b8-6ed7-489a-893a-e288926cc1f7-cert\") pod \"ingress-canary-gbf74\" (UID: \"143dd7b8-6ed7-489a-893a-e288926cc1f7\") " pod="openshift-ingress-canary/ingress-canary-gbf74" Jul 07 23:13:37.665768 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:37.665747 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qk6v5\" (UniqueName: \"kubernetes.io/projected/bd5aafbd-ed3c-46fa-bc87-661f74ba728a-kube-api-access-qk6v5\") pod \"dns-default-4th55\" (UID: \"bd5aafbd-ed3c-46fa-bc87-661f74ba728a\") " pod="openshift-dns/dns-default-4th55" Jul 07 23:13:37.671323 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:37.671185 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/cf5100ad-da85-4da6-b099-d10259a611d2-bound-sa-token\") pod \"image-registry-5668757489-j9qbl\" (UID: \"cf5100ad-da85-4da6-b099-d10259a611d2\") " pod="openshift-image-registry/image-registry-5668757489-j9qbl" Jul 07 23:13:37.671433 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:37.671347 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sgwb9\" (UniqueName: \"kubernetes.io/projected/143dd7b8-6ed7-489a-893a-e288926cc1f7-kube-api-access-sgwb9\") pod \"ingress-canary-gbf74\" (UID: \"143dd7b8-6ed7-489a-893a-e288926cc1f7\") " pod="openshift-ingress-canary/ingress-canary-gbf74" Jul 07 23:13:37.671745 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:37.671678 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xvdfj\" (UniqueName: \"kubernetes.io/projected/cf5100ad-da85-4da6-b099-d10259a611d2-kube-api-access-xvdfj\") pod \"image-registry-5668757489-j9qbl\" (UID: \"cf5100ad-da85-4da6-b099-d10259a611d2\") " pod="openshift-image-registry/image-registry-5668757489-j9qbl" Jul 07 23:13:37.673065 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:37.673031 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:13:37.732177 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:37.732145 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-gbf74" Jul 07 23:13:38.091416 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:38.091379 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-p27rw" Jul 07 23:13:38.091416 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:38.091403 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-84hdc" Jul 07 23:13:38.091416 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:38.091403 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ndz2q" Jul 07 23:13:38.094767 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:38.094744 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 07 23:13:38.094767 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:38.094762 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 07 23:13:38.094900 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:38.094769 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 07 23:13:38.094900 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:38.094762 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 07 23:13:38.095100 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:38.095085 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-nhfdw\"" Jul 07 23:13:38.095156 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:38.095087 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-hxkkf\"" Jul 07 23:13:38.161081 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:38.161051 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/cf5100ad-da85-4da6-b099-d10259a611d2-registry-tls\") pod \"image-registry-5668757489-j9qbl\" (UID: \"cf5100ad-da85-4da6-b099-d10259a611d2\") " pod="openshift-image-registry/image-registry-5668757489-j9qbl" Jul 07 23:13:38.161242 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:38.161103 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/bd5aafbd-ed3c-46fa-bc87-661f74ba728a-metrics-tls\") pod \"dns-default-4th55\" (UID: \"bd5aafbd-ed3c-46fa-bc87-661f74ba728a\") " pod="openshift-dns/dns-default-4th55" Jul 07 23:13:38.161242 ip-10-0-134-140 kubenswrapper[2570]: E0707 23:13:38.161178 2570 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 07 23:13:38.161242 ip-10-0-134-140 kubenswrapper[2570]: E0707 23:13:38.161198 2570 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5668757489-j9qbl: secret "image-registry-tls" not found Jul 07 23:13:38.161385 ip-10-0-134-140 kubenswrapper[2570]: E0707 23:13:38.161267 2570 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 07 23:13:38.161385 ip-10-0-134-140 kubenswrapper[2570]: E0707 23:13:38.161293 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/cf5100ad-da85-4da6-b099-d10259a611d2-registry-tls podName:cf5100ad-da85-4da6-b099-d10259a611d2 nodeName:}" failed. No retries permitted until 2026-07-07 23:13:39.161269287 +0000 UTC m=+36.558270797 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/cf5100ad-da85-4da6-b099-d10259a611d2-registry-tls") pod "image-registry-5668757489-j9qbl" (UID: "cf5100ad-da85-4da6-b099-d10259a611d2") : secret "image-registry-tls" not found Jul 07 23:13:38.161385 ip-10-0-134-140 kubenswrapper[2570]: E0707 23:13:38.161323 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bd5aafbd-ed3c-46fa-bc87-661f74ba728a-metrics-tls podName:bd5aafbd-ed3c-46fa-bc87-661f74ba728a nodeName:}" failed. No retries permitted until 2026-07-07 23:13:39.16130418 +0000 UTC m=+36.558305671 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/bd5aafbd-ed3c-46fa-bc87-661f74ba728a-metrics-tls") pod "dns-default-4th55" (UID: "bd5aafbd-ed3c-46fa-bc87-661f74ba728a") : secret "dns-default-metrics-tls" not found Jul 07 23:13:38.499066 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:38.498945 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-gbf74"] Jul 07 23:13:38.502124 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:13:38.502102 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod143dd7b8_6ed7_489a_893a_e288926cc1f7.slice/crio-922b58dcb03522c90ac044e763a645ff4f5bb68e74c05b1126f80bf5578b4f4a WatchSource:0}: Error finding container 922b58dcb03522c90ac044e763a645ff4f5bb68e74c05b1126f80bf5578b4f4a: Status 404 returned error can't find the container with id 922b58dcb03522c90ac044e763a645ff4f5bb68e74c05b1126f80bf5578b4f4a Jul 07 23:13:39.169400 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:39.169357 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/bd5aafbd-ed3c-46fa-bc87-661f74ba728a-metrics-tls\") pod \"dns-default-4th55\" (UID: \"bd5aafbd-ed3c-46fa-bc87-661f74ba728a\") " pod="openshift-dns/dns-default-4th55" Jul 07 23:13:39.169598 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:39.169485 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/cf5100ad-da85-4da6-b099-d10259a611d2-registry-tls\") pod \"image-registry-5668757489-j9qbl\" (UID: \"cf5100ad-da85-4da6-b099-d10259a611d2\") " pod="openshift-image-registry/image-registry-5668757489-j9qbl" Jul 07 23:13:39.169598 ip-10-0-134-140 kubenswrapper[2570]: E0707 23:13:39.169520 2570 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 07 23:13:39.169598 ip-10-0-134-140 kubenswrapper[2570]: E0707 23:13:39.169597 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bd5aafbd-ed3c-46fa-bc87-661f74ba728a-metrics-tls podName:bd5aafbd-ed3c-46fa-bc87-661f74ba728a nodeName:}" failed. No retries permitted until 2026-07-07 23:13:41.169574246 +0000 UTC m=+38.566575742 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/bd5aafbd-ed3c-46fa-bc87-661f74ba728a-metrics-tls") pod "dns-default-4th55" (UID: "bd5aafbd-ed3c-46fa-bc87-661f74ba728a") : secret "dns-default-metrics-tls" not found Jul 07 23:13:39.169736 ip-10-0-134-140 kubenswrapper[2570]: E0707 23:13:39.169605 2570 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 07 23:13:39.169736 ip-10-0-134-140 kubenswrapper[2570]: E0707 23:13:39.169619 2570 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5668757489-j9qbl: secret "image-registry-tls" not found Jul 07 23:13:39.169736 ip-10-0-134-140 kubenswrapper[2570]: E0707 23:13:39.169665 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/cf5100ad-da85-4da6-b099-d10259a611d2-registry-tls podName:cf5100ad-da85-4da6-b099-d10259a611d2 nodeName:}" failed. No retries permitted until 2026-07-07 23:13:41.169650068 +0000 UTC m=+38.566651565 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/cf5100ad-da85-4da6-b099-d10259a611d2-registry-tls") pod "image-registry-5668757489-j9qbl" (UID: "cf5100ad-da85-4da6-b099-d10259a611d2") : secret "image-registry-tls" not found Jul 07 23:13:39.185526 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:39.185499 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:13:39.268989 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:39.268901 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:13:39.271377 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:39.271345 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-gbf74" event={"ID":"143dd7b8-6ed7-489a-893a-e288926cc1f7","Type":"ContainerStarted","Data":"922b58dcb03522c90ac044e763a645ff4f5bb68e74c05b1126f80bf5578b4f4a"} Jul 07 23:13:39.274482 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:39.274437 2570 generic.go:356] "Generic (PLEG): container finished" podID="573e90ac-1d7d-4584-b231-6635ac4e12e2" containerID="7d2f987e06371387c50777aa77edab94c95ac3aff3fe81ab15f78a6261056eab" exitCode=0 Jul 07 23:13:39.274575 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:39.274492 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vztph" event={"ID":"573e90ac-1d7d-4584-b231-6635ac4e12e2","Type":"ContainerDied","Data":"7d2f987e06371387c50777aa77edab94c95ac3aff3fe81ab15f78a6261056eab"} Jul 07 23:13:39.420303 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:39.420236 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:13:39.521741 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:39.521705 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:13:39.581036 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:39.580999 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:13:39.850337 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:39.850301 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:13:39.923300 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:39.923259 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:13:40.281399 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:40.281368 2570 generic.go:356] "Generic (PLEG): container finished" podID="573e90ac-1d7d-4584-b231-6635ac4e12e2" containerID="9a98cd5685252e1688f8aa64a350947cd867f93154074c63ae85b10f808c8aa9" exitCode=0 Jul 07 23:13:40.281592 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:40.281410 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vztph" event={"ID":"573e90ac-1d7d-4584-b231-6635ac4e12e2","Type":"ContainerDied","Data":"9a98cd5685252e1688f8aa64a350947cd867f93154074c63ae85b10f808c8aa9"} Jul 07 23:13:41.185706 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:41.185672 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/cf5100ad-da85-4da6-b099-d10259a611d2-registry-tls\") pod \"image-registry-5668757489-j9qbl\" (UID: \"cf5100ad-da85-4da6-b099-d10259a611d2\") " pod="openshift-image-registry/image-registry-5668757489-j9qbl" Jul 07 23:13:41.186097 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:41.185743 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/bd5aafbd-ed3c-46fa-bc87-661f74ba728a-metrics-tls\") pod \"dns-default-4th55\" (UID: \"bd5aafbd-ed3c-46fa-bc87-661f74ba728a\") " pod="openshift-dns/dns-default-4th55" Jul 07 23:13:41.186097 ip-10-0-134-140 kubenswrapper[2570]: E0707 23:13:41.185828 2570 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 07 23:13:41.186097 ip-10-0-134-140 kubenswrapper[2570]: E0707 23:13:41.185844 2570 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5668757489-j9qbl: secret "image-registry-tls" not found Jul 07 23:13:41.186097 ip-10-0-134-140 kubenswrapper[2570]: E0707 23:13:41.185880 2570 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 07 23:13:41.186097 ip-10-0-134-140 kubenswrapper[2570]: E0707 23:13:41.185900 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/cf5100ad-da85-4da6-b099-d10259a611d2-registry-tls podName:cf5100ad-da85-4da6-b099-d10259a611d2 nodeName:}" failed. No retries permitted until 2026-07-07 23:13:45.185884211 +0000 UTC m=+42.582885701 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/cf5100ad-da85-4da6-b099-d10259a611d2-registry-tls") pod "image-registry-5668757489-j9qbl" (UID: "cf5100ad-da85-4da6-b099-d10259a611d2") : secret "image-registry-tls" not found Jul 07 23:13:41.186097 ip-10-0-134-140 kubenswrapper[2570]: E0707 23:13:41.185927 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bd5aafbd-ed3c-46fa-bc87-661f74ba728a-metrics-tls podName:bd5aafbd-ed3c-46fa-bc87-661f74ba728a nodeName:}" failed. No retries permitted until 2026-07-07 23:13:45.185914467 +0000 UTC m=+42.582915957 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/bd5aafbd-ed3c-46fa-bc87-661f74ba728a-metrics-tls") pod "dns-default-4th55" (UID: "bd5aafbd-ed3c-46fa-bc87-661f74ba728a") : secret "dns-default-metrics-tls" not found Jul 07 23:13:41.290496 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:41.290443 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vztph" event={"ID":"573e90ac-1d7d-4584-b231-6635ac4e12e2","Type":"ContainerStarted","Data":"750bd089f046f8c33348b3ca41c1fe91b4f971356e773f3f9919a8b8f2211cdb"} Jul 07 23:13:41.291935 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:41.291910 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-gbf74" event={"ID":"143dd7b8-6ed7-489a-893a-e288926cc1f7","Type":"ContainerStarted","Data":"fc5dd4b9fb299ebed41236083527a98250f0aa1a88180017315f57a15ef305f9"} Jul 07 23:13:41.325960 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:41.325918 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-vztph" podStartSLOduration=8.67985292 podStartE2EDuration="38.325905284s" totalImagesPullingTime="29.646052364s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 23:13:03 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 23:13:41.323968942 +0000 UTC m=+38.720970467" watchObservedRunningTime="2026-07-07 23:13:41.325905284 +0000 UTC m=+38.722906796" Jul 07 23:13:41.556966 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:41.556928 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:13:41.659574 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:41.659550 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:13:41.759561 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:41.759529 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:13:42.750090 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:42.749918 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:13:42.804589 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:42.804559 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:13:44.633088 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:44.633055 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:13:44.734430 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:44.734392 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:13:45.216232 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:45.216200 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/bd5aafbd-ed3c-46fa-bc87-661f74ba728a-metrics-tls\") pod \"dns-default-4th55\" (UID: \"bd5aafbd-ed3c-46fa-bc87-661f74ba728a\") " pod="openshift-dns/dns-default-4th55" Jul 07 23:13:45.216444 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:45.216264 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/cf5100ad-da85-4da6-b099-d10259a611d2-registry-tls\") pod \"image-registry-5668757489-j9qbl\" (UID: \"cf5100ad-da85-4da6-b099-d10259a611d2\") " pod="openshift-image-registry/image-registry-5668757489-j9qbl" Jul 07 23:13:45.216444 ip-10-0-134-140 kubenswrapper[2570]: E0707 23:13:45.216351 2570 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 07 23:13:45.216444 ip-10-0-134-140 kubenswrapper[2570]: E0707 23:13:45.216358 2570 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 07 23:13:45.216444 ip-10-0-134-140 kubenswrapper[2570]: E0707 23:13:45.216422 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bd5aafbd-ed3c-46fa-bc87-661f74ba728a-metrics-tls podName:bd5aafbd-ed3c-46fa-bc87-661f74ba728a nodeName:}" failed. No retries permitted until 2026-07-07 23:13:53.216406022 +0000 UTC m=+50.613407516 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/bd5aafbd-ed3c-46fa-bc87-661f74ba728a-metrics-tls") pod "dns-default-4th55" (UID: "bd5aafbd-ed3c-46fa-bc87-661f74ba728a") : secret "dns-default-metrics-tls" not found Jul 07 23:13:45.216444 ip-10-0-134-140 kubenswrapper[2570]: E0707 23:13:45.216361 2570 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5668757489-j9qbl: secret "image-registry-tls" not found Jul 07 23:13:45.216618 ip-10-0-134-140 kubenswrapper[2570]: E0707 23:13:45.216510 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/cf5100ad-da85-4da6-b099-d10259a611d2-registry-tls podName:cf5100ad-da85-4da6-b099-d10259a611d2 nodeName:}" failed. No retries permitted until 2026-07-07 23:13:53.216497516 +0000 UTC m=+50.613499007 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/cf5100ad-da85-4da6-b099-d10259a611d2-registry-tls") pod "image-registry-5668757489-j9qbl" (UID: "cf5100ad-da85-4da6-b099-d10259a611d2") : secret "image-registry-tls" not found Jul 07 23:13:53.272692 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:53.272658 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/bd5aafbd-ed3c-46fa-bc87-661f74ba728a-metrics-tls\") pod \"dns-default-4th55\" (UID: \"bd5aafbd-ed3c-46fa-bc87-661f74ba728a\") " pod="openshift-dns/dns-default-4th55" Jul 07 23:13:53.273181 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:53.272726 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/cf5100ad-da85-4da6-b099-d10259a611d2-registry-tls\") pod \"image-registry-5668757489-j9qbl\" (UID: \"cf5100ad-da85-4da6-b099-d10259a611d2\") " pod="openshift-image-registry/image-registry-5668757489-j9qbl" Jul 07 23:13:53.273181 ip-10-0-134-140 kubenswrapper[2570]: E0707 23:13:53.272806 2570 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 07 23:13:53.273181 ip-10-0-134-140 kubenswrapper[2570]: E0707 23:13:53.272816 2570 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5668757489-j9qbl: secret "image-registry-tls" not found Jul 07 23:13:53.273181 ip-10-0-134-140 kubenswrapper[2570]: E0707 23:13:53.272816 2570 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 07 23:13:53.273181 ip-10-0-134-140 kubenswrapper[2570]: E0707 23:13:53.272857 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/cf5100ad-da85-4da6-b099-d10259a611d2-registry-tls podName:cf5100ad-da85-4da6-b099-d10259a611d2 nodeName:}" failed. No retries permitted until 2026-07-07 23:14:09.272845982 +0000 UTC m=+66.669847472 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/cf5100ad-da85-4da6-b099-d10259a611d2-registry-tls") pod "image-registry-5668757489-j9qbl" (UID: "cf5100ad-da85-4da6-b099-d10259a611d2") : secret "image-registry-tls" not found Jul 07 23:13:53.273181 ip-10-0-134-140 kubenswrapper[2570]: E0707 23:13:53.272891 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bd5aafbd-ed3c-46fa-bc87-661f74ba728a-metrics-tls podName:bd5aafbd-ed3c-46fa-bc87-661f74ba728a nodeName:}" failed. No retries permitted until 2026-07-07 23:14:09.272872505 +0000 UTC m=+66.669874011 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/bd5aafbd-ed3c-46fa-bc87-661f74ba728a-metrics-tls") pod "dns-default-4th55" (UID: "bd5aafbd-ed3c-46fa-bc87-661f74ba728a") : secret "dns-default-metrics-tls" not found Jul 07 23:13:58.469075 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:58.469039 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:13:58.530603 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:58.530570 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:13:58.595833 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:58.595800 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:13:59.345014 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:59.344983 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:13:59.420177 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:13:59.420152 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:14:02.372202 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:02.372161 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:14:02.434942 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:02.434906 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:14:03.270977 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:03.270950 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-w9gmc" Jul 07 23:14:03.296344 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:03.296309 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:14:03.307363 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:03.307312 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-gbf74" podStartSLOduration=23.872491242 podStartE2EDuration="26.30729734s" totalImagesPullingTime="2.434806098s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 23:13:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 23:13:41.343585843 +0000 UTC m=+38.740587356" watchObservedRunningTime="2026-07-07 23:14:03.30729734 +0000 UTC m=+60.704298869" Jul 07 23:14:03.424399 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:03.424351 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:14:03.524988 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:03.524957 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:14:03.626141 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:03.626106 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:14:03.738744 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:03.738713 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:14:03.830007 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:03.829932 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:14:04.338937 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:04.338898 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:14:06.770015 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:06.769983 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-64786d9cbb-4qp2l"] Jul 07 23:14:06.772959 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:06.772937 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-4qp2l" Jul 07 23:14:06.773075 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:06.772962 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-5c847dcfc4-r6r4j"] Jul 07 23:14:06.774915 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:06.774895 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5c847dcfc4-r6r4j" Jul 07 23:14:06.775242 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:06.775226 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\"" Jul 07 23:14:06.775412 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:06.775365 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 07 23:14:06.775508 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:06.775492 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 07 23:14:06.778410 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:06.778387 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-k4v5j\"" Jul 07 23:14:06.780895 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:06.780874 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\"" Jul 07 23:14:06.781010 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:06.780921 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 07 23:14:06.781010 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:06.780954 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\"" Jul 07 23:14:06.781174 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:06.781157 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 07 23:14:06.781244 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:06.781160 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\"" Jul 07 23:14:06.781429 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:06.781407 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-kzrlw\"" Jul 07 23:14:06.783532 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:06.783506 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-64786d9cbb-4qp2l"] Jul 07 23:14:06.784744 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:06.784706 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\"" Jul 07 23:14:06.786366 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:06.786346 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5c847dcfc4-r6r4j"] Jul 07 23:14:06.872879 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:06.872850 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/12b98da0-429d-4232-a0d6-b5c2699ec31a-serving-cert\") pod \"insights-operator-5c847dcfc4-r6r4j\" (UID: \"12b98da0-429d-4232-a0d6-b5c2699ec31a\") " pod="openshift-insights/insights-operator-5c847dcfc4-r6r4j" Jul 07 23:14:06.873049 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:06.872898 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/12b98da0-429d-4232-a0d6-b5c2699ec31a-service-ca-bundle\") pod \"insights-operator-5c847dcfc4-r6r4j\" (UID: \"12b98da0-429d-4232-a0d6-b5c2699ec31a\") " pod="openshift-insights/insights-operator-5c847dcfc4-r6r4j" Jul 07 23:14:06.873049 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:06.872920 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/dbb27ecc-1295-4648-851e-0daefb42af23-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-4qp2l\" (UID: \"dbb27ecc-1295-4648-851e-0daefb42af23\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-4qp2l" Jul 07 23:14:06.873049 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:06.872942 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fnmqd\" (UniqueName: \"kubernetes.io/projected/dbb27ecc-1295-4648-851e-0daefb42af23-kube-api-access-fnmqd\") pod \"cluster-monitoring-operator-64786d9cbb-4qp2l\" (UID: \"dbb27ecc-1295-4648-851e-0daefb42af23\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-4qp2l" Jul 07 23:14:06.873049 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:06.873002 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/dbb27ecc-1295-4648-851e-0daefb42af23-telemetry-config\") pod \"cluster-monitoring-operator-64786d9cbb-4qp2l\" (UID: \"dbb27ecc-1295-4648-851e-0daefb42af23\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-4qp2l" Jul 07 23:14:06.873049 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:06.873017 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/12b98da0-429d-4232-a0d6-b5c2699ec31a-tmp\") pod \"insights-operator-5c847dcfc4-r6r4j\" (UID: \"12b98da0-429d-4232-a0d6-b5c2699ec31a\") " pod="openshift-insights/insights-operator-5c847dcfc4-r6r4j" Jul 07 23:14:06.873049 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:06.873032 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6s75l\" (UniqueName: \"kubernetes.io/projected/12b98da0-429d-4232-a0d6-b5c2699ec31a-kube-api-access-6s75l\") pod \"insights-operator-5c847dcfc4-r6r4j\" (UID: \"12b98da0-429d-4232-a0d6-b5c2699ec31a\") " pod="openshift-insights/insights-operator-5c847dcfc4-r6r4j" Jul 07 23:14:06.873277 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:06.873073 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/12b98da0-429d-4232-a0d6-b5c2699ec31a-snapshots\") pod \"insights-operator-5c847dcfc4-r6r4j\" (UID: \"12b98da0-429d-4232-a0d6-b5c2699ec31a\") " pod="openshift-insights/insights-operator-5c847dcfc4-r6r4j" Jul 07 23:14:06.873277 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:06.873088 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/12b98da0-429d-4232-a0d6-b5c2699ec31a-trusted-ca-bundle\") pod \"insights-operator-5c847dcfc4-r6r4j\" (UID: \"12b98da0-429d-4232-a0d6-b5c2699ec31a\") " pod="openshift-insights/insights-operator-5c847dcfc4-r6r4j" Jul 07 23:14:06.876958 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:06.876934 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-65d5dfb6cf-84llt"] Jul 07 23:14:06.878719 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:06.878703 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-65d5dfb6cf-84llt" Jul 07 23:14:06.881369 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:06.881352 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\"" Jul 07 23:14:06.881835 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:06.881819 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\"" Jul 07 23:14:06.882165 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:06.882147 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-qfmls\"" Jul 07 23:14:06.882219 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:06.882167 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 07 23:14:06.882254 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:06.882214 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 07 23:14:06.882254 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:06.882217 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\"" Jul 07 23:14:06.882324 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:06.882156 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\"" Jul 07 23:14:06.893632 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:06.893611 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-65d5dfb6cf-84llt"] Jul 07 23:14:06.973558 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:06.973522 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gq7lc\" (UniqueName: \"kubernetes.io/projected/40bbfb8b-2c2b-43c5-8419-3dba3777f888-kube-api-access-gq7lc\") pod \"router-default-65d5dfb6cf-84llt\" (UID: \"40bbfb8b-2c2b-43c5-8419-3dba3777f888\") " pod="openshift-ingress/router-default-65d5dfb6cf-84llt" Jul 07 23:14:06.973558 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:06.973560 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/12b98da0-429d-4232-a0d6-b5c2699ec31a-tmp\") pod \"insights-operator-5c847dcfc4-r6r4j\" (UID: \"12b98da0-429d-4232-a0d6-b5c2699ec31a\") " pod="openshift-insights/insights-operator-5c847dcfc4-r6r4j" Jul 07 23:14:06.973783 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:06.973617 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6s75l\" (UniqueName: \"kubernetes.io/projected/12b98da0-429d-4232-a0d6-b5c2699ec31a-kube-api-access-6s75l\") pod \"insights-operator-5c847dcfc4-r6r4j\" (UID: \"12b98da0-429d-4232-a0d6-b5c2699ec31a\") " pod="openshift-insights/insights-operator-5c847dcfc4-r6r4j" Jul 07 23:14:06.973783 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:06.973666 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/40bbfb8b-2c2b-43c5-8419-3dba3777f888-metrics-certs\") pod \"router-default-65d5dfb6cf-84llt\" (UID: \"40bbfb8b-2c2b-43c5-8419-3dba3777f888\") " pod="openshift-ingress/router-default-65d5dfb6cf-84llt" Jul 07 23:14:06.973783 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:06.973704 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/12b98da0-429d-4232-a0d6-b5c2699ec31a-serving-cert\") pod \"insights-operator-5c847dcfc4-r6r4j\" (UID: \"12b98da0-429d-4232-a0d6-b5c2699ec31a\") " pod="openshift-insights/insights-operator-5c847dcfc4-r6r4j" Jul 07 23:14:06.973783 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:06.973761 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/12b98da0-429d-4232-a0d6-b5c2699ec31a-trusted-ca-bundle\") pod \"insights-operator-5c847dcfc4-r6r4j\" (UID: \"12b98da0-429d-4232-a0d6-b5c2699ec31a\") " pod="openshift-insights/insights-operator-5c847dcfc4-r6r4j" Jul 07 23:14:06.973974 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:06.973795 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/12b98da0-429d-4232-a0d6-b5c2699ec31a-snapshots\") pod \"insights-operator-5c847dcfc4-r6r4j\" (UID: \"12b98da0-429d-4232-a0d6-b5c2699ec31a\") " pod="openshift-insights/insights-operator-5c847dcfc4-r6r4j" Jul 07 23:14:06.973974 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:06.973827 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/40bbfb8b-2c2b-43c5-8419-3dba3777f888-stats-auth\") pod \"router-default-65d5dfb6cf-84llt\" (UID: \"40bbfb8b-2c2b-43c5-8419-3dba3777f888\") " pod="openshift-ingress/router-default-65d5dfb6cf-84llt" Jul 07 23:14:06.973974 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:06.973855 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/12b98da0-429d-4232-a0d6-b5c2699ec31a-service-ca-bundle\") pod \"insights-operator-5c847dcfc4-r6r4j\" (UID: \"12b98da0-429d-4232-a0d6-b5c2699ec31a\") " pod="openshift-insights/insights-operator-5c847dcfc4-r6r4j" Jul 07 23:14:06.973974 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:06.973886 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/dbb27ecc-1295-4648-851e-0daefb42af23-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-4qp2l\" (UID: \"dbb27ecc-1295-4648-851e-0daefb42af23\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-4qp2l" Jul 07 23:14:06.973974 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:06.973943 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fnmqd\" (UniqueName: \"kubernetes.io/projected/dbb27ecc-1295-4648-851e-0daefb42af23-kube-api-access-fnmqd\") pod \"cluster-monitoring-operator-64786d9cbb-4qp2l\" (UID: \"dbb27ecc-1295-4648-851e-0daefb42af23\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-4qp2l" Jul 07 23:14:06.974186 ip-10-0-134-140 kubenswrapper[2570]: E0707 23:14:06.973978 2570 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 07 23:14:06.974186 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:06.973989 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/12b98da0-429d-4232-a0d6-b5c2699ec31a-tmp\") pod \"insights-operator-5c847dcfc4-r6r4j\" (UID: \"12b98da0-429d-4232-a0d6-b5c2699ec31a\") " pod="openshift-insights/insights-operator-5c847dcfc4-r6r4j" Jul 07 23:14:06.974186 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:06.973998 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/dbb27ecc-1295-4648-851e-0daefb42af23-telemetry-config\") pod \"cluster-monitoring-operator-64786d9cbb-4qp2l\" (UID: \"dbb27ecc-1295-4648-851e-0daefb42af23\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-4qp2l" Jul 07 23:14:06.974186 ip-10-0-134-140 kubenswrapper[2570]: E0707 23:14:06.974032 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dbb27ecc-1295-4648-851e-0daefb42af23-cluster-monitoring-operator-tls podName:dbb27ecc-1295-4648-851e-0daefb42af23 nodeName:}" failed. No retries permitted until 2026-07-07 23:14:07.474015881 +0000 UTC m=+64.871017384 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/dbb27ecc-1295-4648-851e-0daefb42af23-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-4qp2l" (UID: "dbb27ecc-1295-4648-851e-0daefb42af23") : secret "cluster-monitoring-operator-tls" not found Jul 07 23:14:06.974186 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:06.974163 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/40bbfb8b-2c2b-43c5-8419-3dba3777f888-service-ca-bundle\") pod \"router-default-65d5dfb6cf-84llt\" (UID: \"40bbfb8b-2c2b-43c5-8419-3dba3777f888\") " pod="openshift-ingress/router-default-65d5dfb6cf-84llt" Jul 07 23:14:06.974419 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:06.974225 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/40bbfb8b-2c2b-43c5-8419-3dba3777f888-default-certificate\") pod \"router-default-65d5dfb6cf-84llt\" (UID: \"40bbfb8b-2c2b-43c5-8419-3dba3777f888\") " pod="openshift-ingress/router-default-65d5dfb6cf-84llt" Jul 07 23:14:06.974525 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:06.974505 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/12b98da0-429d-4232-a0d6-b5c2699ec31a-snapshots\") pod \"insights-operator-5c847dcfc4-r6r4j\" (UID: \"12b98da0-429d-4232-a0d6-b5c2699ec31a\") " pod="openshift-insights/insights-operator-5c847dcfc4-r6r4j" Jul 07 23:14:06.974614 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:06.974595 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/dbb27ecc-1295-4648-851e-0daefb42af23-telemetry-config\") pod \"cluster-monitoring-operator-64786d9cbb-4qp2l\" (UID: \"dbb27ecc-1295-4648-851e-0daefb42af23\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-4qp2l" Jul 07 23:14:06.974979 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:06.974963 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/12b98da0-429d-4232-a0d6-b5c2699ec31a-service-ca-bundle\") pod \"insights-operator-5c847dcfc4-r6r4j\" (UID: \"12b98da0-429d-4232-a0d6-b5c2699ec31a\") " pod="openshift-insights/insights-operator-5c847dcfc4-r6r4j" Jul 07 23:14:06.975350 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:06.975332 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/12b98da0-429d-4232-a0d6-b5c2699ec31a-trusted-ca-bundle\") pod \"insights-operator-5c847dcfc4-r6r4j\" (UID: \"12b98da0-429d-4232-a0d6-b5c2699ec31a\") " pod="openshift-insights/insights-operator-5c847dcfc4-r6r4j" Jul 07 23:14:06.977845 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:06.977828 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/12b98da0-429d-4232-a0d6-b5c2699ec31a-serving-cert\") pod \"insights-operator-5c847dcfc4-r6r4j\" (UID: \"12b98da0-429d-4232-a0d6-b5c2699ec31a\") " pod="openshift-insights/insights-operator-5c847dcfc4-r6r4j" Jul 07 23:14:06.985590 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:06.985563 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6s75l\" (UniqueName: \"kubernetes.io/projected/12b98da0-429d-4232-a0d6-b5c2699ec31a-kube-api-access-6s75l\") pod \"insights-operator-5c847dcfc4-r6r4j\" (UID: \"12b98da0-429d-4232-a0d6-b5c2699ec31a\") " pod="openshift-insights/insights-operator-5c847dcfc4-r6r4j" Jul 07 23:14:06.985675 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:06.985573 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fnmqd\" (UniqueName: \"kubernetes.io/projected/dbb27ecc-1295-4648-851e-0daefb42af23-kube-api-access-fnmqd\") pod \"cluster-monitoring-operator-64786d9cbb-4qp2l\" (UID: \"dbb27ecc-1295-4648-851e-0daefb42af23\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-4qp2l" Jul 07 23:14:07.038676 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:07.038651 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:14:07.074814 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:07.074782 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/40bbfb8b-2c2b-43c5-8419-3dba3777f888-stats-auth\") pod \"router-default-65d5dfb6cf-84llt\" (UID: \"40bbfb8b-2c2b-43c5-8419-3dba3777f888\") " pod="openshift-ingress/router-default-65d5dfb6cf-84llt" Jul 07 23:14:07.074965 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:07.074855 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/40bbfb8b-2c2b-43c5-8419-3dba3777f888-service-ca-bundle\") pod \"router-default-65d5dfb6cf-84llt\" (UID: \"40bbfb8b-2c2b-43c5-8419-3dba3777f888\") " pod="openshift-ingress/router-default-65d5dfb6cf-84llt" Jul 07 23:14:07.074965 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:07.074911 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/40bbfb8b-2c2b-43c5-8419-3dba3777f888-default-certificate\") pod \"router-default-65d5dfb6cf-84llt\" (UID: \"40bbfb8b-2c2b-43c5-8419-3dba3777f888\") " pod="openshift-ingress/router-default-65d5dfb6cf-84llt" Jul 07 23:14:07.075078 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:07.074961 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gq7lc\" (UniqueName: \"kubernetes.io/projected/40bbfb8b-2c2b-43c5-8419-3dba3777f888-kube-api-access-gq7lc\") pod \"router-default-65d5dfb6cf-84llt\" (UID: \"40bbfb8b-2c2b-43c5-8419-3dba3777f888\") " pod="openshift-ingress/router-default-65d5dfb6cf-84llt" Jul 07 23:14:07.075078 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:07.074990 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/40bbfb8b-2c2b-43c5-8419-3dba3777f888-metrics-certs\") pod \"router-default-65d5dfb6cf-84llt\" (UID: \"40bbfb8b-2c2b-43c5-8419-3dba3777f888\") " pod="openshift-ingress/router-default-65d5dfb6cf-84llt" Jul 07 23:14:07.075078 ip-10-0-134-140 kubenswrapper[2570]: E0707 23:14:07.075026 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/40bbfb8b-2c2b-43c5-8419-3dba3777f888-service-ca-bundle podName:40bbfb8b-2c2b-43c5-8419-3dba3777f888 nodeName:}" failed. No retries permitted until 2026-07-07 23:14:07.575005702 +0000 UTC m=+64.972007196 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/40bbfb8b-2c2b-43c5-8419-3dba3777f888-service-ca-bundle") pod "router-default-65d5dfb6cf-84llt" (UID: "40bbfb8b-2c2b-43c5-8419-3dba3777f888") : configmap references non-existent config key: service-ca.crt Jul 07 23:14:07.075224 ip-10-0-134-140 kubenswrapper[2570]: E0707 23:14:07.075079 2570 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 07 23:14:07.075224 ip-10-0-134-140 kubenswrapper[2570]: E0707 23:14:07.075147 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/40bbfb8b-2c2b-43c5-8419-3dba3777f888-metrics-certs podName:40bbfb8b-2c2b-43c5-8419-3dba3777f888 nodeName:}" failed. No retries permitted until 2026-07-07 23:14:07.575131038 +0000 UTC m=+64.972132532 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/40bbfb8b-2c2b-43c5-8419-3dba3777f888-metrics-certs") pod "router-default-65d5dfb6cf-84llt" (UID: "40bbfb8b-2c2b-43c5-8419-3dba3777f888") : secret "router-metrics-certs-default" not found Jul 07 23:14:07.077207 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:07.077182 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/40bbfb8b-2c2b-43c5-8419-3dba3777f888-default-certificate\") pod \"router-default-65d5dfb6cf-84llt\" (UID: \"40bbfb8b-2c2b-43c5-8419-3dba3777f888\") " pod="openshift-ingress/router-default-65d5dfb6cf-84llt" Jul 07 23:14:07.077207 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:07.077191 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/40bbfb8b-2c2b-43c5-8419-3dba3777f888-stats-auth\") pod \"router-default-65d5dfb6cf-84llt\" (UID: \"40bbfb8b-2c2b-43c5-8419-3dba3777f888\") " pod="openshift-ingress/router-default-65d5dfb6cf-84llt" Jul 07 23:14:07.083495 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:07.083458 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gq7lc\" (UniqueName: \"kubernetes.io/projected/40bbfb8b-2c2b-43c5-8419-3dba3777f888-kube-api-access-gq7lc\") pod \"router-default-65d5dfb6cf-84llt\" (UID: \"40bbfb8b-2c2b-43c5-8419-3dba3777f888\") " pod="openshift-ingress/router-default-65d5dfb6cf-84llt" Jul 07 23:14:07.114969 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:07.114947 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5c847dcfc4-r6r4j" Jul 07 23:14:07.139727 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:07.139691 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:14:07.236339 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:07.236304 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5c847dcfc4-r6r4j"] Jul 07 23:14:07.238975 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:14:07.238945 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod12b98da0_429d_4232_a0d6_b5c2699ec31a.slice/crio-9fbf4802523c0aa616242b26c7f549fc445f0ba2ce8fafb8c655cd4ad1eaa12b WatchSource:0}: Error finding container 9fbf4802523c0aa616242b26c7f549fc445f0ba2ce8fafb8c655cd4ad1eaa12b: Status 404 returned error can't find the container with id 9fbf4802523c0aa616242b26c7f549fc445f0ba2ce8fafb8c655cd4ad1eaa12b Jul 07 23:14:07.347563 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:07.347488 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5c847dcfc4-r6r4j" event={"ID":"12b98da0-429d-4232-a0d6-b5c2699ec31a","Type":"ContainerStarted","Data":"9fbf4802523c0aa616242b26c7f549fc445f0ba2ce8fafb8c655cd4ad1eaa12b"} Jul 07 23:14:07.477900 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:07.477863 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/dbb27ecc-1295-4648-851e-0daefb42af23-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-4qp2l\" (UID: \"dbb27ecc-1295-4648-851e-0daefb42af23\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-4qp2l" Jul 07 23:14:07.478074 ip-10-0-134-140 kubenswrapper[2570]: E0707 23:14:07.478034 2570 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 07 23:14:07.478123 ip-10-0-134-140 kubenswrapper[2570]: E0707 23:14:07.478097 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dbb27ecc-1295-4648-851e-0daefb42af23-cluster-monitoring-operator-tls podName:dbb27ecc-1295-4648-851e-0daefb42af23 nodeName:}" failed. No retries permitted until 2026-07-07 23:14:08.478082275 +0000 UTC m=+65.875083765 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/dbb27ecc-1295-4648-851e-0daefb42af23-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-4qp2l" (UID: "dbb27ecc-1295-4648-851e-0daefb42af23") : secret "cluster-monitoring-operator-tls" not found Jul 07 23:14:07.578903 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:07.578871 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/40bbfb8b-2c2b-43c5-8419-3dba3777f888-service-ca-bundle\") pod \"router-default-65d5dfb6cf-84llt\" (UID: \"40bbfb8b-2c2b-43c5-8419-3dba3777f888\") " pod="openshift-ingress/router-default-65d5dfb6cf-84llt" Jul 07 23:14:07.579032 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:07.578964 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/40bbfb8b-2c2b-43c5-8419-3dba3777f888-metrics-certs\") pod \"router-default-65d5dfb6cf-84llt\" (UID: \"40bbfb8b-2c2b-43c5-8419-3dba3777f888\") " pod="openshift-ingress/router-default-65d5dfb6cf-84llt" Jul 07 23:14:07.579085 ip-10-0-134-140 kubenswrapper[2570]: E0707 23:14:07.579046 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/40bbfb8b-2c2b-43c5-8419-3dba3777f888-service-ca-bundle podName:40bbfb8b-2c2b-43c5-8419-3dba3777f888 nodeName:}" failed. No retries permitted until 2026-07-07 23:14:08.579028047 +0000 UTC m=+65.976029536 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/40bbfb8b-2c2b-43c5-8419-3dba3777f888-service-ca-bundle") pod "router-default-65d5dfb6cf-84llt" (UID: "40bbfb8b-2c2b-43c5-8419-3dba3777f888") : configmap references non-existent config key: service-ca.crt Jul 07 23:14:07.579085 ip-10-0-134-140 kubenswrapper[2570]: E0707 23:14:07.579075 2570 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 07 23:14:07.579150 ip-10-0-134-140 kubenswrapper[2570]: E0707 23:14:07.579113 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/40bbfb8b-2c2b-43c5-8419-3dba3777f888-metrics-certs podName:40bbfb8b-2c2b-43c5-8419-3dba3777f888 nodeName:}" failed. No retries permitted until 2026-07-07 23:14:08.579102856 +0000 UTC m=+65.976104347 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/40bbfb8b-2c2b-43c5-8419-3dba3777f888-metrics-certs") pod "router-default-65d5dfb6cf-84llt" (UID: "40bbfb8b-2c2b-43c5-8419-3dba3777f888") : secret "router-metrics-certs-default" not found Jul 07 23:14:07.745931 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:07.745841 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-rwtl7"] Jul 07 23:14:07.747796 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:07.747774 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-svhp9"] Jul 07 23:14:07.747932 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:07.747907 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-rwtl7" Jul 07 23:14:07.749770 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:07.749745 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-svhp9" Jul 07 23:14:07.751397 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:07.751379 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\"" Jul 07 23:14:07.751528 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:07.751488 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-7x26g\"" Jul 07 23:14:07.752278 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:07.752254 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\"" Jul 07 23:14:07.752384 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:07.752351 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-n8t6s\"" Jul 07 23:14:07.752384 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:07.752365 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\"" Jul 07 23:14:07.755380 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:07.752571 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\"" Jul 07 23:14:07.755380 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:07.752901 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\"" Jul 07 23:14:07.763088 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:07.762926 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-svhp9"] Jul 07 23:14:07.763930 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:07.763910 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-rwtl7"] Jul 07 23:14:07.780367 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:07.780345 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wnkb5\" (UniqueName: \"kubernetes.io/projected/d557aa86-8e99-41fa-9d29-80b527cd8711-kube-api-access-wnkb5\") pod \"cluster-samples-operator-56fcf86bf-svhp9\" (UID: \"d557aa86-8e99-41fa-9d29-80b527cd8711\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-svhp9" Jul 07 23:14:07.780704 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:07.780417 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dgcw2\" (UniqueName: \"kubernetes.io/projected/01587fde-e08c-441e-a73e-644b4d50b608-kube-api-access-dgcw2\") pod \"volume-data-source-validator-5d456fdbcd-rwtl7\" (UID: \"01587fde-e08c-441e-a73e-644b4d50b608\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-rwtl7" Jul 07 23:14:07.780704 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:07.780511 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/d557aa86-8e99-41fa-9d29-80b527cd8711-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-svhp9\" (UID: \"d557aa86-8e99-41fa-9d29-80b527cd8711\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-svhp9" Jul 07 23:14:07.881693 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:07.881661 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wnkb5\" (UniqueName: \"kubernetes.io/projected/d557aa86-8e99-41fa-9d29-80b527cd8711-kube-api-access-wnkb5\") pod \"cluster-samples-operator-56fcf86bf-svhp9\" (UID: \"d557aa86-8e99-41fa-9d29-80b527cd8711\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-svhp9" Jul 07 23:14:07.881866 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:07.881730 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dgcw2\" (UniqueName: \"kubernetes.io/projected/01587fde-e08c-441e-a73e-644b4d50b608-kube-api-access-dgcw2\") pod \"volume-data-source-validator-5d456fdbcd-rwtl7\" (UID: \"01587fde-e08c-441e-a73e-644b4d50b608\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-rwtl7" Jul 07 23:14:07.881866 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:07.881797 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/d557aa86-8e99-41fa-9d29-80b527cd8711-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-svhp9\" (UID: \"d557aa86-8e99-41fa-9d29-80b527cd8711\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-svhp9" Jul 07 23:14:07.881984 ip-10-0-134-140 kubenswrapper[2570]: E0707 23:14:07.881887 2570 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 07 23:14:07.881984 ip-10-0-134-140 kubenswrapper[2570]: E0707 23:14:07.881944 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d557aa86-8e99-41fa-9d29-80b527cd8711-samples-operator-tls podName:d557aa86-8e99-41fa-9d29-80b527cd8711 nodeName:}" failed. No retries permitted until 2026-07-07 23:14:08.381925497 +0000 UTC m=+65.778926988 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/d557aa86-8e99-41fa-9d29-80b527cd8711-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-svhp9" (UID: "d557aa86-8e99-41fa-9d29-80b527cd8711") : secret "samples-operator-tls" not found Jul 07 23:14:07.892574 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:07.892544 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wnkb5\" (UniqueName: \"kubernetes.io/projected/d557aa86-8e99-41fa-9d29-80b527cd8711-kube-api-access-wnkb5\") pod \"cluster-samples-operator-56fcf86bf-svhp9\" (UID: \"d557aa86-8e99-41fa-9d29-80b527cd8711\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-svhp9" Jul 07 23:14:07.892719 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:07.892609 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dgcw2\" (UniqueName: \"kubernetes.io/projected/01587fde-e08c-441e-a73e-644b4d50b608-kube-api-access-dgcw2\") pod \"volume-data-source-validator-5d456fdbcd-rwtl7\" (UID: \"01587fde-e08c-441e-a73e-644b4d50b608\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-rwtl7" Jul 07 23:14:08.062619 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:08.062582 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-rwtl7" Jul 07 23:14:08.200665 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:08.200613 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-rwtl7"] Jul 07 23:14:08.202643 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:14:08.202610 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod01587fde_e08c_441e_a73e_644b4d50b608.slice/crio-35d3a410c48eca75756e296dbf574ce2f20373ee831aabfd2c328d0372c533d2 WatchSource:0}: Error finding container 35d3a410c48eca75756e296dbf574ce2f20373ee831aabfd2c328d0372c533d2: Status 404 returned error can't find the container with id 35d3a410c48eca75756e296dbf574ce2f20373ee831aabfd2c328d0372c533d2 Jul 07 23:14:08.351609 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:08.351522 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-rwtl7" event={"ID":"01587fde-e08c-441e-a73e-644b4d50b608","Type":"ContainerStarted","Data":"35d3a410c48eca75756e296dbf574ce2f20373ee831aabfd2c328d0372c533d2"} Jul 07 23:14:08.387245 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:08.387211 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/d557aa86-8e99-41fa-9d29-80b527cd8711-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-svhp9\" (UID: \"d557aa86-8e99-41fa-9d29-80b527cd8711\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-svhp9" Jul 07 23:14:08.387414 ip-10-0-134-140 kubenswrapper[2570]: E0707 23:14:08.387394 2570 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 07 23:14:08.387506 ip-10-0-134-140 kubenswrapper[2570]: E0707 23:14:08.387494 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d557aa86-8e99-41fa-9d29-80b527cd8711-samples-operator-tls podName:d557aa86-8e99-41fa-9d29-80b527cd8711 nodeName:}" failed. No retries permitted until 2026-07-07 23:14:09.387453006 +0000 UTC m=+66.784454515 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/d557aa86-8e99-41fa-9d29-80b527cd8711-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-svhp9" (UID: "d557aa86-8e99-41fa-9d29-80b527cd8711") : secret "samples-operator-tls" not found Jul 07 23:14:08.488432 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:08.488389 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/dbb27ecc-1295-4648-851e-0daefb42af23-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-4qp2l\" (UID: \"dbb27ecc-1295-4648-851e-0daefb42af23\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-4qp2l" Jul 07 23:14:08.488606 ip-10-0-134-140 kubenswrapper[2570]: E0707 23:14:08.488564 2570 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 07 23:14:08.488650 ip-10-0-134-140 kubenswrapper[2570]: E0707 23:14:08.488644 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dbb27ecc-1295-4648-851e-0daefb42af23-cluster-monitoring-operator-tls podName:dbb27ecc-1295-4648-851e-0daefb42af23 nodeName:}" failed. No retries permitted until 2026-07-07 23:14:10.488627844 +0000 UTC m=+67.885629334 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/dbb27ecc-1295-4648-851e-0daefb42af23-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-4qp2l" (UID: "dbb27ecc-1295-4648-851e-0daefb42af23") : secret "cluster-monitoring-operator-tls" not found Jul 07 23:14:08.589015 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:08.588985 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/40bbfb8b-2c2b-43c5-8419-3dba3777f888-metrics-certs\") pod \"router-default-65d5dfb6cf-84llt\" (UID: \"40bbfb8b-2c2b-43c5-8419-3dba3777f888\") " pod="openshift-ingress/router-default-65d5dfb6cf-84llt" Jul 07 23:14:08.589172 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:08.589073 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/40bbfb8b-2c2b-43c5-8419-3dba3777f888-service-ca-bundle\") pod \"router-default-65d5dfb6cf-84llt\" (UID: \"40bbfb8b-2c2b-43c5-8419-3dba3777f888\") " pod="openshift-ingress/router-default-65d5dfb6cf-84llt" Jul 07 23:14:08.589172 ip-10-0-134-140 kubenswrapper[2570]: E0707 23:14:08.589124 2570 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 07 23:14:08.589285 ip-10-0-134-140 kubenswrapper[2570]: E0707 23:14:08.589198 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/40bbfb8b-2c2b-43c5-8419-3dba3777f888-metrics-certs podName:40bbfb8b-2c2b-43c5-8419-3dba3777f888 nodeName:}" failed. No retries permitted until 2026-07-07 23:14:10.589181205 +0000 UTC m=+67.986182706 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/40bbfb8b-2c2b-43c5-8419-3dba3777f888-metrics-certs") pod "router-default-65d5dfb6cf-84llt" (UID: "40bbfb8b-2c2b-43c5-8419-3dba3777f888") : secret "router-metrics-certs-default" not found Jul 07 23:14:08.589285 ip-10-0-134-140 kubenswrapper[2570]: E0707 23:14:08.589220 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/40bbfb8b-2c2b-43c5-8419-3dba3777f888-service-ca-bundle podName:40bbfb8b-2c2b-43c5-8419-3dba3777f888 nodeName:}" failed. No retries permitted until 2026-07-07 23:14:10.589208745 +0000 UTC m=+67.986210238 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/40bbfb8b-2c2b-43c5-8419-3dba3777f888-service-ca-bundle") pod "router-default-65d5dfb6cf-84llt" (UID: "40bbfb8b-2c2b-43c5-8419-3dba3777f888") : configmap references non-existent config key: service-ca.crt Jul 07 23:14:08.749667 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:08.749583 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-68b757fb67-8hvxr"] Jul 07 23:14:08.751955 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:08.751927 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-68b757fb67-8hvxr" Jul 07 23:14:08.754656 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:08.754631 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\"" Jul 07 23:14:08.755029 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:08.755011 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-jbv6x\"" Jul 07 23:14:08.755029 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:08.755019 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\"" Jul 07 23:14:08.755193 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:08.755079 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\"" Jul 07 23:14:08.755193 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:08.755118 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\"" Jul 07 23:14:08.760539 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:08.760519 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\"" Jul 07 23:14:08.764012 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:08.763994 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-68b757fb67-8hvxr"] Jul 07 23:14:08.791085 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:08.791048 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/faae2bd8-cbbd-4cfb-bce6-7954977b7ebc-serving-cert\") pod \"console-operator-68b757fb67-8hvxr\" (UID: \"faae2bd8-cbbd-4cfb-bce6-7954977b7ebc\") " pod="openshift-console-operator/console-operator-68b757fb67-8hvxr" Jul 07 23:14:08.791455 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:08.791109 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tqrnt\" (UniqueName: \"kubernetes.io/projected/faae2bd8-cbbd-4cfb-bce6-7954977b7ebc-kube-api-access-tqrnt\") pod \"console-operator-68b757fb67-8hvxr\" (UID: \"faae2bd8-cbbd-4cfb-bce6-7954977b7ebc\") " pod="openshift-console-operator/console-operator-68b757fb67-8hvxr" Jul 07 23:14:08.791455 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:08.791165 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/14406dc0-620c-4615-b98a-b7205dc72966-metrics-certs\") pod \"network-metrics-daemon-ndz2q\" (UID: \"14406dc0-620c-4615-b98a-b7205dc72966\") " pod="openshift-multus/network-metrics-daemon-ndz2q" Jul 07 23:14:08.791455 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:08.791281 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/faae2bd8-cbbd-4cfb-bce6-7954977b7ebc-trusted-ca\") pod \"console-operator-68b757fb67-8hvxr\" (UID: \"faae2bd8-cbbd-4cfb-bce6-7954977b7ebc\") " pod="openshift-console-operator/console-operator-68b757fb67-8hvxr" Jul 07 23:14:08.791455 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:08.791355 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/faae2bd8-cbbd-4cfb-bce6-7954977b7ebc-config\") pod \"console-operator-68b757fb67-8hvxr\" (UID: \"faae2bd8-cbbd-4cfb-bce6-7954977b7ebc\") " pod="openshift-console-operator/console-operator-68b757fb67-8hvxr" Jul 07 23:14:08.793971 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:08.793952 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 07 23:14:08.802177 ip-10-0-134-140 kubenswrapper[2570]: E0707 23:14:08.802158 2570 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jul 07 23:14:08.802300 ip-10-0-134-140 kubenswrapper[2570]: E0707 23:14:08.802216 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/14406dc0-620c-4615-b98a-b7205dc72966-metrics-certs podName:14406dc0-620c-4615-b98a-b7205dc72966 nodeName:}" failed. No retries permitted until 2026-07-07 23:15:12.802198489 +0000 UTC m=+130.199199979 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/14406dc0-620c-4615-b98a-b7205dc72966-metrics-certs") pod "network-metrics-daemon-ndz2q" (UID: "14406dc0-620c-4615-b98a-b7205dc72966") : secret "metrics-daemon-secret" not found Jul 07 23:14:08.892655 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:08.892625 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/faae2bd8-cbbd-4cfb-bce6-7954977b7ebc-trusted-ca\") pod \"console-operator-68b757fb67-8hvxr\" (UID: \"faae2bd8-cbbd-4cfb-bce6-7954977b7ebc\") " pod="openshift-console-operator/console-operator-68b757fb67-8hvxr" Jul 07 23:14:08.892655 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:08.892662 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bkbfq\" (UniqueName: \"kubernetes.io/projected/f47fd470-2c7a-4ea8-bf73-e3f94320f117-kube-api-access-bkbfq\") pod \"network-check-target-p27rw\" (UID: \"f47fd470-2c7a-4ea8-bf73-e3f94320f117\") " pod="openshift-network-diagnostics/network-check-target-p27rw" Jul 07 23:14:08.892898 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:08.892705 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/faae2bd8-cbbd-4cfb-bce6-7954977b7ebc-config\") pod \"console-operator-68b757fb67-8hvxr\" (UID: \"faae2bd8-cbbd-4cfb-bce6-7954977b7ebc\") " pod="openshift-console-operator/console-operator-68b757fb67-8hvxr" Jul 07 23:14:08.892898 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:08.892722 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1496ff7f-366b-4cd2-8396-2cdbfed841fa-original-pull-secret\") pod \"global-pull-secret-syncer-84hdc\" (UID: \"1496ff7f-366b-4cd2-8396-2cdbfed841fa\") " pod="kube-system/global-pull-secret-syncer-84hdc" Jul 07 23:14:08.892898 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:08.892741 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/faae2bd8-cbbd-4cfb-bce6-7954977b7ebc-serving-cert\") pod \"console-operator-68b757fb67-8hvxr\" (UID: \"faae2bd8-cbbd-4cfb-bce6-7954977b7ebc\") " pod="openshift-console-operator/console-operator-68b757fb67-8hvxr" Jul 07 23:14:08.892898 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:08.892787 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tqrnt\" (UniqueName: \"kubernetes.io/projected/faae2bd8-cbbd-4cfb-bce6-7954977b7ebc-kube-api-access-tqrnt\") pod \"console-operator-68b757fb67-8hvxr\" (UID: \"faae2bd8-cbbd-4cfb-bce6-7954977b7ebc\") " pod="openshift-console-operator/console-operator-68b757fb67-8hvxr" Jul 07 23:14:08.893446 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:08.893417 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/faae2bd8-cbbd-4cfb-bce6-7954977b7ebc-config\") pod \"console-operator-68b757fb67-8hvxr\" (UID: \"faae2bd8-cbbd-4cfb-bce6-7954977b7ebc\") " pod="openshift-console-operator/console-operator-68b757fb67-8hvxr" Jul 07 23:14:08.893630 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:08.893610 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/faae2bd8-cbbd-4cfb-bce6-7954977b7ebc-trusted-ca\") pod \"console-operator-68b757fb67-8hvxr\" (UID: \"faae2bd8-cbbd-4cfb-bce6-7954977b7ebc\") " pod="openshift-console-operator/console-operator-68b757fb67-8hvxr" Jul 07 23:14:08.895213 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:08.895184 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/faae2bd8-cbbd-4cfb-bce6-7954977b7ebc-serving-cert\") pod \"console-operator-68b757fb67-8hvxr\" (UID: \"faae2bd8-cbbd-4cfb-bce6-7954977b7ebc\") " pod="openshift-console-operator/console-operator-68b757fb67-8hvxr" Jul 07 23:14:08.895678 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:08.895659 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 07 23:14:08.895779 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:08.895722 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 07 23:14:08.901694 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:08.901676 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tqrnt\" (UniqueName: \"kubernetes.io/projected/faae2bd8-cbbd-4cfb-bce6-7954977b7ebc-kube-api-access-tqrnt\") pod \"console-operator-68b757fb67-8hvxr\" (UID: \"faae2bd8-cbbd-4cfb-bce6-7954977b7ebc\") " pod="openshift-console-operator/console-operator-68b757fb67-8hvxr" Jul 07 23:14:08.905302 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:08.905282 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1496ff7f-366b-4cd2-8396-2cdbfed841fa-original-pull-secret\") pod \"global-pull-secret-syncer-84hdc\" (UID: \"1496ff7f-366b-4cd2-8396-2cdbfed841fa\") " pod="kube-system/global-pull-secret-syncer-84hdc" Jul 07 23:14:08.905892 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:08.905876 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 07 23:14:08.915745 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:08.915727 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bkbfq\" (UniqueName: \"kubernetes.io/projected/f47fd470-2c7a-4ea8-bf73-e3f94320f117-kube-api-access-bkbfq\") pod \"network-check-target-p27rw\" (UID: \"f47fd470-2c7a-4ea8-bf73-e3f94320f117\") " pod="openshift-network-diagnostics/network-check-target-p27rw" Jul 07 23:14:09.005644 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:09.005566 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-nhfdw\"" Jul 07 23:14:09.008770 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:09.008749 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-84hdc" Jul 07 23:14:09.013488 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:09.013460 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-p27rw" Jul 07 23:14:09.063501 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:09.063304 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-68b757fb67-8hvxr" Jul 07 23:14:09.165122 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:09.165051 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-84hdc"] Jul 07 23:14:09.171396 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:14:09.171348 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1496ff7f_366b_4cd2_8396_2cdbfed841fa.slice/crio-d263e915a220dd781f4ba917a535d7caea99fccfd99b96f1abd27f3a9f977cd2 WatchSource:0}: Error finding container d263e915a220dd781f4ba917a535d7caea99fccfd99b96f1abd27f3a9f977cd2: Status 404 returned error can't find the container with id d263e915a220dd781f4ba917a535d7caea99fccfd99b96f1abd27f3a9f977cd2 Jul 07 23:14:09.186755 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:09.186586 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-p27rw"] Jul 07 23:14:09.190506 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:14:09.190479 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf47fd470_2c7a_4ea8_bf73_e3f94320f117.slice/crio-d67f5fc7f2b7c7859fcf8eabbbc3f77c1adda0e5faeddf08c72d386ab77e2392 WatchSource:0}: Error finding container d67f5fc7f2b7c7859fcf8eabbbc3f77c1adda0e5faeddf08c72d386ab77e2392: Status 404 returned error can't find the container with id d67f5fc7f2b7c7859fcf8eabbbc3f77c1adda0e5faeddf08c72d386ab77e2392 Jul 07 23:14:09.220813 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:09.220786 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-68b757fb67-8hvxr"] Jul 07 23:14:09.223870 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:14:09.223844 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfaae2bd8_cbbd_4cfb_bce6_7954977b7ebc.slice/crio-acfda5b04f630745e19d6aaa49d864380f8316545ce8c22186be1f9d66ca8c61 WatchSource:0}: Error finding container acfda5b04f630745e19d6aaa49d864380f8316545ce8c22186be1f9d66ca8c61: Status 404 returned error can't find the container with id acfda5b04f630745e19d6aaa49d864380f8316545ce8c22186be1f9d66ca8c61 Jul 07 23:14:09.296978 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:09.296951 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/bd5aafbd-ed3c-46fa-bc87-661f74ba728a-metrics-tls\") pod \"dns-default-4th55\" (UID: \"bd5aafbd-ed3c-46fa-bc87-661f74ba728a\") " pod="openshift-dns/dns-default-4th55" Jul 07 23:14:09.297140 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:09.297087 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/cf5100ad-da85-4da6-b099-d10259a611d2-registry-tls\") pod \"image-registry-5668757489-j9qbl\" (UID: \"cf5100ad-da85-4da6-b099-d10259a611d2\") " pod="openshift-image-registry/image-registry-5668757489-j9qbl" Jul 07 23:14:09.297140 ip-10-0-134-140 kubenswrapper[2570]: E0707 23:14:09.297120 2570 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 07 23:14:09.297258 ip-10-0-134-140 kubenswrapper[2570]: E0707 23:14:09.297198 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bd5aafbd-ed3c-46fa-bc87-661f74ba728a-metrics-tls podName:bd5aafbd-ed3c-46fa-bc87-661f74ba728a nodeName:}" failed. No retries permitted until 2026-07-07 23:14:41.297181784 +0000 UTC m=+98.694183274 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/bd5aafbd-ed3c-46fa-bc87-661f74ba728a-metrics-tls") pod "dns-default-4th55" (UID: "bd5aafbd-ed3c-46fa-bc87-661f74ba728a") : secret "dns-default-metrics-tls" not found Jul 07 23:14:09.297258 ip-10-0-134-140 kubenswrapper[2570]: E0707 23:14:09.297208 2570 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 07 23:14:09.297258 ip-10-0-134-140 kubenswrapper[2570]: E0707 23:14:09.297224 2570 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5668757489-j9qbl: secret "image-registry-tls" not found Jul 07 23:14:09.297422 ip-10-0-134-140 kubenswrapper[2570]: E0707 23:14:09.297291 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/cf5100ad-da85-4da6-b099-d10259a611d2-registry-tls podName:cf5100ad-da85-4da6-b099-d10259a611d2 nodeName:}" failed. No retries permitted until 2026-07-07 23:14:41.297274503 +0000 UTC m=+98.694276015 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/cf5100ad-da85-4da6-b099-d10259a611d2-registry-tls") pod "image-registry-5668757489-j9qbl" (UID: "cf5100ad-da85-4da6-b099-d10259a611d2") : secret "image-registry-tls" not found Jul 07 23:14:09.355510 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:09.355454 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-84hdc" event={"ID":"1496ff7f-366b-4cd2-8396-2cdbfed841fa","Type":"ContainerStarted","Data":"d263e915a220dd781f4ba917a535d7caea99fccfd99b96f1abd27f3a9f977cd2"} Jul 07 23:14:09.357149 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:09.357123 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5c847dcfc4-r6r4j" event={"ID":"12b98da0-429d-4232-a0d6-b5c2699ec31a","Type":"ContainerStarted","Data":"6c12dcb40a67812930af4647cbda07711caa9258e2abf6b0bc8264a36e343c3d"} Jul 07 23:14:09.358423 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:09.358398 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-8hvxr" event={"ID":"faae2bd8-cbbd-4cfb-bce6-7954977b7ebc","Type":"ContainerStarted","Data":"acfda5b04f630745e19d6aaa49d864380f8316545ce8c22186be1f9d66ca8c61"} Jul 07 23:14:09.359571 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:09.359552 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-p27rw" event={"ID":"f47fd470-2c7a-4ea8-bf73-e3f94320f117","Type":"ContainerStarted","Data":"d67f5fc7f2b7c7859fcf8eabbbc3f77c1adda0e5faeddf08c72d386ab77e2392"} Jul 07 23:14:09.374816 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:09.374777 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-5c847dcfc4-r6r4j" podStartSLOduration=1.689148312 podStartE2EDuration="3.374763938s" totalImagesPullingTime="1.685615626s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 23:14:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 23:14:09.373918634 +0000 UTC m=+66.770920167" watchObservedRunningTime="2026-07-07 23:14:09.374763938 +0000 UTC m=+66.771765450" Jul 07 23:14:09.398735 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:09.398707 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/d557aa86-8e99-41fa-9d29-80b527cd8711-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-svhp9\" (UID: \"d557aa86-8e99-41fa-9d29-80b527cd8711\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-svhp9" Jul 07 23:14:09.398874 ip-10-0-134-140 kubenswrapper[2570]: E0707 23:14:09.398821 2570 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 07 23:14:09.398955 ip-10-0-134-140 kubenswrapper[2570]: E0707 23:14:09.398887 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d557aa86-8e99-41fa-9d29-80b527cd8711-samples-operator-tls podName:d557aa86-8e99-41fa-9d29-80b527cd8711 nodeName:}" failed. No retries permitted until 2026-07-07 23:14:11.398868979 +0000 UTC m=+68.795870469 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/d557aa86-8e99-41fa-9d29-80b527cd8711-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-svhp9" (UID: "d557aa86-8e99-41fa-9d29-80b527cd8711") : secret "samples-operator-tls" not found Jul 07 23:14:09.754181 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:09.754105 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-n9t7q"] Jul 07 23:14:09.779810 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:09.779772 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-n9t7q"] Jul 07 23:14:09.779946 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:09.779899 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-n9t7q" Jul 07 23:14:09.785444 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:09.785419 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-wxtn5\"" Jul 07 23:14:09.786058 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:09.785781 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 07 23:14:09.786058 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:09.785783 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 07 23:14:09.833441 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:09.833410 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:14:09.899193 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:09.899160 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:14:09.903143 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:09.903120 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ccb9c548-3ffa-4ec0-8871-a7b956ff502f-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-n9t7q\" (UID: \"ccb9c548-3ffa-4ec0-8871-a7b956ff502f\") " pod="openshift-insights/insights-runtime-extractor-n9t7q" Jul 07 23:14:09.903259 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:09.903176 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k2v5n\" (UniqueName: \"kubernetes.io/projected/ccb9c548-3ffa-4ec0-8871-a7b956ff502f-kube-api-access-k2v5n\") pod \"insights-runtime-extractor-n9t7q\" (UID: \"ccb9c548-3ffa-4ec0-8871-a7b956ff502f\") " pod="openshift-insights/insights-runtime-extractor-n9t7q" Jul 07 23:14:09.903259 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:09.903245 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/ccb9c548-3ffa-4ec0-8871-a7b956ff502f-crio-socket\") pod \"insights-runtime-extractor-n9t7q\" (UID: \"ccb9c548-3ffa-4ec0-8871-a7b956ff502f\") " pod="openshift-insights/insights-runtime-extractor-n9t7q" Jul 07 23:14:09.903332 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:09.903296 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ccb9c548-3ffa-4ec0-8871-a7b956ff502f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-n9t7q\" (UID: \"ccb9c548-3ffa-4ec0-8871-a7b956ff502f\") " pod="openshift-insights/insights-runtime-extractor-n9t7q" Jul 07 23:14:09.903332 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:09.903319 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/ccb9c548-3ffa-4ec0-8871-a7b956ff502f-data-volume\") pod \"insights-runtime-extractor-n9t7q\" (UID: \"ccb9c548-3ffa-4ec0-8871-a7b956ff502f\") " pod="openshift-insights/insights-runtime-extractor-n9t7q" Jul 07 23:14:10.003912 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:10.003878 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ccb9c548-3ffa-4ec0-8871-a7b956ff502f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-n9t7q\" (UID: \"ccb9c548-3ffa-4ec0-8871-a7b956ff502f\") " pod="openshift-insights/insights-runtime-extractor-n9t7q" Jul 07 23:14:10.003912 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:10.003913 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/ccb9c548-3ffa-4ec0-8871-a7b956ff502f-data-volume\") pod \"insights-runtime-extractor-n9t7q\" (UID: \"ccb9c548-3ffa-4ec0-8871-a7b956ff502f\") " pod="openshift-insights/insights-runtime-extractor-n9t7q" Jul 07 23:14:10.004110 ip-10-0-134-140 kubenswrapper[2570]: E0707 23:14:10.004022 2570 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 07 23:14:10.004110 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:10.004090 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ccb9c548-3ffa-4ec0-8871-a7b956ff502f-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-n9t7q\" (UID: \"ccb9c548-3ffa-4ec0-8871-a7b956ff502f\") " pod="openshift-insights/insights-runtime-extractor-n9t7q" Jul 07 23:14:10.004232 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:10.004151 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k2v5n\" (UniqueName: \"kubernetes.io/projected/ccb9c548-3ffa-4ec0-8871-a7b956ff502f-kube-api-access-k2v5n\") pod \"insights-runtime-extractor-n9t7q\" (UID: \"ccb9c548-3ffa-4ec0-8871-a7b956ff502f\") " pod="openshift-insights/insights-runtime-extractor-n9t7q" Jul 07 23:14:10.004232 ip-10-0-134-140 kubenswrapper[2570]: E0707 23:14:10.004165 2570 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 07 23:14:10.004232 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:10.004195 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/ccb9c548-3ffa-4ec0-8871-a7b956ff502f-data-volume\") pod \"insights-runtime-extractor-n9t7q\" (UID: \"ccb9c548-3ffa-4ec0-8871-a7b956ff502f\") " pod="openshift-insights/insights-runtime-extractor-n9t7q" Jul 07 23:14:10.004232 ip-10-0-134-140 kubenswrapper[2570]: E0707 23:14:10.004201 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ccb9c548-3ffa-4ec0-8871-a7b956ff502f-insights-runtime-extractor-tls podName:ccb9c548-3ffa-4ec0-8871-a7b956ff502f nodeName:}" failed. No retries permitted until 2026-07-07 23:14:10.504180253 +0000 UTC m=+67.901181757 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/ccb9c548-3ffa-4ec0-8871-a7b956ff502f-insights-runtime-extractor-tls") pod "insights-runtime-extractor-n9t7q" (UID: "ccb9c548-3ffa-4ec0-8871-a7b956ff502f") : secret "insights-runtime-extractor-tls" not found Jul 07 23:14:10.004426 ip-10-0-134-140 kubenswrapper[2570]: E0707 23:14:10.004240 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ccb9c548-3ffa-4ec0-8871-a7b956ff502f-kube-rbac-proxy-cm podName:ccb9c548-3ffa-4ec0-8871-a7b956ff502f nodeName:}" failed. No retries permitted until 2026-07-07 23:14:10.504230985 +0000 UTC m=+67.901232477 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/ccb9c548-3ffa-4ec0-8871-a7b956ff502f-kube-rbac-proxy-cm") pod "insights-runtime-extractor-n9t7q" (UID: "ccb9c548-3ffa-4ec0-8871-a7b956ff502f") : configmap "kube-rbac-proxy" not found Jul 07 23:14:10.004426 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:10.004270 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/ccb9c548-3ffa-4ec0-8871-a7b956ff502f-crio-socket\") pod \"insights-runtime-extractor-n9t7q\" (UID: \"ccb9c548-3ffa-4ec0-8871-a7b956ff502f\") " pod="openshift-insights/insights-runtime-extractor-n9t7q" Jul 07 23:14:10.004426 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:10.004339 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/ccb9c548-3ffa-4ec0-8871-a7b956ff502f-crio-socket\") pod \"insights-runtime-extractor-n9t7q\" (UID: \"ccb9c548-3ffa-4ec0-8871-a7b956ff502f\") " pod="openshift-insights/insights-runtime-extractor-n9t7q" Jul 07 23:14:10.016345 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:10.016322 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k2v5n\" (UniqueName: \"kubernetes.io/projected/ccb9c548-3ffa-4ec0-8871-a7b956ff502f-kube-api-access-k2v5n\") pod \"insights-runtime-extractor-n9t7q\" (UID: \"ccb9c548-3ffa-4ec0-8871-a7b956ff502f\") " pod="openshift-insights/insights-runtime-extractor-n9t7q" Jul 07 23:14:10.365813 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:10.365746 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-rwtl7" event={"ID":"01587fde-e08c-441e-a73e-644b4d50b608","Type":"ContainerStarted","Data":"8539e2d0ab0e6600b4fcbbc262bfaefe4b8923b7a8092bc69df812bc9e77a73b"} Jul 07 23:14:10.386380 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:10.386043 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-rwtl7" podStartSLOduration=1.5420248490000001 podStartE2EDuration="3.386021951s" totalImagesPullingTime="1.843997102s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 23:14:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 23:14:10.383919869 +0000 UTC m=+67.780921379" watchObservedRunningTime="2026-07-07 23:14:10.386021951 +0000 UTC m=+67.783023464" Jul 07 23:14:10.419820 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:10.419786 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:14:10.507924 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:10.507883 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/dbb27ecc-1295-4648-851e-0daefb42af23-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-4qp2l\" (UID: \"dbb27ecc-1295-4648-851e-0daefb42af23\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-4qp2l" Jul 07 23:14:10.508111 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:10.507938 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ccb9c548-3ffa-4ec0-8871-a7b956ff502f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-n9t7q\" (UID: \"ccb9c548-3ffa-4ec0-8871-a7b956ff502f\") " pod="openshift-insights/insights-runtime-extractor-n9t7q" Jul 07 23:14:10.508111 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:10.508070 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ccb9c548-3ffa-4ec0-8871-a7b956ff502f-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-n9t7q\" (UID: \"ccb9c548-3ffa-4ec0-8871-a7b956ff502f\") " pod="openshift-insights/insights-runtime-extractor-n9t7q" Jul 07 23:14:10.508111 ip-10-0-134-140 kubenswrapper[2570]: E0707 23:14:10.508078 2570 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 07 23:14:10.508111 ip-10-0-134-140 kubenswrapper[2570]: E0707 23:14:10.508108 2570 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 07 23:14:10.508315 ip-10-0-134-140 kubenswrapper[2570]: E0707 23:14:10.508147 2570 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 07 23:14:10.508315 ip-10-0-134-140 kubenswrapper[2570]: E0707 23:14:10.508168 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ccb9c548-3ffa-4ec0-8871-a7b956ff502f-insights-runtime-extractor-tls podName:ccb9c548-3ffa-4ec0-8871-a7b956ff502f nodeName:}" failed. No retries permitted until 2026-07-07 23:14:11.508150119 +0000 UTC m=+68.905151627 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/ccb9c548-3ffa-4ec0-8871-a7b956ff502f-insights-runtime-extractor-tls") pod "insights-runtime-extractor-n9t7q" (UID: "ccb9c548-3ffa-4ec0-8871-a7b956ff502f") : secret "insights-runtime-extractor-tls" not found Jul 07 23:14:10.508315 ip-10-0-134-140 kubenswrapper[2570]: E0707 23:14:10.508184 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ccb9c548-3ffa-4ec0-8871-a7b956ff502f-kube-rbac-proxy-cm podName:ccb9c548-3ffa-4ec0-8871-a7b956ff502f nodeName:}" failed. No retries permitted until 2026-07-07 23:14:11.508174523 +0000 UTC m=+68.905176028 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/ccb9c548-3ffa-4ec0-8871-a7b956ff502f-kube-rbac-proxy-cm") pod "insights-runtime-extractor-n9t7q" (UID: "ccb9c548-3ffa-4ec0-8871-a7b956ff502f") : configmap "kube-rbac-proxy" not found Jul 07 23:14:10.508315 ip-10-0-134-140 kubenswrapper[2570]: E0707 23:14:10.508198 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dbb27ecc-1295-4648-851e-0daefb42af23-cluster-monitoring-operator-tls podName:dbb27ecc-1295-4648-851e-0daefb42af23 nodeName:}" failed. No retries permitted until 2026-07-07 23:14:14.508191156 +0000 UTC m=+71.905192650 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/dbb27ecc-1295-4648-851e-0daefb42af23-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-4qp2l" (UID: "dbb27ecc-1295-4648-851e-0daefb42af23") : secret "cluster-monitoring-operator-tls" not found Jul 07 23:14:10.519043 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:10.519011 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:14:10.609379 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:10.609334 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/40bbfb8b-2c2b-43c5-8419-3dba3777f888-service-ca-bundle\") pod \"router-default-65d5dfb6cf-84llt\" (UID: \"40bbfb8b-2c2b-43c5-8419-3dba3777f888\") " pod="openshift-ingress/router-default-65d5dfb6cf-84llt" Jul 07 23:14:10.609560 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:10.609488 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/40bbfb8b-2c2b-43c5-8419-3dba3777f888-metrics-certs\") pod \"router-default-65d5dfb6cf-84llt\" (UID: \"40bbfb8b-2c2b-43c5-8419-3dba3777f888\") " pod="openshift-ingress/router-default-65d5dfb6cf-84llt" Jul 07 23:14:10.609644 ip-10-0-134-140 kubenswrapper[2570]: E0707 23:14:10.609622 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/40bbfb8b-2c2b-43c5-8419-3dba3777f888-service-ca-bundle podName:40bbfb8b-2c2b-43c5-8419-3dba3777f888 nodeName:}" failed. No retries permitted until 2026-07-07 23:14:14.609599528 +0000 UTC m=+72.006601023 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/40bbfb8b-2c2b-43c5-8419-3dba3777f888-service-ca-bundle") pod "router-default-65d5dfb6cf-84llt" (UID: "40bbfb8b-2c2b-43c5-8419-3dba3777f888") : configmap references non-existent config key: service-ca.crt Jul 07 23:14:10.609730 ip-10-0-134-140 kubenswrapper[2570]: E0707 23:14:10.609663 2570 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 07 23:14:10.609730 ip-10-0-134-140 kubenswrapper[2570]: E0707 23:14:10.609706 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/40bbfb8b-2c2b-43c5-8419-3dba3777f888-metrics-certs podName:40bbfb8b-2c2b-43c5-8419-3dba3777f888 nodeName:}" failed. No retries permitted until 2026-07-07 23:14:14.609694533 +0000 UTC m=+72.006696023 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/40bbfb8b-2c2b-43c5-8419-3dba3777f888-metrics-certs") pod "router-default-65d5dfb6cf-84llt" (UID: "40bbfb8b-2c2b-43c5-8419-3dba3777f888") : secret "router-metrics-certs-default" not found Jul 07 23:14:10.621612 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:10.621534 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:14:10.941271 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:10.941162 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-qgcpr_c02c0747-b1ae-452c-81aa-5101d7125fc9/dns-node-resolver/0.log" Jul 07 23:14:11.416715 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:11.416666 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/d557aa86-8e99-41fa-9d29-80b527cd8711-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-svhp9\" (UID: \"d557aa86-8e99-41fa-9d29-80b527cd8711\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-svhp9" Jul 07 23:14:11.416906 ip-10-0-134-140 kubenswrapper[2570]: E0707 23:14:11.416874 2570 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 07 23:14:11.416977 ip-10-0-134-140 kubenswrapper[2570]: E0707 23:14:11.416937 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d557aa86-8e99-41fa-9d29-80b527cd8711-samples-operator-tls podName:d557aa86-8e99-41fa-9d29-80b527cd8711 nodeName:}" failed. No retries permitted until 2026-07-07 23:14:15.416918642 +0000 UTC m=+72.813920134 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/d557aa86-8e99-41fa-9d29-80b527cd8711-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-svhp9" (UID: "d557aa86-8e99-41fa-9d29-80b527cd8711") : secret "samples-operator-tls" not found Jul 07 23:14:11.517359 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:11.517322 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ccb9c548-3ffa-4ec0-8871-a7b956ff502f-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-n9t7q\" (UID: \"ccb9c548-3ffa-4ec0-8871-a7b956ff502f\") " pod="openshift-insights/insights-runtime-extractor-n9t7q" Jul 07 23:14:11.517548 ip-10-0-134-140 kubenswrapper[2570]: E0707 23:14:11.517408 2570 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 07 23:14:11.517548 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:11.517444 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ccb9c548-3ffa-4ec0-8871-a7b956ff502f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-n9t7q\" (UID: \"ccb9c548-3ffa-4ec0-8871-a7b956ff502f\") " pod="openshift-insights/insights-runtime-extractor-n9t7q" Jul 07 23:14:11.517548 ip-10-0-134-140 kubenswrapper[2570]: E0707 23:14:11.517493 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ccb9c548-3ffa-4ec0-8871-a7b956ff502f-kube-rbac-proxy-cm podName:ccb9c548-3ffa-4ec0-8871-a7b956ff502f nodeName:}" failed. No retries permitted until 2026-07-07 23:14:13.517458207 +0000 UTC m=+70.914459707 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/ccb9c548-3ffa-4ec0-8871-a7b956ff502f-kube-rbac-proxy-cm") pod "insights-runtime-extractor-n9t7q" (UID: "ccb9c548-3ffa-4ec0-8871-a7b956ff502f") : configmap "kube-rbac-proxy" not found Jul 07 23:14:11.517548 ip-10-0-134-140 kubenswrapper[2570]: E0707 23:14:11.517547 2570 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 07 23:14:11.517790 ip-10-0-134-140 kubenswrapper[2570]: E0707 23:14:11.517582 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ccb9c548-3ffa-4ec0-8871-a7b956ff502f-insights-runtime-extractor-tls podName:ccb9c548-3ffa-4ec0-8871-a7b956ff502f nodeName:}" failed. No retries permitted until 2026-07-07 23:14:13.517573 +0000 UTC m=+70.914574491 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/ccb9c548-3ffa-4ec0-8871-a7b956ff502f-insights-runtime-extractor-tls") pod "insights-runtime-extractor-n9t7q" (UID: "ccb9c548-3ffa-4ec0-8871-a7b956ff502f") : secret "insights-runtime-extractor-tls" not found Jul 07 23:14:11.549592 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:11.549560 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-7z6hc_df04fe1d-eb1a-4711-80ea-72d68116bbca/node-ca/0.log" Jul 07 23:14:12.343098 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:12.343069 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-gbf74_143dd7b8-6ed7-489a-893a-e288926cc1f7/serve-healthcheck-canary/0.log" Jul 07 23:14:13.535680 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:13.535639 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ccb9c548-3ffa-4ec0-8871-a7b956ff502f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-n9t7q\" (UID: \"ccb9c548-3ffa-4ec0-8871-a7b956ff502f\") " pod="openshift-insights/insights-runtime-extractor-n9t7q" Jul 07 23:14:13.536155 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:13.535745 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ccb9c548-3ffa-4ec0-8871-a7b956ff502f-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-n9t7q\" (UID: \"ccb9c548-3ffa-4ec0-8871-a7b956ff502f\") " pod="openshift-insights/insights-runtime-extractor-n9t7q" Jul 07 23:14:13.536155 ip-10-0-134-140 kubenswrapper[2570]: E0707 23:14:13.535788 2570 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 07 23:14:13.536155 ip-10-0-134-140 kubenswrapper[2570]: E0707 23:14:13.535854 2570 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 07 23:14:13.536155 ip-10-0-134-140 kubenswrapper[2570]: E0707 23:14:13.535867 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ccb9c548-3ffa-4ec0-8871-a7b956ff502f-insights-runtime-extractor-tls podName:ccb9c548-3ffa-4ec0-8871-a7b956ff502f nodeName:}" failed. No retries permitted until 2026-07-07 23:14:17.535846285 +0000 UTC m=+74.932847792 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/ccb9c548-3ffa-4ec0-8871-a7b956ff502f-insights-runtime-extractor-tls") pod "insights-runtime-extractor-n9t7q" (UID: "ccb9c548-3ffa-4ec0-8871-a7b956ff502f") : secret "insights-runtime-extractor-tls" not found Jul 07 23:14:13.536155 ip-10-0-134-140 kubenswrapper[2570]: E0707 23:14:13.535891 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ccb9c548-3ffa-4ec0-8871-a7b956ff502f-kube-rbac-proxy-cm podName:ccb9c548-3ffa-4ec0-8871-a7b956ff502f nodeName:}" failed. No retries permitted until 2026-07-07 23:14:17.535880729 +0000 UTC m=+74.932882232 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/ccb9c548-3ffa-4ec0-8871-a7b956ff502f-kube-rbac-proxy-cm") pod "insights-runtime-extractor-n9t7q" (UID: "ccb9c548-3ffa-4ec0-8871-a7b956ff502f") : configmap "kube-rbac-proxy" not found Jul 07 23:14:14.285207 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:14.285167 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:14:14.356819 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:14.356784 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:14:14.546159 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:14.546071 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/dbb27ecc-1295-4648-851e-0daefb42af23-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-4qp2l\" (UID: \"dbb27ecc-1295-4648-851e-0daefb42af23\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-4qp2l" Jul 07 23:14:14.546525 ip-10-0-134-140 kubenswrapper[2570]: E0707 23:14:14.546230 2570 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 07 23:14:14.546525 ip-10-0-134-140 kubenswrapper[2570]: E0707 23:14:14.546296 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dbb27ecc-1295-4648-851e-0daefb42af23-cluster-monitoring-operator-tls podName:dbb27ecc-1295-4648-851e-0daefb42af23 nodeName:}" failed. No retries permitted until 2026-07-07 23:14:22.54628146 +0000 UTC m=+79.943282950 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/dbb27ecc-1295-4648-851e-0daefb42af23-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-4qp2l" (UID: "dbb27ecc-1295-4648-851e-0daefb42af23") : secret "cluster-monitoring-operator-tls" not found Jul 07 23:14:14.647128 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:14.647094 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/40bbfb8b-2c2b-43c5-8419-3dba3777f888-service-ca-bundle\") pod \"router-default-65d5dfb6cf-84llt\" (UID: \"40bbfb8b-2c2b-43c5-8419-3dba3777f888\") " pod="openshift-ingress/router-default-65d5dfb6cf-84llt" Jul 07 23:14:14.647318 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:14.647183 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/40bbfb8b-2c2b-43c5-8419-3dba3777f888-metrics-certs\") pod \"router-default-65d5dfb6cf-84llt\" (UID: \"40bbfb8b-2c2b-43c5-8419-3dba3777f888\") " pod="openshift-ingress/router-default-65d5dfb6cf-84llt" Jul 07 23:14:14.647318 ip-10-0-134-140 kubenswrapper[2570]: E0707 23:14:14.647285 2570 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 07 23:14:14.647318 ip-10-0-134-140 kubenswrapper[2570]: E0707 23:14:14.647296 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/40bbfb8b-2c2b-43c5-8419-3dba3777f888-service-ca-bundle podName:40bbfb8b-2c2b-43c5-8419-3dba3777f888 nodeName:}" failed. No retries permitted until 2026-07-07 23:14:22.64727557 +0000 UTC m=+80.044277061 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/40bbfb8b-2c2b-43c5-8419-3dba3777f888-service-ca-bundle") pod "router-default-65d5dfb6cf-84llt" (UID: "40bbfb8b-2c2b-43c5-8419-3dba3777f888") : configmap references non-existent config key: service-ca.crt Jul 07 23:14:14.647508 ip-10-0-134-140 kubenswrapper[2570]: E0707 23:14:14.647339 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/40bbfb8b-2c2b-43c5-8419-3dba3777f888-metrics-certs podName:40bbfb8b-2c2b-43c5-8419-3dba3777f888 nodeName:}" failed. No retries permitted until 2026-07-07 23:14:22.647327891 +0000 UTC m=+80.044329381 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/40bbfb8b-2c2b-43c5-8419-3dba3777f888-metrics-certs") pod "router-default-65d5dfb6cf-84llt" (UID: "40bbfb8b-2c2b-43c5-8419-3dba3777f888") : secret "router-metrics-certs-default" not found Jul 07 23:14:15.381648 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:15.381611 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-84hdc" event={"ID":"1496ff7f-366b-4cd2-8396-2cdbfed841fa","Type":"ContainerStarted","Data":"52ce3eb12ac18ef931595d5bcb46baf8435741c51b32bde8900ab13f8d7fc09b"} Jul 07 23:14:15.383098 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:15.383077 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-8hvxr_faae2bd8-cbbd-4cfb-bce6-7954977b7ebc/console-operator/0.log" Jul 07 23:14:15.383213 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:15.383113 2570 generic.go:356] "Generic (PLEG): container finished" podID="faae2bd8-cbbd-4cfb-bce6-7954977b7ebc" containerID="302b8e4cfca4fa11f4e3dc2f8536b3d126f0e95e892c9d4fe9db61c08205cd3f" exitCode=255 Jul 07 23:14:15.383213 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:15.383172 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-8hvxr" event={"ID":"faae2bd8-cbbd-4cfb-bce6-7954977b7ebc","Type":"ContainerDied","Data":"302b8e4cfca4fa11f4e3dc2f8536b3d126f0e95e892c9d4fe9db61c08205cd3f"} Jul 07 23:14:15.383351 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:15.383333 2570 scope.go:117] "RemoveContainer" containerID="302b8e4cfca4fa11f4e3dc2f8536b3d126f0e95e892c9d4fe9db61c08205cd3f" Jul 07 23:14:15.384637 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:15.384616 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-p27rw" event={"ID":"f47fd470-2c7a-4ea8-bf73-e3f94320f117","Type":"ContainerStarted","Data":"076a581799f2cb67f7f192d479b6a79ee9877191d7d60d35b3a7ab04547c361a"} Jul 07 23:14:15.384748 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:15.384737 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-p27rw" Jul 07 23:14:15.409157 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:15.407311 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-84hdc" podStartSLOduration=67.19254441 podStartE2EDuration="1m12.407293044s" totalImagesPullingTime="5.214748634s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 23:13:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 23:14:15.406015673 +0000 UTC m=+72.803017186" watchObservedRunningTime="2026-07-07 23:14:15.407293044 +0000 UTC m=+72.804294557" Jul 07 23:14:15.445111 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:15.444998 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-p27rw" podStartSLOduration=67.253816423 podStartE2EDuration="1m12.444984232s" totalImagesPullingTime="5.191167809s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 23:13:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 23:14:15.444533789 +0000 UTC m=+72.841535322" watchObservedRunningTime="2026-07-07 23:14:15.444984232 +0000 UTC m=+72.841985743" Jul 07 23:14:15.455160 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:15.455042 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/d557aa86-8e99-41fa-9d29-80b527cd8711-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-svhp9\" (UID: \"d557aa86-8e99-41fa-9d29-80b527cd8711\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-svhp9" Jul 07 23:14:15.455539 ip-10-0-134-140 kubenswrapper[2570]: E0707 23:14:15.455511 2570 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 07 23:14:15.455753 ip-10-0-134-140 kubenswrapper[2570]: E0707 23:14:15.455606 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d557aa86-8e99-41fa-9d29-80b527cd8711-samples-operator-tls podName:d557aa86-8e99-41fa-9d29-80b527cd8711 nodeName:}" failed. No retries permitted until 2026-07-07 23:14:23.455583856 +0000 UTC m=+80.852585362 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/d557aa86-8e99-41fa-9d29-80b527cd8711-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-svhp9" (UID: "d557aa86-8e99-41fa-9d29-80b527cd8711") : secret "samples-operator-tls" not found Jul 07 23:14:15.976336 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:15.976295 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:14:16.049345 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:16.049307 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:14:16.389802 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:16.389773 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-8hvxr_faae2bd8-cbbd-4cfb-bce6-7954977b7ebc/console-operator/1.log" Jul 07 23:14:16.390139 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:16.390123 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-8hvxr_faae2bd8-cbbd-4cfb-bce6-7954977b7ebc/console-operator/0.log" Jul 07 23:14:16.390246 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:16.390161 2570 generic.go:356] "Generic (PLEG): container finished" podID="faae2bd8-cbbd-4cfb-bce6-7954977b7ebc" containerID="528b774aea7e273a5c12ee6862fd3cf2ad5f81d7973fa54855db85179bd685d3" exitCode=255 Jul 07 23:14:16.390303 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:16.390250 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-8hvxr" event={"ID":"faae2bd8-cbbd-4cfb-bce6-7954977b7ebc","Type":"ContainerDied","Data":"528b774aea7e273a5c12ee6862fd3cf2ad5f81d7973fa54855db85179bd685d3"} Jul 07 23:14:16.390359 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:16.390327 2570 scope.go:117] "RemoveContainer" containerID="302b8e4cfca4fa11f4e3dc2f8536b3d126f0e95e892c9d4fe9db61c08205cd3f" Jul 07 23:14:16.390565 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:16.390541 2570 scope.go:117] "RemoveContainer" containerID="528b774aea7e273a5c12ee6862fd3cf2ad5f81d7973fa54855db85179bd685d3" Jul 07 23:14:16.390781 ip-10-0-134-140 kubenswrapper[2570]: E0707 23:14:16.390740 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-68b757fb67-8hvxr_openshift-console-operator(faae2bd8-cbbd-4cfb-bce6-7954977b7ebc)\"" pod="openshift-console-operator/console-operator-68b757fb67-8hvxr" podUID="faae2bd8-cbbd-4cfb-bce6-7954977b7ebc" Jul 07 23:14:16.395700 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:16.395676 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:14:16.498393 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:16.498366 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:14:17.394122 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:17.394097 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-8hvxr_faae2bd8-cbbd-4cfb-bce6-7954977b7ebc/console-operator/1.log" Jul 07 23:14:17.394599 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:17.394524 2570 scope.go:117] "RemoveContainer" containerID="528b774aea7e273a5c12ee6862fd3cf2ad5f81d7973fa54855db85179bd685d3" Jul 07 23:14:17.394724 ip-10-0-134-140 kubenswrapper[2570]: E0707 23:14:17.394703 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-68b757fb67-8hvxr_openshift-console-operator(faae2bd8-cbbd-4cfb-bce6-7954977b7ebc)\"" pod="openshift-console-operator/console-operator-68b757fb67-8hvxr" podUID="faae2bd8-cbbd-4cfb-bce6-7954977b7ebc" Jul 07 23:14:17.575182 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:17.575148 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ccb9c548-3ffa-4ec0-8871-a7b956ff502f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-n9t7q\" (UID: \"ccb9c548-3ffa-4ec0-8871-a7b956ff502f\") " pod="openshift-insights/insights-runtime-extractor-n9t7q" Jul 07 23:14:17.575380 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:17.575260 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ccb9c548-3ffa-4ec0-8871-a7b956ff502f-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-n9t7q\" (UID: \"ccb9c548-3ffa-4ec0-8871-a7b956ff502f\") " pod="openshift-insights/insights-runtime-extractor-n9t7q" Jul 07 23:14:17.575380 ip-10-0-134-140 kubenswrapper[2570]: E0707 23:14:17.575294 2570 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 07 23:14:17.575380 ip-10-0-134-140 kubenswrapper[2570]: E0707 23:14:17.575367 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ccb9c548-3ffa-4ec0-8871-a7b956ff502f-insights-runtime-extractor-tls podName:ccb9c548-3ffa-4ec0-8871-a7b956ff502f nodeName:}" failed. No retries permitted until 2026-07-07 23:14:25.575351479 +0000 UTC m=+82.972352972 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/ccb9c548-3ffa-4ec0-8871-a7b956ff502f-insights-runtime-extractor-tls") pod "insights-runtime-extractor-n9t7q" (UID: "ccb9c548-3ffa-4ec0-8871-a7b956ff502f") : secret "insights-runtime-extractor-tls" not found Jul 07 23:14:17.575380 ip-10-0-134-140 kubenswrapper[2570]: E0707 23:14:17.575371 2570 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 07 23:14:17.575566 ip-10-0-134-140 kubenswrapper[2570]: E0707 23:14:17.575434 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ccb9c548-3ffa-4ec0-8871-a7b956ff502f-kube-rbac-proxy-cm podName:ccb9c548-3ffa-4ec0-8871-a7b956ff502f nodeName:}" failed. No retries permitted until 2026-07-07 23:14:25.575416786 +0000 UTC m=+82.972418278 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/ccb9c548-3ffa-4ec0-8871-a7b956ff502f-kube-rbac-proxy-cm") pod "insights-runtime-extractor-n9t7q" (UID: "ccb9c548-3ffa-4ec0-8871-a7b956ff502f") : configmap "kube-rbac-proxy" not found Jul 07 23:14:18.460566 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:18.460532 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:14:18.561766 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:18.561732 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:14:19.064095 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:19.064049 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-68b757fb67-8hvxr" Jul 07 23:14:19.064270 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:19.064134 2570 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-68b757fb67-8hvxr" Jul 07 23:14:19.064442 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:19.064430 2570 scope.go:117] "RemoveContainer" containerID="528b774aea7e273a5c12ee6862fd3cf2ad5f81d7973fa54855db85179bd685d3" Jul 07 23:14:19.064626 ip-10-0-134-140 kubenswrapper[2570]: E0707 23:14:19.064606 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-68b757fb67-8hvxr_openshift-console-operator(faae2bd8-cbbd-4cfb-bce6-7954977b7ebc)\"" pod="openshift-console-operator/console-operator-68b757fb67-8hvxr" podUID="faae2bd8-cbbd-4cfb-bce6-7954977b7ebc" Jul 07 23:14:19.953073 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:19.953040 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:14:20.041799 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:20.041759 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:14:20.100446 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:20.100417 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:14:21.614461 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:21.614428 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:14:21.713102 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:21.713072 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:14:22.334678 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:22.334648 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:14:22.397059 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:22.397026 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:14:22.619298 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:22.619211 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/dbb27ecc-1295-4648-851e-0daefb42af23-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-4qp2l\" (UID: \"dbb27ecc-1295-4648-851e-0daefb42af23\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-4qp2l" Jul 07 23:14:22.619691 ip-10-0-134-140 kubenswrapper[2570]: E0707 23:14:22.619328 2570 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 07 23:14:22.619691 ip-10-0-134-140 kubenswrapper[2570]: E0707 23:14:22.619382 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dbb27ecc-1295-4648-851e-0daefb42af23-cluster-monitoring-operator-tls podName:dbb27ecc-1295-4648-851e-0daefb42af23 nodeName:}" failed. No retries permitted until 2026-07-07 23:14:38.619367971 +0000 UTC m=+96.016369461 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/dbb27ecc-1295-4648-851e-0daefb42af23-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-4qp2l" (UID: "dbb27ecc-1295-4648-851e-0daefb42af23") : secret "cluster-monitoring-operator-tls" not found Jul 07 23:14:22.720412 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:22.720380 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/40bbfb8b-2c2b-43c5-8419-3dba3777f888-service-ca-bundle\") pod \"router-default-65d5dfb6cf-84llt\" (UID: \"40bbfb8b-2c2b-43c5-8419-3dba3777f888\") " pod="openshift-ingress/router-default-65d5dfb6cf-84llt" Jul 07 23:14:22.720560 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:22.720497 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/40bbfb8b-2c2b-43c5-8419-3dba3777f888-metrics-certs\") pod \"router-default-65d5dfb6cf-84llt\" (UID: \"40bbfb8b-2c2b-43c5-8419-3dba3777f888\") " pod="openshift-ingress/router-default-65d5dfb6cf-84llt" Jul 07 23:14:22.720597 ip-10-0-134-140 kubenswrapper[2570]: E0707 23:14:22.720566 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/40bbfb8b-2c2b-43c5-8419-3dba3777f888-service-ca-bundle podName:40bbfb8b-2c2b-43c5-8419-3dba3777f888 nodeName:}" failed. No retries permitted until 2026-07-07 23:14:38.720549986 +0000 UTC m=+96.117551476 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/40bbfb8b-2c2b-43c5-8419-3dba3777f888-service-ca-bundle") pod "router-default-65d5dfb6cf-84llt" (UID: "40bbfb8b-2c2b-43c5-8419-3dba3777f888") : configmap references non-existent config key: service-ca.crt Jul 07 23:14:22.720638 ip-10-0-134-140 kubenswrapper[2570]: E0707 23:14:22.720605 2570 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 07 23:14:22.720673 ip-10-0-134-140 kubenswrapper[2570]: E0707 23:14:22.720666 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/40bbfb8b-2c2b-43c5-8419-3dba3777f888-metrics-certs podName:40bbfb8b-2c2b-43c5-8419-3dba3777f888 nodeName:}" failed. No retries permitted until 2026-07-07 23:14:38.720651606 +0000 UTC m=+96.117653096 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/40bbfb8b-2c2b-43c5-8419-3dba3777f888-metrics-certs") pod "router-default-65d5dfb6cf-84llt" (UID: "40bbfb8b-2c2b-43c5-8419-3dba3777f888") : secret "router-metrics-certs-default" not found Jul 07 23:14:22.955937 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:22.955856 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:14:23.393654 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:23.393604 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:14:23.451380 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:23.451350 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:14:23.529308 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:23.529275 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/d557aa86-8e99-41fa-9d29-80b527cd8711-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-svhp9\" (UID: \"d557aa86-8e99-41fa-9d29-80b527cd8711\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-svhp9" Jul 07 23:14:23.529491 ip-10-0-134-140 kubenswrapper[2570]: E0707 23:14:23.529414 2570 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 07 23:14:23.529535 ip-10-0-134-140 kubenswrapper[2570]: E0707 23:14:23.529489 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d557aa86-8e99-41fa-9d29-80b527cd8711-samples-operator-tls podName:d557aa86-8e99-41fa-9d29-80b527cd8711 nodeName:}" failed. No retries permitted until 2026-07-07 23:14:39.529454336 +0000 UTC m=+96.926455829 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/d557aa86-8e99-41fa-9d29-80b527cd8711-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-svhp9" (UID: "d557aa86-8e99-41fa-9d29-80b527cd8711") : secret "samples-operator-tls" not found Jul 07 23:14:23.977551 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:23.977516 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:14:24.045187 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:24.045153 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:14:25.255052 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:25.254927 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:14:25.646569 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:25.646522 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ccb9c548-3ffa-4ec0-8871-a7b956ff502f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-n9t7q\" (UID: \"ccb9c548-3ffa-4ec0-8871-a7b956ff502f\") " pod="openshift-insights/insights-runtime-extractor-n9t7q" Jul 07 23:14:25.646754 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:25.646645 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ccb9c548-3ffa-4ec0-8871-a7b956ff502f-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-n9t7q\" (UID: \"ccb9c548-3ffa-4ec0-8871-a7b956ff502f\") " pod="openshift-insights/insights-runtime-extractor-n9t7q" Jul 07 23:14:25.646754 ip-10-0-134-140 kubenswrapper[2570]: E0707 23:14:25.646687 2570 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 07 23:14:25.646824 ip-10-0-134-140 kubenswrapper[2570]: E0707 23:14:25.646767 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ccb9c548-3ffa-4ec0-8871-a7b956ff502f-insights-runtime-extractor-tls podName:ccb9c548-3ffa-4ec0-8871-a7b956ff502f nodeName:}" failed. No retries permitted until 2026-07-07 23:14:41.64674675 +0000 UTC m=+99.043748251 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/ccb9c548-3ffa-4ec0-8871-a7b956ff502f-insights-runtime-extractor-tls") pod "insights-runtime-extractor-n9t7q" (UID: "ccb9c548-3ffa-4ec0-8871-a7b956ff502f") : secret "insights-runtime-extractor-tls" not found Jul 07 23:14:25.647213 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:25.647194 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ccb9c548-3ffa-4ec0-8871-a7b956ff502f-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-n9t7q\" (UID: \"ccb9c548-3ffa-4ec0-8871-a7b956ff502f\") " pod="openshift-insights/insights-runtime-extractor-n9t7q" Jul 07 23:14:26.183795 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:26.183763 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:14:26.405125 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:26.405083 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:14:26.570041 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:26.570013 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:14:26.727090 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:26.727057 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:14:26.894630 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:26.894533 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:14:26.995399 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:26.995372 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:14:27.184217 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:27.184128 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:14:27.285678 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:27.285650 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:14:27.867170 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:27.867134 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:14:28.358715 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:28.358680 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:14:28.665078 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:28.664989 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:14:28.878344 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:28.878310 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:14:29.274589 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:29.274546 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:14:29.467046 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:29.467003 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:14:29.632598 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:29.632522 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:14:29.733192 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:29.733154 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:14:29.862880 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:29.862839 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:14:29.922642 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:29.922563 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:14:30.269314 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:30.269237 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:14:30.442444 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:30.442407 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:14:31.060483 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:31.060435 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:14:31.121288 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:31.121250 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:14:31.266022 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:31.265991 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:14:31.685319 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:31.685277 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:14:31.869585 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:31.869545 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:14:32.465057 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:32.465012 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:14:32.638822 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:32.638782 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:14:33.063940 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:33.063905 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:14:33.264500 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:33.264456 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:14:33.462043 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:33.461964 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:14:33.870739 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:33.870706 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:14:34.091674 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:34.091634 2570 scope.go:117] "RemoveContainer" containerID="528b774aea7e273a5c12ee6862fd3cf2ad5f81d7973fa54855db85179bd685d3" Jul 07 23:14:34.444493 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:34.444399 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-8hvxr_faae2bd8-cbbd-4cfb-bce6-7954977b7ebc/console-operator/2.log" Jul 07 23:14:34.444830 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:34.444815 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-8hvxr_faae2bd8-cbbd-4cfb-bce6-7954977b7ebc/console-operator/1.log" Jul 07 23:14:34.444880 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:34.444847 2570 generic.go:356] "Generic (PLEG): container finished" podID="faae2bd8-cbbd-4cfb-bce6-7954977b7ebc" containerID="04852c942546ac352acd95c7eaa0b4fbe754266a3a8bf8bd0b38e564363b903b" exitCode=255 Jul 07 23:14:34.444925 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:34.444911 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-8hvxr" event={"ID":"faae2bd8-cbbd-4cfb-bce6-7954977b7ebc","Type":"ContainerDied","Data":"04852c942546ac352acd95c7eaa0b4fbe754266a3a8bf8bd0b38e564363b903b"} Jul 07 23:14:34.444972 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:34.444960 2570 scope.go:117] "RemoveContainer" containerID="528b774aea7e273a5c12ee6862fd3cf2ad5f81d7973fa54855db85179bd685d3" Jul 07 23:14:34.445262 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:34.445237 2570 scope.go:117] "RemoveContainer" containerID="04852c942546ac352acd95c7eaa0b4fbe754266a3a8bf8bd0b38e564363b903b" Jul 07 23:14:34.445446 ip-10-0-134-140 kubenswrapper[2570]: E0707 23:14:34.445424 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-68b757fb67-8hvxr_openshift-console-operator(faae2bd8-cbbd-4cfb-bce6-7954977b7ebc)\"" pod="openshift-console-operator/console-operator-68b757fb67-8hvxr" podUID="faae2bd8-cbbd-4cfb-bce6-7954977b7ebc" Jul 07 23:14:34.675902 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:34.675870 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:14:34.861930 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:34.861893 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:14:35.055142 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:35.055100 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:14:35.451271 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:35.451245 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-8hvxr_faae2bd8-cbbd-4cfb-bce6-7954977b7ebc/console-operator/2.log" Jul 07 23:14:35.470968 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:35.470941 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:14:36.268794 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:36.268751 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:14:36.475845 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:36.475810 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:14:36.664539 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:36.664503 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:14:36.864639 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:36.864603 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:14:36.964222 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:36.964134 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:14:38.654598 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:38.654556 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/dbb27ecc-1295-4648-851e-0daefb42af23-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-4qp2l\" (UID: \"dbb27ecc-1295-4648-851e-0daefb42af23\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-4qp2l" Jul 07 23:14:38.656885 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:38.656868 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/dbb27ecc-1295-4648-851e-0daefb42af23-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-4qp2l\" (UID: \"dbb27ecc-1295-4648-851e-0daefb42af23\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-4qp2l" Jul 07 23:14:38.755417 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:38.755380 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/40bbfb8b-2c2b-43c5-8419-3dba3777f888-service-ca-bundle\") pod \"router-default-65d5dfb6cf-84llt\" (UID: \"40bbfb8b-2c2b-43c5-8419-3dba3777f888\") " pod="openshift-ingress/router-default-65d5dfb6cf-84llt" Jul 07 23:14:38.755602 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:38.755501 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/40bbfb8b-2c2b-43c5-8419-3dba3777f888-metrics-certs\") pod \"router-default-65d5dfb6cf-84llt\" (UID: \"40bbfb8b-2c2b-43c5-8419-3dba3777f888\") " pod="openshift-ingress/router-default-65d5dfb6cf-84llt" Jul 07 23:14:38.756045 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:38.756022 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/40bbfb8b-2c2b-43c5-8419-3dba3777f888-service-ca-bundle\") pod \"router-default-65d5dfb6cf-84llt\" (UID: \"40bbfb8b-2c2b-43c5-8419-3dba3777f888\") " pod="openshift-ingress/router-default-65d5dfb6cf-84llt" Jul 07 23:14:38.757728 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:38.757705 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/40bbfb8b-2c2b-43c5-8419-3dba3777f888-metrics-certs\") pod \"router-default-65d5dfb6cf-84llt\" (UID: \"40bbfb8b-2c2b-43c5-8419-3dba3777f888\") " pod="openshift-ingress/router-default-65d5dfb6cf-84llt" Jul 07 23:14:38.886241 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:38.886207 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-4qp2l" Jul 07 23:14:38.986914 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:38.986886 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-65d5dfb6cf-84llt" Jul 07 23:14:39.015727 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:39.015696 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-64786d9cbb-4qp2l"] Jul 07 23:14:39.018257 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:14:39.018230 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddbb27ecc_1295_4648_851e_0daefb42af23.slice/crio-c60c5dc3089600ba532bcc74b77045b59f8d9bd30908032565924dbd7daee919 WatchSource:0}: Error finding container c60c5dc3089600ba532bcc74b77045b59f8d9bd30908032565924dbd7daee919: Status 404 returned error can't find the container with id c60c5dc3089600ba532bcc74b77045b59f8d9bd30908032565924dbd7daee919 Jul 07 23:14:39.063636 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:39.063615 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-68b757fb67-8hvxr" Jul 07 23:14:39.063729 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:39.063668 2570 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-68b757fb67-8hvxr" Jul 07 23:14:39.064342 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:39.064314 2570 scope.go:117] "RemoveContainer" containerID="04852c942546ac352acd95c7eaa0b4fbe754266a3a8bf8bd0b38e564363b903b" Jul 07 23:14:39.064582 ip-10-0-134-140 kubenswrapper[2570]: E0707 23:14:39.064554 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-68b757fb67-8hvxr_openshift-console-operator(faae2bd8-cbbd-4cfb-bce6-7954977b7ebc)\"" pod="openshift-console-operator/console-operator-68b757fb67-8hvxr" podUID="faae2bd8-cbbd-4cfb-bce6-7954977b7ebc" Jul 07 23:14:39.118255 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:39.118216 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-65d5dfb6cf-84llt"] Jul 07 23:14:39.120865 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:14:39.120841 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod40bbfb8b_2c2b_43c5_8419_3dba3777f888.slice/crio-6d7e7a166518d6cba1107329ed8bca69d587d7816bcc8c20f51cccfa3da79f1d WatchSource:0}: Error finding container 6d7e7a166518d6cba1107329ed8bca69d587d7816bcc8c20f51cccfa3da79f1d: Status 404 returned error can't find the container with id 6d7e7a166518d6cba1107329ed8bca69d587d7816bcc8c20f51cccfa3da79f1d Jul 07 23:14:39.464425 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:39.464385 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-4qp2l" event={"ID":"dbb27ecc-1295-4648-851e-0daefb42af23","Type":"ContainerStarted","Data":"c60c5dc3089600ba532bcc74b77045b59f8d9bd30908032565924dbd7daee919"} Jul 07 23:14:39.465886 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:39.465862 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-65d5dfb6cf-84llt" event={"ID":"40bbfb8b-2c2b-43c5-8419-3dba3777f888","Type":"ContainerStarted","Data":"458e5916a20da2807498f781f87dd354b34ed7bef812ee09cd3220eac4aa37a1"} Jul 07 23:14:39.466013 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:39.465904 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-65d5dfb6cf-84llt" event={"ID":"40bbfb8b-2c2b-43c5-8419-3dba3777f888","Type":"ContainerStarted","Data":"6d7e7a166518d6cba1107329ed8bca69d587d7816bcc8c20f51cccfa3da79f1d"} Jul 07 23:14:39.494930 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:39.494889 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-65d5dfb6cf-84llt" podStartSLOduration=33.494876182 podStartE2EDuration="33.494876182s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 23:14:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 23:14:39.493097706 +0000 UTC m=+96.890099217" watchObservedRunningTime="2026-07-07 23:14:39.494876182 +0000 UTC m=+96.891877694" Jul 07 23:14:39.560768 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:39.560736 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/d557aa86-8e99-41fa-9d29-80b527cd8711-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-svhp9\" (UID: \"d557aa86-8e99-41fa-9d29-80b527cd8711\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-svhp9" Jul 07 23:14:39.563452 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:39.563428 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/d557aa86-8e99-41fa-9d29-80b527cd8711-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-svhp9\" (UID: \"d557aa86-8e99-41fa-9d29-80b527cd8711\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-svhp9" Jul 07 23:14:39.570314 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:39.570296 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-svhp9" Jul 07 23:14:39.700543 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:39.700508 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-svhp9"] Jul 07 23:14:39.987389 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:39.987309 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-65d5dfb6cf-84llt" Jul 07 23:14:39.990183 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:39.990155 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-65d5dfb6cf-84llt" Jul 07 23:14:40.470791 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:40.470748 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-svhp9" event={"ID":"d557aa86-8e99-41fa-9d29-80b527cd8711","Type":"ContainerStarted","Data":"a81802ce344e4b436d3bf201d002e5cb6e4887eb2d0a061cedd06e210b4b2a36"} Jul 07 23:14:40.471059 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:40.471032 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-65d5dfb6cf-84llt" Jul 07 23:14:40.472453 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:40.472432 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-65d5dfb6cf-84llt" Jul 07 23:14:41.047419 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:41.047385 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:14:41.148290 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:41.147983 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:14:41.246826 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:41.246794 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:14:41.375451 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:41.375371 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/cf5100ad-da85-4da6-b099-d10259a611d2-registry-tls\") pod \"image-registry-5668757489-j9qbl\" (UID: \"cf5100ad-da85-4da6-b099-d10259a611d2\") " pod="openshift-image-registry/image-registry-5668757489-j9qbl" Jul 07 23:14:41.375451 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:41.375417 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/bd5aafbd-ed3c-46fa-bc87-661f74ba728a-metrics-tls\") pod \"dns-default-4th55\" (UID: \"bd5aafbd-ed3c-46fa-bc87-661f74ba728a\") " pod="openshift-dns/dns-default-4th55" Jul 07 23:14:41.377781 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:41.377749 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/bd5aafbd-ed3c-46fa-bc87-661f74ba728a-metrics-tls\") pod \"dns-default-4th55\" (UID: \"bd5aafbd-ed3c-46fa-bc87-661f74ba728a\") " pod="openshift-dns/dns-default-4th55" Jul 07 23:14:41.377911 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:41.377759 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/cf5100ad-da85-4da6-b099-d10259a611d2-registry-tls\") pod \"image-registry-5668757489-j9qbl\" (UID: \"cf5100ad-da85-4da6-b099-d10259a611d2\") " pod="openshift-image-registry/image-registry-5668757489-j9qbl" Jul 07 23:14:41.424236 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:41.424206 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:14:41.475339 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:41.475305 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-4qp2l" event={"ID":"dbb27ecc-1295-4648-851e-0daefb42af23","Type":"ContainerStarted","Data":"d49c990da32bf392eb9aaa824898c6dc67a678b0f7976f3179fee537a73b11f2"} Jul 07 23:14:41.493876 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:41.493833 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-4qp2l" podStartSLOduration=33.859850458 podStartE2EDuration="35.493819307s" totalImagesPullingTime="1.633968849s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 23:14:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 23:14:41.493079298 +0000 UTC m=+98.890080830" watchObservedRunningTime="2026-07-07 23:14:41.493819307 +0000 UTC m=+98.890820818" Jul 07 23:14:41.613908 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:41.613875 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-wgd2s\"" Jul 07 23:14:41.622139 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:41.622112 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5668757489-j9qbl" Jul 07 23:14:41.626314 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:41.626257 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-94qrf\"" Jul 07 23:14:41.633679 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:41.633660 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-4th55" Jul 07 23:14:41.677106 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:41.677074 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ccb9c548-3ffa-4ec0-8871-a7b956ff502f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-n9t7q\" (UID: \"ccb9c548-3ffa-4ec0-8871-a7b956ff502f\") " pod="openshift-insights/insights-runtime-extractor-n9t7q" Jul 07 23:14:41.679591 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:41.679563 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ccb9c548-3ffa-4ec0-8871-a7b956ff502f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-n9t7q\" (UID: \"ccb9c548-3ffa-4ec0-8871-a7b956ff502f\") " pod="openshift-insights/insights-runtime-extractor-n9t7q" Jul 07 23:14:41.888525 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:41.888423 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-n9t7q" Jul 07 23:14:42.182336 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:42.182309 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-5668757489-j9qbl"] Jul 07 23:14:42.185286 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:14:42.185261 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcf5100ad_da85_4da6_b099_d10259a611d2.slice/crio-b599ca3139a8150dcb36dd3bf4faf09973ab24a4f056079a870759ec0f49eb6f WatchSource:0}: Error finding container b599ca3139a8150dcb36dd3bf4faf09973ab24a4f056079a870759ec0f49eb6f: Status 404 returned error can't find the container with id b599ca3139a8150dcb36dd3bf4faf09973ab24a4f056079a870759ec0f49eb6f Jul 07 23:14:42.408673 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:42.408546 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-n9t7q"] Jul 07 23:14:42.411085 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:42.411055 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-4th55"] Jul 07 23:14:42.411733 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:14:42.411706 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podccb9c548_3ffa_4ec0_8871_a7b956ff502f.slice/crio-c57ae5dcd5d47e34811118c26c4972ae0abfca97f015dbbdbae8315a24357fca WatchSource:0}: Error finding container c57ae5dcd5d47e34811118c26c4972ae0abfca97f015dbbdbae8315a24357fca: Status 404 returned error can't find the container with id c57ae5dcd5d47e34811118c26c4972ae0abfca97f015dbbdbae8315a24357fca Jul 07 23:14:42.413882 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:14:42.413849 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbd5aafbd_ed3c_46fa_bc87_661f74ba728a.slice/crio-b4758ee17ba477b725d3bf8a0e01202cbd10b7a660d82aa40b76444a8792eee7 WatchSource:0}: Error finding container b4758ee17ba477b725d3bf8a0e01202cbd10b7a660d82aa40b76444a8792eee7: Status 404 returned error can't find the container with id b4758ee17ba477b725d3bf8a0e01202cbd10b7a660d82aa40b76444a8792eee7 Jul 07 23:14:42.480346 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:42.480315 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5668757489-j9qbl" event={"ID":"cf5100ad-da85-4da6-b099-d10259a611d2","Type":"ContainerStarted","Data":"cb9f7ccb7ab0b1c9f3d6902021225aea29675a4cfa0fd26596c46b8514398e05"} Jul 07 23:14:42.480539 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:42.480364 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5668757489-j9qbl" event={"ID":"cf5100ad-da85-4da6-b099-d10259a611d2","Type":"ContainerStarted","Data":"b599ca3139a8150dcb36dd3bf4faf09973ab24a4f056079a870759ec0f49eb6f"} Jul 07 23:14:42.480539 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:42.480497 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-5668757489-j9qbl" Jul 07 23:14:42.482312 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:42.482284 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-svhp9" event={"ID":"d557aa86-8e99-41fa-9d29-80b527cd8711","Type":"ContainerStarted","Data":"16bb171bc7ab1f2e7d2eab7860181340feddf1ce09de0f6e773d9e5c04ccbba4"} Jul 07 23:14:42.482430 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:42.482342 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-svhp9" event={"ID":"d557aa86-8e99-41fa-9d29-80b527cd8711","Type":"ContainerStarted","Data":"afb7d935e89726b85d97112645b79de588e11128153475bbf62abf3d23e932e3"} Jul 07 23:14:42.483927 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:42.483426 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-n9t7q" event={"ID":"ccb9c548-3ffa-4ec0-8871-a7b956ff502f","Type":"ContainerStarted","Data":"c57ae5dcd5d47e34811118c26c4972ae0abfca97f015dbbdbae8315a24357fca"} Jul 07 23:14:42.484919 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:42.484888 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-4th55" event={"ID":"bd5aafbd-ed3c-46fa-bc87-661f74ba728a","Type":"ContainerStarted","Data":"b4758ee17ba477b725d3bf8a0e01202cbd10b7a660d82aa40b76444a8792eee7"} Jul 07 23:14:42.503476 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:42.503417 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-5668757489-j9qbl" podStartSLOduration=77.50340423 podStartE2EDuration="1m17.50340423s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 23:13:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 23:14:42.50248013 +0000 UTC m=+99.899481656" watchObservedRunningTime="2026-07-07 23:14:42.50340423 +0000 UTC m=+99.900405748" Jul 07 23:14:42.524068 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:42.524023 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-svhp9" podStartSLOduration=33.212807036 podStartE2EDuration="35.524007067s" totalImagesPullingTime="2.311200031s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 23:14:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 23:14:42.523826986 +0000 UTC m=+99.920828522" watchObservedRunningTime="2026-07-07 23:14:42.524007067 +0000 UTC m=+99.921008581" Jul 07 23:14:44.639431 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:44.639391 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:14:44.698562 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:44.698518 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:14:45.863392 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:45.863349 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-7lwcs"] Jul 07 23:14:45.875261 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:45.875239 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-6bffdf9778-qqf65"] Jul 07 23:14:45.875404 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:45.875351 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-7lwcs" Jul 07 23:14:45.878443 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:45.878424 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-7jdv5\"" Jul 07 23:14:45.878443 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:45.878433 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\"" Jul 07 23:14:45.887817 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:45.887794 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6bffdf9778-qqf65"] Jul 07 23:14:45.887970 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:45.887821 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-7lwcs"] Jul 07 23:14:45.887970 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:45.887893 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6bffdf9778-qqf65" Jul 07 23:14:45.890812 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:45.890794 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-mgznt\"" Jul 07 23:14:45.890919 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:45.890860 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\"" Jul 07 23:14:45.891455 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:45.891439 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\"" Jul 07 23:14:45.895185 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:45.895161 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:14:45.911750 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:45.911726 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/51966fcb-7a64-464e-ab66-1d049570bcd3-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-qqf65\" (UID: \"51966fcb-7a64-464e-ab66-1d049570bcd3\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-qqf65" Jul 07 23:14:45.911837 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:45.911796 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/51966fcb-7a64-464e-ab66-1d049570bcd3-nginx-conf\") pod \"networking-console-plugin-6bffdf9778-qqf65\" (UID: \"51966fcb-7a64-464e-ab66-1d049570bcd3\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-qqf65" Jul 07 23:14:45.911837 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:45.911819 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/1b5943d1-f098-4857-ae9f-158dcbcb7b79-tls-certificates\") pod \"prometheus-operator-admission-webhook-d559d66d5-7lwcs\" (UID: \"1b5943d1-f098-4857-ae9f-158dcbcb7b79\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-7lwcs" Jul 07 23:14:45.992436 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:45.992356 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:14:46.013147 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:46.013122 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/51966fcb-7a64-464e-ab66-1d049570bcd3-nginx-conf\") pod \"networking-console-plugin-6bffdf9778-qqf65\" (UID: \"51966fcb-7a64-464e-ab66-1d049570bcd3\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-qqf65" Jul 07 23:14:46.013328 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:46.013156 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/1b5943d1-f098-4857-ae9f-158dcbcb7b79-tls-certificates\") pod \"prometheus-operator-admission-webhook-d559d66d5-7lwcs\" (UID: \"1b5943d1-f098-4857-ae9f-158dcbcb7b79\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-7lwcs" Jul 07 23:14:46.013328 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:46.013186 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/51966fcb-7a64-464e-ab66-1d049570bcd3-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-qqf65\" (UID: \"51966fcb-7a64-464e-ab66-1d049570bcd3\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-qqf65" Jul 07 23:14:46.013897 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:46.013868 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/51966fcb-7a64-464e-ab66-1d049570bcd3-nginx-conf\") pod \"networking-console-plugin-6bffdf9778-qqf65\" (UID: \"51966fcb-7a64-464e-ab66-1d049570bcd3\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-qqf65" Jul 07 23:14:46.016033 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:46.016009 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/1b5943d1-f098-4857-ae9f-158dcbcb7b79-tls-certificates\") pod \"prometheus-operator-admission-webhook-d559d66d5-7lwcs\" (UID: \"1b5943d1-f098-4857-ae9f-158dcbcb7b79\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-7lwcs" Jul 07 23:14:46.016153 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:46.016108 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/51966fcb-7a64-464e-ab66-1d049570bcd3-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-qqf65\" (UID: \"51966fcb-7a64-464e-ab66-1d049570bcd3\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-qqf65" Jul 07 23:14:46.124327 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:46.124284 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:14:46.185495 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:46.185452 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-7lwcs" Jul 07 23:14:46.196601 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:46.196570 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6bffdf9778-qqf65" Jul 07 23:14:46.217383 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:46.217349 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:14:46.392880 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:46.392847 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-p27rw" Jul 07 23:14:47.087795 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:47.087763 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-7lwcs"] Jul 07 23:14:47.090610 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:14:47.090580 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1b5943d1_f098_4857_ae9f_158dcbcb7b79.slice/crio-26fb68bc9105fc07ab29d4e42f854df911f30cdf8e67aeacb92f345f734f5cfa WatchSource:0}: Error finding container 26fb68bc9105fc07ab29d4e42f854df911f30cdf8e67aeacb92f345f734f5cfa: Status 404 returned error can't find the container with id 26fb68bc9105fc07ab29d4e42f854df911f30cdf8e67aeacb92f345f734f5cfa Jul 07 23:14:47.108632 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:47.108610 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6bffdf9778-qqf65"] Jul 07 23:14:47.520574 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:47.520534 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-n9t7q" event={"ID":"ccb9c548-3ffa-4ec0-8871-a7b956ff502f","Type":"ContainerStarted","Data":"cb82a404189a9b0e1023bba5b4c2dda1173e979299618e62e1343f23eb28ae9a"} Jul 07 23:14:47.521900 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:47.521877 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-7lwcs" event={"ID":"1b5943d1-f098-4857-ae9f-158dcbcb7b79","Type":"ContainerStarted","Data":"26fb68bc9105fc07ab29d4e42f854df911f30cdf8e67aeacb92f345f734f5cfa"} Jul 07 23:14:47.522369 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:47.522335 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:14:47.523220 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:47.523198 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6bffdf9778-qqf65" event={"ID":"51966fcb-7a64-464e-ab66-1d049570bcd3","Type":"ContainerStarted","Data":"3d89ec4386d984a032c1181d8c605894e0cd85401da08df357c96da9c46acda4"} Jul 07 23:14:47.524775 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:47.524757 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-4th55" event={"ID":"bd5aafbd-ed3c-46fa-bc87-661f74ba728a","Type":"ContainerStarted","Data":"eb649e5b483a0a3c0061792025c702ca5d5a397e62fe7a93415c468c32aba6c4"} Jul 07 23:14:47.524882 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:47.524791 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-4th55" event={"ID":"bd5aafbd-ed3c-46fa-bc87-661f74ba728a","Type":"ContainerStarted","Data":"ed60a67a4b12da34f89989453172d4fd326738a56b00b0be9da55cf45e714d71"} Jul 07 23:14:47.524942 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:47.524899 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-4th55" Jul 07 23:14:47.546018 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:47.545942 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-4th55" podStartSLOduration=66.012005483 podStartE2EDuration="1m10.545928534s" totalImagesPullingTime="4.533923051s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 23:13:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 23:14:47.545558323 +0000 UTC m=+104.942559857" watchObservedRunningTime="2026-07-07 23:14:47.545928534 +0000 UTC m=+104.942930068" Jul 07 23:14:47.622275 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:47.622237 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:14:49.533856 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:49.533822 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-n9t7q" event={"ID":"ccb9c548-3ffa-4ec0-8871-a7b956ff502f","Type":"ContainerStarted","Data":"2dbf34392d7b5bae2803de2bd529a9c520c48cab50f1087d4e325800d7caccf5"} Jul 07 23:14:49.535087 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:49.535067 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-7lwcs" event={"ID":"1b5943d1-f098-4857-ae9f-158dcbcb7b79","Type":"ContainerStarted","Data":"06a428f3aa3436861d35fac5c76a25341b11ea6b61d6ba938cfb6703b472bcb6"} Jul 07 23:14:49.535316 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:49.535282 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-7lwcs" Jul 07 23:14:49.536660 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:49.536639 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6bffdf9778-qqf65" event={"ID":"51966fcb-7a64-464e-ab66-1d049570bcd3","Type":"ContainerStarted","Data":"3709fa9bab6a9cc02c298ca792b26808d2ab27c14ec0c5d6994c2c6ab574571e"} Jul 07 23:14:49.540144 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:49.540124 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-7lwcs" Jul 07 23:14:49.554631 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:49.554595 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-7lwcs" podStartSLOduration=2.290878941 podStartE2EDuration="4.554582722s" totalImagesPullingTime="2.263703781s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 23:14:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 23:14:49.554138039 +0000 UTC m=+106.951139550" watchObservedRunningTime="2026-07-07 23:14:49.554582722 +0000 UTC m=+106.951584212" Jul 07 23:14:49.594675 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:49.594632 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-6bffdf9778-qqf65" podStartSLOduration=2.353935488 podStartE2EDuration="4.59461869s" totalImagesPullingTime="2.240683202s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 23:14:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 23:14:49.594060226 +0000 UTC m=+106.991061739" watchObservedRunningTime="2026-07-07 23:14:49.59461869 +0000 UTC m=+106.991620215" Jul 07 23:14:49.733376 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:49.733320 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:14:49.814575 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:49.814489 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:14:50.253286 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:50.253199 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:14:50.333414 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:50.333365 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:14:51.041103 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:51.041059 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:14:51.091872 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:51.091839 2570 scope.go:117] "RemoveContainer" containerID="04852c942546ac352acd95c7eaa0b4fbe754266a3a8bf8bd0b38e564363b903b" Jul 07 23:14:51.092068 ip-10-0-134-140 kubenswrapper[2570]: E0707 23:14:51.092041 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-68b757fb67-8hvxr_openshift-console-operator(faae2bd8-cbbd-4cfb-bce6-7954977b7ebc)\"" pod="openshift-console-operator/console-operator-68b757fb67-8hvxr" podUID="faae2bd8-cbbd-4cfb-bce6-7954977b7ebc" Jul 07 23:14:52.549524 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:52.549485 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-n9t7q" event={"ID":"ccb9c548-3ffa-4ec0-8871-a7b956ff502f","Type":"ContainerStarted","Data":"6e5e1a4dfbd2aaf52bff152a13cca70ee5217fd55ddb58c9fed191f7be131593"} Jul 07 23:14:52.570113 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:52.570064 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-n9t7q" podStartSLOduration=34.180712648 podStartE2EDuration="43.570050835s" totalImagesPullingTime="9.389338187s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 23:14:09 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 23:14:52.568557085 +0000 UTC m=+109.965558617" watchObservedRunningTime="2026-07-07 23:14:52.570050835 +0000 UTC m=+109.967052347" Jul 07 23:14:54.841396 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:54.841362 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:14:57.530659 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:57.530624 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-4th55" Jul 07 23:14:57.967979 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:14:57.967887 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:15:00.390103 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:00.390063 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:15:00.627827 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:00.627790 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:15:00.688502 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:00.688398 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-54cf96dfcf-j8m6n"] Jul 07 23:15:00.690595 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:00.690572 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-j8m6n" Jul 07 23:15:00.693538 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:00.693519 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\"" Jul 07 23:15:00.693676 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:00.693519 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-s6rkm\"" Jul 07 23:15:00.693676 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:00.693515 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\"" Jul 07 23:15:00.693676 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:00.693521 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 07 23:15:00.700711 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:00.700691 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-7c9dbb8b7d-fkvsb"] Jul 07 23:15:00.702675 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:00.702660 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-fkvsb" Jul 07 23:15:00.704654 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:00.704636 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\"" Jul 07 23:15:00.704749 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:00.704653 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\"" Jul 07 23:15:00.704806 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:00.704760 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-kljmh\"" Jul 07 23:15:00.704972 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:00.704952 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\"" Jul 07 23:15:00.714338 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:00.714318 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-54cf96dfcf-j8m6n"] Jul 07 23:15:00.719226 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:00.719203 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-7c9dbb8b7d-fkvsb"] Jul 07 23:15:00.721890 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:00.721869 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-xctgs"] Jul 07 23:15:00.724557 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:00.724537 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-xctgs" Jul 07 23:15:00.727001 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:00.726983 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 07 23:15:00.727086 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:00.727041 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 07 23:15:00.727170 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:00.727096 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 07 23:15:00.727335 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:00.727319 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-9nqnm\"" Jul 07 23:15:00.730068 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:00.730050 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/0e28cfdd-d891-4c6e-b4a3-808c02e3df64-openshift-state-metrics-tls\") pod \"openshift-state-metrics-54cf96dfcf-j8m6n\" (UID: \"0e28cfdd-d891-4c6e-b4a3-808c02e3df64\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-j8m6n" Jul 07 23:15:00.730160 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:00.730073 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ksz9x\" (UniqueName: \"kubernetes.io/projected/0e28cfdd-d891-4c6e-b4a3-808c02e3df64-kube-api-access-ksz9x\") pod \"openshift-state-metrics-54cf96dfcf-j8m6n\" (UID: \"0e28cfdd-d891-4c6e-b4a3-808c02e3df64\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-j8m6n" Jul 07 23:15:00.730160 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:00.730111 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0e28cfdd-d891-4c6e-b4a3-808c02e3df64-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-54cf96dfcf-j8m6n\" (UID: \"0e28cfdd-d891-4c6e-b4a3-808c02e3df64\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-j8m6n" Jul 07 23:15:00.730259 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:00.730193 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0e28cfdd-d891-4c6e-b4a3-808c02e3df64-metrics-client-ca\") pod \"openshift-state-metrics-54cf96dfcf-j8m6n\" (UID: \"0e28cfdd-d891-4c6e-b4a3-808c02e3df64\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-j8m6n" Jul 07 23:15:00.737683 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:00.737662 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:15:00.807137 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:00.807099 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:15:00.831146 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:00.831123 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/58a67519-a641-49b9-ba37-aac45e387515-sys\") pod \"node-exporter-xctgs\" (UID: \"58a67519-a641-49b9-ba37-aac45e387515\") " pod="openshift-monitoring/node-exporter-xctgs" Jul 07 23:15:00.831280 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:00.831152 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/58a67519-a641-49b9-ba37-aac45e387515-node-exporter-accelerators-collector-config\") pod \"node-exporter-xctgs\" (UID: \"58a67519-a641-49b9-ba37-aac45e387515\") " pod="openshift-monitoring/node-exporter-xctgs" Jul 07 23:15:00.831280 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:00.831177 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/93aecf5b-2135-4f91-8ed2-3828b00f4b2f-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-7c9dbb8b7d-fkvsb\" (UID: \"93aecf5b-2135-4f91-8ed2-3828b00f4b2f\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-fkvsb" Jul 07 23:15:00.831280 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:00.831208 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/58a67519-a641-49b9-ba37-aac45e387515-node-exporter-textfile\") pod \"node-exporter-xctgs\" (UID: \"58a67519-a641-49b9-ba37-aac45e387515\") " pod="openshift-monitoring/node-exporter-xctgs" Jul 07 23:15:00.831280 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:00.831272 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/0e28cfdd-d891-4c6e-b4a3-808c02e3df64-openshift-state-metrics-tls\") pod \"openshift-state-metrics-54cf96dfcf-j8m6n\" (UID: \"0e28cfdd-d891-4c6e-b4a3-808c02e3df64\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-j8m6n" Jul 07 23:15:00.831517 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:00.831328 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ksz9x\" (UniqueName: \"kubernetes.io/projected/0e28cfdd-d891-4c6e-b4a3-808c02e3df64-kube-api-access-ksz9x\") pod \"openshift-state-metrics-54cf96dfcf-j8m6n\" (UID: \"0e28cfdd-d891-4c6e-b4a3-808c02e3df64\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-j8m6n" Jul 07 23:15:00.831517 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:00.831364 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/93aecf5b-2135-4f91-8ed2-3828b00f4b2f-kube-state-metrics-tls\") pod \"kube-state-metrics-7c9dbb8b7d-fkvsb\" (UID: \"93aecf5b-2135-4f91-8ed2-3828b00f4b2f\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-fkvsb" Jul 07 23:15:00.831517 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:00.831394 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/58a67519-a641-49b9-ba37-aac45e387515-metrics-client-ca\") pod \"node-exporter-xctgs\" (UID: \"58a67519-a641-49b9-ba37-aac45e387515\") " pod="openshift-monitoring/node-exporter-xctgs" Jul 07 23:15:00.831517 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:00.831459 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0e28cfdd-d891-4c6e-b4a3-808c02e3df64-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-54cf96dfcf-j8m6n\" (UID: \"0e28cfdd-d891-4c6e-b4a3-808c02e3df64\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-j8m6n" Jul 07 23:15:00.831694 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:00.831520 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/58a67519-a641-49b9-ba37-aac45e387515-node-exporter-tls\") pod \"node-exporter-xctgs\" (UID: \"58a67519-a641-49b9-ba37-aac45e387515\") " pod="openshift-monitoring/node-exporter-xctgs" Jul 07 23:15:00.831694 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:00.831564 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/93aecf5b-2135-4f91-8ed2-3828b00f4b2f-volume-directive-shadow\") pod \"kube-state-metrics-7c9dbb8b7d-fkvsb\" (UID: \"93aecf5b-2135-4f91-8ed2-3828b00f4b2f\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-fkvsb" Jul 07 23:15:00.831694 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:00.831592 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/93aecf5b-2135-4f91-8ed2-3828b00f4b2f-metrics-client-ca\") pod \"kube-state-metrics-7c9dbb8b7d-fkvsb\" (UID: \"93aecf5b-2135-4f91-8ed2-3828b00f4b2f\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-fkvsb" Jul 07 23:15:00.831694 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:00.831617 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/58a67519-a641-49b9-ba37-aac45e387515-root\") pod \"node-exporter-xctgs\" (UID: \"58a67519-a641-49b9-ba37-aac45e387515\") " pod="openshift-monitoring/node-exporter-xctgs" Jul 07 23:15:00.831694 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:00.831667 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lf27p\" (UniqueName: \"kubernetes.io/projected/93aecf5b-2135-4f91-8ed2-3828b00f4b2f-kube-api-access-lf27p\") pod \"kube-state-metrics-7c9dbb8b7d-fkvsb\" (UID: \"93aecf5b-2135-4f91-8ed2-3828b00f4b2f\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-fkvsb" Jul 07 23:15:00.831864 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:00.831701 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/58a67519-a641-49b9-ba37-aac45e387515-node-exporter-wtmp\") pod \"node-exporter-xctgs\" (UID: \"58a67519-a641-49b9-ba37-aac45e387515\") " pod="openshift-monitoring/node-exporter-xctgs" Jul 07 23:15:00.831864 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:00.831726 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/58a67519-a641-49b9-ba37-aac45e387515-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-xctgs\" (UID: \"58a67519-a641-49b9-ba37-aac45e387515\") " pod="openshift-monitoring/node-exporter-xctgs" Jul 07 23:15:00.831864 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:00.831783 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0e28cfdd-d891-4c6e-b4a3-808c02e3df64-metrics-client-ca\") pod \"openshift-state-metrics-54cf96dfcf-j8m6n\" (UID: \"0e28cfdd-d891-4c6e-b4a3-808c02e3df64\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-j8m6n" Jul 07 23:15:00.831864 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:00.831840 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/93aecf5b-2135-4f91-8ed2-3828b00f4b2f-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-7c9dbb8b7d-fkvsb\" (UID: \"93aecf5b-2135-4f91-8ed2-3828b00f4b2f\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-fkvsb" Jul 07 23:15:00.831999 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:00.831874 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j8f2d\" (UniqueName: \"kubernetes.io/projected/58a67519-a641-49b9-ba37-aac45e387515-kube-api-access-j8f2d\") pod \"node-exporter-xctgs\" (UID: \"58a67519-a641-49b9-ba37-aac45e387515\") " pod="openshift-monitoring/node-exporter-xctgs" Jul 07 23:15:00.832328 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:00.832311 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0e28cfdd-d891-4c6e-b4a3-808c02e3df64-metrics-client-ca\") pod \"openshift-state-metrics-54cf96dfcf-j8m6n\" (UID: \"0e28cfdd-d891-4c6e-b4a3-808c02e3df64\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-j8m6n" Jul 07 23:15:00.833632 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:00.833613 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/0e28cfdd-d891-4c6e-b4a3-808c02e3df64-openshift-state-metrics-tls\") pod \"openshift-state-metrics-54cf96dfcf-j8m6n\" (UID: \"0e28cfdd-d891-4c6e-b4a3-808c02e3df64\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-j8m6n" Jul 07 23:15:00.833787 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:00.833770 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0e28cfdd-d891-4c6e-b4a3-808c02e3df64-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-54cf96dfcf-j8m6n\" (UID: \"0e28cfdd-d891-4c6e-b4a3-808c02e3df64\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-j8m6n" Jul 07 23:15:00.839976 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:00.839955 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ksz9x\" (UniqueName: \"kubernetes.io/projected/0e28cfdd-d891-4c6e-b4a3-808c02e3df64-kube-api-access-ksz9x\") pod \"openshift-state-metrics-54cf96dfcf-j8m6n\" (UID: \"0e28cfdd-d891-4c6e-b4a3-808c02e3df64\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-j8m6n" Jul 07 23:15:00.932768 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:00.932739 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/58a67519-a641-49b9-ba37-aac45e387515-node-exporter-tls\") pod \"node-exporter-xctgs\" (UID: \"58a67519-a641-49b9-ba37-aac45e387515\") " pod="openshift-monitoring/node-exporter-xctgs" Jul 07 23:15:00.932911 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:00.932787 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/93aecf5b-2135-4f91-8ed2-3828b00f4b2f-volume-directive-shadow\") pod \"kube-state-metrics-7c9dbb8b7d-fkvsb\" (UID: \"93aecf5b-2135-4f91-8ed2-3828b00f4b2f\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-fkvsb" Jul 07 23:15:00.932911 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:00.932817 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/93aecf5b-2135-4f91-8ed2-3828b00f4b2f-metrics-client-ca\") pod \"kube-state-metrics-7c9dbb8b7d-fkvsb\" (UID: \"93aecf5b-2135-4f91-8ed2-3828b00f4b2f\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-fkvsb" Jul 07 23:15:00.932911 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:00.932843 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/58a67519-a641-49b9-ba37-aac45e387515-root\") pod \"node-exporter-xctgs\" (UID: \"58a67519-a641-49b9-ba37-aac45e387515\") " pod="openshift-monitoring/node-exporter-xctgs" Jul 07 23:15:00.933033 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:00.932913 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/58a67519-a641-49b9-ba37-aac45e387515-root\") pod \"node-exporter-xctgs\" (UID: \"58a67519-a641-49b9-ba37-aac45e387515\") " pod="openshift-monitoring/node-exporter-xctgs" Jul 07 23:15:00.933033 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:00.932950 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lf27p\" (UniqueName: \"kubernetes.io/projected/93aecf5b-2135-4f91-8ed2-3828b00f4b2f-kube-api-access-lf27p\") pod \"kube-state-metrics-7c9dbb8b7d-fkvsb\" (UID: \"93aecf5b-2135-4f91-8ed2-3828b00f4b2f\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-fkvsb" Jul 07 23:15:00.933033 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:00.932973 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/58a67519-a641-49b9-ba37-aac45e387515-node-exporter-wtmp\") pod \"node-exporter-xctgs\" (UID: \"58a67519-a641-49b9-ba37-aac45e387515\") " pod="openshift-monitoring/node-exporter-xctgs" Jul 07 23:15:00.933033 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:00.932998 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/58a67519-a641-49b9-ba37-aac45e387515-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-xctgs\" (UID: \"58a67519-a641-49b9-ba37-aac45e387515\") " pod="openshift-monitoring/node-exporter-xctgs" Jul 07 23:15:00.934112 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:00.933045 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/93aecf5b-2135-4f91-8ed2-3828b00f4b2f-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-7c9dbb8b7d-fkvsb\" (UID: \"93aecf5b-2135-4f91-8ed2-3828b00f4b2f\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-fkvsb" Jul 07 23:15:00.934112 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:00.933106 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j8f2d\" (UniqueName: \"kubernetes.io/projected/58a67519-a641-49b9-ba37-aac45e387515-kube-api-access-j8f2d\") pod \"node-exporter-xctgs\" (UID: \"58a67519-a641-49b9-ba37-aac45e387515\") " pod="openshift-monitoring/node-exporter-xctgs" Jul 07 23:15:00.934112 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:00.933154 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/58a67519-a641-49b9-ba37-aac45e387515-sys\") pod \"node-exporter-xctgs\" (UID: \"58a67519-a641-49b9-ba37-aac45e387515\") " pod="openshift-monitoring/node-exporter-xctgs" Jul 07 23:15:00.934112 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:00.933178 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/58a67519-a641-49b9-ba37-aac45e387515-node-exporter-wtmp\") pod \"node-exporter-xctgs\" (UID: \"58a67519-a641-49b9-ba37-aac45e387515\") " pod="openshift-monitoring/node-exporter-xctgs" Jul 07 23:15:00.934112 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:00.933183 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/58a67519-a641-49b9-ba37-aac45e387515-node-exporter-accelerators-collector-config\") pod \"node-exporter-xctgs\" (UID: \"58a67519-a641-49b9-ba37-aac45e387515\") " pod="openshift-monitoring/node-exporter-xctgs" Jul 07 23:15:00.934112 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:00.933218 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/93aecf5b-2135-4f91-8ed2-3828b00f4b2f-volume-directive-shadow\") pod \"kube-state-metrics-7c9dbb8b7d-fkvsb\" (UID: \"93aecf5b-2135-4f91-8ed2-3828b00f4b2f\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-fkvsb" Jul 07 23:15:00.934112 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:00.933250 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/93aecf5b-2135-4f91-8ed2-3828b00f4b2f-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-7c9dbb8b7d-fkvsb\" (UID: \"93aecf5b-2135-4f91-8ed2-3828b00f4b2f\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-fkvsb" Jul 07 23:15:00.934112 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:00.933278 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/58a67519-a641-49b9-ba37-aac45e387515-node-exporter-textfile\") pod \"node-exporter-xctgs\" (UID: \"58a67519-a641-49b9-ba37-aac45e387515\") " pod="openshift-monitoring/node-exporter-xctgs" Jul 07 23:15:00.934112 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:00.933319 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/93aecf5b-2135-4f91-8ed2-3828b00f4b2f-kube-state-metrics-tls\") pod \"kube-state-metrics-7c9dbb8b7d-fkvsb\" (UID: \"93aecf5b-2135-4f91-8ed2-3828b00f4b2f\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-fkvsb" Jul 07 23:15:00.934112 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:00.933343 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/58a67519-a641-49b9-ba37-aac45e387515-metrics-client-ca\") pod \"node-exporter-xctgs\" (UID: \"58a67519-a641-49b9-ba37-aac45e387515\") " pod="openshift-monitoring/node-exporter-xctgs" Jul 07 23:15:00.934112 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:00.933753 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/93aecf5b-2135-4f91-8ed2-3828b00f4b2f-metrics-client-ca\") pod \"kube-state-metrics-7c9dbb8b7d-fkvsb\" (UID: \"93aecf5b-2135-4f91-8ed2-3828b00f4b2f\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-fkvsb" Jul 07 23:15:00.934112 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:00.933806 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/93aecf5b-2135-4f91-8ed2-3828b00f4b2f-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-7c9dbb8b7d-fkvsb\" (UID: \"93aecf5b-2135-4f91-8ed2-3828b00f4b2f\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-fkvsb" Jul 07 23:15:00.934112 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:00.933820 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/58a67519-a641-49b9-ba37-aac45e387515-node-exporter-accelerators-collector-config\") pod \"node-exporter-xctgs\" (UID: \"58a67519-a641-49b9-ba37-aac45e387515\") " pod="openshift-monitoring/node-exporter-xctgs" Jul 07 23:15:00.934112 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:00.933814 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/58a67519-a641-49b9-ba37-aac45e387515-sys\") pod \"node-exporter-xctgs\" (UID: \"58a67519-a641-49b9-ba37-aac45e387515\") " pod="openshift-monitoring/node-exporter-xctgs" Jul 07 23:15:00.934112 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:00.933831 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/58a67519-a641-49b9-ba37-aac45e387515-node-exporter-textfile\") pod \"node-exporter-xctgs\" (UID: \"58a67519-a641-49b9-ba37-aac45e387515\") " pod="openshift-monitoring/node-exporter-xctgs" Jul 07 23:15:00.934112 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:00.933886 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/58a67519-a641-49b9-ba37-aac45e387515-metrics-client-ca\") pod \"node-exporter-xctgs\" (UID: \"58a67519-a641-49b9-ba37-aac45e387515\") " pod="openshift-monitoring/node-exporter-xctgs" Jul 07 23:15:00.939581 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:00.935605 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/58a67519-a641-49b9-ba37-aac45e387515-node-exporter-tls\") pod \"node-exporter-xctgs\" (UID: \"58a67519-a641-49b9-ba37-aac45e387515\") " pod="openshift-monitoring/node-exporter-xctgs" Jul 07 23:15:00.939581 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:00.935701 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/58a67519-a641-49b9-ba37-aac45e387515-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-xctgs\" (UID: \"58a67519-a641-49b9-ba37-aac45e387515\") " pod="openshift-monitoring/node-exporter-xctgs" Jul 07 23:15:00.939581 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:00.935958 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/93aecf5b-2135-4f91-8ed2-3828b00f4b2f-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-7c9dbb8b7d-fkvsb\" (UID: \"93aecf5b-2135-4f91-8ed2-3828b00f4b2f\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-fkvsb" Jul 07 23:15:00.941379 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:00.941357 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/93aecf5b-2135-4f91-8ed2-3828b00f4b2f-kube-state-metrics-tls\") pod \"kube-state-metrics-7c9dbb8b7d-fkvsb\" (UID: \"93aecf5b-2135-4f91-8ed2-3828b00f4b2f\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-fkvsb" Jul 07 23:15:00.943973 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:00.943953 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j8f2d\" (UniqueName: \"kubernetes.io/projected/58a67519-a641-49b9-ba37-aac45e387515-kube-api-access-j8f2d\") pod \"node-exporter-xctgs\" (UID: \"58a67519-a641-49b9-ba37-aac45e387515\") " pod="openshift-monitoring/node-exporter-xctgs" Jul 07 23:15:00.945429 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:00.945411 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lf27p\" (UniqueName: \"kubernetes.io/projected/93aecf5b-2135-4f91-8ed2-3828b00f4b2f-kube-api-access-lf27p\") pod \"kube-state-metrics-7c9dbb8b7d-fkvsb\" (UID: \"93aecf5b-2135-4f91-8ed2-3828b00f4b2f\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-fkvsb" Jul 07 23:15:00.996702 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:00.996668 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:15:00.998983 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:00.998959 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-j8m6n" Jul 07 23:15:01.011182 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:01.011163 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-fkvsb" Jul 07 23:15:01.033194 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:01.033165 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-xctgs" Jul 07 23:15:01.044082 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:15:01.044055 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod58a67519_a641_49b9_ba37_aac45e387515.slice/crio-29d391c03e6308b5338cc02def0854805c35afd3d5d599234dfd784e64195b03 WatchSource:0}: Error finding container 29d391c03e6308b5338cc02def0854805c35afd3d5d599234dfd784e64195b03: Status 404 returned error can't find the container with id 29d391c03e6308b5338cc02def0854805c35afd3d5d599234dfd784e64195b03 Jul 07 23:15:01.143422 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:01.143386 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-54cf96dfcf-j8m6n"] Jul 07 23:15:01.146068 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:15:01.146041 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0e28cfdd_d891_4c6e_b4a3_808c02e3df64.slice/crio-944fd1659803e71d3af7b4ac5da9b7c5c99ae4e8176c59b46f0353e4a0ef8709 WatchSource:0}: Error finding container 944fd1659803e71d3af7b4ac5da9b7c5c99ae4e8176c59b46f0353e4a0ef8709: Status 404 returned error can't find the container with id 944fd1659803e71d3af7b4ac5da9b7c5c99ae4e8176c59b46f0353e4a0ef8709 Jul 07 23:15:01.162273 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:01.162255 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-7c9dbb8b7d-fkvsb"] Jul 07 23:15:01.164659 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:15:01.164637 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod93aecf5b_2135_4f91_8ed2_3828b00f4b2f.slice/crio-dd7148e3315e4a1d64ca15870ca68564f55abce88f0a9f2825b1c4dcec7989ec WatchSource:0}: Error finding container dd7148e3315e4a1d64ca15870ca68564f55abce88f0a9f2825b1c4dcec7989ec: Status 404 returned error can't find the container with id dd7148e3315e4a1d64ca15870ca68564f55abce88f0a9f2825b1c4dcec7989ec Jul 07 23:15:01.331160 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:01.331130 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:15:01.577815 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:01.577775 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-xctgs" event={"ID":"58a67519-a641-49b9-ba37-aac45e387515","Type":"ContainerStarted","Data":"29d391c03e6308b5338cc02def0854805c35afd3d5d599234dfd784e64195b03"} Jul 07 23:15:01.579713 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:01.579680 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-j8m6n" event={"ID":"0e28cfdd-d891-4c6e-b4a3-808c02e3df64","Type":"ContainerStarted","Data":"c0d40534a1a44ec013641beaceb0c9587402f75b11657450d0447dee78c66275"} Jul 07 23:15:01.579830 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:01.579744 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-j8m6n" event={"ID":"0e28cfdd-d891-4c6e-b4a3-808c02e3df64","Type":"ContainerStarted","Data":"d587a5d36def10c9b2a4d347f9646ddd8e934ecb5a79bf796e2449a613a94946"} Jul 07 23:15:01.579830 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:01.579778 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-j8m6n" event={"ID":"0e28cfdd-d891-4c6e-b4a3-808c02e3df64","Type":"ContainerStarted","Data":"944fd1659803e71d3af7b4ac5da9b7c5c99ae4e8176c59b46f0353e4a0ef8709"} Jul 07 23:15:01.580698 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:01.580678 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-fkvsb" event={"ID":"93aecf5b-2135-4f91-8ed2-3828b00f4b2f","Type":"ContainerStarted","Data":"dd7148e3315e4a1d64ca15870ca68564f55abce88f0a9f2825b1c4dcec7989ec"} Jul 07 23:15:01.658536 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:01.658497 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:15:01.751622 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:01.751585 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:15:01.755944 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:01.755771 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 07 23:15:01.759345 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:01.759322 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 07 23:15:01.762277 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:01.761711 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jul 07 23:15:01.762277 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:01.761984 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jul 07 23:15:01.762277 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:01.761987 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-rd4kk\"" Jul 07 23:15:01.762277 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:01.762181 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jul 07 23:15:01.762609 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:01.762348 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jul 07 23:15:01.762609 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:01.762556 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jul 07 23:15:01.763091 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:01.762766 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jul 07 23:15:01.763091 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:01.762812 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jul 07 23:15:01.763091 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:01.762944 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jul 07 23:15:01.763091 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:01.763053 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jul 07 23:15:01.776172 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:01.776118 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 07 23:15:01.842534 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:01.842434 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/5b8a514b-0b1a-4914-8629-cd4df7e35512-web-config\") pod \"alertmanager-main-0\" (UID: \"5b8a514b-0b1a-4914-8629-cd4df7e35512\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 23:15:01.842534 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:01.842520 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/5b8a514b-0b1a-4914-8629-cd4df7e35512-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"5b8a514b-0b1a-4914-8629-cd4df7e35512\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 23:15:01.842743 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:01.842549 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/5b8a514b-0b1a-4914-8629-cd4df7e35512-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"5b8a514b-0b1a-4914-8629-cd4df7e35512\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 23:15:01.842743 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:01.842575 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/5b8a514b-0b1a-4914-8629-cd4df7e35512-config-out\") pod \"alertmanager-main-0\" (UID: \"5b8a514b-0b1a-4914-8629-cd4df7e35512\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 23:15:01.842743 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:01.842603 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5b8a514b-0b1a-4914-8629-cd4df7e35512-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"5b8a514b-0b1a-4914-8629-cd4df7e35512\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 23:15:01.842743 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:01.842674 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/5b8a514b-0b1a-4914-8629-cd4df7e35512-config-volume\") pod \"alertmanager-main-0\" (UID: \"5b8a514b-0b1a-4914-8629-cd4df7e35512\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 23:15:01.842743 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:01.842713 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/5b8a514b-0b1a-4914-8629-cd4df7e35512-tls-assets\") pod \"alertmanager-main-0\" (UID: \"5b8a514b-0b1a-4914-8629-cd4df7e35512\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 23:15:01.842994 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:01.842755 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/5b8a514b-0b1a-4914-8629-cd4df7e35512-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"5b8a514b-0b1a-4914-8629-cd4df7e35512\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 23:15:01.842994 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:01.842780 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/5b8a514b-0b1a-4914-8629-cd4df7e35512-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"5b8a514b-0b1a-4914-8629-cd4df7e35512\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 23:15:01.842994 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:01.842816 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cdpdd\" (UniqueName: \"kubernetes.io/projected/5b8a514b-0b1a-4914-8629-cd4df7e35512-kube-api-access-cdpdd\") pod \"alertmanager-main-0\" (UID: \"5b8a514b-0b1a-4914-8629-cd4df7e35512\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 23:15:01.842994 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:01.842845 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/5b8a514b-0b1a-4914-8629-cd4df7e35512-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"5b8a514b-0b1a-4914-8629-cd4df7e35512\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 23:15:01.842994 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:01.842878 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/5b8a514b-0b1a-4914-8629-cd4df7e35512-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"5b8a514b-0b1a-4914-8629-cd4df7e35512\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 23:15:01.842994 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:01.842903 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5b8a514b-0b1a-4914-8629-cd4df7e35512-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"5b8a514b-0b1a-4914-8629-cd4df7e35512\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 23:15:01.881293 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:01.881113 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:15:01.943899 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:01.943859 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cdpdd\" (UniqueName: \"kubernetes.io/projected/5b8a514b-0b1a-4914-8629-cd4df7e35512-kube-api-access-cdpdd\") pod \"alertmanager-main-0\" (UID: \"5b8a514b-0b1a-4914-8629-cd4df7e35512\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 23:15:01.944083 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:01.943912 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/5b8a514b-0b1a-4914-8629-cd4df7e35512-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"5b8a514b-0b1a-4914-8629-cd4df7e35512\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 23:15:01.944083 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:01.943949 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/5b8a514b-0b1a-4914-8629-cd4df7e35512-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"5b8a514b-0b1a-4914-8629-cd4df7e35512\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 23:15:01.944083 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:01.943979 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5b8a514b-0b1a-4914-8629-cd4df7e35512-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"5b8a514b-0b1a-4914-8629-cd4df7e35512\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 23:15:01.944083 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:01.944010 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/5b8a514b-0b1a-4914-8629-cd4df7e35512-web-config\") pod \"alertmanager-main-0\" (UID: \"5b8a514b-0b1a-4914-8629-cd4df7e35512\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 23:15:01.944324 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:01.944294 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/5b8a514b-0b1a-4914-8629-cd4df7e35512-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"5b8a514b-0b1a-4914-8629-cd4df7e35512\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 23:15:01.944379 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:01.944346 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/5b8a514b-0b1a-4914-8629-cd4df7e35512-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"5b8a514b-0b1a-4914-8629-cd4df7e35512\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 23:15:01.944437 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:01.944378 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/5b8a514b-0b1a-4914-8629-cd4df7e35512-config-out\") pod \"alertmanager-main-0\" (UID: \"5b8a514b-0b1a-4914-8629-cd4df7e35512\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 23:15:01.944437 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:01.944402 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5b8a514b-0b1a-4914-8629-cd4df7e35512-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"5b8a514b-0b1a-4914-8629-cd4df7e35512\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 23:15:01.944569 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:01.944503 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/5b8a514b-0b1a-4914-8629-cd4df7e35512-config-volume\") pod \"alertmanager-main-0\" (UID: \"5b8a514b-0b1a-4914-8629-cd4df7e35512\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 23:15:01.944569 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:01.944551 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/5b8a514b-0b1a-4914-8629-cd4df7e35512-tls-assets\") pod \"alertmanager-main-0\" (UID: \"5b8a514b-0b1a-4914-8629-cd4df7e35512\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 23:15:01.944667 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:01.944598 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/5b8a514b-0b1a-4914-8629-cd4df7e35512-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"5b8a514b-0b1a-4914-8629-cd4df7e35512\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 23:15:01.944667 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:01.944622 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/5b8a514b-0b1a-4914-8629-cd4df7e35512-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"5b8a514b-0b1a-4914-8629-cd4df7e35512\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 23:15:01.944968 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:01.944943 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/5b8a514b-0b1a-4914-8629-cd4df7e35512-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"5b8a514b-0b1a-4914-8629-cd4df7e35512\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 23:15:01.945058 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:01.945005 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5b8a514b-0b1a-4914-8629-cd4df7e35512-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"5b8a514b-0b1a-4914-8629-cd4df7e35512\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 23:15:01.945759 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:01.945606 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5b8a514b-0b1a-4914-8629-cd4df7e35512-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"5b8a514b-0b1a-4914-8629-cd4df7e35512\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 23:15:01.950972 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:01.950613 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/5b8a514b-0b1a-4914-8629-cd4df7e35512-config-out\") pod \"alertmanager-main-0\" (UID: \"5b8a514b-0b1a-4914-8629-cd4df7e35512\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 23:15:01.950972 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:01.950698 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/5b8a514b-0b1a-4914-8629-cd4df7e35512-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"5b8a514b-0b1a-4914-8629-cd4df7e35512\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 23:15:01.951183 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:01.951161 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/5b8a514b-0b1a-4914-8629-cd4df7e35512-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"5b8a514b-0b1a-4914-8629-cd4df7e35512\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 23:15:01.951966 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:01.951944 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/5b8a514b-0b1a-4914-8629-cd4df7e35512-web-config\") pod \"alertmanager-main-0\" (UID: \"5b8a514b-0b1a-4914-8629-cd4df7e35512\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 23:15:01.952669 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:01.952291 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/5b8a514b-0b1a-4914-8629-cd4df7e35512-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"5b8a514b-0b1a-4914-8629-cd4df7e35512\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 23:15:01.952669 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:01.952446 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/5b8a514b-0b1a-4914-8629-cd4df7e35512-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"5b8a514b-0b1a-4914-8629-cd4df7e35512\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 23:15:01.952669 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:01.952634 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/5b8a514b-0b1a-4914-8629-cd4df7e35512-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"5b8a514b-0b1a-4914-8629-cd4df7e35512\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 23:15:01.953268 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:01.953230 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/5b8a514b-0b1a-4914-8629-cd4df7e35512-tls-assets\") pod \"alertmanager-main-0\" (UID: \"5b8a514b-0b1a-4914-8629-cd4df7e35512\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 23:15:01.955319 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:01.955280 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cdpdd\" (UniqueName: \"kubernetes.io/projected/5b8a514b-0b1a-4914-8629-cd4df7e35512-kube-api-access-cdpdd\") pod \"alertmanager-main-0\" (UID: \"5b8a514b-0b1a-4914-8629-cd4df7e35512\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 23:15:01.956607 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:01.956548 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/5b8a514b-0b1a-4914-8629-cd4df7e35512-config-volume\") pod \"alertmanager-main-0\" (UID: \"5b8a514b-0b1a-4914-8629-cd4df7e35512\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 23:15:02.073326 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:02.073289 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 07 23:15:02.320253 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:02.320153 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 07 23:15:02.536651 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:15:02.536616 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5b8a514b_0b1a_4914_8629_cd4df7e35512.slice/crio-04680da060bc71722a5ccca835abb89e66b5531d24cdb6bdcbaf639cdd017ef7 WatchSource:0}: Error finding container 04680da060bc71722a5ccca835abb89e66b5531d24cdb6bdcbaf639cdd017ef7: Status 404 returned error can't find the container with id 04680da060bc71722a5ccca835abb89e66b5531d24cdb6bdcbaf639cdd017ef7 Jul 07 23:15:02.586132 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:02.586093 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"5b8a514b-0b1a-4914-8629-cd4df7e35512","Type":"ContainerStarted","Data":"04680da060bc71722a5ccca835abb89e66b5531d24cdb6bdcbaf639cdd017ef7"} Jul 07 23:15:03.093868 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:03.093782 2570 scope.go:117] "RemoveContainer" containerID="04852c942546ac352acd95c7eaa0b4fbe754266a3a8bf8bd0b38e564363b903b" Jul 07 23:15:03.497751 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:03.497658 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-5668757489-j9qbl" Jul 07 23:15:03.593072 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:03.593034 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-fkvsb" event={"ID":"93aecf5b-2135-4f91-8ed2-3828b00f4b2f","Type":"ContainerStarted","Data":"a0f3df551a55bf48ae490f52501a73a1ff57acd5bc1838f18507e185a71f6754"} Jul 07 23:15:03.593548 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:03.593104 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-fkvsb" event={"ID":"93aecf5b-2135-4f91-8ed2-3828b00f4b2f","Type":"ContainerStarted","Data":"6a6c1c0faabc13f5ab8b440a65e0ca8b8f5f14779a7de434ef8ee96bbfc559ed"} Jul 07 23:15:03.593548 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:03.593131 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-fkvsb" event={"ID":"93aecf5b-2135-4f91-8ed2-3828b00f4b2f","Type":"ContainerStarted","Data":"0cb4c9a89a848327d239ec251d2f876b687013077fa9070ab687904b9bc0c1c0"} Jul 07 23:15:03.595602 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:03.595563 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-8hvxr_faae2bd8-cbbd-4cfb-bce6-7954977b7ebc/console-operator/2.log" Jul 07 23:15:03.595720 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:03.595688 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-8hvxr" event={"ID":"faae2bd8-cbbd-4cfb-bce6-7954977b7ebc","Type":"ContainerStarted","Data":"f8ebf333a7b634f8daf8533ac259aff2492dfeadbe46920e99e5b49e6c26a7da"} Jul 07 23:15:03.596102 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:03.595946 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-68b757fb67-8hvxr" Jul 07 23:15:03.598157 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:03.597973 2570 generic.go:356] "Generic (PLEG): container finished" podID="58a67519-a641-49b9-ba37-aac45e387515" containerID="b3783c275590b9d7e56054a7e9dd89a98e0230e45d6bf52712c0c6c37dcd58c9" exitCode=0 Jul 07 23:15:03.598157 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:03.598061 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-xctgs" event={"ID":"58a67519-a641-49b9-ba37-aac45e387515","Type":"ContainerDied","Data":"b3783c275590b9d7e56054a7e9dd89a98e0230e45d6bf52712c0c6c37dcd58c9"} Jul 07 23:15:03.600770 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:03.600713 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-j8m6n" event={"ID":"0e28cfdd-d891-4c6e-b4a3-808c02e3df64","Type":"ContainerStarted","Data":"dfd8b77cbcb75803a093d6c4efb3ce7932b0d61f55b1f155197084bb78052756"} Jul 07 23:15:03.616673 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:03.616643 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:15:03.623226 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:03.622685 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-fkvsb" podStartSLOduration=2.017898403 podStartE2EDuration="3.622665258s" totalImagesPullingTime="1.604766855s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 23:15:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 23:15:03.621895068 +0000 UTC m=+121.018896584" watchObservedRunningTime="2026-07-07 23:15:03.622665258 +0000 UTC m=+121.019666770" Jul 07 23:15:03.625776 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:03.625749 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-68b757fb67-8hvxr" Jul 07 23:15:03.650655 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:03.650601 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-j8m6n" podStartSLOduration=2.131714866 podStartE2EDuration="3.650583278s" totalImagesPullingTime="1.518868412s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 23:15:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 23:15:03.648964869 +0000 UTC m=+121.045966411" watchObservedRunningTime="2026-07-07 23:15:03.650583278 +0000 UTC m=+121.047584795" Jul 07 23:15:03.677393 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:03.677330 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-68b757fb67-8hvxr" podStartSLOduration=50.528177091 podStartE2EDuration="55.677313853s" totalImagesPullingTime="5.149136762s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 23:14:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 23:15:03.674379886 +0000 UTC m=+121.071381417" watchObservedRunningTime="2026-07-07 23:15:03.677313853 +0000 UTC m=+121.074315366" Jul 07 23:15:03.753168 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:03.753038 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:15:03.884314 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:03.884270 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:15:04.074157 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:04.074121 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:15:04.156639 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:04.156606 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:15:04.302680 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:04.302630 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:15:04.367488 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:04.367382 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:15:04.606848 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:04.606807 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-xctgs" event={"ID":"58a67519-a641-49b9-ba37-aac45e387515","Type":"ContainerStarted","Data":"869bd30dd0ba917c15495fe864edcea5d6ed99909d1b87f632192426d1e34cbc"} Jul 07 23:15:04.607260 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:04.606902 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-xctgs" event={"ID":"58a67519-a641-49b9-ba37-aac45e387515","Type":"ContainerStarted","Data":"ab0b29e54926c1430d032c7ea72d50658d2dffaf481d1f5ef564f1210791b90a"} Jul 07 23:15:04.608232 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:04.608205 2570 generic.go:356] "Generic (PLEG): container finished" podID="5b8a514b-0b1a-4914-8629-cd4df7e35512" containerID="0950b95b7cda66668ca633f211d0abfd56cc63552dd86e5380152fecde7326f9" exitCode=0 Jul 07 23:15:04.608345 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:04.608298 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"5b8a514b-0b1a-4914-8629-cd4df7e35512","Type":"ContainerDied","Data":"0950b95b7cda66668ca633f211d0abfd56cc63552dd86e5380152fecde7326f9"} Jul 07 23:15:04.629575 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:04.629459 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-xctgs" podStartSLOduration=2.82050919 podStartE2EDuration="4.629443131s" totalImagesPullingTime="1.720414157s" totalInitContainerRuntime="88.519784ms" isStatefulPod=true podCreationTimestamp="2026-07-07 23:15:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 23:15:04.6278265 +0000 UTC m=+122.024828013" watchObservedRunningTime="2026-07-07 23:15:04.629443131 +0000 UTC m=+122.026444644" Jul 07 23:15:04.713226 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:04.713185 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:15:05.056589 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:05.056548 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:15:05.425136 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:05.425062 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:15:05.511604 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:05.511545 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:15:05.583992 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:05.583956 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:15:05.805380 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:05.805342 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:15:06.233573 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:06.233491 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:15:06.618053 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:06.618021 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"5b8a514b-0b1a-4914-8629-cd4df7e35512","Type":"ContainerStarted","Data":"0eb86e210ed0d8f6aa83ff35645458b6bacc06f1f9c4dad68d2b343e12454d37"} Jul 07 23:15:06.618157 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:06.618090 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"5b8a514b-0b1a-4914-8629-cd4df7e35512","Type":"ContainerStarted","Data":"afc0814a7deb893adb5b56cd3bdfa0d4eace767b04b9ea30969e75ce84ab1c23"} Jul 07 23:15:06.618157 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:06.618120 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"5b8a514b-0b1a-4914-8629-cd4df7e35512","Type":"ContainerStarted","Data":"2ac24188b9b50c58c8e4cc0f75c62fbec8df79cd46cc4a3d31042c631af60b3f"} Jul 07 23:15:06.618157 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:06.618147 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"5b8a514b-0b1a-4914-8629-cd4df7e35512","Type":"ContainerStarted","Data":"7f3321ed31282d7770736f9381ad57bf89dbe8a8b13d836d8522929262473454"} Jul 07 23:15:06.753268 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:06.753184 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:15:06.863496 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:06.863443 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:15:06.943998 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:06.943963 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:15:07.001547 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:07.001505 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:15:07.625009 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:07.624975 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"5b8a514b-0b1a-4914-8629-cd4df7e35512","Type":"ContainerStarted","Data":"078b93de67102186596acb479778e1484d1dcc8ec1a36180a529e063c316ff74"} Jul 07 23:15:07.625186 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:07.625041 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"5b8a514b-0b1a-4914-8629-cd4df7e35512","Type":"ContainerStarted","Data":"244617530e4181cfa1ec9b570685651a42e04b30df6bae462192289c7a1fc065"} Jul 07 23:15:07.661303 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:07.661242 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.937104817 podStartE2EDuration="6.661226583s" totalImagesPullingTime="3.706809557s" totalInitContainerRuntime="17.312209ms" isStatefulPod=false podCreationTimestamp="2026-07-07 23:15:01 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 23:15:07.660150023 +0000 UTC m=+125.057151518" watchObservedRunningTime="2026-07-07 23:15:07.661226583 +0000 UTC m=+125.058228095" Jul 07 23:15:09.548902 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:09.548871 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-5668757489-j9qbl"] Jul 07 23:15:09.737097 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:09.737064 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:15:09.833344 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:09.833259 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:15:09.933876 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:09.933834 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:15:10.031410 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:10.031374 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:15:10.161012 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:10.160930 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:15:10.267436 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:10.267406 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:15:12.844798 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:12.844758 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/14406dc0-620c-4615-b98a-b7205dc72966-metrics-certs\") pod \"network-metrics-daemon-ndz2q\" (UID: \"14406dc0-620c-4615-b98a-b7205dc72966\") " pod="openshift-multus/network-metrics-daemon-ndz2q" Jul 07 23:15:12.847066 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:12.847043 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/14406dc0-620c-4615-b98a-b7205dc72966-metrics-certs\") pod \"network-metrics-daemon-ndz2q\" (UID: \"14406dc0-620c-4615-b98a-b7205dc72966\") " pod="openshift-multus/network-metrics-daemon-ndz2q" Jul 07 23:15:12.918561 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:12.918527 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-hxkkf\"" Jul 07 23:15:12.926665 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:12.926649 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ndz2q" Jul 07 23:15:13.062201 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:13.062170 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-ndz2q"] Jul 07 23:15:13.065293 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:15:13.065267 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod14406dc0_620c_4615_b98a_b7205dc72966.slice/crio-1f527558b9e63bb00665c079285428f64bae136ef8a8574987b5bb471d5fefd8 WatchSource:0}: Error finding container 1f527558b9e63bb00665c079285428f64bae136ef8a8574987b5bb471d5fefd8: Status 404 returned error can't find the container with id 1f527558b9e63bb00665c079285428f64bae136ef8a8574987b5bb471d5fefd8 Jul 07 23:15:13.366328 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:13.366296 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-868585db77-hhcwd"] Jul 07 23:15:13.370765 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:13.370742 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-868585db77-hhcwd" Jul 07 23:15:13.373201 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:13.373182 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 07 23:15:13.373379 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:13.373365 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 07 23:15:13.373961 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:13.373926 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 07 23:15:13.373961 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:13.373948 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 07 23:15:13.374120 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:13.373962 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 07 23:15:13.374120 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:13.373994 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-m6lx8\"" Jul 07 23:15:13.374120 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:13.374003 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 07 23:15:13.374120 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:13.373951 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 07 23:15:13.382502 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:13.382480 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-868585db77-hhcwd"] Jul 07 23:15:13.449412 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:13.449378 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/991c6bc8-4897-4ccd-a50c-5baef9ea21bb-console-config\") pod \"console-868585db77-hhcwd\" (UID: \"991c6bc8-4897-4ccd-a50c-5baef9ea21bb\") " pod="openshift-console/console-868585db77-hhcwd" Jul 07 23:15:13.449412 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:13.449414 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/991c6bc8-4897-4ccd-a50c-5baef9ea21bb-console-serving-cert\") pod \"console-868585db77-hhcwd\" (UID: \"991c6bc8-4897-4ccd-a50c-5baef9ea21bb\") " pod="openshift-console/console-868585db77-hhcwd" Jul 07 23:15:13.449616 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:13.449439 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/991c6bc8-4897-4ccd-a50c-5baef9ea21bb-console-oauth-config\") pod \"console-868585db77-hhcwd\" (UID: \"991c6bc8-4897-4ccd-a50c-5baef9ea21bb\") " pod="openshift-console/console-868585db77-hhcwd" Jul 07 23:15:13.449616 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:13.449566 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/991c6bc8-4897-4ccd-a50c-5baef9ea21bb-oauth-serving-cert\") pod \"console-868585db77-hhcwd\" (UID: \"991c6bc8-4897-4ccd-a50c-5baef9ea21bb\") " pod="openshift-console/console-868585db77-hhcwd" Jul 07 23:15:13.449616 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:13.449593 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wlvln\" (UniqueName: \"kubernetes.io/projected/991c6bc8-4897-4ccd-a50c-5baef9ea21bb-kube-api-access-wlvln\") pod \"console-868585db77-hhcwd\" (UID: \"991c6bc8-4897-4ccd-a50c-5baef9ea21bb\") " pod="openshift-console/console-868585db77-hhcwd" Jul 07 23:15:13.449718 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:13.449615 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/991c6bc8-4897-4ccd-a50c-5baef9ea21bb-service-ca\") pod \"console-868585db77-hhcwd\" (UID: \"991c6bc8-4897-4ccd-a50c-5baef9ea21bb\") " pod="openshift-console/console-868585db77-hhcwd" Jul 07 23:15:13.550667 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:13.550634 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/991c6bc8-4897-4ccd-a50c-5baef9ea21bb-console-config\") pod \"console-868585db77-hhcwd\" (UID: \"991c6bc8-4897-4ccd-a50c-5baef9ea21bb\") " pod="openshift-console/console-868585db77-hhcwd" Jul 07 23:15:13.550838 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:13.550675 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/991c6bc8-4897-4ccd-a50c-5baef9ea21bb-console-serving-cert\") pod \"console-868585db77-hhcwd\" (UID: \"991c6bc8-4897-4ccd-a50c-5baef9ea21bb\") " pod="openshift-console/console-868585db77-hhcwd" Jul 07 23:15:13.550838 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:13.550717 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/991c6bc8-4897-4ccd-a50c-5baef9ea21bb-console-oauth-config\") pod \"console-868585db77-hhcwd\" (UID: \"991c6bc8-4897-4ccd-a50c-5baef9ea21bb\") " pod="openshift-console/console-868585db77-hhcwd" Jul 07 23:15:13.550838 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:13.550779 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/991c6bc8-4897-4ccd-a50c-5baef9ea21bb-oauth-serving-cert\") pod \"console-868585db77-hhcwd\" (UID: \"991c6bc8-4897-4ccd-a50c-5baef9ea21bb\") " pod="openshift-console/console-868585db77-hhcwd" Jul 07 23:15:13.550838 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:13.550802 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wlvln\" (UniqueName: \"kubernetes.io/projected/991c6bc8-4897-4ccd-a50c-5baef9ea21bb-kube-api-access-wlvln\") pod \"console-868585db77-hhcwd\" (UID: \"991c6bc8-4897-4ccd-a50c-5baef9ea21bb\") " pod="openshift-console/console-868585db77-hhcwd" Jul 07 23:15:13.550838 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:13.550832 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/991c6bc8-4897-4ccd-a50c-5baef9ea21bb-service-ca\") pod \"console-868585db77-hhcwd\" (UID: \"991c6bc8-4897-4ccd-a50c-5baef9ea21bb\") " pod="openshift-console/console-868585db77-hhcwd" Jul 07 23:15:13.551580 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:13.551553 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/991c6bc8-4897-4ccd-a50c-5baef9ea21bb-console-config\") pod \"console-868585db77-hhcwd\" (UID: \"991c6bc8-4897-4ccd-a50c-5baef9ea21bb\") " pod="openshift-console/console-868585db77-hhcwd" Jul 07 23:15:13.551711 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:13.551688 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/991c6bc8-4897-4ccd-a50c-5baef9ea21bb-service-ca\") pod \"console-868585db77-hhcwd\" (UID: \"991c6bc8-4897-4ccd-a50c-5baef9ea21bb\") " pod="openshift-console/console-868585db77-hhcwd" Jul 07 23:15:13.551777 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:13.551695 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/991c6bc8-4897-4ccd-a50c-5baef9ea21bb-oauth-serving-cert\") pod \"console-868585db77-hhcwd\" (UID: \"991c6bc8-4897-4ccd-a50c-5baef9ea21bb\") " pod="openshift-console/console-868585db77-hhcwd" Jul 07 23:15:13.553763 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:13.553734 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/991c6bc8-4897-4ccd-a50c-5baef9ea21bb-console-serving-cert\") pod \"console-868585db77-hhcwd\" (UID: \"991c6bc8-4897-4ccd-a50c-5baef9ea21bb\") " pod="openshift-console/console-868585db77-hhcwd" Jul 07 23:15:13.553859 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:13.553794 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/991c6bc8-4897-4ccd-a50c-5baef9ea21bb-console-oauth-config\") pod \"console-868585db77-hhcwd\" (UID: \"991c6bc8-4897-4ccd-a50c-5baef9ea21bb\") " pod="openshift-console/console-868585db77-hhcwd" Jul 07 23:15:13.568413 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:13.568387 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wlvln\" (UniqueName: \"kubernetes.io/projected/991c6bc8-4897-4ccd-a50c-5baef9ea21bb-kube-api-access-wlvln\") pod \"console-868585db77-hhcwd\" (UID: \"991c6bc8-4897-4ccd-a50c-5baef9ea21bb\") " pod="openshift-console/console-868585db77-hhcwd" Jul 07 23:15:13.650394 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:13.650301 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-ndz2q" event={"ID":"14406dc0-620c-4615-b98a-b7205dc72966","Type":"ContainerStarted","Data":"1f527558b9e63bb00665c079285428f64bae136ef8a8574987b5bb471d5fefd8"} Jul 07 23:15:13.680650 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:13.680610 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-868585db77-hhcwd" Jul 07 23:15:13.851894 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:13.851863 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-868585db77-hhcwd"] Jul 07 23:15:13.936185 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:15:13.936106 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod991c6bc8_4897_4ccd_a50c_5baef9ea21bb.slice/crio-2c3a33e564914ab9a8ba221b5cae540f401a55c08778c5c060876e161698157a WatchSource:0}: Error finding container 2c3a33e564914ab9a8ba221b5cae540f401a55c08778c5c060876e161698157a: Status 404 returned error can't find the container with id 2c3a33e564914ab9a8ba221b5cae540f401a55c08778c5c060876e161698157a Jul 07 23:15:14.621634 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:14.621583 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:15:14.662919 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:14.662871 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-868585db77-hhcwd" event={"ID":"991c6bc8-4897-4ccd-a50c-5baef9ea21bb","Type":"ContainerStarted","Data":"2c3a33e564914ab9a8ba221b5cae540f401a55c08778c5c060876e161698157a"} Jul 07 23:15:14.664850 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:14.664815 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-ndz2q" event={"ID":"14406dc0-620c-4615-b98a-b7205dc72966","Type":"ContainerStarted","Data":"f0d9f6c970cbea7e49f3686faa454946eafa617fa281ea103007567dace8d412"} Jul 07 23:15:14.664968 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:14.664864 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-ndz2q" event={"ID":"14406dc0-620c-4615-b98a-b7205dc72966","Type":"ContainerStarted","Data":"45ff05e9f3a6bb140477ea92d6f92d34131902622cc8e4abdb6214d2b9027be4"} Jul 07 23:15:14.694376 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:14.694326 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-ndz2q" podStartSLOduration=130.78040394 podStartE2EDuration="2m11.694311541s" totalImagesPullingTime="913.907601ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 23:13:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 23:15:14.691805784 +0000 UTC m=+132.088807320" watchObservedRunningTime="2026-07-07 23:15:14.694311541 +0000 UTC m=+132.091313053" Jul 07 23:15:14.720852 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:14.720813 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:15:14.894156 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:14.893984 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:15:15.029831 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:15.029796 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:15:15.713002 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:15.712968 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:15:15.775394 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:15.775358 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:15:17.678982 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:17.678886 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-868585db77-hhcwd" event={"ID":"991c6bc8-4897-4ccd-a50c-5baef9ea21bb","Type":"ContainerStarted","Data":"91a5d6daa33dda1dcfaac830671272e00e9bb125a279b3a12978aa934038f9d7"} Jul 07 23:15:17.701217 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:17.701156 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-868585db77-hhcwd" podStartSLOduration=1.3377001769999999 podStartE2EDuration="4.701137686s" totalImagesPullingTime="3.363437509s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 23:15:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 23:15:17.700874902 +0000 UTC m=+135.097876441" watchObservedRunningTime="2026-07-07 23:15:17.701137686 +0000 UTC m=+135.098139198" Jul 07 23:15:19.165407 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:19.165375 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:15:19.268733 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:19.268701 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:15:21.349337 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:21.349298 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:15:21.453806 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:21.453773 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:15:21.557694 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:21.557658 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:15:22.563623 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:22.563589 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:15:22.619966 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:22.619933 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:15:23.680880 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:23.680842 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-868585db77-hhcwd" Jul 07 23:15:23.681256 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:23.680908 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-868585db77-hhcwd" Jul 07 23:15:23.685750 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:23.685722 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-868585db77-hhcwd" Jul 07 23:15:23.706080 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:23.706051 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-868585db77-hhcwd" Jul 07 23:15:27.361389 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:27.361354 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:15:28.351063 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:28.351032 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:15:28.409457 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:28.409429 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:15:29.637581 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:29.637546 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:15:29.736736 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:29.736698 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:15:32.619982 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:32.619949 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-868585db77-hhcwd"] Jul 07 23:15:32.728364 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:32.728327 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:15:32.786579 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:32.786547 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:15:33.191857 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:33.191824 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:15:33.246723 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:33.246689 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:15:34.568923 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:34.568861 2570 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-image-registry/image-registry-5668757489-j9qbl" podUID="cf5100ad-da85-4da6-b099-d10259a611d2" containerName="registry" containerID="cri-o://cb9f7ccb7ab0b1c9f3d6902021225aea29675a4cfa0fd26596c46b8514398e05" gracePeriod=30 Jul 07 23:15:34.743851 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:34.743818 2570 generic.go:356] "Generic (PLEG): container finished" podID="cf5100ad-da85-4da6-b099-d10259a611d2" containerID="cb9f7ccb7ab0b1c9f3d6902021225aea29675a4cfa0fd26596c46b8514398e05" exitCode=0 Jul 07 23:15:34.743851 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:34.743838 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5668757489-j9qbl" event={"ID":"cf5100ad-da85-4da6-b099-d10259a611d2","Type":"ContainerDied","Data":"cb9f7ccb7ab0b1c9f3d6902021225aea29675a4cfa0fd26596c46b8514398e05"} Jul 07 23:15:34.822736 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:34.822678 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5668757489-j9qbl" Jul 07 23:15:34.928143 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:34.928101 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/cf5100ad-da85-4da6-b099-d10259a611d2-trusted-ca\") pod \"cf5100ad-da85-4da6-b099-d10259a611d2\" (UID: \"cf5100ad-da85-4da6-b099-d10259a611d2\") " Jul 07 23:15:34.928143 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:34.928145 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/cf5100ad-da85-4da6-b099-d10259a611d2-registry-tls\") pod \"cf5100ad-da85-4da6-b099-d10259a611d2\" (UID: \"cf5100ad-da85-4da6-b099-d10259a611d2\") " Jul 07 23:15:34.928393 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:34.928170 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xvdfj\" (UniqueName: \"kubernetes.io/projected/cf5100ad-da85-4da6-b099-d10259a611d2-kube-api-access-xvdfj\") pod \"cf5100ad-da85-4da6-b099-d10259a611d2\" (UID: \"cf5100ad-da85-4da6-b099-d10259a611d2\") " Jul 07 23:15:34.928393 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:34.928188 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/cf5100ad-da85-4da6-b099-d10259a611d2-registry-certificates\") pod \"cf5100ad-da85-4da6-b099-d10259a611d2\" (UID: \"cf5100ad-da85-4da6-b099-d10259a611d2\") " Jul 07 23:15:34.928393 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:34.928219 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/cf5100ad-da85-4da6-b099-d10259a611d2-installation-pull-secrets\") pod \"cf5100ad-da85-4da6-b099-d10259a611d2\" (UID: \"cf5100ad-da85-4da6-b099-d10259a611d2\") " Jul 07 23:15:34.928393 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:34.928246 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/cf5100ad-da85-4da6-b099-d10259a611d2-ca-trust-extracted\") pod \"cf5100ad-da85-4da6-b099-d10259a611d2\" (UID: \"cf5100ad-da85-4da6-b099-d10259a611d2\") " Jul 07 23:15:34.928393 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:34.928289 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/cf5100ad-da85-4da6-b099-d10259a611d2-bound-sa-token\") pod \"cf5100ad-da85-4da6-b099-d10259a611d2\" (UID: \"cf5100ad-da85-4da6-b099-d10259a611d2\") " Jul 07 23:15:34.928393 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:34.928318 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/cf5100ad-da85-4da6-b099-d10259a611d2-image-registry-private-configuration\") pod \"cf5100ad-da85-4da6-b099-d10259a611d2\" (UID: \"cf5100ad-da85-4da6-b099-d10259a611d2\") " Jul 07 23:15:34.928701 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:34.928608 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cf5100ad-da85-4da6-b099-d10259a611d2-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "cf5100ad-da85-4da6-b099-d10259a611d2" (UID: "cf5100ad-da85-4da6-b099-d10259a611d2"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 23:15:34.928752 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:34.928704 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cf5100ad-da85-4da6-b099-d10259a611d2-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "cf5100ad-da85-4da6-b099-d10259a611d2" (UID: "cf5100ad-da85-4da6-b099-d10259a611d2"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 23:15:34.930820 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:34.930787 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cf5100ad-da85-4da6-b099-d10259a611d2-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "cf5100ad-da85-4da6-b099-d10259a611d2" (UID: "cf5100ad-da85-4da6-b099-d10259a611d2"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 23:15:34.930972 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:34.930950 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cf5100ad-da85-4da6-b099-d10259a611d2-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "cf5100ad-da85-4da6-b099-d10259a611d2" (UID: "cf5100ad-da85-4da6-b099-d10259a611d2"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 23:15:34.931031 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:34.930994 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cf5100ad-da85-4da6-b099-d10259a611d2-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "cf5100ad-da85-4da6-b099-d10259a611d2" (UID: "cf5100ad-da85-4da6-b099-d10259a611d2"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 23:15:34.931079 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:34.931030 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cf5100ad-da85-4da6-b099-d10259a611d2-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "cf5100ad-da85-4da6-b099-d10259a611d2" (UID: "cf5100ad-da85-4da6-b099-d10259a611d2"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 23:15:34.931079 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:34.931040 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cf5100ad-da85-4da6-b099-d10259a611d2-kube-api-access-xvdfj" (OuterVolumeSpecName: "kube-api-access-xvdfj") pod "cf5100ad-da85-4da6-b099-d10259a611d2" (UID: "cf5100ad-da85-4da6-b099-d10259a611d2"). InnerVolumeSpecName "kube-api-access-xvdfj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 23:15:34.937208 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:34.937184 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cf5100ad-da85-4da6-b099-d10259a611d2-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "cf5100ad-da85-4da6-b099-d10259a611d2" (UID: "cf5100ad-da85-4da6-b099-d10259a611d2"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 23:15:35.029366 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:35.029334 2570 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/cf5100ad-da85-4da6-b099-d10259a611d2-trusted-ca\") on node \"ip-10-0-134-140.ec2.internal\" DevicePath \"\"" Jul 07 23:15:35.029366 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:35.029359 2570 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/cf5100ad-da85-4da6-b099-d10259a611d2-registry-tls\") on node \"ip-10-0-134-140.ec2.internal\" DevicePath \"\"" Jul 07 23:15:35.029366 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:35.029369 2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xvdfj\" (UniqueName: \"kubernetes.io/projected/cf5100ad-da85-4da6-b099-d10259a611d2-kube-api-access-xvdfj\") on node \"ip-10-0-134-140.ec2.internal\" DevicePath \"\"" Jul 07 23:15:35.029603 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:35.029378 2570 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/cf5100ad-da85-4da6-b099-d10259a611d2-registry-certificates\") on node \"ip-10-0-134-140.ec2.internal\" DevicePath \"\"" Jul 07 23:15:35.029603 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:35.029387 2570 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/cf5100ad-da85-4da6-b099-d10259a611d2-installation-pull-secrets\") on node \"ip-10-0-134-140.ec2.internal\" DevicePath \"\"" Jul 07 23:15:35.029603 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:35.029395 2570 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/cf5100ad-da85-4da6-b099-d10259a611d2-ca-trust-extracted\") on node \"ip-10-0-134-140.ec2.internal\" DevicePath \"\"" Jul 07 23:15:35.029603 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:35.029403 2570 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/cf5100ad-da85-4da6-b099-d10259a611d2-bound-sa-token\") on node \"ip-10-0-134-140.ec2.internal\" DevicePath \"\"" Jul 07 23:15:35.029603 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:35.029412 2570 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/cf5100ad-da85-4da6-b099-d10259a611d2-image-registry-private-configuration\") on node \"ip-10-0-134-140.ec2.internal\" DevicePath \"\"" Jul 07 23:15:35.748594 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:35.748567 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5668757489-j9qbl" Jul 07 23:15:35.749002 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:35.748568 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5668757489-j9qbl" event={"ID":"cf5100ad-da85-4da6-b099-d10259a611d2","Type":"ContainerDied","Data":"b599ca3139a8150dcb36dd3bf4faf09973ab24a4f056079a870759ec0f49eb6f"} Jul 07 23:15:35.749002 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:35.748692 2570 scope.go:117] "RemoveContainer" containerID="cb9f7ccb7ab0b1c9f3d6902021225aea29675a4cfa0fd26596c46b8514398e05" Jul 07 23:15:35.765210 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:35.765187 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-5668757489-j9qbl"] Jul 07 23:15:35.768986 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:35.768965 2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-5668757489-j9qbl"] Jul 07 23:15:37.095519 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:37.095489 2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cf5100ad-da85-4da6-b099-d10259a611d2" path="/var/lib/kubelet/pods/cf5100ad-da85-4da6-b099-d10259a611d2/volumes" Jul 07 23:15:39.764537 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:39.764444 2570 generic.go:356] "Generic (PLEG): container finished" podID="12b98da0-429d-4232-a0d6-b5c2699ec31a" containerID="6c12dcb40a67812930af4647cbda07711caa9258e2abf6b0bc8264a36e343c3d" exitCode=0 Jul 07 23:15:39.764537 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:39.764502 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5c847dcfc4-r6r4j" event={"ID":"12b98da0-429d-4232-a0d6-b5c2699ec31a","Type":"ContainerDied","Data":"6c12dcb40a67812930af4647cbda07711caa9258e2abf6b0bc8264a36e343c3d"} Jul 07 23:15:39.764906 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:39.764834 2570 scope.go:117] "RemoveContainer" containerID="6c12dcb40a67812930af4647cbda07711caa9258e2abf6b0bc8264a36e343c3d" Jul 07 23:15:40.769947 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:40.769912 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5c847dcfc4-r6r4j" event={"ID":"12b98da0-429d-4232-a0d6-b5c2699ec31a","Type":"ContainerStarted","Data":"3e4721872475aab595cbe395f2c605c46b9ac366333cd1062fe2e80605d315a7"} Jul 07 23:15:44.636373 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:44.636337 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:15:44.738660 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:44.738626 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:15:57.641637 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:57.641545 2570 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-868585db77-hhcwd" podUID="991c6bc8-4897-4ccd-a50c-5baef9ea21bb" containerName="console" containerID="cri-o://91a5d6daa33dda1dcfaac830671272e00e9bb125a279b3a12978aa934038f9d7" gracePeriod=15 Jul 07 23:15:57.841961 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:57.841928 2570 generic.go:356] "Generic (PLEG): container finished" podID="991c6bc8-4897-4ccd-a50c-5baef9ea21bb" containerID="91a5d6daa33dda1dcfaac830671272e00e9bb125a279b3a12978aa934038f9d7" exitCode=0 Jul 07 23:15:57.842116 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:57.841999 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-868585db77-hhcwd" event={"ID":"991c6bc8-4897-4ccd-a50c-5baef9ea21bb","Type":"ContainerDied","Data":"91a5d6daa33dda1dcfaac830671272e00e9bb125a279b3a12978aa934038f9d7"} Jul 07 23:15:57.881642 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:57.881619 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-868585db77-hhcwd" Jul 07 23:15:58.019819 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:58.019743 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/991c6bc8-4897-4ccd-a50c-5baef9ea21bb-console-config\") pod \"991c6bc8-4897-4ccd-a50c-5baef9ea21bb\" (UID: \"991c6bc8-4897-4ccd-a50c-5baef9ea21bb\") " Jul 07 23:15:58.019819 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:58.019780 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wlvln\" (UniqueName: \"kubernetes.io/projected/991c6bc8-4897-4ccd-a50c-5baef9ea21bb-kube-api-access-wlvln\") pod \"991c6bc8-4897-4ccd-a50c-5baef9ea21bb\" (UID: \"991c6bc8-4897-4ccd-a50c-5baef9ea21bb\") " Jul 07 23:15:58.019819 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:58.019810 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/991c6bc8-4897-4ccd-a50c-5baef9ea21bb-console-oauth-config\") pod \"991c6bc8-4897-4ccd-a50c-5baef9ea21bb\" (UID: \"991c6bc8-4897-4ccd-a50c-5baef9ea21bb\") " Jul 07 23:15:58.020043 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:58.019835 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/991c6bc8-4897-4ccd-a50c-5baef9ea21bb-console-serving-cert\") pod \"991c6bc8-4897-4ccd-a50c-5baef9ea21bb\" (UID: \"991c6bc8-4897-4ccd-a50c-5baef9ea21bb\") " Jul 07 23:15:58.020043 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:58.019910 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/991c6bc8-4897-4ccd-a50c-5baef9ea21bb-oauth-serving-cert\") pod \"991c6bc8-4897-4ccd-a50c-5baef9ea21bb\" (UID: \"991c6bc8-4897-4ccd-a50c-5baef9ea21bb\") " Jul 07 23:15:58.020043 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:58.019942 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/991c6bc8-4897-4ccd-a50c-5baef9ea21bb-service-ca\") pod \"991c6bc8-4897-4ccd-a50c-5baef9ea21bb\" (UID: \"991c6bc8-4897-4ccd-a50c-5baef9ea21bb\") " Jul 07 23:15:58.020287 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:58.020233 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/991c6bc8-4897-4ccd-a50c-5baef9ea21bb-console-config" (OuterVolumeSpecName: "console-config") pod "991c6bc8-4897-4ccd-a50c-5baef9ea21bb" (UID: "991c6bc8-4897-4ccd-a50c-5baef9ea21bb"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 23:15:58.020413 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:58.020343 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/991c6bc8-4897-4ccd-a50c-5baef9ea21bb-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "991c6bc8-4897-4ccd-a50c-5baef9ea21bb" (UID: "991c6bc8-4897-4ccd-a50c-5baef9ea21bb"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 23:15:58.020413 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:58.020395 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/991c6bc8-4897-4ccd-a50c-5baef9ea21bb-service-ca" (OuterVolumeSpecName: "service-ca") pod "991c6bc8-4897-4ccd-a50c-5baef9ea21bb" (UID: "991c6bc8-4897-4ccd-a50c-5baef9ea21bb"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 23:15:58.022131 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:58.022105 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/991c6bc8-4897-4ccd-a50c-5baef9ea21bb-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "991c6bc8-4897-4ccd-a50c-5baef9ea21bb" (UID: "991c6bc8-4897-4ccd-a50c-5baef9ea21bb"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 23:15:58.022448 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:58.022424 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/991c6bc8-4897-4ccd-a50c-5baef9ea21bb-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "991c6bc8-4897-4ccd-a50c-5baef9ea21bb" (UID: "991c6bc8-4897-4ccd-a50c-5baef9ea21bb"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 23:15:58.022448 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:58.022436 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/991c6bc8-4897-4ccd-a50c-5baef9ea21bb-kube-api-access-wlvln" (OuterVolumeSpecName: "kube-api-access-wlvln") pod "991c6bc8-4897-4ccd-a50c-5baef9ea21bb" (UID: "991c6bc8-4897-4ccd-a50c-5baef9ea21bb"). InnerVolumeSpecName "kube-api-access-wlvln". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 23:15:58.120694 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:58.120661 2570 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/991c6bc8-4897-4ccd-a50c-5baef9ea21bb-console-config\") on node \"ip-10-0-134-140.ec2.internal\" DevicePath \"\"" Jul 07 23:15:58.120694 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:58.120687 2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wlvln\" (UniqueName: \"kubernetes.io/projected/991c6bc8-4897-4ccd-a50c-5baef9ea21bb-kube-api-access-wlvln\") on node \"ip-10-0-134-140.ec2.internal\" DevicePath \"\"" Jul 07 23:15:58.120694 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:58.120697 2570 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/991c6bc8-4897-4ccd-a50c-5baef9ea21bb-console-oauth-config\") on node \"ip-10-0-134-140.ec2.internal\" DevicePath \"\"" Jul 07 23:15:58.120900 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:58.120706 2570 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/991c6bc8-4897-4ccd-a50c-5baef9ea21bb-console-serving-cert\") on node \"ip-10-0-134-140.ec2.internal\" DevicePath \"\"" Jul 07 23:15:58.120900 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:58.120715 2570 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/991c6bc8-4897-4ccd-a50c-5baef9ea21bb-oauth-serving-cert\") on node \"ip-10-0-134-140.ec2.internal\" DevicePath \"\"" Jul 07 23:15:58.120900 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:58.120722 2570 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/991c6bc8-4897-4ccd-a50c-5baef9ea21bb-service-ca\") on node \"ip-10-0-134-140.ec2.internal\" DevicePath \"\"" Jul 07 23:15:58.847476 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:58.847443 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-868585db77-hhcwd" event={"ID":"991c6bc8-4897-4ccd-a50c-5baef9ea21bb","Type":"ContainerDied","Data":"2c3a33e564914ab9a8ba221b5cae540f401a55c08778c5c060876e161698157a"} Jul 07 23:15:58.847875 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:58.847495 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-868585db77-hhcwd" Jul 07 23:15:58.847875 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:58.847521 2570 scope.go:117] "RemoveContainer" containerID="91a5d6daa33dda1dcfaac830671272e00e9bb125a279b3a12978aa934038f9d7" Jul 07 23:15:58.868414 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:58.868389 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-868585db77-hhcwd"] Jul 07 23:15:58.872486 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:58.872450 2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-868585db77-hhcwd"] Jul 07 23:15:59.095343 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:15:59.095310 2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="991c6bc8-4897-4ccd-a50c-5baef9ea21bb" path="/var/lib/kubelet/pods/991c6bc8-4897-4ccd-a50c-5baef9ea21bb/volumes" Jul 07 23:16:02.899219 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:16:02.899178 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:16:19.938330 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:16:19.938277 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:16:20.381894 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:16:20.381859 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:16:20.992678 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:16:20.992648 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 07 23:16:20.993056 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:16:20.992943 2570 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="5b8a514b-0b1a-4914-8629-cd4df7e35512" containerName="alertmanager" containerID="cri-o://7f3321ed31282d7770736f9381ad57bf89dbe8a8b13d836d8522929262473454" gracePeriod=120 Jul 07 23:16:20.993056 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:16:20.993016 2570 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="5b8a514b-0b1a-4914-8629-cd4df7e35512" containerName="kube-rbac-proxy-metric" containerID="cri-o://244617530e4181cfa1ec9b570685651a42e04b30df6bae462192289c7a1fc065" gracePeriod=120 Jul 07 23:16:20.993170 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:16:20.993071 2570 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="5b8a514b-0b1a-4914-8629-cd4df7e35512" containerName="kube-rbac-proxy" containerID="cri-o://0eb86e210ed0d8f6aa83ff35645458b6bacc06f1f9c4dad68d2b343e12454d37" gracePeriod=120 Jul 07 23:16:20.993170 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:16:20.993065 2570 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="5b8a514b-0b1a-4914-8629-cd4df7e35512" containerName="prom-label-proxy" containerID="cri-o://078b93de67102186596acb479778e1484d1dcc8ec1a36180a529e063c316ff74" gracePeriod=120 Jul 07 23:16:20.993170 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:16:20.993021 2570 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="5b8a514b-0b1a-4914-8629-cd4df7e35512" containerName="kube-rbac-proxy-web" containerID="cri-o://afc0814a7deb893adb5b56cd3bdfa0d4eace767b04b9ea30969e75ce84ab1c23" gracePeriod=120 Jul 07 23:16:20.993170 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:16:20.993089 2570 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="5b8a514b-0b1a-4914-8629-cd4df7e35512" containerName="config-reloader" containerID="cri-o://2ac24188b9b50c58c8e4cc0f75c62fbec8df79cd46cc4a3d31042c631af60b3f" gracePeriod=120 Jul 07 23:16:21.944867 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:16:21.944828 2570 generic.go:356] "Generic (PLEG): container finished" podID="5b8a514b-0b1a-4914-8629-cd4df7e35512" containerID="078b93de67102186596acb479778e1484d1dcc8ec1a36180a529e063c316ff74" exitCode=0 Jul 07 23:16:21.944867 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:16:21.944852 2570 generic.go:356] "Generic (PLEG): container finished" podID="5b8a514b-0b1a-4914-8629-cd4df7e35512" containerID="244617530e4181cfa1ec9b570685651a42e04b30df6bae462192289c7a1fc065" exitCode=0 Jul 07 23:16:21.944867 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:16:21.944859 2570 generic.go:356] "Generic (PLEG): container finished" podID="5b8a514b-0b1a-4914-8629-cd4df7e35512" containerID="0eb86e210ed0d8f6aa83ff35645458b6bacc06f1f9c4dad68d2b343e12454d37" exitCode=0 Jul 07 23:16:21.944867 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:16:21.944865 2570 generic.go:356] "Generic (PLEG): container finished" podID="5b8a514b-0b1a-4914-8629-cd4df7e35512" containerID="2ac24188b9b50c58c8e4cc0f75c62fbec8df79cd46cc4a3d31042c631af60b3f" exitCode=0 Jul 07 23:16:21.944867 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:16:21.944870 2570 generic.go:356] "Generic (PLEG): container finished" podID="5b8a514b-0b1a-4914-8629-cd4df7e35512" containerID="7f3321ed31282d7770736f9381ad57bf89dbe8a8b13d836d8522929262473454" exitCode=0 Jul 07 23:16:21.945242 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:16:21.944909 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"5b8a514b-0b1a-4914-8629-cd4df7e35512","Type":"ContainerDied","Data":"078b93de67102186596acb479778e1484d1dcc8ec1a36180a529e063c316ff74"} Jul 07 23:16:21.945242 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:16:21.944973 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"5b8a514b-0b1a-4914-8629-cd4df7e35512","Type":"ContainerDied","Data":"244617530e4181cfa1ec9b570685651a42e04b30df6bae462192289c7a1fc065"} Jul 07 23:16:21.945242 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:16:21.944998 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"5b8a514b-0b1a-4914-8629-cd4df7e35512","Type":"ContainerDied","Data":"0eb86e210ed0d8f6aa83ff35645458b6bacc06f1f9c4dad68d2b343e12454d37"} Jul 07 23:16:21.945242 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:16:21.945022 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"5b8a514b-0b1a-4914-8629-cd4df7e35512","Type":"ContainerDied","Data":"2ac24188b9b50c58c8e4cc0f75c62fbec8df79cd46cc4a3d31042c631af60b3f"} Jul 07 23:16:21.945242 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:16:21.945045 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"5b8a514b-0b1a-4914-8629-cd4df7e35512","Type":"ContainerDied","Data":"7f3321ed31282d7770736f9381ad57bf89dbe8a8b13d836d8522929262473454"} Jul 07 23:16:22.250485 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:16:22.250440 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 07 23:16:22.325932 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:16:22.325897 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/5b8a514b-0b1a-4914-8629-cd4df7e35512-config-volume\") pod \"5b8a514b-0b1a-4914-8629-cd4df7e35512\" (UID: \"5b8a514b-0b1a-4914-8629-cd4df7e35512\") " Jul 07 23:16:22.325932 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:16:22.325934 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/5b8a514b-0b1a-4914-8629-cd4df7e35512-tls-assets\") pod \"5b8a514b-0b1a-4914-8629-cd4df7e35512\" (UID: \"5b8a514b-0b1a-4914-8629-cd4df7e35512\") " Jul 07 23:16:22.326160 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:16:22.325962 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/5b8a514b-0b1a-4914-8629-cd4df7e35512-cluster-tls-config\") pod \"5b8a514b-0b1a-4914-8629-cd4df7e35512\" (UID: \"5b8a514b-0b1a-4914-8629-cd4df7e35512\") " Jul 07 23:16:22.326160 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:16:22.326075 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cdpdd\" (UniqueName: \"kubernetes.io/projected/5b8a514b-0b1a-4914-8629-cd4df7e35512-kube-api-access-cdpdd\") pod \"5b8a514b-0b1a-4914-8629-cd4df7e35512\" (UID: \"5b8a514b-0b1a-4914-8629-cd4df7e35512\") " Jul 07 23:16:22.326160 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:16:22.326132 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/5b8a514b-0b1a-4914-8629-cd4df7e35512-secret-alertmanager-kube-rbac-proxy-metric\") pod \"5b8a514b-0b1a-4914-8629-cd4df7e35512\" (UID: \"5b8a514b-0b1a-4914-8629-cd4df7e35512\") " Jul 07 23:16:22.326315 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:16:22.326172 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/5b8a514b-0b1a-4914-8629-cd4df7e35512-secret-alertmanager-kube-rbac-proxy\") pod \"5b8a514b-0b1a-4914-8629-cd4df7e35512\" (UID: \"5b8a514b-0b1a-4914-8629-cd4df7e35512\") " Jul 07 23:16:22.326315 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:16:22.326269 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/5b8a514b-0b1a-4914-8629-cd4df7e35512-web-config\") pod \"5b8a514b-0b1a-4914-8629-cd4df7e35512\" (UID: \"5b8a514b-0b1a-4914-8629-cd4df7e35512\") " Jul 07 23:16:22.326315 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:16:22.326302 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5b8a514b-0b1a-4914-8629-cd4df7e35512-metrics-client-ca\") pod \"5b8a514b-0b1a-4914-8629-cd4df7e35512\" (UID: \"5b8a514b-0b1a-4914-8629-cd4df7e35512\") " Jul 07 23:16:22.326479 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:16:22.326338 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/5b8a514b-0b1a-4914-8629-cd4df7e35512-secret-alertmanager-kube-rbac-proxy-web\") pod \"5b8a514b-0b1a-4914-8629-cd4df7e35512\" (UID: \"5b8a514b-0b1a-4914-8629-cd4df7e35512\") " Jul 07 23:16:22.326479 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:16:22.326378 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/5b8a514b-0b1a-4914-8629-cd4df7e35512-config-out\") pod \"5b8a514b-0b1a-4914-8629-cd4df7e35512\" (UID: \"5b8a514b-0b1a-4914-8629-cd4df7e35512\") " Jul 07 23:16:22.326479 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:16:22.326443 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/5b8a514b-0b1a-4914-8629-cd4df7e35512-secret-alertmanager-main-tls\") pod \"5b8a514b-0b1a-4914-8629-cd4df7e35512\" (UID: \"5b8a514b-0b1a-4914-8629-cd4df7e35512\") " Jul 07 23:16:22.326800 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:16:22.326769 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5b8a514b-0b1a-4914-8629-cd4df7e35512-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "5b8a514b-0b1a-4914-8629-cd4df7e35512" (UID: "5b8a514b-0b1a-4914-8629-cd4df7e35512"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 23:16:22.328950 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:16:22.328893 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5b8a514b-0b1a-4914-8629-cd4df7e35512-config-volume" (OuterVolumeSpecName: "config-volume") pod "5b8a514b-0b1a-4914-8629-cd4df7e35512" (UID: "5b8a514b-0b1a-4914-8629-cd4df7e35512"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 23:16:22.329110 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:16:22.328985 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5b8a514b-0b1a-4914-8629-cd4df7e35512-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "5b8a514b-0b1a-4914-8629-cd4df7e35512" (UID: "5b8a514b-0b1a-4914-8629-cd4df7e35512"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 23:16:22.329518 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:16:22.329422 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5b8a514b-0b1a-4914-8629-cd4df7e35512-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "5b8a514b-0b1a-4914-8629-cd4df7e35512" (UID: "5b8a514b-0b1a-4914-8629-cd4df7e35512"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 23:16:22.329640 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:16:22.329547 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5b8a514b-0b1a-4914-8629-cd4df7e35512-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "5b8a514b-0b1a-4914-8629-cd4df7e35512" (UID: "5b8a514b-0b1a-4914-8629-cd4df7e35512"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 23:16:22.329640 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:16:22.329572 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5b8a514b-0b1a-4914-8629-cd4df7e35512-kube-api-access-cdpdd" (OuterVolumeSpecName: "kube-api-access-cdpdd") pod "5b8a514b-0b1a-4914-8629-cd4df7e35512" (UID: "5b8a514b-0b1a-4914-8629-cd4df7e35512"). InnerVolumeSpecName "kube-api-access-cdpdd". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 23:16:22.330417 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:16:22.330388 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5b8a514b-0b1a-4914-8629-cd4df7e35512-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "5b8a514b-0b1a-4914-8629-cd4df7e35512" (UID: "5b8a514b-0b1a-4914-8629-cd4df7e35512"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 23:16:22.330584 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:16:22.330559 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5b8a514b-0b1a-4914-8629-cd4df7e35512-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "5b8a514b-0b1a-4914-8629-cd4df7e35512" (UID: "5b8a514b-0b1a-4914-8629-cd4df7e35512"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 23:16:22.330706 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:16:22.330687 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5b8a514b-0b1a-4914-8629-cd4df7e35512-config-out" (OuterVolumeSpecName: "config-out") pod "5b8a514b-0b1a-4914-8629-cd4df7e35512" (UID: "5b8a514b-0b1a-4914-8629-cd4df7e35512"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 23:16:22.334210 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:16:22.334188 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5b8a514b-0b1a-4914-8629-cd4df7e35512-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "5b8a514b-0b1a-4914-8629-cd4df7e35512" (UID: "5b8a514b-0b1a-4914-8629-cd4df7e35512"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 23:16:22.341242 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:16:22.341221 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5b8a514b-0b1a-4914-8629-cd4df7e35512-web-config" (OuterVolumeSpecName: "web-config") pod "5b8a514b-0b1a-4914-8629-cd4df7e35512" (UID: "5b8a514b-0b1a-4914-8629-cd4df7e35512"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 23:16:22.427703 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:16:22.427663 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5b8a514b-0b1a-4914-8629-cd4df7e35512-alertmanager-trusted-ca-bundle\") pod \"5b8a514b-0b1a-4914-8629-cd4df7e35512\" (UID: \"5b8a514b-0b1a-4914-8629-cd4df7e35512\") " Jul 07 23:16:22.427703 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:16:22.427704 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/5b8a514b-0b1a-4914-8629-cd4df7e35512-alertmanager-main-db\") pod \"5b8a514b-0b1a-4914-8629-cd4df7e35512\" (UID: \"5b8a514b-0b1a-4914-8629-cd4df7e35512\") " Jul 07 23:16:22.427933 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:16:22.427918 2570 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/5b8a514b-0b1a-4914-8629-cd4df7e35512-tls-assets\") on node \"ip-10-0-134-140.ec2.internal\" DevicePath \"\"" Jul 07 23:16:22.427966 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:16:22.427941 2570 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/5b8a514b-0b1a-4914-8629-cd4df7e35512-cluster-tls-config\") on node \"ip-10-0-134-140.ec2.internal\" DevicePath \"\"" Jul 07 23:16:22.427966 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:16:22.427956 2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cdpdd\" (UniqueName: \"kubernetes.io/projected/5b8a514b-0b1a-4914-8629-cd4df7e35512-kube-api-access-cdpdd\") on node \"ip-10-0-134-140.ec2.internal\" DevicePath \"\"" Jul 07 23:16:22.428055 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:16:22.427973 2570 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/5b8a514b-0b1a-4914-8629-cd4df7e35512-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-134-140.ec2.internal\" DevicePath \"\"" Jul 07 23:16:22.428055 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:16:22.427990 2570 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/5b8a514b-0b1a-4914-8629-cd4df7e35512-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-134-140.ec2.internal\" DevicePath \"\"" Jul 07 23:16:22.428055 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:16:22.428010 2570 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/5b8a514b-0b1a-4914-8629-cd4df7e35512-web-config\") on node \"ip-10-0-134-140.ec2.internal\" DevicePath \"\"" Jul 07 23:16:22.428055 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:16:22.428023 2570 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5b8a514b-0b1a-4914-8629-cd4df7e35512-metrics-client-ca\") on node \"ip-10-0-134-140.ec2.internal\" DevicePath \"\"" Jul 07 23:16:22.428055 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:16:22.428025 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5b8a514b-0b1a-4914-8629-cd4df7e35512-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "5b8a514b-0b1a-4914-8629-cd4df7e35512" (UID: "5b8a514b-0b1a-4914-8629-cd4df7e35512"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 23:16:22.428055 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:16:22.428036 2570 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/5b8a514b-0b1a-4914-8629-cd4df7e35512-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-134-140.ec2.internal\" DevicePath \"\"" Jul 07 23:16:22.428055 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:16:22.428051 2570 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/5b8a514b-0b1a-4914-8629-cd4df7e35512-config-out\") on node \"ip-10-0-134-140.ec2.internal\" DevicePath \"\"" Jul 07 23:16:22.428259 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:16:22.428065 2570 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/5b8a514b-0b1a-4914-8629-cd4df7e35512-secret-alertmanager-main-tls\") on node \"ip-10-0-134-140.ec2.internal\" DevicePath \"\"" Jul 07 23:16:22.428259 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:16:22.428082 2570 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/5b8a514b-0b1a-4914-8629-cd4df7e35512-config-volume\") on node \"ip-10-0-134-140.ec2.internal\" DevicePath \"\"" Jul 07 23:16:22.428259 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:16:22.428098 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5b8a514b-0b1a-4914-8629-cd4df7e35512-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "5b8a514b-0b1a-4914-8629-cd4df7e35512" (UID: "5b8a514b-0b1a-4914-8629-cd4df7e35512"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 23:16:22.528827 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:16:22.528803 2570 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5b8a514b-0b1a-4914-8629-cd4df7e35512-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-134-140.ec2.internal\" DevicePath \"\"" Jul 07 23:16:22.528827 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:16:22.528826 2570 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/5b8a514b-0b1a-4914-8629-cd4df7e35512-alertmanager-main-db\") on node \"ip-10-0-134-140.ec2.internal\" DevicePath \"\"" Jul 07 23:16:22.951731 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:16:22.951647 2570 generic.go:356] "Generic (PLEG): container finished" podID="5b8a514b-0b1a-4914-8629-cd4df7e35512" containerID="afc0814a7deb893adb5b56cd3bdfa0d4eace767b04b9ea30969e75ce84ab1c23" exitCode=0 Jul 07 23:16:22.951731 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:16:22.951722 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"5b8a514b-0b1a-4914-8629-cd4df7e35512","Type":"ContainerDied","Data":"afc0814a7deb893adb5b56cd3bdfa0d4eace767b04b9ea30969e75ce84ab1c23"} Jul 07 23:16:22.951899 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:16:22.951737 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 07 23:16:22.951899 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:16:22.951789 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"5b8a514b-0b1a-4914-8629-cd4df7e35512","Type":"ContainerDied","Data":"04680da060bc71722a5ccca835abb89e66b5531d24cdb6bdcbaf639cdd017ef7"} Jul 07 23:16:22.951899 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:16:22.951823 2570 scope.go:117] "RemoveContainer" containerID="078b93de67102186596acb479778e1484d1dcc8ec1a36180a529e063c316ff74" Jul 07 23:16:22.959308 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:16:22.959290 2570 scope.go:117] "RemoveContainer" containerID="244617530e4181cfa1ec9b570685651a42e04b30df6bae462192289c7a1fc065" Jul 07 23:16:22.966351 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:16:22.966334 2570 scope.go:117] "RemoveContainer" containerID="0eb86e210ed0d8f6aa83ff35645458b6bacc06f1f9c4dad68d2b343e12454d37" Jul 07 23:16:22.973256 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:16:22.973239 2570 scope.go:117] "RemoveContainer" containerID="afc0814a7deb893adb5b56cd3bdfa0d4eace767b04b9ea30969e75ce84ab1c23" Jul 07 23:16:22.976234 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:16:22.976210 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 07 23:16:22.980571 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:16:22.980441 2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 07 23:16:22.980646 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:16:22.980625 2570 scope.go:117] "RemoveContainer" containerID="2ac24188b9b50c58c8e4cc0f75c62fbec8df79cd46cc4a3d31042c631af60b3f" Jul 07 23:16:22.987626 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:16:22.987606 2570 scope.go:117] "RemoveContainer" containerID="7f3321ed31282d7770736f9381ad57bf89dbe8a8b13d836d8522929262473454" Jul 07 23:16:22.994241 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:16:22.994227 2570 scope.go:117] "RemoveContainer" containerID="0950b95b7cda66668ca633f211d0abfd56cc63552dd86e5380152fecde7326f9" Jul 07 23:16:23.000660 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:16:23.000607 2570 scope.go:117] "RemoveContainer" containerID="078b93de67102186596acb479778e1484d1dcc8ec1a36180a529e063c316ff74" Jul 07 23:16:23.000921 ip-10-0-134-140 kubenswrapper[2570]: E0707 23:16:23.000895 2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"078b93de67102186596acb479778e1484d1dcc8ec1a36180a529e063c316ff74\": container with ID starting with 078b93de67102186596acb479778e1484d1dcc8ec1a36180a529e063c316ff74 not found: ID does not exist" containerID="078b93de67102186596acb479778e1484d1dcc8ec1a36180a529e063c316ff74" Jul 07 23:16:23.000998 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:16:23.000931 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"078b93de67102186596acb479778e1484d1dcc8ec1a36180a529e063c316ff74"} err="failed to get container status \"078b93de67102186596acb479778e1484d1dcc8ec1a36180a529e063c316ff74\": rpc error: code = NotFound desc = could not find container \"078b93de67102186596acb479778e1484d1dcc8ec1a36180a529e063c316ff74\": container with ID starting with 078b93de67102186596acb479778e1484d1dcc8ec1a36180a529e063c316ff74 not found: ID does not exist" Jul 07 23:16:23.000998 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:16:23.000971 2570 scope.go:117] "RemoveContainer" containerID="244617530e4181cfa1ec9b570685651a42e04b30df6bae462192289c7a1fc065" Jul 07 23:16:23.001275 ip-10-0-134-140 kubenswrapper[2570]: E0707 23:16:23.001248 2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"244617530e4181cfa1ec9b570685651a42e04b30df6bae462192289c7a1fc065\": container with ID starting with 244617530e4181cfa1ec9b570685651a42e04b30df6bae462192289c7a1fc065 not found: ID does not exist" containerID="244617530e4181cfa1ec9b570685651a42e04b30df6bae462192289c7a1fc065" Jul 07 23:16:23.001340 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:16:23.001273 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"244617530e4181cfa1ec9b570685651a42e04b30df6bae462192289c7a1fc065"} err="failed to get container status \"244617530e4181cfa1ec9b570685651a42e04b30df6bae462192289c7a1fc065\": rpc error: code = NotFound desc = could not find container \"244617530e4181cfa1ec9b570685651a42e04b30df6bae462192289c7a1fc065\": container with ID starting with 244617530e4181cfa1ec9b570685651a42e04b30df6bae462192289c7a1fc065 not found: ID does not exist" Jul 07 23:16:23.001340 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:16:23.001290 2570 scope.go:117] "RemoveContainer" containerID="0eb86e210ed0d8f6aa83ff35645458b6bacc06f1f9c4dad68d2b343e12454d37" Jul 07 23:16:23.001585 ip-10-0-134-140 kubenswrapper[2570]: E0707 23:16:23.001566 2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0eb86e210ed0d8f6aa83ff35645458b6bacc06f1f9c4dad68d2b343e12454d37\": container with ID starting with 0eb86e210ed0d8f6aa83ff35645458b6bacc06f1f9c4dad68d2b343e12454d37 not found: ID does not exist" containerID="0eb86e210ed0d8f6aa83ff35645458b6bacc06f1f9c4dad68d2b343e12454d37" Jul 07 23:16:23.001646 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:16:23.001593 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0eb86e210ed0d8f6aa83ff35645458b6bacc06f1f9c4dad68d2b343e12454d37"} err="failed to get container status \"0eb86e210ed0d8f6aa83ff35645458b6bacc06f1f9c4dad68d2b343e12454d37\": rpc error: code = NotFound desc = could not find container \"0eb86e210ed0d8f6aa83ff35645458b6bacc06f1f9c4dad68d2b343e12454d37\": container with ID starting with 0eb86e210ed0d8f6aa83ff35645458b6bacc06f1f9c4dad68d2b343e12454d37 not found: ID does not exist" Jul 07 23:16:23.001646 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:16:23.001611 2570 scope.go:117] "RemoveContainer" containerID="afc0814a7deb893adb5b56cd3bdfa0d4eace767b04b9ea30969e75ce84ab1c23" Jul 07 23:16:23.001884 ip-10-0-134-140 kubenswrapper[2570]: E0707 23:16:23.001867 2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"afc0814a7deb893adb5b56cd3bdfa0d4eace767b04b9ea30969e75ce84ab1c23\": container with ID starting with afc0814a7deb893adb5b56cd3bdfa0d4eace767b04b9ea30969e75ce84ab1c23 not found: ID does not exist" containerID="afc0814a7deb893adb5b56cd3bdfa0d4eace767b04b9ea30969e75ce84ab1c23" Jul 07 23:16:23.001944 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:16:23.001889 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"afc0814a7deb893adb5b56cd3bdfa0d4eace767b04b9ea30969e75ce84ab1c23"} err="failed to get container status \"afc0814a7deb893adb5b56cd3bdfa0d4eace767b04b9ea30969e75ce84ab1c23\": rpc error: code = NotFound desc = could not find container \"afc0814a7deb893adb5b56cd3bdfa0d4eace767b04b9ea30969e75ce84ab1c23\": container with ID starting with afc0814a7deb893adb5b56cd3bdfa0d4eace767b04b9ea30969e75ce84ab1c23 not found: ID does not exist" Jul 07 23:16:23.001944 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:16:23.001904 2570 scope.go:117] "RemoveContainer" containerID="2ac24188b9b50c58c8e4cc0f75c62fbec8df79cd46cc4a3d31042c631af60b3f" Jul 07 23:16:23.002142 ip-10-0-134-140 kubenswrapper[2570]: E0707 23:16:23.002124 2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2ac24188b9b50c58c8e4cc0f75c62fbec8df79cd46cc4a3d31042c631af60b3f\": container with ID starting with 2ac24188b9b50c58c8e4cc0f75c62fbec8df79cd46cc4a3d31042c631af60b3f not found: ID does not exist" containerID="2ac24188b9b50c58c8e4cc0f75c62fbec8df79cd46cc4a3d31042c631af60b3f" Jul 07 23:16:23.002197 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:16:23.002151 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2ac24188b9b50c58c8e4cc0f75c62fbec8df79cd46cc4a3d31042c631af60b3f"} err="failed to get container status \"2ac24188b9b50c58c8e4cc0f75c62fbec8df79cd46cc4a3d31042c631af60b3f\": rpc error: code = NotFound desc = could not find container \"2ac24188b9b50c58c8e4cc0f75c62fbec8df79cd46cc4a3d31042c631af60b3f\": container with ID starting with 2ac24188b9b50c58c8e4cc0f75c62fbec8df79cd46cc4a3d31042c631af60b3f not found: ID does not exist" Jul 07 23:16:23.002197 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:16:23.002172 2570 scope.go:117] "RemoveContainer" containerID="7f3321ed31282d7770736f9381ad57bf89dbe8a8b13d836d8522929262473454" Jul 07 23:16:23.002382 ip-10-0-134-140 kubenswrapper[2570]: E0707 23:16:23.002368 2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7f3321ed31282d7770736f9381ad57bf89dbe8a8b13d836d8522929262473454\": container with ID starting with 7f3321ed31282d7770736f9381ad57bf89dbe8a8b13d836d8522929262473454 not found: ID does not exist" containerID="7f3321ed31282d7770736f9381ad57bf89dbe8a8b13d836d8522929262473454" Jul 07 23:16:23.002440 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:16:23.002389 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7f3321ed31282d7770736f9381ad57bf89dbe8a8b13d836d8522929262473454"} err="failed to get container status \"7f3321ed31282d7770736f9381ad57bf89dbe8a8b13d836d8522929262473454\": rpc error: code = NotFound desc = could not find container \"7f3321ed31282d7770736f9381ad57bf89dbe8a8b13d836d8522929262473454\": container with ID starting with 7f3321ed31282d7770736f9381ad57bf89dbe8a8b13d836d8522929262473454 not found: ID does not exist" Jul 07 23:16:23.002440 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:16:23.002409 2570 scope.go:117] "RemoveContainer" containerID="0950b95b7cda66668ca633f211d0abfd56cc63552dd86e5380152fecde7326f9" Jul 07 23:16:23.002688 ip-10-0-134-140 kubenswrapper[2570]: E0707 23:16:23.002669 2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0950b95b7cda66668ca633f211d0abfd56cc63552dd86e5380152fecde7326f9\": container with ID starting with 0950b95b7cda66668ca633f211d0abfd56cc63552dd86e5380152fecde7326f9 not found: ID does not exist" containerID="0950b95b7cda66668ca633f211d0abfd56cc63552dd86e5380152fecde7326f9" Jul 07 23:16:23.002740 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:16:23.002693 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0950b95b7cda66668ca633f211d0abfd56cc63552dd86e5380152fecde7326f9"} err="failed to get container status \"0950b95b7cda66668ca633f211d0abfd56cc63552dd86e5380152fecde7326f9\": rpc error: code = NotFound desc = could not find container \"0950b95b7cda66668ca633f211d0abfd56cc63552dd86e5380152fecde7326f9\": container with ID starting with 0950b95b7cda66668ca633f211d0abfd56cc63552dd86e5380152fecde7326f9 not found: ID does not exist" Jul 07 23:16:23.011146 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:16:23.011117 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 07 23:16:23.011410 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:16:23.011383 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="991c6bc8-4897-4ccd-a50c-5baef9ea21bb" containerName="console" Jul 07 23:16:23.011410 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:16:23.011397 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="991c6bc8-4897-4ccd-a50c-5baef9ea21bb" containerName="console" Jul 07 23:16:23.011410 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:16:23.011409 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5b8a514b-0b1a-4914-8629-cd4df7e35512" containerName="alertmanager" Jul 07 23:16:23.011547 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:16:23.011415 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="5b8a514b-0b1a-4914-8629-cd4df7e35512" containerName="alertmanager" Jul 07 23:16:23.011547 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:16:23.011429 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5b8a514b-0b1a-4914-8629-cd4df7e35512" containerName="kube-rbac-proxy-web" Jul 07 23:16:23.011547 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:16:23.011434 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="5b8a514b-0b1a-4914-8629-cd4df7e35512" containerName="kube-rbac-proxy-web" Jul 07 23:16:23.011547 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:16:23.011441 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5b8a514b-0b1a-4914-8629-cd4df7e35512" containerName="kube-rbac-proxy" Jul 07 23:16:23.011547 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:16:23.011446 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="5b8a514b-0b1a-4914-8629-cd4df7e35512" containerName="kube-rbac-proxy" Jul 07 23:16:23.011547 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:16:23.011455 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5b8a514b-0b1a-4914-8629-cd4df7e35512" containerName="config-reloader" Jul 07 23:16:23.011547 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:16:23.011460 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="5b8a514b-0b1a-4914-8629-cd4df7e35512" containerName="config-reloader" Jul 07 23:16:23.011547 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:16:23.011487 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5b8a514b-0b1a-4914-8629-cd4df7e35512" containerName="init-config-reloader" Jul 07 23:16:23.011547 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:16:23.011492 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="5b8a514b-0b1a-4914-8629-cd4df7e35512" containerName="init-config-reloader" Jul 07 23:16:23.011547 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:16:23.011500 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5b8a514b-0b1a-4914-8629-cd4df7e35512" containerName="kube-rbac-proxy-metric" Jul 07 23:16:23.011547 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:16:23.011507 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="5b8a514b-0b1a-4914-8629-cd4df7e35512" containerName="kube-rbac-proxy-metric" Jul 07 23:16:23.011547 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:16:23.011514 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cf5100ad-da85-4da6-b099-d10259a611d2" containerName="registry" Jul 07 23:16:23.011547 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:16:23.011520 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="cf5100ad-da85-4da6-b099-d10259a611d2" containerName="registry" Jul 07 23:16:23.011547 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:16:23.011527 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5b8a514b-0b1a-4914-8629-cd4df7e35512" containerName="prom-label-proxy" Jul 07 23:16:23.011547 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:16:23.011532 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="5b8a514b-0b1a-4914-8629-cd4df7e35512" containerName="prom-label-proxy" Jul 07 23:16:23.011948 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:16:23.011576 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="5b8a514b-0b1a-4914-8629-cd4df7e35512" containerName="prom-label-proxy" Jul 07 23:16:23.011948 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:16:23.011583 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="5b8a514b-0b1a-4914-8629-cd4df7e35512" containerName="kube-rbac-proxy-metric" Jul 07 23:16:23.011948 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:16:23.011591 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="5b8a514b-0b1a-4914-8629-cd4df7e35512" containerName="config-reloader" Jul 07 23:16:23.011948 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:16:23.011597 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="5b8a514b-0b1a-4914-8629-cd4df7e35512" containerName="alertmanager" Jul 07 23:16:23.011948 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:16:23.011604 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="5b8a514b-0b1a-4914-8629-cd4df7e35512" containerName="kube-rbac-proxy-web" Jul 07 23:16:23.011948 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:16:23.011610 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="5b8a514b-0b1a-4914-8629-cd4df7e35512" containerName="kube-rbac-proxy" Jul 07 23:16:23.011948 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:16:23.011615 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="cf5100ad-da85-4da6-b099-d10259a611d2" containerName="registry" Jul 07 23:16:23.011948 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:16:23.011620 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="991c6bc8-4897-4ccd-a50c-5baef9ea21bb" containerName="console" Jul 07 23:16:23.016897 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:16:23.016878 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 07 23:16:23.020334 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:16:23.020312 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jul 07 23:16:23.020542 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:16:23.020521 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jul 07 23:16:23.020665 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:16:23.020648 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jul 07 23:16:23.020802 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:16:23.020785 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jul 07 23:16:23.020913 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:16:23.020856 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jul 07 23:16:23.020973 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:16:23.020946 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jul 07 23:16:23.021118 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:16:23.021104 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jul 07 23:16:23.021449 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:16:23.021432 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jul 07 23:16:23.021546 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:16:23.021503 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-rd4kk\"" Jul 07 23:16:23.029512 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:16:23.028360 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jul 07 23:16:23.031989 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:16:23.031969 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/ae3fe82c-0bfa-4507-a1db-ade9479e1c26-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"ae3fe82c-0bfa-4507-a1db-ade9479e1c26\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 23:16:23.032152 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:16:23.032011 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ae3fe82c-0bfa-4507-a1db-ade9479e1c26-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"ae3fe82c-0bfa-4507-a1db-ade9479e1c26\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 23:16:23.032152 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:16:23.032039 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/ae3fe82c-0bfa-4507-a1db-ade9479e1c26-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"ae3fe82c-0bfa-4507-a1db-ade9479e1c26\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 23:16:23.032152 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:16:23.032102 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/ae3fe82c-0bfa-4507-a1db-ade9479e1c26-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"ae3fe82c-0bfa-4507-a1db-ade9479e1c26\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 23:16:23.032286 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:16:23.032148 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/ae3fe82c-0bfa-4507-a1db-ade9479e1c26-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"ae3fe82c-0bfa-4507-a1db-ade9479e1c26\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 23:16:23.032286 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:16:23.032183 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/ae3fe82c-0bfa-4507-a1db-ade9479e1c26-tls-assets\") pod \"alertmanager-main-0\" (UID: \"ae3fe82c-0bfa-4507-a1db-ade9479e1c26\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 23:16:23.032286 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:16:23.032238 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/ae3fe82c-0bfa-4507-a1db-ade9479e1c26-config-volume\") pod \"alertmanager-main-0\" (UID: \"ae3fe82c-0bfa-4507-a1db-ade9479e1c26\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 23:16:23.032286 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:16:23.032272 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/ae3fe82c-0bfa-4507-a1db-ade9479e1c26-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"ae3fe82c-0bfa-4507-a1db-ade9479e1c26\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 23:16:23.032437 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:16:23.032289 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/ae3fe82c-0bfa-4507-a1db-ade9479e1c26-web-config\") pod \"alertmanager-main-0\" (UID: \"ae3fe82c-0bfa-4507-a1db-ade9479e1c26\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 23:16:23.032437 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:16:23.032330 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/ae3fe82c-0bfa-4507-a1db-ade9479e1c26-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"ae3fe82c-0bfa-4507-a1db-ade9479e1c26\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 23:16:23.032437 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:16:23.032360 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ae3fe82c-0bfa-4507-a1db-ade9479e1c26-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"ae3fe82c-0bfa-4507-a1db-ade9479e1c26\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 23:16:23.032437 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:16:23.032397 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jm52x\" (UniqueName: \"kubernetes.io/projected/ae3fe82c-0bfa-4507-a1db-ade9479e1c26-kube-api-access-jm52x\") pod \"alertmanager-main-0\" (UID: \"ae3fe82c-0bfa-4507-a1db-ade9479e1c26\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 23:16:23.032655 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:16:23.032449 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/ae3fe82c-0bfa-4507-a1db-ade9479e1c26-config-out\") pod \"alertmanager-main-0\" (UID: \"ae3fe82c-0bfa-4507-a1db-ade9479e1c26\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 23:16:23.035339 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:16:23.035314 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 07 23:16:23.095169 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:16:23.095138 2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5b8a514b-0b1a-4914-8629-cd4df7e35512" path="/var/lib/kubelet/pods/5b8a514b-0b1a-4914-8629-cd4df7e35512/volumes" Jul 07 23:16:23.133694 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:16:23.133667 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ae3fe82c-0bfa-4507-a1db-ade9479e1c26-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"ae3fe82c-0bfa-4507-a1db-ade9479e1c26\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 23:16:23.133783 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:16:23.133723 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jm52x\" (UniqueName: \"kubernetes.io/projected/ae3fe82c-0bfa-4507-a1db-ade9479e1c26-kube-api-access-jm52x\") pod \"alertmanager-main-0\" (UID: \"ae3fe82c-0bfa-4507-a1db-ade9479e1c26\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 23:16:23.133783 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:16:23.133754 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/ae3fe82c-0bfa-4507-a1db-ade9479e1c26-config-out\") pod \"alertmanager-main-0\" (UID: \"ae3fe82c-0bfa-4507-a1db-ade9479e1c26\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 23:16:23.133868 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:16:23.133840 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/ae3fe82c-0bfa-4507-a1db-ade9479e1c26-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"ae3fe82c-0bfa-4507-a1db-ade9479e1c26\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 23:16:23.133916 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:16:23.133884 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ae3fe82c-0bfa-4507-a1db-ade9479e1c26-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"ae3fe82c-0bfa-4507-a1db-ade9479e1c26\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 23:16:23.133953 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:16:23.133914 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/ae3fe82c-0bfa-4507-a1db-ade9479e1c26-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"ae3fe82c-0bfa-4507-a1db-ade9479e1c26\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 23:16:23.133953 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:16:23.133934 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/ae3fe82c-0bfa-4507-a1db-ade9479e1c26-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"ae3fe82c-0bfa-4507-a1db-ade9479e1c26\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 23:16:23.134132 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:16:23.133963 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/ae3fe82c-0bfa-4507-a1db-ade9479e1c26-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"ae3fe82c-0bfa-4507-a1db-ade9479e1c26\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 23:16:23.134132 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:16:23.133999 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/ae3fe82c-0bfa-4507-a1db-ade9479e1c26-tls-assets\") pod \"alertmanager-main-0\" (UID: \"ae3fe82c-0bfa-4507-a1db-ade9479e1c26\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 23:16:23.134132 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:16:23.134029 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/ae3fe82c-0bfa-4507-a1db-ade9479e1c26-config-volume\") pod \"alertmanager-main-0\" (UID: \"ae3fe82c-0bfa-4507-a1db-ade9479e1c26\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 23:16:23.134132 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:16:23.134052 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/ae3fe82c-0bfa-4507-a1db-ade9479e1c26-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"ae3fe82c-0bfa-4507-a1db-ade9479e1c26\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 23:16:23.134132 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:16:23.134077 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/ae3fe82c-0bfa-4507-a1db-ade9479e1c26-web-config\") pod \"alertmanager-main-0\" (UID: \"ae3fe82c-0bfa-4507-a1db-ade9479e1c26\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 23:16:23.134132 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:16:23.134110 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/ae3fe82c-0bfa-4507-a1db-ade9479e1c26-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"ae3fe82c-0bfa-4507-a1db-ade9479e1c26\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 23:16:23.134620 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:16:23.134203 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/ae3fe82c-0bfa-4507-a1db-ade9479e1c26-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"ae3fe82c-0bfa-4507-a1db-ade9479e1c26\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 23:16:23.134620 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:16:23.134517 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ae3fe82c-0bfa-4507-a1db-ade9479e1c26-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"ae3fe82c-0bfa-4507-a1db-ade9479e1c26\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 23:16:23.135013 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:16:23.134985 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ae3fe82c-0bfa-4507-a1db-ade9479e1c26-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"ae3fe82c-0bfa-4507-a1db-ade9479e1c26\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 23:16:23.136931 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:16:23.136884 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/ae3fe82c-0bfa-4507-a1db-ade9479e1c26-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"ae3fe82c-0bfa-4507-a1db-ade9479e1c26\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 23:16:23.137199 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:16:23.137176 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/ae3fe82c-0bfa-4507-a1db-ade9479e1c26-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"ae3fe82c-0bfa-4507-a1db-ade9479e1c26\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 23:16:23.137263 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:16:23.137194 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/ae3fe82c-0bfa-4507-a1db-ade9479e1c26-web-config\") pod \"alertmanager-main-0\" (UID: \"ae3fe82c-0bfa-4507-a1db-ade9479e1c26\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 23:16:23.137446 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:16:23.137425 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/ae3fe82c-0bfa-4507-a1db-ade9479e1c26-config-out\") pod \"alertmanager-main-0\" (UID: \"ae3fe82c-0bfa-4507-a1db-ade9479e1c26\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 23:16:23.137707 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:16:23.137686 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/ae3fe82c-0bfa-4507-a1db-ade9479e1c26-config-volume\") pod \"alertmanager-main-0\" (UID: \"ae3fe82c-0bfa-4507-a1db-ade9479e1c26\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 23:16:23.137774 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:16:23.137694 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/ae3fe82c-0bfa-4507-a1db-ade9479e1c26-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"ae3fe82c-0bfa-4507-a1db-ade9479e1c26\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 23:16:23.138107 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:16:23.138086 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/ae3fe82c-0bfa-4507-a1db-ade9479e1c26-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"ae3fe82c-0bfa-4507-a1db-ade9479e1c26\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 23:16:23.138149 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:16:23.138091 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/ae3fe82c-0bfa-4507-a1db-ade9479e1c26-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"ae3fe82c-0bfa-4507-a1db-ade9479e1c26\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 23:16:23.138520 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:16:23.138504 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/ae3fe82c-0bfa-4507-a1db-ade9479e1c26-tls-assets\") pod \"alertmanager-main-0\" (UID: \"ae3fe82c-0bfa-4507-a1db-ade9479e1c26\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 23:16:23.144801 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:16:23.144785 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jm52x\" (UniqueName: \"kubernetes.io/projected/ae3fe82c-0bfa-4507-a1db-ade9479e1c26-kube-api-access-jm52x\") pod \"alertmanager-main-0\" (UID: \"ae3fe82c-0bfa-4507-a1db-ade9479e1c26\") " pod="openshift-monitoring/alertmanager-main-0" Jul 07 23:16:23.326482 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:16:23.326435 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 07 23:16:23.466654 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:16:23.466622 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 07 23:16:23.469497 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:16:23.469449 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podae3fe82c_0bfa_4507_a1db_ade9479e1c26.slice/crio-6eba405a8ae9010e6390cb7c8703a7184d0012b3c6947ddfb8f432c171591f9a WatchSource:0}: Error finding container 6eba405a8ae9010e6390cb7c8703a7184d0012b3c6947ddfb8f432c171591f9a: Status 404 returned error can't find the container with id 6eba405a8ae9010e6390cb7c8703a7184d0012b3c6947ddfb8f432c171591f9a Jul 07 23:16:23.956645 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:16:23.956614 2570 generic.go:356] "Generic (PLEG): container finished" podID="ae3fe82c-0bfa-4507-a1db-ade9479e1c26" containerID="8c9c9cbfb959e854ad6360d8e15f14b5e86bf6beb632f3b567b2968e4ded9fa9" exitCode=0 Jul 07 23:16:23.956806 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:16:23.956695 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ae3fe82c-0bfa-4507-a1db-ade9479e1c26","Type":"ContainerDied","Data":"8c9c9cbfb959e854ad6360d8e15f14b5e86bf6beb632f3b567b2968e4ded9fa9"} Jul 07 23:16:23.956806 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:16:23.956761 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ae3fe82c-0bfa-4507-a1db-ade9479e1c26","Type":"ContainerStarted","Data":"6eba405a8ae9010e6390cb7c8703a7184d0012b3c6947ddfb8f432c171591f9a"} Jul 07 23:16:24.964927 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:16:24.964887 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ae3fe82c-0bfa-4507-a1db-ade9479e1c26","Type":"ContainerStarted","Data":"97d0b07d1b7428cbdfb72d5c768be4ee80c7d1c26c2a7781ca3cd2a164f3436b"} Jul 07 23:16:24.965376 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:16:24.965036 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ae3fe82c-0bfa-4507-a1db-ade9479e1c26","Type":"ContainerStarted","Data":"55c561318bed6fc83fce9279188f206110f9d30d299db9235f2a12e5dfffb183"} Jul 07 23:16:24.965376 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:16:24.965152 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ae3fe82c-0bfa-4507-a1db-ade9479e1c26","Type":"ContainerStarted","Data":"b2f1de6b4b50244a4aa19bc8934febbcd446184b9080a43a5e8a42d2e716a6cd"} Jul 07 23:16:24.965376 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:16:24.965194 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ae3fe82c-0bfa-4507-a1db-ade9479e1c26","Type":"ContainerStarted","Data":"0e676ab08cff93bbf0cc96e7c95493ae1186299e0e0683cad15db65debb77400"} Jul 07 23:16:24.965376 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:16:24.965231 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ae3fe82c-0bfa-4507-a1db-ade9479e1c26","Type":"ContainerStarted","Data":"e8ad729ce5edc070f82c33def100e5b57461b781423e547d78980a62e804e7c1"} Jul 07 23:16:24.965376 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:16:24.965274 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ae3fe82c-0bfa-4507-a1db-ade9479e1c26","Type":"ContainerStarted","Data":"fe0bc02b90faf230b2da7f799327ad9ca2672800dc6b1946d80c450820cdb133"} Jul 07 23:16:25.000273 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:16:25.000214 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.983611492 podStartE2EDuration="3.000198626s" totalImagesPullingTime="0s" totalInitContainerRuntime="16.587134ms" isStatefulPod=false podCreationTimestamp="2026-07-07 23:16:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 23:16:24.998987311 +0000 UTC m=+202.395988845" watchObservedRunningTime="2026-07-07 23:16:25.000198626 +0000 UTC m=+202.397200148" Jul 07 23:16:25.067553 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:16:25.067518 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-9f6c8d65b-pxb5q"] Jul 07 23:16:25.070950 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:16:25.070925 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-9f6c8d65b-pxb5q" Jul 07 23:16:25.077839 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:16:25.077812 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\"" Jul 07 23:16:25.078078 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:16:25.078060 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\"" Jul 07 23:16:25.078372 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:16:25.078351 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-njf6p\"" Jul 07 23:16:25.078372 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:16:25.078363 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\"" Jul 07 23:16:25.078592 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:16:25.078346 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\"" Jul 07 23:16:25.078699 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:16:25.078682 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\"" Jul 07 23:16:25.084622 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:16:25.084603 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\"" Jul 07 23:16:25.095835 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:16:25.095814 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-9f6c8d65b-pxb5q"] Jul 07 23:16:25.150687 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:16:25.150658 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/073840f8-716e-426b-8516-494fae621efb-serving-certs-ca-bundle\") pod \"telemeter-client-9f6c8d65b-pxb5q\" (UID: \"073840f8-716e-426b-8516-494fae621efb\") " pod="openshift-monitoring/telemeter-client-9f6c8d65b-pxb5q" Jul 07 23:16:25.150837 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:16:25.150694 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/073840f8-716e-426b-8516-494fae621efb-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-9f6c8d65b-pxb5q\" (UID: \"073840f8-716e-426b-8516-494fae621efb\") " pod="openshift-monitoring/telemeter-client-9f6c8d65b-pxb5q" Jul 07 23:16:25.150837 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:16:25.150717 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/073840f8-716e-426b-8516-494fae621efb-metrics-client-ca\") pod \"telemeter-client-9f6c8d65b-pxb5q\" (UID: \"073840f8-716e-426b-8516-494fae621efb\") " pod="openshift-monitoring/telemeter-client-9f6c8d65b-pxb5q" Jul 07 23:16:25.150837 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:16:25.150740 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vbcrj\" (UniqueName: \"kubernetes.io/projected/073840f8-716e-426b-8516-494fae621efb-kube-api-access-vbcrj\") pod \"telemeter-client-9f6c8d65b-pxb5q\" (UID: \"073840f8-716e-426b-8516-494fae621efb\") " pod="openshift-monitoring/telemeter-client-9f6c8d65b-pxb5q" Jul 07 23:16:25.150837 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:16:25.150791 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/073840f8-716e-426b-8516-494fae621efb-federate-client-tls\") pod \"telemeter-client-9f6c8d65b-pxb5q\" (UID: \"073840f8-716e-426b-8516-494fae621efb\") " pod="openshift-monitoring/telemeter-client-9f6c8d65b-pxb5q" Jul 07 23:16:25.150837 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:16:25.150807 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/073840f8-716e-426b-8516-494fae621efb-secret-telemeter-client\") pod \"telemeter-client-9f6c8d65b-pxb5q\" (UID: \"073840f8-716e-426b-8516-494fae621efb\") " pod="openshift-monitoring/telemeter-client-9f6c8d65b-pxb5q" Jul 07 23:16:25.151021 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:16:25.150842 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/073840f8-716e-426b-8516-494fae621efb-telemeter-client-tls\") pod \"telemeter-client-9f6c8d65b-pxb5q\" (UID: \"073840f8-716e-426b-8516-494fae621efb\") " pod="openshift-monitoring/telemeter-client-9f6c8d65b-pxb5q" Jul 07 23:16:25.151021 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:16:25.150858 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/073840f8-716e-426b-8516-494fae621efb-telemeter-trusted-ca-bundle\") pod \"telemeter-client-9f6c8d65b-pxb5q\" (UID: \"073840f8-716e-426b-8516-494fae621efb\") " pod="openshift-monitoring/telemeter-client-9f6c8d65b-pxb5q" Jul 07 23:16:25.251601 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:16:25.251511 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/073840f8-716e-426b-8516-494fae621efb-federate-client-tls\") pod \"telemeter-client-9f6c8d65b-pxb5q\" (UID: \"073840f8-716e-426b-8516-494fae621efb\") " pod="openshift-monitoring/telemeter-client-9f6c8d65b-pxb5q" Jul 07 23:16:25.251601 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:16:25.251552 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/073840f8-716e-426b-8516-494fae621efb-secret-telemeter-client\") pod \"telemeter-client-9f6c8d65b-pxb5q\" (UID: \"073840f8-716e-426b-8516-494fae621efb\") " pod="openshift-monitoring/telemeter-client-9f6c8d65b-pxb5q" Jul 07 23:16:25.251601 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:16:25.251573 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/073840f8-716e-426b-8516-494fae621efb-telemeter-client-tls\") pod \"telemeter-client-9f6c8d65b-pxb5q\" (UID: \"073840f8-716e-426b-8516-494fae621efb\") " pod="openshift-monitoring/telemeter-client-9f6c8d65b-pxb5q" Jul 07 23:16:25.251851 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:16:25.251630 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/073840f8-716e-426b-8516-494fae621efb-telemeter-trusted-ca-bundle\") pod \"telemeter-client-9f6c8d65b-pxb5q\" (UID: \"073840f8-716e-426b-8516-494fae621efb\") " pod="openshift-monitoring/telemeter-client-9f6c8d65b-pxb5q" Jul 07 23:16:25.251851 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:16:25.251693 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/073840f8-716e-426b-8516-494fae621efb-serving-certs-ca-bundle\") pod \"telemeter-client-9f6c8d65b-pxb5q\" (UID: \"073840f8-716e-426b-8516-494fae621efb\") " pod="openshift-monitoring/telemeter-client-9f6c8d65b-pxb5q" Jul 07 23:16:25.251851 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:16:25.251723 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/073840f8-716e-426b-8516-494fae621efb-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-9f6c8d65b-pxb5q\" (UID: \"073840f8-716e-426b-8516-494fae621efb\") " pod="openshift-monitoring/telemeter-client-9f6c8d65b-pxb5q" Jul 07 23:16:25.251851 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:16:25.251749 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/073840f8-716e-426b-8516-494fae621efb-metrics-client-ca\") pod \"telemeter-client-9f6c8d65b-pxb5q\" (UID: \"073840f8-716e-426b-8516-494fae621efb\") " pod="openshift-monitoring/telemeter-client-9f6c8d65b-pxb5q" Jul 07 23:16:25.252060 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:16:25.251911 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vbcrj\" (UniqueName: \"kubernetes.io/projected/073840f8-716e-426b-8516-494fae621efb-kube-api-access-vbcrj\") pod \"telemeter-client-9f6c8d65b-pxb5q\" (UID: \"073840f8-716e-426b-8516-494fae621efb\") " pod="openshift-monitoring/telemeter-client-9f6c8d65b-pxb5q" Jul 07 23:16:25.252528 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:16:25.252500 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/073840f8-716e-426b-8516-494fae621efb-serving-certs-ca-bundle\") pod \"telemeter-client-9f6c8d65b-pxb5q\" (UID: \"073840f8-716e-426b-8516-494fae621efb\") " pod="openshift-monitoring/telemeter-client-9f6c8d65b-pxb5q" Jul 07 23:16:25.252652 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:16:25.252545 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/073840f8-716e-426b-8516-494fae621efb-metrics-client-ca\") pod \"telemeter-client-9f6c8d65b-pxb5q\" (UID: \"073840f8-716e-426b-8516-494fae621efb\") " pod="openshift-monitoring/telemeter-client-9f6c8d65b-pxb5q" Jul 07 23:16:25.252652 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:16:25.252621 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/073840f8-716e-426b-8516-494fae621efb-telemeter-trusted-ca-bundle\") pod \"telemeter-client-9f6c8d65b-pxb5q\" (UID: \"073840f8-716e-426b-8516-494fae621efb\") " pod="openshift-monitoring/telemeter-client-9f6c8d65b-pxb5q" Jul 07 23:16:25.254203 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:16:25.254175 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/073840f8-716e-426b-8516-494fae621efb-telemeter-client-tls\") pod \"telemeter-client-9f6c8d65b-pxb5q\" (UID: \"073840f8-716e-426b-8516-494fae621efb\") " pod="openshift-monitoring/telemeter-client-9f6c8d65b-pxb5q" Jul 07 23:16:25.254203 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:16:25.254197 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/073840f8-716e-426b-8516-494fae621efb-federate-client-tls\") pod \"telemeter-client-9f6c8d65b-pxb5q\" (UID: \"073840f8-716e-426b-8516-494fae621efb\") " pod="openshift-monitoring/telemeter-client-9f6c8d65b-pxb5q" Jul 07 23:16:25.254319 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:16:25.254262 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/073840f8-716e-426b-8516-494fae621efb-secret-telemeter-client\") pod \"telemeter-client-9f6c8d65b-pxb5q\" (UID: \"073840f8-716e-426b-8516-494fae621efb\") " pod="openshift-monitoring/telemeter-client-9f6c8d65b-pxb5q" Jul 07 23:16:25.254503 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:16:25.254462 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/073840f8-716e-426b-8516-494fae621efb-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-9f6c8d65b-pxb5q\" (UID: \"073840f8-716e-426b-8516-494fae621efb\") " pod="openshift-monitoring/telemeter-client-9f6c8d65b-pxb5q" Jul 07 23:16:25.263977 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:16:25.263958 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vbcrj\" (UniqueName: \"kubernetes.io/projected/073840f8-716e-426b-8516-494fae621efb-kube-api-access-vbcrj\") pod \"telemeter-client-9f6c8d65b-pxb5q\" (UID: \"073840f8-716e-426b-8516-494fae621efb\") " pod="openshift-monitoring/telemeter-client-9f6c8d65b-pxb5q" Jul 07 23:16:25.381238 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:16:25.381203 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-9f6c8d65b-pxb5q" Jul 07 23:16:25.528631 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:16:25.528605 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-9f6c8d65b-pxb5q"] Jul 07 23:16:25.530928 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:16:25.530904 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod073840f8_716e_426b_8516_494fae621efb.slice/crio-4f95cb0cae4a6d88d17540abd8c53199535c377073361c0a841e1ad54eef6cd4 WatchSource:0}: Error finding container 4f95cb0cae4a6d88d17540abd8c53199535c377073361c0a841e1ad54eef6cd4: Status 404 returned error can't find the container with id 4f95cb0cae4a6d88d17540abd8c53199535c377073361c0a841e1ad54eef6cd4 Jul 07 23:16:25.971056 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:16:25.970964 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-9f6c8d65b-pxb5q" event={"ID":"073840f8-716e-426b-8516-494fae621efb","Type":"ContainerStarted","Data":"4f95cb0cae4a6d88d17540abd8c53199535c377073361c0a841e1ad54eef6cd4"} Jul 07 23:16:27.981060 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:16:27.981019 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-9f6c8d65b-pxb5q" event={"ID":"073840f8-716e-426b-8516-494fae621efb","Type":"ContainerStarted","Data":"a899a2dbd923d505eadd12121a8ab97f2db4e1b28bcb0e1cba4cc946bfb0140c"} Jul 07 23:16:27.981410 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:16:27.981116 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-9f6c8d65b-pxb5q" event={"ID":"073840f8-716e-426b-8516-494fae621efb","Type":"ContainerStarted","Data":"ad5e267377cef2d5a5e4de9357478329934841dc7043c5a637beaee0d0025c92"} Jul 07 23:16:27.981410 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:16:27.981148 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-9f6c8d65b-pxb5q" event={"ID":"073840f8-716e-426b-8516-494fae621efb","Type":"ContainerStarted","Data":"465574d6d49bec872010ad7a13e63f3714674e5d1ca19e149877a010f4b7007e"} Jul 07 23:16:28.009260 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:16:28.009209 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-9f6c8d65b-pxb5q" podStartSLOduration=1.126147655 podStartE2EDuration="3.009195899s" totalImagesPullingTime="1.883048244s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 23:16:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 23:16:28.007612339 +0000 UTC m=+205.404613852" watchObservedRunningTime="2026-07-07 23:16:28.009195899 +0000 UTC m=+205.406197411" Jul 07 23:16:28.816575 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:16:28.816544 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-85d8699b74-258rk"] Jul 07 23:16:28.819988 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:16:28.819958 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-85d8699b74-258rk" Jul 07 23:16:28.824865 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:16:28.824845 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 07 23:16:28.825090 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:16:28.825064 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 07 23:16:28.825157 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:16:28.825139 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 07 23:16:28.826453 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:16:28.826259 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-m6lx8\"" Jul 07 23:16:28.826601 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:16:28.826491 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 07 23:16:28.826735 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:16:28.826711 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 07 23:16:28.826920 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:16:28.826892 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 07 23:16:28.827003 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:16:28.826931 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 07 23:16:28.833411 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:16:28.833386 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jul 07 23:16:28.848706 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:16:28.848680 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-85d8699b74-258rk"] Jul 07 23:16:28.881066 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:16:28.881040 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1dadeb8f-3214-4232-b430-17471b6ad9c5-service-ca\") pod \"console-85d8699b74-258rk\" (UID: \"1dadeb8f-3214-4232-b430-17471b6ad9c5\") " pod="openshift-console/console-85d8699b74-258rk" Jul 07 23:16:28.881210 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:16:28.881074 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nrj6f\" (UniqueName: \"kubernetes.io/projected/1dadeb8f-3214-4232-b430-17471b6ad9c5-kube-api-access-nrj6f\") pod \"console-85d8699b74-258rk\" (UID: \"1dadeb8f-3214-4232-b430-17471b6ad9c5\") " pod="openshift-console/console-85d8699b74-258rk" Jul 07 23:16:28.881210 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:16:28.881123 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1dadeb8f-3214-4232-b430-17471b6ad9c5-console-oauth-config\") pod \"console-85d8699b74-258rk\" (UID: \"1dadeb8f-3214-4232-b430-17471b6ad9c5\") " pod="openshift-console/console-85d8699b74-258rk" Jul 07 23:16:28.881210 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:16:28.881147 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1dadeb8f-3214-4232-b430-17471b6ad9c5-console-config\") pod \"console-85d8699b74-258rk\" (UID: \"1dadeb8f-3214-4232-b430-17471b6ad9c5\") " pod="openshift-console/console-85d8699b74-258rk" Jul 07 23:16:28.881210 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:16:28.881170 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1dadeb8f-3214-4232-b430-17471b6ad9c5-trusted-ca-bundle\") pod \"console-85d8699b74-258rk\" (UID: \"1dadeb8f-3214-4232-b430-17471b6ad9c5\") " pod="openshift-console/console-85d8699b74-258rk" Jul 07 23:16:28.881376 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:16:28.881230 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1dadeb8f-3214-4232-b430-17471b6ad9c5-console-serving-cert\") pod \"console-85d8699b74-258rk\" (UID: \"1dadeb8f-3214-4232-b430-17471b6ad9c5\") " pod="openshift-console/console-85d8699b74-258rk" Jul 07 23:16:28.881376 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:16:28.881265 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1dadeb8f-3214-4232-b430-17471b6ad9c5-oauth-serving-cert\") pod \"console-85d8699b74-258rk\" (UID: \"1dadeb8f-3214-4232-b430-17471b6ad9c5\") " pod="openshift-console/console-85d8699b74-258rk" Jul 07 23:16:28.925085 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:16:28.925059 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:16:28.981683 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:16:28.981658 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1dadeb8f-3214-4232-b430-17471b6ad9c5-oauth-serving-cert\") pod \"console-85d8699b74-258rk\" (UID: \"1dadeb8f-3214-4232-b430-17471b6ad9c5\") " pod="openshift-console/console-85d8699b74-258rk" Jul 07 23:16:28.982050 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:16:28.981747 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1dadeb8f-3214-4232-b430-17471b6ad9c5-service-ca\") pod \"console-85d8699b74-258rk\" (UID: \"1dadeb8f-3214-4232-b430-17471b6ad9c5\") " pod="openshift-console/console-85d8699b74-258rk" Jul 07 23:16:28.982050 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:16:28.981774 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nrj6f\" (UniqueName: \"kubernetes.io/projected/1dadeb8f-3214-4232-b430-17471b6ad9c5-kube-api-access-nrj6f\") pod \"console-85d8699b74-258rk\" (UID: \"1dadeb8f-3214-4232-b430-17471b6ad9c5\") " pod="openshift-console/console-85d8699b74-258rk" Jul 07 23:16:28.982050 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:16:28.981816 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1dadeb8f-3214-4232-b430-17471b6ad9c5-console-oauth-config\") pod \"console-85d8699b74-258rk\" (UID: \"1dadeb8f-3214-4232-b430-17471b6ad9c5\") " pod="openshift-console/console-85d8699b74-258rk" Jul 07 23:16:28.982050 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:16:28.981936 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1dadeb8f-3214-4232-b430-17471b6ad9c5-console-config\") pod \"console-85d8699b74-258rk\" (UID: \"1dadeb8f-3214-4232-b430-17471b6ad9c5\") " pod="openshift-console/console-85d8699b74-258rk" Jul 07 23:16:28.982050 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:16:28.981971 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1dadeb8f-3214-4232-b430-17471b6ad9c5-trusted-ca-bundle\") pod \"console-85d8699b74-258rk\" (UID: \"1dadeb8f-3214-4232-b430-17471b6ad9c5\") " pod="openshift-console/console-85d8699b74-258rk" Jul 07 23:16:28.982317 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:16:28.982092 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1dadeb8f-3214-4232-b430-17471b6ad9c5-console-serving-cert\") pod \"console-85d8699b74-258rk\" (UID: \"1dadeb8f-3214-4232-b430-17471b6ad9c5\") " pod="openshift-console/console-85d8699b74-258rk" Jul 07 23:16:28.982448 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:16:28.982426 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1dadeb8f-3214-4232-b430-17471b6ad9c5-oauth-serving-cert\") pod \"console-85d8699b74-258rk\" (UID: \"1dadeb8f-3214-4232-b430-17471b6ad9c5\") " pod="openshift-console/console-85d8699b74-258rk" Jul 07 23:16:28.982624 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:16:28.982596 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1dadeb8f-3214-4232-b430-17471b6ad9c5-service-ca\") pod \"console-85d8699b74-258rk\" (UID: \"1dadeb8f-3214-4232-b430-17471b6ad9c5\") " pod="openshift-console/console-85d8699b74-258rk" Jul 07 23:16:28.982778 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:16:28.982757 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1dadeb8f-3214-4232-b430-17471b6ad9c5-console-config\") pod \"console-85d8699b74-258rk\" (UID: \"1dadeb8f-3214-4232-b430-17471b6ad9c5\") " pod="openshift-console/console-85d8699b74-258rk" Jul 07 23:16:28.982894 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:16:28.982873 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1dadeb8f-3214-4232-b430-17471b6ad9c5-trusted-ca-bundle\") pod \"console-85d8699b74-258rk\" (UID: \"1dadeb8f-3214-4232-b430-17471b6ad9c5\") " pod="openshift-console/console-85d8699b74-258rk" Jul 07 23:16:28.984444 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:16:28.984416 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1dadeb8f-3214-4232-b430-17471b6ad9c5-console-oauth-config\") pod \"console-85d8699b74-258rk\" (UID: \"1dadeb8f-3214-4232-b430-17471b6ad9c5\") " pod="openshift-console/console-85d8699b74-258rk" Jul 07 23:16:28.984590 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:16:28.984568 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1dadeb8f-3214-4232-b430-17471b6ad9c5-console-serving-cert\") pod \"console-85d8699b74-258rk\" (UID: \"1dadeb8f-3214-4232-b430-17471b6ad9c5\") " pod="openshift-console/console-85d8699b74-258rk" Jul 07 23:16:28.991528 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:16:28.991503 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nrj6f\" (UniqueName: \"kubernetes.io/projected/1dadeb8f-3214-4232-b430-17471b6ad9c5-kube-api-access-nrj6f\") pod \"console-85d8699b74-258rk\" (UID: \"1dadeb8f-3214-4232-b430-17471b6ad9c5\") " pod="openshift-console/console-85d8699b74-258rk" Jul 07 23:16:28.997931 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:16:28.997905 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:16:29.132591 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:16:29.132517 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-85d8699b74-258rk" Jul 07 23:16:29.284550 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:16:29.284516 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-85d8699b74-258rk"] Jul 07 23:16:29.287837 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:16:29.287806 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1dadeb8f_3214_4232_b430_17471b6ad9c5.slice/crio-1f739c4dec5b5198445926814939b019d570902543d460afb83c00c10e1a8f6b WatchSource:0}: Error finding container 1f739c4dec5b5198445926814939b019d570902543d460afb83c00c10e1a8f6b: Status 404 returned error can't find the container with id 1f739c4dec5b5198445926814939b019d570902543d460afb83c00c10e1a8f6b Jul 07 23:16:29.990600 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:16:29.990559 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-85d8699b74-258rk" event={"ID":"1dadeb8f-3214-4232-b430-17471b6ad9c5","Type":"ContainerStarted","Data":"4a1281edd37eed3fc659f6ad804ea3d3ebb61369588ff16f63f950465b08ca39"} Jul 07 23:16:29.991035 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:16:29.990628 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-85d8699b74-258rk" event={"ID":"1dadeb8f-3214-4232-b430-17471b6ad9c5","Type":"ContainerStarted","Data":"1f739c4dec5b5198445926814939b019d570902543d460afb83c00c10e1a8f6b"} Jul 07 23:16:30.012431 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:16:30.012385 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-85d8699b74-258rk" podStartSLOduration=2.012370787 podStartE2EDuration="2.012370787s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 23:16:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 23:16:30.010921211 +0000 UTC m=+207.407922756" watchObservedRunningTime="2026-07-07 23:16:30.012370787 +0000 UTC m=+207.409372357" Jul 07 23:16:39.133202 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:16:39.133171 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-85d8699b74-258rk" Jul 07 23:16:39.133627 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:16:39.133437 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-85d8699b74-258rk" Jul 07 23:16:39.138095 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:16:39.138070 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-85d8699b74-258rk" Jul 07 23:16:40.033953 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:16:40.033692 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-85d8699b74-258rk" Jul 07 23:16:40.260871 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:16:40.260816 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:16:40.365435 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:16:40.365351 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:16:43.985034 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:16:43.985000 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:16:54.529990 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:16:54.529953 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:17:10.621233 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:17:10.621199 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:17:10.678669 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:17:10.678627 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:17:35.230305 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:17:35.230259 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:17:35.331066 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:17:35.331033 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:17:47.261886 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:17:47.261851 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:17:47.354053 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:17:47.354015 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:17:47.847210 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:17:47.847176 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-6b7b7c5b77-zl5n5"] Jul 07 23:17:47.850702 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:17:47.850668 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6b7b7c5b77-zl5n5" Jul 07 23:17:47.862864 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:17:47.862841 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6b7b7c5b77-zl5n5"] Jul 07 23:17:47.899032 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:17:47.898994 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:17:47.946271 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:17:47.946244 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a53651fd-186d-4c98-9823-bbea9451593f-oauth-serving-cert\") pod \"console-6b7b7c5b77-zl5n5\" (UID: \"a53651fd-186d-4c98-9823-bbea9451593f\") " pod="openshift-console/console-6b7b7c5b77-zl5n5" Jul 07 23:17:47.946415 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:17:47.946279 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a53651fd-186d-4c98-9823-bbea9451593f-console-oauth-config\") pod \"console-6b7b7c5b77-zl5n5\" (UID: \"a53651fd-186d-4c98-9823-bbea9451593f\") " pod="openshift-console/console-6b7b7c5b77-zl5n5" Jul 07 23:17:47.946415 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:17:47.946305 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a53651fd-186d-4c98-9823-bbea9451593f-console-serving-cert\") pod \"console-6b7b7c5b77-zl5n5\" (UID: \"a53651fd-186d-4c98-9823-bbea9451593f\") " pod="openshift-console/console-6b7b7c5b77-zl5n5" Jul 07 23:17:47.946415 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:17:47.946350 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a53651fd-186d-4c98-9823-bbea9451593f-trusted-ca-bundle\") pod \"console-6b7b7c5b77-zl5n5\" (UID: \"a53651fd-186d-4c98-9823-bbea9451593f\") " pod="openshift-console/console-6b7b7c5b77-zl5n5" Jul 07 23:17:47.946415 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:17:47.946392 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a53651fd-186d-4c98-9823-bbea9451593f-service-ca\") pod \"console-6b7b7c5b77-zl5n5\" (UID: \"a53651fd-186d-4c98-9823-bbea9451593f\") " pod="openshift-console/console-6b7b7c5b77-zl5n5" Jul 07 23:17:47.946585 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:17:47.946454 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a53651fd-186d-4c98-9823-bbea9451593f-console-config\") pod \"console-6b7b7c5b77-zl5n5\" (UID: \"a53651fd-186d-4c98-9823-bbea9451593f\") " pod="openshift-console/console-6b7b7c5b77-zl5n5" Jul 07 23:17:47.946585 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:17:47.946507 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nbnqj\" (UniqueName: \"kubernetes.io/projected/a53651fd-186d-4c98-9823-bbea9451593f-kube-api-access-nbnqj\") pod \"console-6b7b7c5b77-zl5n5\" (UID: \"a53651fd-186d-4c98-9823-bbea9451593f\") " pod="openshift-console/console-6b7b7c5b77-zl5n5" Jul 07 23:17:47.957311 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:17:47.957280 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:17:48.047214 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:17:48.047177 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a53651fd-186d-4c98-9823-bbea9451593f-oauth-serving-cert\") pod \"console-6b7b7c5b77-zl5n5\" (UID: \"a53651fd-186d-4c98-9823-bbea9451593f\") " pod="openshift-console/console-6b7b7c5b77-zl5n5" Jul 07 23:17:48.047214 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:17:48.047220 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a53651fd-186d-4c98-9823-bbea9451593f-console-oauth-config\") pod \"console-6b7b7c5b77-zl5n5\" (UID: \"a53651fd-186d-4c98-9823-bbea9451593f\") " pod="openshift-console/console-6b7b7c5b77-zl5n5" Jul 07 23:17:48.047449 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:17:48.047251 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a53651fd-186d-4c98-9823-bbea9451593f-console-serving-cert\") pod \"console-6b7b7c5b77-zl5n5\" (UID: \"a53651fd-186d-4c98-9823-bbea9451593f\") " pod="openshift-console/console-6b7b7c5b77-zl5n5" Jul 07 23:17:48.047449 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:17:48.047276 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a53651fd-186d-4c98-9823-bbea9451593f-trusted-ca-bundle\") pod \"console-6b7b7c5b77-zl5n5\" (UID: \"a53651fd-186d-4c98-9823-bbea9451593f\") " pod="openshift-console/console-6b7b7c5b77-zl5n5" Jul 07 23:17:48.047449 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:17:48.047312 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a53651fd-186d-4c98-9823-bbea9451593f-service-ca\") pod \"console-6b7b7c5b77-zl5n5\" (UID: \"a53651fd-186d-4c98-9823-bbea9451593f\") " pod="openshift-console/console-6b7b7c5b77-zl5n5" Jul 07 23:17:48.047449 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:17:48.047346 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a53651fd-186d-4c98-9823-bbea9451593f-console-config\") pod \"console-6b7b7c5b77-zl5n5\" (UID: \"a53651fd-186d-4c98-9823-bbea9451593f\") " pod="openshift-console/console-6b7b7c5b77-zl5n5" Jul 07 23:17:48.047700 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:17:48.047500 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nbnqj\" (UniqueName: \"kubernetes.io/projected/a53651fd-186d-4c98-9823-bbea9451593f-kube-api-access-nbnqj\") pod \"console-6b7b7c5b77-zl5n5\" (UID: \"a53651fd-186d-4c98-9823-bbea9451593f\") " pod="openshift-console/console-6b7b7c5b77-zl5n5" Jul 07 23:17:48.048092 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:17:48.048070 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a53651fd-186d-4c98-9823-bbea9451593f-oauth-serving-cert\") pod \"console-6b7b7c5b77-zl5n5\" (UID: \"a53651fd-186d-4c98-9823-bbea9451593f\") " pod="openshift-console/console-6b7b7c5b77-zl5n5" Jul 07 23:17:48.048194 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:17:48.048125 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a53651fd-186d-4c98-9823-bbea9451593f-console-config\") pod \"console-6b7b7c5b77-zl5n5\" (UID: \"a53651fd-186d-4c98-9823-bbea9451593f\") " pod="openshift-console/console-6b7b7c5b77-zl5n5" Jul 07 23:17:48.048194 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:17:48.048157 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a53651fd-186d-4c98-9823-bbea9451593f-service-ca\") pod \"console-6b7b7c5b77-zl5n5\" (UID: \"a53651fd-186d-4c98-9823-bbea9451593f\") " pod="openshift-console/console-6b7b7c5b77-zl5n5" Jul 07 23:17:48.048307 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:17:48.048289 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a53651fd-186d-4c98-9823-bbea9451593f-trusted-ca-bundle\") pod \"console-6b7b7c5b77-zl5n5\" (UID: \"a53651fd-186d-4c98-9823-bbea9451593f\") " pod="openshift-console/console-6b7b7c5b77-zl5n5" Jul 07 23:17:48.049826 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:17:48.049805 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a53651fd-186d-4c98-9823-bbea9451593f-console-serving-cert\") pod \"console-6b7b7c5b77-zl5n5\" (UID: \"a53651fd-186d-4c98-9823-bbea9451593f\") " pod="openshift-console/console-6b7b7c5b77-zl5n5" Jul 07 23:17:48.049920 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:17:48.049857 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a53651fd-186d-4c98-9823-bbea9451593f-console-oauth-config\") pod \"console-6b7b7c5b77-zl5n5\" (UID: \"a53651fd-186d-4c98-9823-bbea9451593f\") " pod="openshift-console/console-6b7b7c5b77-zl5n5" Jul 07 23:17:48.058128 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:17:48.058103 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nbnqj\" (UniqueName: \"kubernetes.io/projected/a53651fd-186d-4c98-9823-bbea9451593f-kube-api-access-nbnqj\") pod \"console-6b7b7c5b77-zl5n5\" (UID: \"a53651fd-186d-4c98-9823-bbea9451593f\") " pod="openshift-console/console-6b7b7c5b77-zl5n5" Jul 07 23:17:48.160367 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:17:48.160297 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6b7b7c5b77-zl5n5" Jul 07 23:17:48.290924 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:17:48.290900 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6b7b7c5b77-zl5n5"] Jul 07 23:17:48.293524 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:17:48.293496 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda53651fd_186d_4c98_9823_bbea9451593f.slice/crio-1e34f5a96dc404915562c6784f36d94ffabfe3ee41b7300a1e9cfd7a04a57877 WatchSource:0}: Error finding container 1e34f5a96dc404915562c6784f36d94ffabfe3ee41b7300a1e9cfd7a04a57877: Status 404 returned error can't find the container with id 1e34f5a96dc404915562c6784f36d94ffabfe3ee41b7300a1e9cfd7a04a57877 Jul 07 23:17:49.291270 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:17:49.291236 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6b7b7c5b77-zl5n5" event={"ID":"a53651fd-186d-4c98-9823-bbea9451593f","Type":"ContainerStarted","Data":"ccc1941070b5682188895f10ce6816a0000c7b21e38c6098740e0c62f2a997d7"} Jul 07 23:17:49.291691 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:17:49.291287 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6b7b7c5b77-zl5n5" event={"ID":"a53651fd-186d-4c98-9823-bbea9451593f","Type":"ContainerStarted","Data":"1e34f5a96dc404915562c6784f36d94ffabfe3ee41b7300a1e9cfd7a04a57877"} Jul 07 23:17:49.316781 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:17:49.316733 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-6b7b7c5b77-zl5n5" podStartSLOduration=2.316717867 podStartE2EDuration="2.316717867s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 23:17:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 23:17:49.315867302 +0000 UTC m=+286.712868813" watchObservedRunningTime="2026-07-07 23:17:49.316717867 +0000 UTC m=+286.713719378" Jul 07 23:17:58.161174 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:17:58.161139 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6b7b7c5b77-zl5n5" Jul 07 23:17:58.161605 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:17:58.161197 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-6b7b7c5b77-zl5n5" Jul 07 23:17:58.165844 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:17:58.165820 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-6b7b7c5b77-zl5n5" Jul 07 23:17:58.329484 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:17:58.329441 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-6b7b7c5b77-zl5n5" Jul 07 23:17:58.393202 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:17:58.393168 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-85d8699b74-258rk"] Jul 07 23:17:58.577614 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:17:58.577571 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:17:58.823890 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:17:58.823856 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:17:58.925583 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:17:58.925500 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:17:59.659681 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:17:59.659645 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:17:59.936326 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:17:59.936227 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:18:00.001157 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:18:00.001119 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:18:02.994841 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:18:02.994812 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-8hvxr_faae2bd8-cbbd-4cfb-bce6-7954977b7ebc/console-operator/2.log" Jul 07 23:18:03.000569 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:18:03.000540 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-8hvxr_faae2bd8-cbbd-4cfb-bce6-7954977b7ebc/console-operator/2.log" Jul 07 23:18:03.007340 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:18:03.007294 2570 kubelet.go:1648] "Image garbage collection succeeded" Jul 07 23:18:23.415323 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:18:23.415261 2570 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-85d8699b74-258rk" podUID="1dadeb8f-3214-4232-b430-17471b6ad9c5" containerName="console" containerID="cri-o://4a1281edd37eed3fc659f6ad804ea3d3ebb61369588ff16f63f950465b08ca39" gracePeriod=15 Jul 07 23:18:23.658036 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:18:23.658010 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-85d8699b74-258rk" Jul 07 23:18:23.747743 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:18:23.747666 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1dadeb8f-3214-4232-b430-17471b6ad9c5-oauth-serving-cert\") pod \"1dadeb8f-3214-4232-b430-17471b6ad9c5\" (UID: \"1dadeb8f-3214-4232-b430-17471b6ad9c5\") " Jul 07 23:18:23.747743 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:18:23.747705 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1dadeb8f-3214-4232-b430-17471b6ad9c5-console-config\") pod \"1dadeb8f-3214-4232-b430-17471b6ad9c5\" (UID: \"1dadeb8f-3214-4232-b430-17471b6ad9c5\") " Jul 07 23:18:23.747743 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:18:23.747724 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1dadeb8f-3214-4232-b430-17471b6ad9c5-console-serving-cert\") pod \"1dadeb8f-3214-4232-b430-17471b6ad9c5\" (UID: \"1dadeb8f-3214-4232-b430-17471b6ad9c5\") " Jul 07 23:18:23.748011 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:18:23.747754 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1dadeb8f-3214-4232-b430-17471b6ad9c5-service-ca\") pod \"1dadeb8f-3214-4232-b430-17471b6ad9c5\" (UID: \"1dadeb8f-3214-4232-b430-17471b6ad9c5\") " Jul 07 23:18:23.748011 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:18:23.747777 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1dadeb8f-3214-4232-b430-17471b6ad9c5-console-oauth-config\") pod \"1dadeb8f-3214-4232-b430-17471b6ad9c5\" (UID: \"1dadeb8f-3214-4232-b430-17471b6ad9c5\") " Jul 07 23:18:23.748011 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:18:23.747901 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1dadeb8f-3214-4232-b430-17471b6ad9c5-trusted-ca-bundle\") pod \"1dadeb8f-3214-4232-b430-17471b6ad9c5\" (UID: \"1dadeb8f-3214-4232-b430-17471b6ad9c5\") " Jul 07 23:18:23.748011 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:18:23.747945 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nrj6f\" (UniqueName: \"kubernetes.io/projected/1dadeb8f-3214-4232-b430-17471b6ad9c5-kube-api-access-nrj6f\") pod \"1dadeb8f-3214-4232-b430-17471b6ad9c5\" (UID: \"1dadeb8f-3214-4232-b430-17471b6ad9c5\") " Jul 07 23:18:23.748237 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:18:23.748109 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1dadeb8f-3214-4232-b430-17471b6ad9c5-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "1dadeb8f-3214-4232-b430-17471b6ad9c5" (UID: "1dadeb8f-3214-4232-b430-17471b6ad9c5"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 23:18:23.748237 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:18:23.748117 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1dadeb8f-3214-4232-b430-17471b6ad9c5-console-config" (OuterVolumeSpecName: "console-config") pod "1dadeb8f-3214-4232-b430-17471b6ad9c5" (UID: "1dadeb8f-3214-4232-b430-17471b6ad9c5"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 23:18:23.748346 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:18:23.748266 2570 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1dadeb8f-3214-4232-b430-17471b6ad9c5-oauth-serving-cert\") on node \"ip-10-0-134-140.ec2.internal\" DevicePath \"\"" Jul 07 23:18:23.748346 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:18:23.748280 2570 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1dadeb8f-3214-4232-b430-17471b6ad9c5-console-config\") on node \"ip-10-0-134-140.ec2.internal\" DevicePath \"\"" Jul 07 23:18:23.748346 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:18:23.748289 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1dadeb8f-3214-4232-b430-17471b6ad9c5-service-ca" (OuterVolumeSpecName: "service-ca") pod "1dadeb8f-3214-4232-b430-17471b6ad9c5" (UID: "1dadeb8f-3214-4232-b430-17471b6ad9c5"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 23:18:23.748346 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:18:23.748332 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1dadeb8f-3214-4232-b430-17471b6ad9c5-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "1dadeb8f-3214-4232-b430-17471b6ad9c5" (UID: "1dadeb8f-3214-4232-b430-17471b6ad9c5"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 23:18:23.750016 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:18:23.749983 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1dadeb8f-3214-4232-b430-17471b6ad9c5-kube-api-access-nrj6f" (OuterVolumeSpecName: "kube-api-access-nrj6f") pod "1dadeb8f-3214-4232-b430-17471b6ad9c5" (UID: "1dadeb8f-3214-4232-b430-17471b6ad9c5"). InnerVolumeSpecName "kube-api-access-nrj6f". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 23:18:23.750016 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:18:23.750008 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1dadeb8f-3214-4232-b430-17471b6ad9c5-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "1dadeb8f-3214-4232-b430-17471b6ad9c5" (UID: "1dadeb8f-3214-4232-b430-17471b6ad9c5"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 23:18:23.750168 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:18:23.750032 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1dadeb8f-3214-4232-b430-17471b6ad9c5-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "1dadeb8f-3214-4232-b430-17471b6ad9c5" (UID: "1dadeb8f-3214-4232-b430-17471b6ad9c5"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 23:18:23.849576 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:18:23.849541 2570 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1dadeb8f-3214-4232-b430-17471b6ad9c5-trusted-ca-bundle\") on node \"ip-10-0-134-140.ec2.internal\" DevicePath \"\"" Jul 07 23:18:23.849576 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:18:23.849569 2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nrj6f\" (UniqueName: \"kubernetes.io/projected/1dadeb8f-3214-4232-b430-17471b6ad9c5-kube-api-access-nrj6f\") on node \"ip-10-0-134-140.ec2.internal\" DevicePath \"\"" Jul 07 23:18:23.849762 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:18:23.849583 2570 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1dadeb8f-3214-4232-b430-17471b6ad9c5-console-serving-cert\") on node \"ip-10-0-134-140.ec2.internal\" DevicePath \"\"" Jul 07 23:18:23.849762 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:18:23.849596 2570 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1dadeb8f-3214-4232-b430-17471b6ad9c5-service-ca\") on node \"ip-10-0-134-140.ec2.internal\" DevicePath \"\"" Jul 07 23:18:23.849762 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:18:23.849608 2570 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1dadeb8f-3214-4232-b430-17471b6ad9c5-console-oauth-config\") on node \"ip-10-0-134-140.ec2.internal\" DevicePath \"\"" Jul 07 23:18:24.433207 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:18:24.433170 2570 generic.go:356] "Generic (PLEG): container finished" podID="1dadeb8f-3214-4232-b430-17471b6ad9c5" containerID="4a1281edd37eed3fc659f6ad804ea3d3ebb61369588ff16f63f950465b08ca39" exitCode=0 Jul 07 23:18:24.433645 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:18:24.433213 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-85d8699b74-258rk" Jul 07 23:18:24.433645 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:18:24.433251 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-85d8699b74-258rk" event={"ID":"1dadeb8f-3214-4232-b430-17471b6ad9c5","Type":"ContainerDied","Data":"4a1281edd37eed3fc659f6ad804ea3d3ebb61369588ff16f63f950465b08ca39"} Jul 07 23:18:24.433645 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:18:24.433299 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-85d8699b74-258rk" event={"ID":"1dadeb8f-3214-4232-b430-17471b6ad9c5","Type":"ContainerDied","Data":"1f739c4dec5b5198445926814939b019d570902543d460afb83c00c10e1a8f6b"} Jul 07 23:18:24.433645 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:18:24.433318 2570 scope.go:117] "RemoveContainer" containerID="4a1281edd37eed3fc659f6ad804ea3d3ebb61369588ff16f63f950465b08ca39" Jul 07 23:18:24.441802 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:18:24.441782 2570 scope.go:117] "RemoveContainer" containerID="4a1281edd37eed3fc659f6ad804ea3d3ebb61369588ff16f63f950465b08ca39" Jul 07 23:18:24.442050 ip-10-0-134-140 kubenswrapper[2570]: E0707 23:18:24.442030 2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4a1281edd37eed3fc659f6ad804ea3d3ebb61369588ff16f63f950465b08ca39\": container with ID starting with 4a1281edd37eed3fc659f6ad804ea3d3ebb61369588ff16f63f950465b08ca39 not found: ID does not exist" containerID="4a1281edd37eed3fc659f6ad804ea3d3ebb61369588ff16f63f950465b08ca39" Jul 07 23:18:24.442118 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:18:24.442058 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4a1281edd37eed3fc659f6ad804ea3d3ebb61369588ff16f63f950465b08ca39"} err="failed to get container status \"4a1281edd37eed3fc659f6ad804ea3d3ebb61369588ff16f63f950465b08ca39\": rpc error: code = NotFound desc = could not find container \"4a1281edd37eed3fc659f6ad804ea3d3ebb61369588ff16f63f950465b08ca39\": container with ID starting with 4a1281edd37eed3fc659f6ad804ea3d3ebb61369588ff16f63f950465b08ca39 not found: ID does not exist" Jul 07 23:18:24.457336 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:18:24.457313 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-85d8699b74-258rk"] Jul 07 23:18:24.462562 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:18:24.462543 2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-85d8699b74-258rk"] Jul 07 23:18:25.095689 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:18:25.095654 2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1dadeb8f-3214-4232-b430-17471b6ad9c5" path="/var/lib/kubelet/pods/1dadeb8f-3214-4232-b430-17471b6ad9c5/volumes" Jul 07 23:18:31.800737 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:18:31.800702 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:18:31.899276 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:18:31.899219 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:18:32.480092 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:18:32.480052 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540kkfq6"] Jul 07 23:18:32.480411 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:18:32.480398 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1dadeb8f-3214-4232-b430-17471b6ad9c5" containerName="console" Jul 07 23:18:32.480462 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:18:32.480413 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="1dadeb8f-3214-4232-b430-17471b6ad9c5" containerName="console" Jul 07 23:18:32.480524 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:18:32.480482 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="1dadeb8f-3214-4232-b430-17471b6ad9c5" containerName="console" Jul 07 23:18:32.483329 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:18:32.483310 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540kkfq6" Jul 07 23:18:32.485615 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:18:32.485591 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 07 23:18:32.486287 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:18:32.486275 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 07 23:18:32.486335 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:18:32.486288 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-gtdsm\"" Jul 07 23:18:32.492524 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:18:32.492502 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540kkfq6"] Jul 07 23:18:32.624521 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:18:32.624460 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5e70229c-07f0-464c-9d09-db61cd26a063-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540kkfq6\" (UID: \"5e70229c-07f0-464c-9d09-db61cd26a063\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540kkfq6" Jul 07 23:18:32.624695 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:18:32.624548 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5e70229c-07f0-464c-9d09-db61cd26a063-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540kkfq6\" (UID: \"5e70229c-07f0-464c-9d09-db61cd26a063\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540kkfq6" Jul 07 23:18:32.624695 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:18:32.624599 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s27jc\" (UniqueName: \"kubernetes.io/projected/5e70229c-07f0-464c-9d09-db61cd26a063-kube-api-access-s27jc\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540kkfq6\" (UID: \"5e70229c-07f0-464c-9d09-db61cd26a063\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540kkfq6" Jul 07 23:18:32.725800 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:18:32.725764 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5e70229c-07f0-464c-9d09-db61cd26a063-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540kkfq6\" (UID: \"5e70229c-07f0-464c-9d09-db61cd26a063\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540kkfq6" Jul 07 23:18:32.725944 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:18:32.725819 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s27jc\" (UniqueName: \"kubernetes.io/projected/5e70229c-07f0-464c-9d09-db61cd26a063-kube-api-access-s27jc\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540kkfq6\" (UID: \"5e70229c-07f0-464c-9d09-db61cd26a063\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540kkfq6" Jul 07 23:18:32.725944 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:18:32.725861 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5e70229c-07f0-464c-9d09-db61cd26a063-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540kkfq6\" (UID: \"5e70229c-07f0-464c-9d09-db61cd26a063\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540kkfq6" Jul 07 23:18:32.726164 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:18:32.726144 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5e70229c-07f0-464c-9d09-db61cd26a063-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540kkfq6\" (UID: \"5e70229c-07f0-464c-9d09-db61cd26a063\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540kkfq6" Jul 07 23:18:32.726204 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:18:32.726172 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5e70229c-07f0-464c-9d09-db61cd26a063-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540kkfq6\" (UID: \"5e70229c-07f0-464c-9d09-db61cd26a063\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540kkfq6" Jul 07 23:18:32.734595 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:18:32.734533 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-s27jc\" (UniqueName: \"kubernetes.io/projected/5e70229c-07f0-464c-9d09-db61cd26a063-kube-api-access-s27jc\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540kkfq6\" (UID: \"5e70229c-07f0-464c-9d09-db61cd26a063\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540kkfq6" Jul 07 23:18:32.793166 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:18:32.793139 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540kkfq6" Jul 07 23:18:32.923826 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:18:32.923799 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540kkfq6"] Jul 07 23:18:32.926275 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:18:32.926241 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5e70229c_07f0_464c_9d09_db61cd26a063.slice/crio-754eec5d2459bf278f0d4335f285046e4a7567099afdffbaa271c45ea89c4ee8 WatchSource:0}: Error finding container 754eec5d2459bf278f0d4335f285046e4a7567099afdffbaa271c45ea89c4ee8: Status 404 returned error can't find the container with id 754eec5d2459bf278f0d4335f285046e4a7567099afdffbaa271c45ea89c4ee8 Jul 07 23:18:32.928126 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:18:32.928104 2570 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 07 23:18:33.469020 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:18:33.468978 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540kkfq6" event={"ID":"5e70229c-07f0-464c-9d09-db61cd26a063","Type":"ContainerStarted","Data":"754eec5d2459bf278f0d4335f285046e4a7567099afdffbaa271c45ea89c4ee8"} Jul 07 23:18:40.502103 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:18:40.502011 2570 generic.go:356] "Generic (PLEG): container finished" podID="5e70229c-07f0-464c-9d09-db61cd26a063" containerID="db2fbdcd6194204246101c308a12dbc25d3246460bed78be72feabacd9b20075" exitCode=0 Jul 07 23:18:40.502103 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:18:40.502062 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540kkfq6" event={"ID":"5e70229c-07f0-464c-9d09-db61cd26a063","Type":"ContainerDied","Data":"db2fbdcd6194204246101c308a12dbc25d3246460bed78be72feabacd9b20075"} Jul 07 23:18:44.520590 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:18:44.520552 2570 generic.go:356] "Generic (PLEG): container finished" podID="5e70229c-07f0-464c-9d09-db61cd26a063" containerID="84237ec9fff5f76cf13a903b2c3febe03784d1d782fc2d687ff34217519942b8" exitCode=0 Jul 07 23:18:44.520988 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:18:44.520624 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540kkfq6" event={"ID":"5e70229c-07f0-464c-9d09-db61cd26a063","Type":"ContainerDied","Data":"84237ec9fff5f76cf13a903b2c3febe03784d1d782fc2d687ff34217519942b8"} Jul 07 23:18:53.561169 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:18:53.561135 2570 generic.go:356] "Generic (PLEG): container finished" podID="5e70229c-07f0-464c-9d09-db61cd26a063" containerID="7667362ea1136c8da16484b1203d8eda5b054adced0a8035196aa0bc265fa313" exitCode=0 Jul 07 23:18:53.561594 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:18:53.561183 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540kkfq6" event={"ID":"5e70229c-07f0-464c-9d09-db61cd26a063","Type":"ContainerDied","Data":"7667362ea1136c8da16484b1203d8eda5b054adced0a8035196aa0bc265fa313"} Jul 07 23:18:54.694981 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:18:54.694955 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540kkfq6" Jul 07 23:18:54.822550 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:18:54.817668 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-s27jc\" (UniqueName: \"kubernetes.io/projected/5e70229c-07f0-464c-9d09-db61cd26a063-kube-api-access-s27jc\") pod \"5e70229c-07f0-464c-9d09-db61cd26a063\" (UID: \"5e70229c-07f0-464c-9d09-db61cd26a063\") " Jul 07 23:18:54.822550 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:18:54.817736 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5e70229c-07f0-464c-9d09-db61cd26a063-bundle\") pod \"5e70229c-07f0-464c-9d09-db61cd26a063\" (UID: \"5e70229c-07f0-464c-9d09-db61cd26a063\") " Jul 07 23:18:54.822550 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:18:54.817854 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5e70229c-07f0-464c-9d09-db61cd26a063-util\") pod \"5e70229c-07f0-464c-9d09-db61cd26a063\" (UID: \"5e70229c-07f0-464c-9d09-db61cd26a063\") " Jul 07 23:18:54.823833 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:18:54.823797 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5e70229c-07f0-464c-9d09-db61cd26a063-bundle" (OuterVolumeSpecName: "bundle") pod "5e70229c-07f0-464c-9d09-db61cd26a063" (UID: "5e70229c-07f0-464c-9d09-db61cd26a063"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 23:18:54.824177 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:18:54.824153 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5e70229c-07f0-464c-9d09-db61cd26a063-util" (OuterVolumeSpecName: "util") pod "5e70229c-07f0-464c-9d09-db61cd26a063" (UID: "5e70229c-07f0-464c-9d09-db61cd26a063"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 23:18:54.825234 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:18:54.825213 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5e70229c-07f0-464c-9d09-db61cd26a063-kube-api-access-s27jc" (OuterVolumeSpecName: "kube-api-access-s27jc") pod "5e70229c-07f0-464c-9d09-db61cd26a063" (UID: "5e70229c-07f0-464c-9d09-db61cd26a063"). InnerVolumeSpecName "kube-api-access-s27jc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 23:18:54.919333 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:18:54.919291 2570 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5e70229c-07f0-464c-9d09-db61cd26a063-util\") on node \"ip-10-0-134-140.ec2.internal\" DevicePath \"\"" Jul 07 23:18:54.919333 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:18:54.919330 2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s27jc\" (UniqueName: \"kubernetes.io/projected/5e70229c-07f0-464c-9d09-db61cd26a063-kube-api-access-s27jc\") on node \"ip-10-0-134-140.ec2.internal\" DevicePath \"\"" Jul 07 23:18:54.919333 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:18:54.919341 2570 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5e70229c-07f0-464c-9d09-db61cd26a063-bundle\") on node \"ip-10-0-134-140.ec2.internal\" DevicePath \"\"" Jul 07 23:18:55.570251 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:18:55.570214 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540kkfq6" event={"ID":"5e70229c-07f0-464c-9d09-db61cd26a063","Type":"ContainerDied","Data":"754eec5d2459bf278f0d4335f285046e4a7567099afdffbaa271c45ea89c4ee8"} Jul 07 23:18:55.570449 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:18:55.570273 2570 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="754eec5d2459bf278f0d4335f285046e4a7567099afdffbaa271c45ea89c4ee8" Jul 07 23:18:55.570449 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:18:55.570224 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540kkfq6" Jul 07 23:18:58.589731 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:18:58.589701 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:18:59.807974 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:18:59.807939 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:19:00.886664 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:19:00.886630 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-62qgq"] Jul 07 23:19:00.887067 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:19:00.887006 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5e70229c-07f0-464c-9d09-db61cd26a063" containerName="util" Jul 07 23:19:00.887067 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:19:00.887019 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="5e70229c-07f0-464c-9d09-db61cd26a063" containerName="util" Jul 07 23:19:00.887067 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:19:00.887043 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5e70229c-07f0-464c-9d09-db61cd26a063" containerName="pull" Jul 07 23:19:00.887067 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:19:00.887049 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="5e70229c-07f0-464c-9d09-db61cd26a063" containerName="pull" Jul 07 23:19:00.887067 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:19:00.887058 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5e70229c-07f0-464c-9d09-db61cd26a063" containerName="extract" Jul 07 23:19:00.887067 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:19:00.887063 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="5e70229c-07f0-464c-9d09-db61cd26a063" containerName="extract" Jul 07 23:19:00.887279 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:19:00.887114 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="5e70229c-07f0-464c-9d09-db61cd26a063" containerName="extract" Jul 07 23:19:00.888972 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:19:00.888957 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-62qgq" Jul 07 23:19:00.891359 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:19:00.891337 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\"" Jul 07 23:19:00.891597 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:19:00.891583 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-22ncp\"" Jul 07 23:19:00.891671 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:19:00.891642 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\"" Jul 07 23:19:00.905657 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:19:00.905633 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-62qgq"] Jul 07 23:19:00.966792 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:19:00.966760 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p75fz\" (UniqueName: \"kubernetes.io/projected/2c85dffa-5aa8-44e5-95cd-d41b4469ed4f-kube-api-access-p75fz\") pod \"cert-manager-operator-controller-manager-55c444bf86-62qgq\" (UID: \"2c85dffa-5aa8-44e5-95cd-d41b4469ed4f\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-62qgq" Jul 07 23:19:00.966961 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:19:00.966807 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2c85dffa-5aa8-44e5-95cd-d41b4469ed4f-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-62qgq\" (UID: \"2c85dffa-5aa8-44e5-95cd-d41b4469ed4f\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-62qgq" Jul 07 23:19:01.067978 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:19:01.067943 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2c85dffa-5aa8-44e5-95cd-d41b4469ed4f-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-62qgq\" (UID: \"2c85dffa-5aa8-44e5-95cd-d41b4469ed4f\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-62qgq" Jul 07 23:19:01.068159 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:19:01.068007 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p75fz\" (UniqueName: \"kubernetes.io/projected/2c85dffa-5aa8-44e5-95cd-d41b4469ed4f-kube-api-access-p75fz\") pod \"cert-manager-operator-controller-manager-55c444bf86-62qgq\" (UID: \"2c85dffa-5aa8-44e5-95cd-d41b4469ed4f\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-62qgq" Jul 07 23:19:01.068367 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:19:01.068342 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2c85dffa-5aa8-44e5-95cd-d41b4469ed4f-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-62qgq\" (UID: \"2c85dffa-5aa8-44e5-95cd-d41b4469ed4f\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-62qgq" Jul 07 23:19:01.076557 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:19:01.076530 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p75fz\" (UniqueName: \"kubernetes.io/projected/2c85dffa-5aa8-44e5-95cd-d41b4469ed4f-kube-api-access-p75fz\") pod \"cert-manager-operator-controller-manager-55c444bf86-62qgq\" (UID: \"2c85dffa-5aa8-44e5-95cd-d41b4469ed4f\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-62qgq" Jul 07 23:19:01.198398 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:19:01.198309 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-62qgq" Jul 07 23:19:01.339534 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:19:01.339507 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-62qgq"] Jul 07 23:19:01.341684 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:19:01.341660 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2c85dffa_5aa8_44e5_95cd_d41b4469ed4f.slice/crio-a1bbdd73f643c76095d763677857c623acd2c5c550db38b6e87cdadb1a3ae4a5 WatchSource:0}: Error finding container a1bbdd73f643c76095d763677857c623acd2c5c550db38b6e87cdadb1a3ae4a5: Status 404 returned error can't find the container with id a1bbdd73f643c76095d763677857c623acd2c5c550db38b6e87cdadb1a3ae4a5 Jul 07 23:19:01.594715 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:19:01.594677 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-62qgq" event={"ID":"2c85dffa-5aa8-44e5-95cd-d41b4469ed4f","Type":"ContainerStarted","Data":"a1bbdd73f643c76095d763677857c623acd2c5c550db38b6e87cdadb1a3ae4a5"} Jul 07 23:19:02.786934 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:19:02.786901 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:19:02.888972 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:19:02.888895 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:19:04.610310 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:19:04.610273 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-62qgq" event={"ID":"2c85dffa-5aa8-44e5-95cd-d41b4469ed4f","Type":"ContainerStarted","Data":"0c491b65be2c8fb7c21b1b64070b5901d92d12a276f5c630c9597da01e2ba8b5"} Jul 07 23:19:04.639715 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:19:04.639651 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-62qgq" podStartSLOduration=2.18071367 podStartE2EDuration="4.63962761s" totalImagesPullingTime="2.45891394s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 23:19:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 23:19:04.637856435 +0000 UTC m=+362.034857948" watchObservedRunningTime="2026-07-07 23:19:04.63962761 +0000 UTC m=+362.036629124" Jul 07 23:19:04.810520 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:19:04.810445 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:19:04.908095 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:19:04.907996 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:19:05.032818 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:19:05.032785 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:19:05.143940 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:19:05.143910 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:19:05.240799 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:19:05.240707 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftvc94"] Jul 07 23:19:05.243232 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:19:05.243214 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftvc94" Jul 07 23:19:05.248799 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:19:05.248772 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 07 23:19:05.248903 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:19:05.248816 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-gtdsm\"" Jul 07 23:19:05.249096 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:19:05.249077 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 07 23:19:05.256815 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:19:05.256793 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftvc94"] Jul 07 23:19:05.301300 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:19:05.301269 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/732b9576-ea30-42a5-9deb-b699af114ac4-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftvc94\" (UID: \"732b9576-ea30-42a5-9deb-b699af114ac4\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftvc94" Jul 07 23:19:05.301442 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:19:05.301305 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wpxjm\" (UniqueName: \"kubernetes.io/projected/732b9576-ea30-42a5-9deb-b699af114ac4-kube-api-access-wpxjm\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftvc94\" (UID: \"732b9576-ea30-42a5-9deb-b699af114ac4\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftvc94" Jul 07 23:19:05.301442 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:19:05.301402 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/732b9576-ea30-42a5-9deb-b699af114ac4-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftvc94\" (UID: \"732b9576-ea30-42a5-9deb-b699af114ac4\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftvc94" Jul 07 23:19:05.401811 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:19:05.401769 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/732b9576-ea30-42a5-9deb-b699af114ac4-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftvc94\" (UID: \"732b9576-ea30-42a5-9deb-b699af114ac4\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftvc94" Jul 07 23:19:05.401811 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:19:05.401814 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wpxjm\" (UniqueName: \"kubernetes.io/projected/732b9576-ea30-42a5-9deb-b699af114ac4-kube-api-access-wpxjm\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftvc94\" (UID: \"732b9576-ea30-42a5-9deb-b699af114ac4\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftvc94" Jul 07 23:19:05.402116 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:19:05.401867 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/732b9576-ea30-42a5-9deb-b699af114ac4-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftvc94\" (UID: \"732b9576-ea30-42a5-9deb-b699af114ac4\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftvc94" Jul 07 23:19:05.402234 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:19:05.402207 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/732b9576-ea30-42a5-9deb-b699af114ac4-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftvc94\" (UID: \"732b9576-ea30-42a5-9deb-b699af114ac4\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftvc94" Jul 07 23:19:05.402291 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:19:05.402234 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/732b9576-ea30-42a5-9deb-b699af114ac4-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftvc94\" (UID: \"732b9576-ea30-42a5-9deb-b699af114ac4\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftvc94" Jul 07 23:19:05.421788 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:19:05.421756 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wpxjm\" (UniqueName: \"kubernetes.io/projected/732b9576-ea30-42a5-9deb-b699af114ac4-kube-api-access-wpxjm\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftvc94\" (UID: \"732b9576-ea30-42a5-9deb-b699af114ac4\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftvc94" Jul 07 23:19:05.552079 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:19:05.552042 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftvc94" Jul 07 23:19:05.679926 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:19:05.679895 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftvc94"] Jul 07 23:19:05.682853 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:19:05.682827 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod732b9576_ea30_42a5_9deb_b699af114ac4.slice/crio-9082adce0f24411779305f86ed4ef0de089a6372193316deb6320fa1cecfc80c WatchSource:0}: Error finding container 9082adce0f24411779305f86ed4ef0de089a6372193316deb6320fa1cecfc80c: Status 404 returned error can't find the container with id 9082adce0f24411779305f86ed4ef0de089a6372193316deb6320fa1cecfc80c Jul 07 23:19:06.189529 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:19:06.189490 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-r7xsz"] Jul 07 23:19:06.191791 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:19:06.191773 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-r7xsz" Jul 07 23:19:06.198619 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:19:06.198595 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Jul 07 23:19:06.199568 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:19:06.199551 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Jul 07 23:19:06.199837 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:19:06.199818 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-cnwdd\"" Jul 07 23:19:06.207757 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:19:06.207739 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q2wdq\" (UniqueName: \"kubernetes.io/projected/536649a8-b960-45a3-ad82-cd3d138a2c01-kube-api-access-q2wdq\") pod \"cert-manager-webhook-6486ffd997-r7xsz\" (UID: \"536649a8-b960-45a3-ad82-cd3d138a2c01\") " pod="cert-manager/cert-manager-webhook-6486ffd997-r7xsz" Jul 07 23:19:06.207862 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:19:06.207840 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/536649a8-b960-45a3-ad82-cd3d138a2c01-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-r7xsz\" (UID: \"536649a8-b960-45a3-ad82-cd3d138a2c01\") " pod="cert-manager/cert-manager-webhook-6486ffd997-r7xsz" Jul 07 23:19:06.219517 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:19:06.219492 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-r7xsz"] Jul 07 23:19:06.308983 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:19:06.308951 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/536649a8-b960-45a3-ad82-cd3d138a2c01-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-r7xsz\" (UID: \"536649a8-b960-45a3-ad82-cd3d138a2c01\") " pod="cert-manager/cert-manager-webhook-6486ffd997-r7xsz" Jul 07 23:19:06.309145 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:19:06.309004 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q2wdq\" (UniqueName: \"kubernetes.io/projected/536649a8-b960-45a3-ad82-cd3d138a2c01-kube-api-access-q2wdq\") pod \"cert-manager-webhook-6486ffd997-r7xsz\" (UID: \"536649a8-b960-45a3-ad82-cd3d138a2c01\") " pod="cert-manager/cert-manager-webhook-6486ffd997-r7xsz" Jul 07 23:19:06.318383 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:19:06.318354 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/536649a8-b960-45a3-ad82-cd3d138a2c01-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-r7xsz\" (UID: \"536649a8-b960-45a3-ad82-cd3d138a2c01\") " pod="cert-manager/cert-manager-webhook-6486ffd997-r7xsz" Jul 07 23:19:06.318565 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:19:06.318546 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q2wdq\" (UniqueName: \"kubernetes.io/projected/536649a8-b960-45a3-ad82-cd3d138a2c01-kube-api-access-q2wdq\") pod \"cert-manager-webhook-6486ffd997-r7xsz\" (UID: \"536649a8-b960-45a3-ad82-cd3d138a2c01\") " pod="cert-manager/cert-manager-webhook-6486ffd997-r7xsz" Jul 07 23:19:06.513599 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:19:06.513507 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-r7xsz" Jul 07 23:19:06.624383 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:19:06.624347 2570 generic.go:356] "Generic (PLEG): container finished" podID="732b9576-ea30-42a5-9deb-b699af114ac4" containerID="12a038dd6a215cfc20be000890dbc96a96d65e4b3303f068482dd998231363a2" exitCode=0 Jul 07 23:19:06.624556 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:19:06.624425 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftvc94" event={"ID":"732b9576-ea30-42a5-9deb-b699af114ac4","Type":"ContainerDied","Data":"12a038dd6a215cfc20be000890dbc96a96d65e4b3303f068482dd998231363a2"} Jul 07 23:19:06.624556 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:19:06.624504 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftvc94" event={"ID":"732b9576-ea30-42a5-9deb-b699af114ac4","Type":"ContainerStarted","Data":"9082adce0f24411779305f86ed4ef0de089a6372193316deb6320fa1cecfc80c"} Jul 07 23:19:06.639449 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:19:06.639357 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-r7xsz"] Jul 07 23:19:06.642213 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:19:06.642190 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod536649a8_b960_45a3_ad82_cd3d138a2c01.slice/crio-2e63b04b77ea6f100182e3425ff6d1b0c4c8963a061284b01eb250c21b330ef4 WatchSource:0}: Error finding container 2e63b04b77ea6f100182e3425ff6d1b0c4c8963a061284b01eb250c21b330ef4: Status 404 returned error can't find the container with id 2e63b04b77ea6f100182e3425ff6d1b0c4c8963a061284b01eb250c21b330ef4 Jul 07 23:19:07.217341 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:19:07.217302 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:19:07.426279 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:19:07.426236 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:19:07.630520 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:19:07.630488 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-r7xsz" event={"ID":"536649a8-b960-45a3-ad82-cd3d138a2c01","Type":"ContainerStarted","Data":"2e63b04b77ea6f100182e3425ff6d1b0c4c8963a061284b01eb250c21b330ef4"} Jul 07 23:19:09.573178 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:19:09.573134 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:19:09.644037 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:19:09.643998 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:19:10.647544 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:19:10.647440 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-r7xsz" event={"ID":"536649a8-b960-45a3-ad82-cd3d138a2c01","Type":"ContainerStarted","Data":"dc345ddbd292c614bd6243615292f3acc38ddd1782cc0af073e3f82bff24678f"} Jul 07 23:19:10.647544 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:19:10.647502 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-r7xsz" Jul 07 23:19:10.649237 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:19:10.649211 2570 generic.go:356] "Generic (PLEG): container finished" podID="732b9576-ea30-42a5-9deb-b699af114ac4" containerID="98a3aa16c94df238d177a5a79627116488b71503221f8da693fdad8667b67aa5" exitCode=0 Jul 07 23:19:10.649343 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:19:10.649293 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftvc94" event={"ID":"732b9576-ea30-42a5-9deb-b699af114ac4","Type":"ContainerDied","Data":"98a3aa16c94df238d177a5a79627116488b71503221f8da693fdad8667b67aa5"} Jul 07 23:19:10.668047 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:19:10.667891 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-r7xsz" podStartSLOduration=1.008465201 podStartE2EDuration="4.667872146s" totalImagesPullingTime="3.659406945s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 23:19:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 23:19:10.667246148 +0000 UTC m=+368.064247662" watchObservedRunningTime="2026-07-07 23:19:10.667872146 +0000 UTC m=+368.064873658" Jul 07 23:19:11.655671 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:19:11.655632 2570 generic.go:356] "Generic (PLEG): container finished" podID="732b9576-ea30-42a5-9deb-b699af114ac4" containerID="d5acef8913e61e7be5a3adcf676c1980970de4806982f5a48228bb7800ef24bc" exitCode=0 Jul 07 23:19:11.656061 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:19:11.655726 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftvc94" event={"ID":"732b9576-ea30-42a5-9deb-b699af114ac4","Type":"ContainerDied","Data":"d5acef8913e61e7be5a3adcf676c1980970de4806982f5a48228bb7800ef24bc"} Jul 07 23:19:12.787330 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:19:12.787307 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftvc94" Jul 07 23:19:12.860634 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:19:12.860605 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wpxjm\" (UniqueName: \"kubernetes.io/projected/732b9576-ea30-42a5-9deb-b699af114ac4-kube-api-access-wpxjm\") pod \"732b9576-ea30-42a5-9deb-b699af114ac4\" (UID: \"732b9576-ea30-42a5-9deb-b699af114ac4\") " Jul 07 23:19:12.860804 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:19:12.860681 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/732b9576-ea30-42a5-9deb-b699af114ac4-util\") pod \"732b9576-ea30-42a5-9deb-b699af114ac4\" (UID: \"732b9576-ea30-42a5-9deb-b699af114ac4\") " Jul 07 23:19:12.860804 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:19:12.860707 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/732b9576-ea30-42a5-9deb-b699af114ac4-bundle\") pod \"732b9576-ea30-42a5-9deb-b699af114ac4\" (UID: \"732b9576-ea30-42a5-9deb-b699af114ac4\") " Jul 07 23:19:12.861113 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:19:12.861081 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/732b9576-ea30-42a5-9deb-b699af114ac4-bundle" (OuterVolumeSpecName: "bundle") pod "732b9576-ea30-42a5-9deb-b699af114ac4" (UID: "732b9576-ea30-42a5-9deb-b699af114ac4"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 23:19:12.862619 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:19:12.862594 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/732b9576-ea30-42a5-9deb-b699af114ac4-kube-api-access-wpxjm" (OuterVolumeSpecName: "kube-api-access-wpxjm") pod "732b9576-ea30-42a5-9deb-b699af114ac4" (UID: "732b9576-ea30-42a5-9deb-b699af114ac4"). InnerVolumeSpecName "kube-api-access-wpxjm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 23:19:12.865983 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:19:12.865960 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/732b9576-ea30-42a5-9deb-b699af114ac4-util" (OuterVolumeSpecName: "util") pod "732b9576-ea30-42a5-9deb-b699af114ac4" (UID: "732b9576-ea30-42a5-9deb-b699af114ac4"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 23:19:12.961410 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:19:12.961337 2570 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/732b9576-ea30-42a5-9deb-b699af114ac4-util\") on node \"ip-10-0-134-140.ec2.internal\" DevicePath \"\"" Jul 07 23:19:12.961410 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:19:12.961362 2570 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/732b9576-ea30-42a5-9deb-b699af114ac4-bundle\") on node \"ip-10-0-134-140.ec2.internal\" DevicePath \"\"" Jul 07 23:19:12.961410 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:19:12.961372 2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wpxjm\" (UniqueName: \"kubernetes.io/projected/732b9576-ea30-42a5-9deb-b699af114ac4-kube-api-access-wpxjm\") on node \"ip-10-0-134-140.ec2.internal\" DevicePath \"\"" Jul 07 23:19:13.667133 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:19:13.667094 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftvc94" event={"ID":"732b9576-ea30-42a5-9deb-b699af114ac4","Type":"ContainerDied","Data":"9082adce0f24411779305f86ed4ef0de089a6372193316deb6320fa1cecfc80c"} Jul 07 23:19:13.667334 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:19:13.667149 2570 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="9082adce0f24411779305f86ed4ef0de089a6372193316deb6320fa1cecfc80c" Jul 07 23:19:13.667334 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:19:13.667108 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ftvc94" Jul 07 23:19:14.434892 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:19:14.434860 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:19:16.658268 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:19:16.658234 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-r7xsz" Jul 07 23:19:21.024742 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:19:21.024706 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:19:21.117792 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:19:21.117757 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:19:23.029835 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:19:23.029802 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-8jnrg"] Jul 07 23:19:23.030209 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:19:23.030121 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="732b9576-ea30-42a5-9deb-b699af114ac4" containerName="pull" Jul 07 23:19:23.030209 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:19:23.030132 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="732b9576-ea30-42a5-9deb-b699af114ac4" containerName="pull" Jul 07 23:19:23.030209 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:19:23.030140 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="732b9576-ea30-42a5-9deb-b699af114ac4" containerName="extract" Jul 07 23:19:23.030209 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:19:23.030145 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="732b9576-ea30-42a5-9deb-b699af114ac4" containerName="extract" Jul 07 23:19:23.030209 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:19:23.030154 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="732b9576-ea30-42a5-9deb-b699af114ac4" containerName="util" Jul 07 23:19:23.030209 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:19:23.030159 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="732b9576-ea30-42a5-9deb-b699af114ac4" containerName="util" Jul 07 23:19:23.030397 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:19:23.030214 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="732b9576-ea30-42a5-9deb-b699af114ac4" containerName="extract" Jul 07 23:19:23.061918 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:19:23.061884 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-8jnrg"] Jul 07 23:19:23.062069 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:19:23.061978 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-8jnrg" Jul 07 23:19:23.064563 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:19:23.064537 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-zndz6\"" Jul 07 23:19:23.064690 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:19:23.064602 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 07 23:19:23.065183 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:19:23.065166 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 07 23:19:23.151060 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:19:23.151027 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/68a7079f-266e-4430-aa48-048b11c59f0a-tmp\") pod \"openshift-lws-operator-fd8ccff4c-8jnrg\" (UID: \"68a7079f-266e-4430-aa48-048b11c59f0a\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-8jnrg" Jul 07 23:19:23.151216 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:19:23.151066 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wmr6m\" (UniqueName: \"kubernetes.io/projected/68a7079f-266e-4430-aa48-048b11c59f0a-kube-api-access-wmr6m\") pod \"openshift-lws-operator-fd8ccff4c-8jnrg\" (UID: \"68a7079f-266e-4430-aa48-048b11c59f0a\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-8jnrg" Jul 07 23:19:23.255917 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:19:23.255882 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/68a7079f-266e-4430-aa48-048b11c59f0a-tmp\") pod \"openshift-lws-operator-fd8ccff4c-8jnrg\" (UID: \"68a7079f-266e-4430-aa48-048b11c59f0a\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-8jnrg" Jul 07 23:19:23.256086 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:19:23.255925 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wmr6m\" (UniqueName: \"kubernetes.io/projected/68a7079f-266e-4430-aa48-048b11c59f0a-kube-api-access-wmr6m\") pod \"openshift-lws-operator-fd8ccff4c-8jnrg\" (UID: \"68a7079f-266e-4430-aa48-048b11c59f0a\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-8jnrg" Jul 07 23:19:23.256343 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:19:23.256319 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/68a7079f-266e-4430-aa48-048b11c59f0a-tmp\") pod \"openshift-lws-operator-fd8ccff4c-8jnrg\" (UID: \"68a7079f-266e-4430-aa48-048b11c59f0a\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-8jnrg" Jul 07 23:19:23.274935 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:19:23.274909 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wmr6m\" (UniqueName: \"kubernetes.io/projected/68a7079f-266e-4430-aa48-048b11c59f0a-kube-api-access-wmr6m\") pod \"openshift-lws-operator-fd8ccff4c-8jnrg\" (UID: \"68a7079f-266e-4430-aa48-048b11c59f0a\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-8jnrg" Jul 07 23:19:23.371396 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:19:23.371316 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-8jnrg" Jul 07 23:19:23.525248 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:19:23.525223 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-8jnrg"] Jul 07 23:19:23.527357 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:19:23.527318 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod68a7079f_266e_4430_aa48_048b11c59f0a.slice/crio-746215abf7abd488fdcbf2ca73df0987192974bf3ee99f02d3f5252541da8388 WatchSource:0}: Error finding container 746215abf7abd488fdcbf2ca73df0987192974bf3ee99f02d3f5252541da8388: Status 404 returned error can't find the container with id 746215abf7abd488fdcbf2ca73df0987192974bf3ee99f02d3f5252541da8388 Jul 07 23:19:23.713892 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:19:23.713811 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-8jnrg" event={"ID":"68a7079f-266e-4430-aa48-048b11c59f0a","Type":"ContainerStarted","Data":"746215abf7abd488fdcbf2ca73df0987192974bf3ee99f02d3f5252541da8388"} Jul 07 23:19:25.726347 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:19:25.726262 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-8jnrg" event={"ID":"68a7079f-266e-4430-aa48-048b11c59f0a","Type":"ContainerStarted","Data":"a2e32c75c36d517969f4b7a0bf46bfa591dd2b89fe108159c08ce536609273d7"} Jul 07 23:19:25.750360 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:19:25.750302 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-8jnrg" podStartSLOduration=0.826217788 podStartE2EDuration="2.750281196s" totalImagesPullingTime="1.924063408s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 23:19:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 23:19:25.747749012 +0000 UTC m=+383.144750526" watchObservedRunningTime="2026-07-07 23:19:25.750281196 +0000 UTC m=+383.147282712" Jul 07 23:19:26.520774 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:19:26.520734 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:19:26.622336 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:19:26.622298 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:19:28.816931 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:19:28.816890 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:19:28.923533 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:19:28.923499 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:19:29.039124 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:19:29.039085 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:19:30.905678 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:19:30.905646 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afd9rwj"] Jul 07 23:19:30.908017 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:19:30.908001 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afd9rwj" Jul 07 23:19:30.910765 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:19:30.910741 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-gtdsm\"" Jul 07 23:19:30.911215 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:19:30.911199 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 07 23:19:30.911559 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:19:30.911541 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 07 23:19:30.925060 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:19:30.925033 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afd9rwj"] Jul 07 23:19:31.023224 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:19:31.023187 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2e5a4c7f-4254-4d70-934d-9b612f2df8bc-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afd9rwj\" (UID: \"2e5a4c7f-4254-4d70-934d-9b612f2df8bc\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afd9rwj" Jul 07 23:19:31.023224 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:19:31.023225 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-czc5h\" (UniqueName: \"kubernetes.io/projected/2e5a4c7f-4254-4d70-934d-9b612f2df8bc-kube-api-access-czc5h\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afd9rwj\" (UID: \"2e5a4c7f-4254-4d70-934d-9b612f2df8bc\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afd9rwj" Jul 07 23:19:31.023455 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:19:31.023335 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2e5a4c7f-4254-4d70-934d-9b612f2df8bc-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afd9rwj\" (UID: \"2e5a4c7f-4254-4d70-934d-9b612f2df8bc\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afd9rwj" Jul 07 23:19:31.124657 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:19:31.124623 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2e5a4c7f-4254-4d70-934d-9b612f2df8bc-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afd9rwj\" (UID: \"2e5a4c7f-4254-4d70-934d-9b612f2df8bc\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afd9rwj" Jul 07 23:19:31.124831 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:19:31.124669 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2e5a4c7f-4254-4d70-934d-9b612f2df8bc-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afd9rwj\" (UID: \"2e5a4c7f-4254-4d70-934d-9b612f2df8bc\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afd9rwj" Jul 07 23:19:31.124831 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:19:31.124688 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-czc5h\" (UniqueName: \"kubernetes.io/projected/2e5a4c7f-4254-4d70-934d-9b612f2df8bc-kube-api-access-czc5h\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afd9rwj\" (UID: \"2e5a4c7f-4254-4d70-934d-9b612f2df8bc\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afd9rwj" Jul 07 23:19:31.125006 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:19:31.124984 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2e5a4c7f-4254-4d70-934d-9b612f2df8bc-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afd9rwj\" (UID: \"2e5a4c7f-4254-4d70-934d-9b612f2df8bc\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afd9rwj" Jul 07 23:19:31.125069 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:19:31.125031 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2e5a4c7f-4254-4d70-934d-9b612f2df8bc-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afd9rwj\" (UID: \"2e5a4c7f-4254-4d70-934d-9b612f2df8bc\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afd9rwj" Jul 07 23:19:31.146412 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:19:31.146380 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-czc5h\" (UniqueName: \"kubernetes.io/projected/2e5a4c7f-4254-4d70-934d-9b612f2df8bc-kube-api-access-czc5h\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afd9rwj\" (UID: \"2e5a4c7f-4254-4d70-934d-9b612f2df8bc\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afd9rwj" Jul 07 23:19:31.217127 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:19:31.217046 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afd9rwj" Jul 07 23:19:31.363329 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:19:31.363306 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afd9rwj"] Jul 07 23:19:31.365548 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:19:31.365517 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2e5a4c7f_4254_4d70_934d_9b612f2df8bc.slice/crio-55ed098787aec36529da9bd5cbccd36e80b0d20bdf445169c19b3189c1337705 WatchSource:0}: Error finding container 55ed098787aec36529da9bd5cbccd36e80b0d20bdf445169c19b3189c1337705: Status 404 returned error can't find the container with id 55ed098787aec36529da9bd5cbccd36e80b0d20bdf445169c19b3189c1337705 Jul 07 23:19:31.753810 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:19:31.753775 2570 generic.go:356] "Generic (PLEG): container finished" podID="2e5a4c7f-4254-4d70-934d-9b612f2df8bc" containerID="163e07667b4fd045f65f13b08d429a81491e56e8c04228860c4fb7fa507eb739" exitCode=0 Jul 07 23:19:31.754008 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:19:31.753852 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afd9rwj" event={"ID":"2e5a4c7f-4254-4d70-934d-9b612f2df8bc","Type":"ContainerDied","Data":"163e07667b4fd045f65f13b08d429a81491e56e8c04228860c4fb7fa507eb739"} Jul 07 23:19:31.754008 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:19:31.753893 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afd9rwj" event={"ID":"2e5a4c7f-4254-4d70-934d-9b612f2df8bc","Type":"ContainerStarted","Data":"55ed098787aec36529da9bd5cbccd36e80b0d20bdf445169c19b3189c1337705"} Jul 07 23:19:32.760766 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:19:32.760684 2570 generic.go:356] "Generic (PLEG): container finished" podID="2e5a4c7f-4254-4d70-934d-9b612f2df8bc" containerID="79f2999bb4c91f44d6d8febe072e1ce6e5d4bccf18656af25775fec1374db5f5" exitCode=0 Jul 07 23:19:32.761136 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:19:32.760757 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afd9rwj" event={"ID":"2e5a4c7f-4254-4d70-934d-9b612f2df8bc","Type":"ContainerDied","Data":"79f2999bb4c91f44d6d8febe072e1ce6e5d4bccf18656af25775fec1374db5f5"} Jul 07 23:19:33.768107 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:19:33.768071 2570 generic.go:356] "Generic (PLEG): container finished" podID="2e5a4c7f-4254-4d70-934d-9b612f2df8bc" containerID="f7562035a03a5548322e4c59d553bc57fa06cefc97f0af3b83d126dd99eb4e6e" exitCode=0 Jul 07 23:19:33.768521 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:19:33.768118 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afd9rwj" event={"ID":"2e5a4c7f-4254-4d70-934d-9b612f2df8bc","Type":"ContainerDied","Data":"f7562035a03a5548322e4c59d553bc57fa06cefc97f0af3b83d126dd99eb4e6e"} Jul 07 23:19:34.900187 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:19:34.900164 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afd9rwj" Jul 07 23:19:34.957205 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:19:34.957172 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2e5a4c7f-4254-4d70-934d-9b612f2df8bc-bundle\") pod \"2e5a4c7f-4254-4d70-934d-9b612f2df8bc\" (UID: \"2e5a4c7f-4254-4d70-934d-9b612f2df8bc\") " Jul 07 23:19:34.957357 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:19:34.957244 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-czc5h\" (UniqueName: \"kubernetes.io/projected/2e5a4c7f-4254-4d70-934d-9b612f2df8bc-kube-api-access-czc5h\") pod \"2e5a4c7f-4254-4d70-934d-9b612f2df8bc\" (UID: \"2e5a4c7f-4254-4d70-934d-9b612f2df8bc\") " Jul 07 23:19:34.957357 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:19:34.957338 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2e5a4c7f-4254-4d70-934d-9b612f2df8bc-util\") pod \"2e5a4c7f-4254-4d70-934d-9b612f2df8bc\" (UID: \"2e5a4c7f-4254-4d70-934d-9b612f2df8bc\") " Jul 07 23:19:34.958121 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:19:34.958091 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2e5a4c7f-4254-4d70-934d-9b612f2df8bc-bundle" (OuterVolumeSpecName: "bundle") pod "2e5a4c7f-4254-4d70-934d-9b612f2df8bc" (UID: "2e5a4c7f-4254-4d70-934d-9b612f2df8bc"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 23:19:34.959364 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:19:34.959340 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2e5a4c7f-4254-4d70-934d-9b612f2df8bc-kube-api-access-czc5h" (OuterVolumeSpecName: "kube-api-access-czc5h") pod "2e5a4c7f-4254-4d70-934d-9b612f2df8bc" (UID: "2e5a4c7f-4254-4d70-934d-9b612f2df8bc"). InnerVolumeSpecName "kube-api-access-czc5h". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 23:19:34.963368 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:19:34.963346 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2e5a4c7f-4254-4d70-934d-9b612f2df8bc-util" (OuterVolumeSpecName: "util") pod "2e5a4c7f-4254-4d70-934d-9b612f2df8bc" (UID: "2e5a4c7f-4254-4d70-934d-9b612f2df8bc"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 23:19:35.058588 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:19:35.058560 2570 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2e5a4c7f-4254-4d70-934d-9b612f2df8bc-util\") on node \"ip-10-0-134-140.ec2.internal\" DevicePath \"\"" Jul 07 23:19:35.058588 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:19:35.058588 2570 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2e5a4c7f-4254-4d70-934d-9b612f2df8bc-bundle\") on node \"ip-10-0-134-140.ec2.internal\" DevicePath \"\"" Jul 07 23:19:35.058756 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:19:35.058598 2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-czc5h\" (UniqueName: \"kubernetes.io/projected/2e5a4c7f-4254-4d70-934d-9b612f2df8bc-kube-api-access-czc5h\") on node \"ip-10-0-134-140.ec2.internal\" DevicePath \"\"" Jul 07 23:19:35.779585 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:19:35.779557 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afd9rwj" Jul 07 23:19:35.779761 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:19:35.779548 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afd9rwj" event={"ID":"2e5a4c7f-4254-4d70-934d-9b612f2df8bc","Type":"ContainerDied","Data":"55ed098787aec36529da9bd5cbccd36e80b0d20bdf445169c19b3189c1337705"} Jul 07 23:19:35.779761 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:19:35.779680 2570 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="55ed098787aec36529da9bd5cbccd36e80b0d20bdf445169c19b3189c1337705" Jul 07 23:19:43.088250 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:19:43.088216 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:19:43.206969 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:19:43.206923 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:19:43.637852 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:19:43.637812 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:19:44.055000 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:19:44.054967 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:19:44.157945 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:19:44.157909 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:19:44.256429 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:19:44.256393 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7c9859966d-7bxpt"] Jul 07 23:19:44.256730 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:19:44.256717 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2e5a4c7f-4254-4d70-934d-9b612f2df8bc" containerName="pull" Jul 07 23:19:44.256776 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:19:44.256731 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="2e5a4c7f-4254-4d70-934d-9b612f2df8bc" containerName="pull" Jul 07 23:19:44.256776 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:19:44.256742 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2e5a4c7f-4254-4d70-934d-9b612f2df8bc" containerName="util" Jul 07 23:19:44.256776 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:19:44.256748 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="2e5a4c7f-4254-4d70-934d-9b612f2df8bc" containerName="util" Jul 07 23:19:44.256776 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:19:44.256758 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2e5a4c7f-4254-4d70-934d-9b612f2df8bc" containerName="extract" Jul 07 23:19:44.256776 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:19:44.256763 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="2e5a4c7f-4254-4d70-934d-9b612f2df8bc" containerName="extract" Jul 07 23:19:44.256921 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:19:44.256826 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="2e5a4c7f-4254-4d70-934d-9b612f2df8bc" containerName="extract" Jul 07 23:19:44.259268 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:19:44.259248 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-7c9859966d-7bxpt" Jul 07 23:19:44.265821 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:19:44.265805 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 07 23:19:44.265888 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:19:44.265830 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 07 23:19:44.266331 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:19:44.266315 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 07 23:19:44.266544 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:19:44.266527 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-z2ktg\"" Jul 07 23:19:44.266696 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:19:44.266678 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 07 23:19:44.319950 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:19:44.319871 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7c9859966d-7bxpt"] Jul 07 23:19:44.337299 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:19:44.337271 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/33b45f14-3cc1-4712-bf73-206fa4041802-tmp\") pod \"opendatahub-operator-controller-manager-7c9859966d-7bxpt\" (UID: \"33b45f14-3cc1-4712-bf73-206fa4041802\") " pod="opendatahub/opendatahub-operator-controller-manager-7c9859966d-7bxpt" Jul 07 23:19:44.337422 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:19:44.337305 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/33b45f14-3cc1-4712-bf73-206fa4041802-webhook-cert\") pod \"opendatahub-operator-controller-manager-7c9859966d-7bxpt\" (UID: \"33b45f14-3cc1-4712-bf73-206fa4041802\") " pod="opendatahub/opendatahub-operator-controller-manager-7c9859966d-7bxpt" Jul 07 23:19:44.337422 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:19:44.337334 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/33b45f14-3cc1-4712-bf73-206fa4041802-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7c9859966d-7bxpt\" (UID: \"33b45f14-3cc1-4712-bf73-206fa4041802\") " pod="opendatahub/opendatahub-operator-controller-manager-7c9859966d-7bxpt" Jul 07 23:19:44.337422 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:19:44.337358 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/33b45f14-3cc1-4712-bf73-206fa4041802-manifests\") pod \"opendatahub-operator-controller-manager-7c9859966d-7bxpt\" (UID: \"33b45f14-3cc1-4712-bf73-206fa4041802\") " pod="opendatahub/opendatahub-operator-controller-manager-7c9859966d-7bxpt" Jul 07 23:19:44.337422 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:19:44.337387 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7z98g\" (UniqueName: \"kubernetes.io/projected/33b45f14-3cc1-4712-bf73-206fa4041802-kube-api-access-7z98g\") pod \"opendatahub-operator-controller-manager-7c9859966d-7bxpt\" (UID: \"33b45f14-3cc1-4712-bf73-206fa4041802\") " pod="opendatahub/opendatahub-operator-controller-manager-7c9859966d-7bxpt" Jul 07 23:19:44.438745 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:19:44.438704 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7z98g\" (UniqueName: \"kubernetes.io/projected/33b45f14-3cc1-4712-bf73-206fa4041802-kube-api-access-7z98g\") pod \"opendatahub-operator-controller-manager-7c9859966d-7bxpt\" (UID: \"33b45f14-3cc1-4712-bf73-206fa4041802\") " pod="opendatahub/opendatahub-operator-controller-manager-7c9859966d-7bxpt" Jul 07 23:19:44.438947 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:19:44.438803 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/33b45f14-3cc1-4712-bf73-206fa4041802-tmp\") pod \"opendatahub-operator-controller-manager-7c9859966d-7bxpt\" (UID: \"33b45f14-3cc1-4712-bf73-206fa4041802\") " pod="opendatahub/opendatahub-operator-controller-manager-7c9859966d-7bxpt" Jul 07 23:19:44.438947 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:19:44.438836 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/33b45f14-3cc1-4712-bf73-206fa4041802-webhook-cert\") pod \"opendatahub-operator-controller-manager-7c9859966d-7bxpt\" (UID: \"33b45f14-3cc1-4712-bf73-206fa4041802\") " pod="opendatahub/opendatahub-operator-controller-manager-7c9859966d-7bxpt" Jul 07 23:19:44.438947 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:19:44.438874 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/33b45f14-3cc1-4712-bf73-206fa4041802-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7c9859966d-7bxpt\" (UID: \"33b45f14-3cc1-4712-bf73-206fa4041802\") " pod="opendatahub/opendatahub-operator-controller-manager-7c9859966d-7bxpt" Jul 07 23:19:44.439100 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:19:44.438992 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/33b45f14-3cc1-4712-bf73-206fa4041802-manifests\") pod \"opendatahub-operator-controller-manager-7c9859966d-7bxpt\" (UID: \"33b45f14-3cc1-4712-bf73-206fa4041802\") " pod="opendatahub/opendatahub-operator-controller-manager-7c9859966d-7bxpt" Jul 07 23:19:44.439318 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:19:44.439284 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/33b45f14-3cc1-4712-bf73-206fa4041802-manifests\") pod \"opendatahub-operator-controller-manager-7c9859966d-7bxpt\" (UID: \"33b45f14-3cc1-4712-bf73-206fa4041802\") " pod="opendatahub/opendatahub-operator-controller-manager-7c9859966d-7bxpt" Jul 07 23:19:44.441042 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:19:44.441016 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/33b45f14-3cc1-4712-bf73-206fa4041802-tmp\") pod \"opendatahub-operator-controller-manager-7c9859966d-7bxpt\" (UID: \"33b45f14-3cc1-4712-bf73-206fa4041802\") " pod="opendatahub/opendatahub-operator-controller-manager-7c9859966d-7bxpt" Jul 07 23:19:44.441424 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:19:44.441405 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/33b45f14-3cc1-4712-bf73-206fa4041802-webhook-cert\") pod \"opendatahub-operator-controller-manager-7c9859966d-7bxpt\" (UID: \"33b45f14-3cc1-4712-bf73-206fa4041802\") " pod="opendatahub/opendatahub-operator-controller-manager-7c9859966d-7bxpt" Jul 07 23:19:44.441490 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:19:44.441405 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/33b45f14-3cc1-4712-bf73-206fa4041802-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7c9859966d-7bxpt\" (UID: \"33b45f14-3cc1-4712-bf73-206fa4041802\") " pod="opendatahub/opendatahub-operator-controller-manager-7c9859966d-7bxpt" Jul 07 23:19:44.467207 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:19:44.467182 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7z98g\" (UniqueName: \"kubernetes.io/projected/33b45f14-3cc1-4712-bf73-206fa4041802-kube-api-access-7z98g\") pod \"opendatahub-operator-controller-manager-7c9859966d-7bxpt\" (UID: \"33b45f14-3cc1-4712-bf73-206fa4041802\") " pod="opendatahub/opendatahub-operator-controller-manager-7c9859966d-7bxpt" Jul 07 23:19:44.569023 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:19:44.568985 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-7c9859966d-7bxpt" Jul 07 23:19:44.580923 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:19:44.580895 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:19:44.679447 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:19:44.679415 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:19:44.717812 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:19:44.717657 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7c9859966d-7bxpt"] Jul 07 23:19:44.720246 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:19:44.720223 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod33b45f14_3cc1_4712_bf73_206fa4041802.slice/crio-34f0506d925f5497770cae05d54d7a1f268f0b7317b1edb19cd8d0b1477af97a WatchSource:0}: Error finding container 34f0506d925f5497770cae05d54d7a1f268f0b7317b1edb19cd8d0b1477af97a: Status 404 returned error can't find the container with id 34f0506d925f5497770cae05d54d7a1f268f0b7317b1edb19cd8d0b1477af97a Jul 07 23:19:44.787019 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:19:44.786985 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:19:44.821525 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:19:44.821443 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7c9859966d-7bxpt" event={"ID":"33b45f14-3cc1-4712-bf73-206fa4041802","Type":"ContainerStarted","Data":"34f0506d925f5497770cae05d54d7a1f268f0b7317b1edb19cd8d0b1477af97a"} Jul 07 23:19:44.878833 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:19:44.878793 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:19:44.978610 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:19:44.978568 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:19:45.359269 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:19:45.359196 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:19:45.452717 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:19:45.452684 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:19:47.839838 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:19:47.839804 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7c9859966d-7bxpt" event={"ID":"33b45f14-3cc1-4712-bf73-206fa4041802","Type":"ContainerStarted","Data":"94a9c61ad35002c9b8da8afe9cbdd276a62ea41a879f878b1d08ae1a692f3252"} Jul 07 23:19:48.845508 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:19:48.845460 2570 generic.go:356] "Generic (PLEG): container finished" podID="33b45f14-3cc1-4712-bf73-206fa4041802" containerID="94a9c61ad35002c9b8da8afe9cbdd276a62ea41a879f878b1d08ae1a692f3252" exitCode=0 Jul 07 23:19:48.845824 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:19:48.845503 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7c9859966d-7bxpt" event={"ID":"33b45f14-3cc1-4712-bf73-206fa4041802","Type":"ContainerDied","Data":"94a9c61ad35002c9b8da8afe9cbdd276a62ea41a879f878b1d08ae1a692f3252"} Jul 07 23:19:49.853879 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:19:49.853839 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7c9859966d-7bxpt" event={"ID":"33b45f14-3cc1-4712-bf73-206fa4041802","Type":"ContainerStarted","Data":"2dea622233942acbf22e13b8e213bed74b47f0cf443b7ad9995e78a29bd41648"} Jul 07 23:19:49.854273 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:19:49.853944 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-7c9859966d-7bxpt" Jul 07 23:19:49.880507 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:19:49.880438 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-7c9859966d-7bxpt" podStartSLOduration=1.462235057 podStartE2EDuration="5.880422429s" totalImagesPullingTime="3.312460693s" totalInitContainerRuntime="1.105726679s" isStatefulPod=false podCreationTimestamp="2026-07-07 23:19:44 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 23:19:49.877825802 +0000 UTC m=+407.274827311" watchObservedRunningTime="2026-07-07 23:19:49.880422429 +0000 UTC m=+407.277424006" Jul 07 23:20:00.860930 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:20:00.860896 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-7c9859966d-7bxpt" Jul 07 23:20:01.853571 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:20:01.853541 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-7895d9b5-ct8kv"] Jul 07 23:20:01.858037 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:20:01.858017 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-7895d9b5-ct8kv" Jul 07 23:20:01.862105 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:20:01.862087 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-ghd6g\"" Jul 07 23:20:01.862446 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:20:01.862120 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Jul 07 23:20:01.862446 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:20:01.862137 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Jul 07 23:20:01.862446 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:20:01.862176 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Jul 07 23:20:01.877193 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:20:01.877167 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-7895d9b5-ct8kv"] Jul 07 23:20:01.900106 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:20:01.900079 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5qxll\" (UniqueName: \"kubernetes.io/projected/4fa028e3-4f6a-4053-a9d6-ca798750c5eb-kube-api-access-5qxll\") pod \"lws-controller-manager-7895d9b5-ct8kv\" (UID: \"4fa028e3-4f6a-4053-a9d6-ca798750c5eb\") " pod="openshift-lws-operator/lws-controller-manager-7895d9b5-ct8kv" Jul 07 23:20:01.900234 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:20:01.900165 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/4fa028e3-4f6a-4053-a9d6-ca798750c5eb-manager-config\") pod \"lws-controller-manager-7895d9b5-ct8kv\" (UID: \"4fa028e3-4f6a-4053-a9d6-ca798750c5eb\") " pod="openshift-lws-operator/lws-controller-manager-7895d9b5-ct8kv" Jul 07 23:20:01.900234 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:20:01.900187 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4fa028e3-4f6a-4053-a9d6-ca798750c5eb-cert\") pod \"lws-controller-manager-7895d9b5-ct8kv\" (UID: \"4fa028e3-4f6a-4053-a9d6-ca798750c5eb\") " pod="openshift-lws-operator/lws-controller-manager-7895d9b5-ct8kv" Jul 07 23:20:01.900234 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:20:01.900222 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/4fa028e3-4f6a-4053-a9d6-ca798750c5eb-metrics-cert\") pod \"lws-controller-manager-7895d9b5-ct8kv\" (UID: \"4fa028e3-4f6a-4053-a9d6-ca798750c5eb\") " pod="openshift-lws-operator/lws-controller-manager-7895d9b5-ct8kv" Jul 07 23:20:02.000815 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:20:02.000775 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/4fa028e3-4f6a-4053-a9d6-ca798750c5eb-manager-config\") pod \"lws-controller-manager-7895d9b5-ct8kv\" (UID: \"4fa028e3-4f6a-4053-a9d6-ca798750c5eb\") " pod="openshift-lws-operator/lws-controller-manager-7895d9b5-ct8kv" Jul 07 23:20:02.000815 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:20:02.000815 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4fa028e3-4f6a-4053-a9d6-ca798750c5eb-cert\") pod \"lws-controller-manager-7895d9b5-ct8kv\" (UID: \"4fa028e3-4f6a-4053-a9d6-ca798750c5eb\") " pod="openshift-lws-operator/lws-controller-manager-7895d9b5-ct8kv" Jul 07 23:20:02.001077 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:20:02.000834 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/4fa028e3-4f6a-4053-a9d6-ca798750c5eb-metrics-cert\") pod \"lws-controller-manager-7895d9b5-ct8kv\" (UID: \"4fa028e3-4f6a-4053-a9d6-ca798750c5eb\") " pod="openshift-lws-operator/lws-controller-manager-7895d9b5-ct8kv" Jul 07 23:20:02.001077 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:20:02.000880 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5qxll\" (UniqueName: \"kubernetes.io/projected/4fa028e3-4f6a-4053-a9d6-ca798750c5eb-kube-api-access-5qxll\") pod \"lws-controller-manager-7895d9b5-ct8kv\" (UID: \"4fa028e3-4f6a-4053-a9d6-ca798750c5eb\") " pod="openshift-lws-operator/lws-controller-manager-7895d9b5-ct8kv" Jul 07 23:20:02.001536 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:20:02.001509 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/4fa028e3-4f6a-4053-a9d6-ca798750c5eb-manager-config\") pod \"lws-controller-manager-7895d9b5-ct8kv\" (UID: \"4fa028e3-4f6a-4053-a9d6-ca798750c5eb\") " pod="openshift-lws-operator/lws-controller-manager-7895d9b5-ct8kv" Jul 07 23:20:02.003438 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:20:02.003414 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/4fa028e3-4f6a-4053-a9d6-ca798750c5eb-metrics-cert\") pod \"lws-controller-manager-7895d9b5-ct8kv\" (UID: \"4fa028e3-4f6a-4053-a9d6-ca798750c5eb\") " pod="openshift-lws-operator/lws-controller-manager-7895d9b5-ct8kv" Jul 07 23:20:02.003506 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:20:02.003414 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4fa028e3-4f6a-4053-a9d6-ca798750c5eb-cert\") pod \"lws-controller-manager-7895d9b5-ct8kv\" (UID: \"4fa028e3-4f6a-4053-a9d6-ca798750c5eb\") " pod="openshift-lws-operator/lws-controller-manager-7895d9b5-ct8kv" Jul 07 23:20:02.011192 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:20:02.011167 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5qxll\" (UniqueName: \"kubernetes.io/projected/4fa028e3-4f6a-4053-a9d6-ca798750c5eb-kube-api-access-5qxll\") pod \"lws-controller-manager-7895d9b5-ct8kv\" (UID: \"4fa028e3-4f6a-4053-a9d6-ca798750c5eb\") " pod="openshift-lws-operator/lws-controller-manager-7895d9b5-ct8kv" Jul 07 23:20:02.167417 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:20:02.167323 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-7895d9b5-ct8kv" Jul 07 23:20:02.300291 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:20:02.300267 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-7895d9b5-ct8kv"] Jul 07 23:20:02.302779 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:20:02.302741 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4fa028e3_4f6a_4053_a9d6_ca798750c5eb.slice/crio-513fd6ad2173bcc4c887723267884c1354696a8b3abfed0083014d66ac892bcd WatchSource:0}: Error finding container 513fd6ad2173bcc4c887723267884c1354696a8b3abfed0083014d66ac892bcd: Status 404 returned error can't find the container with id 513fd6ad2173bcc4c887723267884c1354696a8b3abfed0083014d66ac892bcd Jul 07 23:20:02.916312 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:20:02.916282 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-7895d9b5-ct8kv" event={"ID":"4fa028e3-4f6a-4053-a9d6-ca798750c5eb","Type":"ContainerStarted","Data":"513fd6ad2173bcc4c887723267884c1354696a8b3abfed0083014d66ac892bcd"} Jul 07 23:20:03.661384 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:20:03.661341 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:20:03.908047 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:20:03.907999 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:20:04.236636 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:20:04.236600 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:20:04.382260 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:20:04.382203 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:20:04.403599 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:20:04.403567 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-7b7668c465-j5zks"] Jul 07 23:20:04.408072 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:20:04.408050 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-7b7668c465-j5zks" Jul 07 23:20:04.410745 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:20:04.410714 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Jul 07 23:20:04.411753 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:20:04.411733 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-xf9mp\"" Jul 07 23:20:04.411753 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:20:04.411738 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Jul 07 23:20:04.418588 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:20:04.418568 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-spqcv\" (UniqueName: \"kubernetes.io/projected/5157b9da-e2ac-44a1-8ab2-c5f3c787e896-kube-api-access-spqcv\") pod \"kube-auth-proxy-7b7668c465-j5zks\" (UID: \"5157b9da-e2ac-44a1-8ab2-c5f3c787e896\") " pod="openshift-ingress/kube-auth-proxy-7b7668c465-j5zks" Jul 07 23:20:04.418672 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:20:04.418601 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5157b9da-e2ac-44a1-8ab2-c5f3c787e896-tmp\") pod \"kube-auth-proxy-7b7668c465-j5zks\" (UID: \"5157b9da-e2ac-44a1-8ab2-c5f3c787e896\") " pod="openshift-ingress/kube-auth-proxy-7b7668c465-j5zks" Jul 07 23:20:04.418672 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:20:04.418631 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5157b9da-e2ac-44a1-8ab2-c5f3c787e896-tls-certs\") pod \"kube-auth-proxy-7b7668c465-j5zks\" (UID: \"5157b9da-e2ac-44a1-8ab2-c5f3c787e896\") " pod="openshift-ingress/kube-auth-proxy-7b7668c465-j5zks" Jul 07 23:20:04.427959 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:20:04.427928 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-7b7668c465-j5zks"] Jul 07 23:20:04.519936 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:20:04.519838 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5157b9da-e2ac-44a1-8ab2-c5f3c787e896-tls-certs\") pod \"kube-auth-proxy-7b7668c465-j5zks\" (UID: \"5157b9da-e2ac-44a1-8ab2-c5f3c787e896\") " pod="openshift-ingress/kube-auth-proxy-7b7668c465-j5zks" Jul 07 23:20:04.519936 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:20:04.519929 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-spqcv\" (UniqueName: \"kubernetes.io/projected/5157b9da-e2ac-44a1-8ab2-c5f3c787e896-kube-api-access-spqcv\") pod \"kube-auth-proxy-7b7668c465-j5zks\" (UID: \"5157b9da-e2ac-44a1-8ab2-c5f3c787e896\") " pod="openshift-ingress/kube-auth-proxy-7b7668c465-j5zks" Jul 07 23:20:04.520180 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:20:04.519958 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5157b9da-e2ac-44a1-8ab2-c5f3c787e896-tmp\") pod \"kube-auth-proxy-7b7668c465-j5zks\" (UID: \"5157b9da-e2ac-44a1-8ab2-c5f3c787e896\") " pod="openshift-ingress/kube-auth-proxy-7b7668c465-j5zks" Jul 07 23:20:04.522287 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:20:04.522258 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5157b9da-e2ac-44a1-8ab2-c5f3c787e896-tmp\") pod \"kube-auth-proxy-7b7668c465-j5zks\" (UID: \"5157b9da-e2ac-44a1-8ab2-c5f3c787e896\") " pod="openshift-ingress/kube-auth-proxy-7b7668c465-j5zks" Jul 07 23:20:04.522518 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:20:04.522499 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5157b9da-e2ac-44a1-8ab2-c5f3c787e896-tls-certs\") pod \"kube-auth-proxy-7b7668c465-j5zks\" (UID: \"5157b9da-e2ac-44a1-8ab2-c5f3c787e896\") " pod="openshift-ingress/kube-auth-proxy-7b7668c465-j5zks" Jul 07 23:20:04.539157 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:20:04.539125 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-spqcv\" (UniqueName: \"kubernetes.io/projected/5157b9da-e2ac-44a1-8ab2-c5f3c787e896-kube-api-access-spqcv\") pod \"kube-auth-proxy-7b7668c465-j5zks\" (UID: \"5157b9da-e2ac-44a1-8ab2-c5f3c787e896\") " pod="openshift-ingress/kube-auth-proxy-7b7668c465-j5zks" Jul 07 23:20:04.718491 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:20:04.718436 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-7b7668c465-j5zks" Jul 07 23:20:04.791799 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:20:04.791767 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:20:04.886740 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:20:04.886704 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-7b7668c465-j5zks"] Jul 07 23:20:04.888776 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:20:04.888747 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:20:04.892557 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:20:04.890229 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5157b9da_e2ac_44a1_8ab2_c5f3c787e896.slice/crio-0bdfd09e5c37a0221f136817ede990282caa10aba70c500463110e008f27bd6c WatchSource:0}: Error finding container 0bdfd09e5c37a0221f136817ede990282caa10aba70c500463110e008f27bd6c: Status 404 returned error can't find the container with id 0bdfd09e5c37a0221f136817ede990282caa10aba70c500463110e008f27bd6c Jul 07 23:20:04.927868 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:20:04.927829 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-7b7668c465-j5zks" event={"ID":"5157b9da-e2ac-44a1-8ab2-c5f3c787e896","Type":"ContainerStarted","Data":"0bdfd09e5c37a0221f136817ede990282caa10aba70c500463110e008f27bd6c"} Jul 07 23:20:04.964743 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:20:04.964700 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:20:05.212969 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:20:05.212882 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:20:05.652507 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:20:05.652396 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:20:05.764605 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:20:05.764567 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:20:05.938739 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:20:05.937783 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-7895d9b5-ct8kv" event={"ID":"4fa028e3-4f6a-4053-a9d6-ca798750c5eb","Type":"ContainerStarted","Data":"7b4872d3b8cb46df2aecd5f1daab50d3186acdce6e6f6515566a5468bedd443c"} Jul 07 23:20:05.938739 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:20:05.938613 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-7895d9b5-ct8kv" Jul 07 23:20:05.989300 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:20:05.989264 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:20:06.087600 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:20:06.087561 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:20:07.128223 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:20:07.128145 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:20:07.950660 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:20:07.950619 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-7b7668c465-j5zks" event={"ID":"5157b9da-e2ac-44a1-8ab2-c5f3c787e896","Type":"ContainerStarted","Data":"ad22253ef10f6c95b6a06a78d33824e0c44549d76a8b0d49153043394b46e346"} Jul 07 23:20:07.967315 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:20:07.967260 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-7895d9b5-ct8kv" podStartSLOduration=3.8881725080000002 podStartE2EDuration="6.967244941s" totalImagesPullingTime="3.079072433s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 23:20:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 23:20:05.962283104 +0000 UTC m=+423.359284617" watchObservedRunningTime="2026-07-07 23:20:07.967244941 +0000 UTC m=+425.364246456" Jul 07 23:20:07.968556 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:20:07.968514 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-7b7668c465-j5zks" podStartSLOduration=1.37320784 podStartE2EDuration="3.968502939s" totalImagesPullingTime="2.595295099s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 23:20:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 23:20:07.967028661 +0000 UTC m=+425.364030177" watchObservedRunningTime="2026-07-07 23:20:07.968502939 +0000 UTC m=+425.365504451" Jul 07 23:20:13.924686 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:20:13.924642 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:20:14.018490 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:20:14.018444 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:20:14.592188 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:20:14.592157 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:20:14.874642 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:20:14.874549 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:20:14.990255 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:20:14.990210 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:20:16.366891 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:20:16.366848 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:20:16.456755 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:20:16.456724 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:20:16.533094 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:20:16.533060 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:20:16.636138 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:20:16.636043 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:20:16.743421 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:20:16.743388 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:20:16.855663 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:20:16.855628 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:20:16.987762 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:20:16.987665 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:20:17.958521 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:20:17.958487 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-7895d9b5-ct8kv" Jul 07 23:20:53.630223 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:20:53.630187 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:21:05.160192 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:21:05.160156 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:21:05.280967 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:21:05.280926 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:21:05.455557 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:21:05.455448 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:21:06.103888 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:21:06.103846 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:21:06.303064 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:21:06.303035 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:21:21.316401 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:21:21.316369 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e067bwg"] Jul 07 23:21:21.320113 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:21:21.320097 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e067bwg" Jul 07 23:21:21.322824 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:21:21.322799 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 07 23:21:21.322931 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:21:21.322838 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 07 23:21:21.322931 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:21:21.322845 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-zdtnw\"" Jul 07 23:21:21.329701 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:21:21.329678 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e067bwg"] Jul 07 23:21:21.382644 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:21:21.382615 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wjgqx\" (UniqueName: \"kubernetes.io/projected/ae1ba717-f216-48ff-9789-43126b0a9c1f-kube-api-access-wjgqx\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e067bwg\" (UID: \"ae1ba717-f216-48ff-9789-43126b0a9c1f\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e067bwg" Jul 07 23:21:21.382790 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:21:21.382667 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ae1ba717-f216-48ff-9789-43126b0a9c1f-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e067bwg\" (UID: \"ae1ba717-f216-48ff-9789-43126b0a9c1f\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e067bwg" Jul 07 23:21:21.382790 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:21:21.382700 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ae1ba717-f216-48ff-9789-43126b0a9c1f-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e067bwg\" (UID: \"ae1ba717-f216-48ff-9789-43126b0a9c1f\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e067bwg" Jul 07 23:21:21.483792 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:21:21.483762 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wjgqx\" (UniqueName: \"kubernetes.io/projected/ae1ba717-f216-48ff-9789-43126b0a9c1f-kube-api-access-wjgqx\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e067bwg\" (UID: \"ae1ba717-f216-48ff-9789-43126b0a9c1f\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e067bwg" Jul 07 23:21:21.483950 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:21:21.483808 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ae1ba717-f216-48ff-9789-43126b0a9c1f-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e067bwg\" (UID: \"ae1ba717-f216-48ff-9789-43126b0a9c1f\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e067bwg" Jul 07 23:21:21.483950 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:21:21.483832 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ae1ba717-f216-48ff-9789-43126b0a9c1f-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e067bwg\" (UID: \"ae1ba717-f216-48ff-9789-43126b0a9c1f\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e067bwg" Jul 07 23:21:21.484229 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:21:21.484210 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ae1ba717-f216-48ff-9789-43126b0a9c1f-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e067bwg\" (UID: \"ae1ba717-f216-48ff-9789-43126b0a9c1f\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e067bwg" Jul 07 23:21:21.484229 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:21:21.484224 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ae1ba717-f216-48ff-9789-43126b0a9c1f-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e067bwg\" (UID: \"ae1ba717-f216-48ff-9789-43126b0a9c1f\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e067bwg" Jul 07 23:21:21.494011 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:21:21.493989 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wjgqx\" (UniqueName: \"kubernetes.io/projected/ae1ba717-f216-48ff-9789-43126b0a9c1f-kube-api-access-wjgqx\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e067bwg\" (UID: \"ae1ba717-f216-48ff-9789-43126b0a9c1f\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e067bwg" Jul 07 23:21:21.629575 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:21:21.629494 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e067bwg" Jul 07 23:21:21.770541 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:21:21.770517 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e067bwg"] Jul 07 23:21:21.772520 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:21:21.772443 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podae1ba717_f216_48ff_9789_43126b0a9c1f.slice/crio-79f72d92b68936e99ca5b032934dd9609e9f756d39746d18b1bb4f99afac0b14 WatchSource:0}: Error finding container 79f72d92b68936e99ca5b032934dd9609e9f756d39746d18b1bb4f99afac0b14: Status 404 returned error can't find the container with id 79f72d92b68936e99ca5b032934dd9609e9f756d39746d18b1bb4f99afac0b14 Jul 07 23:21:22.109583 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:21:22.109549 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759wgjps"] Jul 07 23:21:22.112989 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:21:22.112973 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759wgjps" Jul 07 23:21:22.122401 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:21:22.122372 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759wgjps"] Jul 07 23:21:22.188397 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:21:22.188361 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tlf82\" (UniqueName: \"kubernetes.io/projected/8ef9793b-9b49-43a9-9b83-cc09c9002479-kube-api-access-tlf82\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759wgjps\" (UID: \"8ef9793b-9b49-43a9-9b83-cc09c9002479\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759wgjps" Jul 07 23:21:22.188577 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:21:22.188417 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8ef9793b-9b49-43a9-9b83-cc09c9002479-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759wgjps\" (UID: \"8ef9793b-9b49-43a9-9b83-cc09c9002479\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759wgjps" Jul 07 23:21:22.188577 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:21:22.188441 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8ef9793b-9b49-43a9-9b83-cc09c9002479-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759wgjps\" (UID: \"8ef9793b-9b49-43a9-9b83-cc09c9002479\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759wgjps" Jul 07 23:21:22.289269 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:21:22.289237 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8ef9793b-9b49-43a9-9b83-cc09c9002479-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759wgjps\" (UID: \"8ef9793b-9b49-43a9-9b83-cc09c9002479\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759wgjps" Jul 07 23:21:22.289459 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:21:22.289300 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tlf82\" (UniqueName: \"kubernetes.io/projected/8ef9793b-9b49-43a9-9b83-cc09c9002479-kube-api-access-tlf82\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759wgjps\" (UID: \"8ef9793b-9b49-43a9-9b83-cc09c9002479\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759wgjps" Jul 07 23:21:22.289459 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:21:22.289348 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8ef9793b-9b49-43a9-9b83-cc09c9002479-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759wgjps\" (UID: \"8ef9793b-9b49-43a9-9b83-cc09c9002479\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759wgjps" Jul 07 23:21:22.289666 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:21:22.289645 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8ef9793b-9b49-43a9-9b83-cc09c9002479-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759wgjps\" (UID: \"8ef9793b-9b49-43a9-9b83-cc09c9002479\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759wgjps" Jul 07 23:21:22.289729 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:21:22.289675 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8ef9793b-9b49-43a9-9b83-cc09c9002479-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759wgjps\" (UID: \"8ef9793b-9b49-43a9-9b83-cc09c9002479\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759wgjps" Jul 07 23:21:22.298177 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:21:22.298149 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tlf82\" (UniqueName: \"kubernetes.io/projected/8ef9793b-9b49-43a9-9b83-cc09c9002479-kube-api-access-tlf82\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759wgjps\" (UID: \"8ef9793b-9b49-43a9-9b83-cc09c9002479\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759wgjps" Jul 07 23:21:22.301235 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:21:22.301211 2570 generic.go:356] "Generic (PLEG): container finished" podID="ae1ba717-f216-48ff-9789-43126b0a9c1f" containerID="db6976bf559e02789915c6bdc98e3a97ee8d0d55396338a6be2c9c15dcae0d14" exitCode=0 Jul 07 23:21:22.301339 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:21:22.301290 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e067bwg" event={"ID":"ae1ba717-f216-48ff-9789-43126b0a9c1f","Type":"ContainerDied","Data":"db6976bf559e02789915c6bdc98e3a97ee8d0d55396338a6be2c9c15dcae0d14"} Jul 07 23:21:22.301395 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:21:22.301340 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e067bwg" event={"ID":"ae1ba717-f216-48ff-9789-43126b0a9c1f","Type":"ContainerStarted","Data":"79f72d92b68936e99ca5b032934dd9609e9f756d39746d18b1bb4f99afac0b14"} Jul 07 23:21:22.422677 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:21:22.422596 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759wgjps" Jul 07 23:21:22.557705 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:21:22.557638 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759wgjps"] Jul 07 23:21:22.559980 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:21:22.559941 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8ef9793b_9b49_43a9_9b83_cc09c9002479.slice/crio-69ddddd3b2089e36dc548315706e2bdb0483af39ec46d7c477dc3e999de70434 WatchSource:0}: Error finding container 69ddddd3b2089e36dc548315706e2bdb0483af39ec46d7c477dc3e999de70434: Status 404 returned error can't find the container with id 69ddddd3b2089e36dc548315706e2bdb0483af39ec46d7c477dc3e999de70434 Jul 07 23:21:22.908648 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:21:22.908622 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1stzl9"] Jul 07 23:21:22.913290 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:21:22.913271 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1stzl9" Jul 07 23:21:22.921112 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:21:22.921089 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1stzl9"] Jul 07 23:21:22.995995 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:21:22.995969 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/807f17fe-22a0-4a3b-82af-b242ca8e2013-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1stzl9\" (UID: \"807f17fe-22a0-4a3b-82af-b242ca8e2013\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1stzl9" Jul 07 23:21:22.996110 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:21:22.996010 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/807f17fe-22a0-4a3b-82af-b242ca8e2013-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1stzl9\" (UID: \"807f17fe-22a0-4a3b-82af-b242ca8e2013\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1stzl9" Jul 07 23:21:22.996110 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:21:22.996027 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cfgw6\" (UniqueName: \"kubernetes.io/projected/807f17fe-22a0-4a3b-82af-b242ca8e2013-kube-api-access-cfgw6\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1stzl9\" (UID: \"807f17fe-22a0-4a3b-82af-b242ca8e2013\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1stzl9" Jul 07 23:21:23.096932 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:21:23.096901 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/807f17fe-22a0-4a3b-82af-b242ca8e2013-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1stzl9\" (UID: \"807f17fe-22a0-4a3b-82af-b242ca8e2013\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1stzl9" Jul 07 23:21:23.097098 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:21:23.096944 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/807f17fe-22a0-4a3b-82af-b242ca8e2013-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1stzl9\" (UID: \"807f17fe-22a0-4a3b-82af-b242ca8e2013\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1stzl9" Jul 07 23:21:23.097098 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:21:23.096966 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cfgw6\" (UniqueName: \"kubernetes.io/projected/807f17fe-22a0-4a3b-82af-b242ca8e2013-kube-api-access-cfgw6\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1stzl9\" (UID: \"807f17fe-22a0-4a3b-82af-b242ca8e2013\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1stzl9" Jul 07 23:21:23.097259 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:21:23.097237 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/807f17fe-22a0-4a3b-82af-b242ca8e2013-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1stzl9\" (UID: \"807f17fe-22a0-4a3b-82af-b242ca8e2013\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1stzl9" Jul 07 23:21:23.097319 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:21:23.097299 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/807f17fe-22a0-4a3b-82af-b242ca8e2013-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1stzl9\" (UID: \"807f17fe-22a0-4a3b-82af-b242ca8e2013\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1stzl9" Jul 07 23:21:23.106520 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:21:23.106498 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cfgw6\" (UniqueName: \"kubernetes.io/projected/807f17fe-22a0-4a3b-82af-b242ca8e2013-kube-api-access-cfgw6\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1stzl9\" (UID: \"807f17fe-22a0-4a3b-82af-b242ca8e2013\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1stzl9" Jul 07 23:21:23.251681 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:21:23.251596 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1stzl9" Jul 07 23:21:23.316048 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:21:23.316001 2570 generic.go:356] "Generic (PLEG): container finished" podID="ae1ba717-f216-48ff-9789-43126b0a9c1f" containerID="09ebc7301b4c4e24cbfc4cc364577b4fe6dd93f193be312dcbc103da672cb088" exitCode=0 Jul 07 23:21:23.316195 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:21:23.316154 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e067bwg" event={"ID":"ae1ba717-f216-48ff-9789-43126b0a9c1f","Type":"ContainerDied","Data":"09ebc7301b4c4e24cbfc4cc364577b4fe6dd93f193be312dcbc103da672cb088"} Jul 07 23:21:23.320496 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:21:23.320448 2570 generic.go:356] "Generic (PLEG): container finished" podID="8ef9793b-9b49-43a9-9b83-cc09c9002479" containerID="f41345390ed19e720be010c694b3b12593a589697683dbaaf712be73fea32e40" exitCode=0 Jul 07 23:21:23.320620 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:21:23.320587 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759wgjps" event={"ID":"8ef9793b-9b49-43a9-9b83-cc09c9002479","Type":"ContainerDied","Data":"f41345390ed19e720be010c694b3b12593a589697683dbaaf712be73fea32e40"} Jul 07 23:21:23.320680 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:21:23.320627 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759wgjps" event={"ID":"8ef9793b-9b49-43a9-9b83-cc09c9002479","Type":"ContainerStarted","Data":"69ddddd3b2089e36dc548315706e2bdb0483af39ec46d7c477dc3e999de70434"} Jul 07 23:21:23.395093 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:21:23.395064 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1stzl9"] Jul 07 23:21:23.397100 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:21:23.397076 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod807f17fe_22a0_4a3b_82af_b242ca8e2013.slice/crio-bc2964c67b3e02cc072379eb62c81d875e5a706d00ed46aa51733f1821db0c75 WatchSource:0}: Error finding container bc2964c67b3e02cc072379eb62c81d875e5a706d00ed46aa51733f1821db0c75: Status 404 returned error can't find the container with id bc2964c67b3e02cc072379eb62c81d875e5a706d00ed46aa51733f1821db0c75 Jul 07 23:21:23.710242 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:21:23.710205 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed735hg4d"] Jul 07 23:21:23.713912 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:21:23.713896 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed735hg4d" Jul 07 23:21:23.722905 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:21:23.722879 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed735hg4d"] Jul 07 23:21:23.802562 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:21:23.802486 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/113878d9-e1e4-4af3-aed1-cdcd089443c4-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed735hg4d\" (UID: \"113878d9-e1e4-4af3-aed1-cdcd089443c4\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed735hg4d" Jul 07 23:21:23.802562 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:21:23.802522 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/113878d9-e1e4-4af3-aed1-cdcd089443c4-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed735hg4d\" (UID: \"113878d9-e1e4-4af3-aed1-cdcd089443c4\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed735hg4d" Jul 07 23:21:23.802731 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:21:23.802697 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zrtwp\" (UniqueName: \"kubernetes.io/projected/113878d9-e1e4-4af3-aed1-cdcd089443c4-kube-api-access-zrtwp\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed735hg4d\" (UID: \"113878d9-e1e4-4af3-aed1-cdcd089443c4\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed735hg4d" Jul 07 23:21:23.903356 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:21:23.903322 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/113878d9-e1e4-4af3-aed1-cdcd089443c4-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed735hg4d\" (UID: \"113878d9-e1e4-4af3-aed1-cdcd089443c4\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed735hg4d" Jul 07 23:21:23.903356 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:21:23.903360 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/113878d9-e1e4-4af3-aed1-cdcd089443c4-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed735hg4d\" (UID: \"113878d9-e1e4-4af3-aed1-cdcd089443c4\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed735hg4d" Jul 07 23:21:23.903576 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:21:23.903455 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zrtwp\" (UniqueName: \"kubernetes.io/projected/113878d9-e1e4-4af3-aed1-cdcd089443c4-kube-api-access-zrtwp\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed735hg4d\" (UID: \"113878d9-e1e4-4af3-aed1-cdcd089443c4\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed735hg4d" Jul 07 23:21:23.903754 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:21:23.903735 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/113878d9-e1e4-4af3-aed1-cdcd089443c4-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed735hg4d\" (UID: \"113878d9-e1e4-4af3-aed1-cdcd089443c4\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed735hg4d" Jul 07 23:21:23.903754 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:21:23.903745 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/113878d9-e1e4-4af3-aed1-cdcd089443c4-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed735hg4d\" (UID: \"113878d9-e1e4-4af3-aed1-cdcd089443c4\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed735hg4d" Jul 07 23:21:23.912703 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:21:23.912677 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zrtwp\" (UniqueName: \"kubernetes.io/projected/113878d9-e1e4-4af3-aed1-cdcd089443c4-kube-api-access-zrtwp\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed735hg4d\" (UID: \"113878d9-e1e4-4af3-aed1-cdcd089443c4\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed735hg4d" Jul 07 23:21:24.023988 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:21:24.023957 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed735hg4d" Jul 07 23:21:24.223242 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:21:24.223220 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed735hg4d"] Jul 07 23:21:24.225288 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:21:24.225256 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod113878d9_e1e4_4af3_aed1_cdcd089443c4.slice/crio-2d1e7be01c7b1a609e8e3a8d194b19fb4471f866af622cb15163a52d21d058b0 WatchSource:0}: Error finding container 2d1e7be01c7b1a609e8e3a8d194b19fb4471f866af622cb15163a52d21d058b0: Status 404 returned error can't find the container with id 2d1e7be01c7b1a609e8e3a8d194b19fb4471f866af622cb15163a52d21d058b0 Jul 07 23:21:24.327864 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:21:24.327830 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed735hg4d" event={"ID":"113878d9-e1e4-4af3-aed1-cdcd089443c4","Type":"ContainerStarted","Data":"1f254e09605ea42866bb4adebb4e58d871854910f797e785af1ddbecfb974960"} Jul 07 23:21:24.327989 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:21:24.327888 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed735hg4d" event={"ID":"113878d9-e1e4-4af3-aed1-cdcd089443c4","Type":"ContainerStarted","Data":"2d1e7be01c7b1a609e8e3a8d194b19fb4471f866af622cb15163a52d21d058b0"} Jul 07 23:21:24.329361 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:21:24.329341 2570 generic.go:356] "Generic (PLEG): container finished" podID="807f17fe-22a0-4a3b-82af-b242ca8e2013" containerID="71147c5cab2e8ed999c6045992b0653aba902008a24d8b9bc85c48f3751d3485" exitCode=0 Jul 07 23:21:24.329460 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:21:24.329426 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1stzl9" event={"ID":"807f17fe-22a0-4a3b-82af-b242ca8e2013","Type":"ContainerDied","Data":"71147c5cab2e8ed999c6045992b0653aba902008a24d8b9bc85c48f3751d3485"} Jul 07 23:21:24.329553 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:21:24.329497 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1stzl9" event={"ID":"807f17fe-22a0-4a3b-82af-b242ca8e2013","Type":"ContainerStarted","Data":"bc2964c67b3e02cc072379eb62c81d875e5a706d00ed46aa51733f1821db0c75"} Jul 07 23:21:24.331695 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:21:24.331674 2570 generic.go:356] "Generic (PLEG): container finished" podID="ae1ba717-f216-48ff-9789-43126b0a9c1f" containerID="035fedaa5c34d31a46681afb2afa06d4415ec16c6da28749ef1e998b9e10d4ff" exitCode=0 Jul 07 23:21:24.331795 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:21:24.331750 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e067bwg" event={"ID":"ae1ba717-f216-48ff-9789-43126b0a9c1f","Type":"ContainerDied","Data":"035fedaa5c34d31a46681afb2afa06d4415ec16c6da28749ef1e998b9e10d4ff"} Jul 07 23:21:24.333586 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:21:24.333539 2570 generic.go:356] "Generic (PLEG): container finished" podID="8ef9793b-9b49-43a9-9b83-cc09c9002479" containerID="e0865df4bea597c3cafafd12fc4250fc79fbc041c03489640fb8e7738bc1bdaa" exitCode=0 Jul 07 23:21:24.333649 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:21:24.333625 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759wgjps" event={"ID":"8ef9793b-9b49-43a9-9b83-cc09c9002479","Type":"ContainerDied","Data":"e0865df4bea597c3cafafd12fc4250fc79fbc041c03489640fb8e7738bc1bdaa"} Jul 07 23:21:25.340626 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:21:25.340549 2570 generic.go:356] "Generic (PLEG): container finished" podID="8ef9793b-9b49-43a9-9b83-cc09c9002479" containerID="6f72043621fddf0d557ac7b4fb373c4106abdfff2efa53e3dbe2b9bec5f72524" exitCode=0 Jul 07 23:21:25.340626 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:21:25.340609 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759wgjps" event={"ID":"8ef9793b-9b49-43a9-9b83-cc09c9002479","Type":"ContainerDied","Data":"6f72043621fddf0d557ac7b4fb373c4106abdfff2efa53e3dbe2b9bec5f72524"} Jul 07 23:21:25.342070 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:21:25.342048 2570 generic.go:356] "Generic (PLEG): container finished" podID="113878d9-e1e4-4af3-aed1-cdcd089443c4" containerID="1f254e09605ea42866bb4adebb4e58d871854910f797e785af1ddbecfb974960" exitCode=0 Jul 07 23:21:25.342197 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:21:25.342113 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed735hg4d" event={"ID":"113878d9-e1e4-4af3-aed1-cdcd089443c4","Type":"ContainerDied","Data":"1f254e09605ea42866bb4adebb4e58d871854910f797e785af1ddbecfb974960"} Jul 07 23:21:25.343911 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:21:25.343889 2570 generic.go:356] "Generic (PLEG): container finished" podID="807f17fe-22a0-4a3b-82af-b242ca8e2013" containerID="648db35f9f9a7dcfafa6d0ff2a69946cff98f856fb9b4bcc28795b013012cecc" exitCode=0 Jul 07 23:21:25.344006 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:21:25.343970 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1stzl9" event={"ID":"807f17fe-22a0-4a3b-82af-b242ca8e2013","Type":"ContainerDied","Data":"648db35f9f9a7dcfafa6d0ff2a69946cff98f856fb9b4bcc28795b013012cecc"} Jul 07 23:21:25.476649 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:21:25.476621 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e067bwg" Jul 07 23:21:25.519005 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:21:25.518980 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wjgqx\" (UniqueName: \"kubernetes.io/projected/ae1ba717-f216-48ff-9789-43126b0a9c1f-kube-api-access-wjgqx\") pod \"ae1ba717-f216-48ff-9789-43126b0a9c1f\" (UID: \"ae1ba717-f216-48ff-9789-43126b0a9c1f\") " Jul 07 23:21:25.519125 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:21:25.519034 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ae1ba717-f216-48ff-9789-43126b0a9c1f-util\") pod \"ae1ba717-f216-48ff-9789-43126b0a9c1f\" (UID: \"ae1ba717-f216-48ff-9789-43126b0a9c1f\") " Jul 07 23:21:25.519125 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:21:25.519102 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ae1ba717-f216-48ff-9789-43126b0a9c1f-bundle\") pod \"ae1ba717-f216-48ff-9789-43126b0a9c1f\" (UID: \"ae1ba717-f216-48ff-9789-43126b0a9c1f\") " Jul 07 23:21:25.519645 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:21:25.519620 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ae1ba717-f216-48ff-9789-43126b0a9c1f-bundle" (OuterVolumeSpecName: "bundle") pod "ae1ba717-f216-48ff-9789-43126b0a9c1f" (UID: "ae1ba717-f216-48ff-9789-43126b0a9c1f"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 23:21:25.521116 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:21:25.521088 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ae1ba717-f216-48ff-9789-43126b0a9c1f-kube-api-access-wjgqx" (OuterVolumeSpecName: "kube-api-access-wjgqx") pod "ae1ba717-f216-48ff-9789-43126b0a9c1f" (UID: "ae1ba717-f216-48ff-9789-43126b0a9c1f"). InnerVolumeSpecName "kube-api-access-wjgqx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 23:21:25.524842 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:21:25.524805 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ae1ba717-f216-48ff-9789-43126b0a9c1f-util" (OuterVolumeSpecName: "util") pod "ae1ba717-f216-48ff-9789-43126b0a9c1f" (UID: "ae1ba717-f216-48ff-9789-43126b0a9c1f"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 23:21:25.620458 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:21:25.620390 2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wjgqx\" (UniqueName: \"kubernetes.io/projected/ae1ba717-f216-48ff-9789-43126b0a9c1f-kube-api-access-wjgqx\") on node \"ip-10-0-134-140.ec2.internal\" DevicePath \"\"" Jul 07 23:21:25.620458 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:21:25.620418 2570 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ae1ba717-f216-48ff-9789-43126b0a9c1f-util\") on node \"ip-10-0-134-140.ec2.internal\" DevicePath \"\"" Jul 07 23:21:25.620458 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:21:25.620430 2570 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ae1ba717-f216-48ff-9789-43126b0a9c1f-bundle\") on node \"ip-10-0-134-140.ec2.internal\" DevicePath \"\"" Jul 07 23:21:26.350991 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:21:26.350950 2570 generic.go:356] "Generic (PLEG): container finished" podID="113878d9-e1e4-4af3-aed1-cdcd089443c4" containerID="9e32bf23bb3a9ad8fde0690b17cfae3ae313721ec0d94092b3c20292fc837ea7" exitCode=0 Jul 07 23:21:26.351369 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:21:26.351009 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed735hg4d" event={"ID":"113878d9-e1e4-4af3-aed1-cdcd089443c4","Type":"ContainerDied","Data":"9e32bf23bb3a9ad8fde0690b17cfae3ae313721ec0d94092b3c20292fc837ea7"} Jul 07 23:21:26.353165 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:21:26.353145 2570 generic.go:356] "Generic (PLEG): container finished" podID="807f17fe-22a0-4a3b-82af-b242ca8e2013" containerID="45d747a9ad2ad8c86236b2420bd99f9035ac48d91c37b93d9aec7fd491fa6ba6" exitCode=0 Jul 07 23:21:26.353270 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:21:26.353217 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1stzl9" event={"ID":"807f17fe-22a0-4a3b-82af-b242ca8e2013","Type":"ContainerDied","Data":"45d747a9ad2ad8c86236b2420bd99f9035ac48d91c37b93d9aec7fd491fa6ba6"} Jul 07 23:21:26.355047 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:21:26.355027 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e067bwg" event={"ID":"ae1ba717-f216-48ff-9789-43126b0a9c1f","Type":"ContainerDied","Data":"79f72d92b68936e99ca5b032934dd9609e9f756d39746d18b1bb4f99afac0b14"} Jul 07 23:21:26.355134 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:21:26.355051 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e067bwg" Jul 07 23:21:26.355134 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:21:26.355068 2570 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="79f72d92b68936e99ca5b032934dd9609e9f756d39746d18b1bb4f99afac0b14" Jul 07 23:21:26.484891 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:21:26.484834 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759wgjps" Jul 07 23:21:26.629546 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:21:26.629513 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8ef9793b-9b49-43a9-9b83-cc09c9002479-util\") pod \"8ef9793b-9b49-43a9-9b83-cc09c9002479\" (UID: \"8ef9793b-9b49-43a9-9b83-cc09c9002479\") " Jul 07 23:21:26.629700 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:21:26.629583 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tlf82\" (UniqueName: \"kubernetes.io/projected/8ef9793b-9b49-43a9-9b83-cc09c9002479-kube-api-access-tlf82\") pod \"8ef9793b-9b49-43a9-9b83-cc09c9002479\" (UID: \"8ef9793b-9b49-43a9-9b83-cc09c9002479\") " Jul 07 23:21:26.629766 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:21:26.629743 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8ef9793b-9b49-43a9-9b83-cc09c9002479-bundle\") pod \"8ef9793b-9b49-43a9-9b83-cc09c9002479\" (UID: \"8ef9793b-9b49-43a9-9b83-cc09c9002479\") " Jul 07 23:21:26.630194 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:21:26.630162 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8ef9793b-9b49-43a9-9b83-cc09c9002479-bundle" (OuterVolumeSpecName: "bundle") pod "8ef9793b-9b49-43a9-9b83-cc09c9002479" (UID: "8ef9793b-9b49-43a9-9b83-cc09c9002479"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 23:21:26.631669 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:21:26.631648 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8ef9793b-9b49-43a9-9b83-cc09c9002479-kube-api-access-tlf82" (OuterVolumeSpecName: "kube-api-access-tlf82") pod "8ef9793b-9b49-43a9-9b83-cc09c9002479" (UID: "8ef9793b-9b49-43a9-9b83-cc09c9002479"). InnerVolumeSpecName "kube-api-access-tlf82". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 23:21:26.635064 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:21:26.635045 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8ef9793b-9b49-43a9-9b83-cc09c9002479-util" (OuterVolumeSpecName: "util") pod "8ef9793b-9b49-43a9-9b83-cc09c9002479" (UID: "8ef9793b-9b49-43a9-9b83-cc09c9002479"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 23:21:26.731081 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:21:26.731046 2570 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8ef9793b-9b49-43a9-9b83-cc09c9002479-bundle\") on node \"ip-10-0-134-140.ec2.internal\" DevicePath \"\"" Jul 07 23:21:26.731081 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:21:26.731075 2570 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8ef9793b-9b49-43a9-9b83-cc09c9002479-util\") on node \"ip-10-0-134-140.ec2.internal\" DevicePath \"\"" Jul 07 23:21:26.731081 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:21:26.731084 2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tlf82\" (UniqueName: \"kubernetes.io/projected/8ef9793b-9b49-43a9-9b83-cc09c9002479-kube-api-access-tlf82\") on node \"ip-10-0-134-140.ec2.internal\" DevicePath \"\"" Jul 07 23:21:27.362323 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:21:27.362235 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759wgjps" Jul 07 23:21:27.362323 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:21:27.362261 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759wgjps" event={"ID":"8ef9793b-9b49-43a9-9b83-cc09c9002479","Type":"ContainerDied","Data":"69ddddd3b2089e36dc548315706e2bdb0483af39ec46d7c477dc3e999de70434"} Jul 07 23:21:27.362323 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:21:27.362314 2570 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="69ddddd3b2089e36dc548315706e2bdb0483af39ec46d7c477dc3e999de70434" Jul 07 23:21:27.364518 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:21:27.364496 2570 generic.go:356] "Generic (PLEG): container finished" podID="113878d9-e1e4-4af3-aed1-cdcd089443c4" containerID="93cefb7fb594ce4b3e3009815a39f9114e5e49767d1c4a997609cc6671b6b01a" exitCode=0 Jul 07 23:21:27.364652 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:21:27.364580 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed735hg4d" event={"ID":"113878d9-e1e4-4af3-aed1-cdcd089443c4","Type":"ContainerDied","Data":"93cefb7fb594ce4b3e3009815a39f9114e5e49767d1c4a997609cc6671b6b01a"} Jul 07 23:21:27.493241 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:21:27.493217 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1stzl9" Jul 07 23:21:27.640202 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:21:27.640112 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/807f17fe-22a0-4a3b-82af-b242ca8e2013-util\") pod \"807f17fe-22a0-4a3b-82af-b242ca8e2013\" (UID: \"807f17fe-22a0-4a3b-82af-b242ca8e2013\") " Jul 07 23:21:27.640202 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:21:27.640175 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cfgw6\" (UniqueName: \"kubernetes.io/projected/807f17fe-22a0-4a3b-82af-b242ca8e2013-kube-api-access-cfgw6\") pod \"807f17fe-22a0-4a3b-82af-b242ca8e2013\" (UID: \"807f17fe-22a0-4a3b-82af-b242ca8e2013\") " Jul 07 23:21:27.640442 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:21:27.640215 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/807f17fe-22a0-4a3b-82af-b242ca8e2013-bundle\") pod \"807f17fe-22a0-4a3b-82af-b242ca8e2013\" (UID: \"807f17fe-22a0-4a3b-82af-b242ca8e2013\") " Jul 07 23:21:27.640763 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:21:27.640728 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/807f17fe-22a0-4a3b-82af-b242ca8e2013-bundle" (OuterVolumeSpecName: "bundle") pod "807f17fe-22a0-4a3b-82af-b242ca8e2013" (UID: "807f17fe-22a0-4a3b-82af-b242ca8e2013"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 23:21:27.642279 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:21:27.642244 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/807f17fe-22a0-4a3b-82af-b242ca8e2013-kube-api-access-cfgw6" (OuterVolumeSpecName: "kube-api-access-cfgw6") pod "807f17fe-22a0-4a3b-82af-b242ca8e2013" (UID: "807f17fe-22a0-4a3b-82af-b242ca8e2013"). InnerVolumeSpecName "kube-api-access-cfgw6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 23:21:27.645012 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:21:27.644990 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/807f17fe-22a0-4a3b-82af-b242ca8e2013-util" (OuterVolumeSpecName: "util") pod "807f17fe-22a0-4a3b-82af-b242ca8e2013" (UID: "807f17fe-22a0-4a3b-82af-b242ca8e2013"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 23:21:27.741333 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:21:27.741282 2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cfgw6\" (UniqueName: \"kubernetes.io/projected/807f17fe-22a0-4a3b-82af-b242ca8e2013-kube-api-access-cfgw6\") on node \"ip-10-0-134-140.ec2.internal\" DevicePath \"\"" Jul 07 23:21:27.741333 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:21:27.741327 2570 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/807f17fe-22a0-4a3b-82af-b242ca8e2013-bundle\") on node \"ip-10-0-134-140.ec2.internal\" DevicePath \"\"" Jul 07 23:21:27.741333 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:21:27.741340 2570 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/807f17fe-22a0-4a3b-82af-b242ca8e2013-util\") on node \"ip-10-0-134-140.ec2.internal\" DevicePath \"\"" Jul 07 23:21:28.371017 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:21:28.370990 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1stzl9" Jul 07 23:21:28.371417 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:21:28.371015 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1stzl9" event={"ID":"807f17fe-22a0-4a3b-82af-b242ca8e2013","Type":"ContainerDied","Data":"bc2964c67b3e02cc072379eb62c81d875e5a706d00ed46aa51733f1821db0c75"} Jul 07 23:21:28.371417 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:21:28.371068 2570 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="bc2964c67b3e02cc072379eb62c81d875e5a706d00ed46aa51733f1821db0c75" Jul 07 23:21:28.503131 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:21:28.503110 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed735hg4d" Jul 07 23:21:28.649556 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:21:28.649454 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zrtwp\" (UniqueName: \"kubernetes.io/projected/113878d9-e1e4-4af3-aed1-cdcd089443c4-kube-api-access-zrtwp\") pod \"113878d9-e1e4-4af3-aed1-cdcd089443c4\" (UID: \"113878d9-e1e4-4af3-aed1-cdcd089443c4\") " Jul 07 23:21:28.649556 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:21:28.649514 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/113878d9-e1e4-4af3-aed1-cdcd089443c4-util\") pod \"113878d9-e1e4-4af3-aed1-cdcd089443c4\" (UID: \"113878d9-e1e4-4af3-aed1-cdcd089443c4\") " Jul 07 23:21:28.649775 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:21:28.649577 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/113878d9-e1e4-4af3-aed1-cdcd089443c4-bundle\") pod \"113878d9-e1e4-4af3-aed1-cdcd089443c4\" (UID: \"113878d9-e1e4-4af3-aed1-cdcd089443c4\") " Jul 07 23:21:28.650314 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:21:28.650286 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/113878d9-e1e4-4af3-aed1-cdcd089443c4-bundle" (OuterVolumeSpecName: "bundle") pod "113878d9-e1e4-4af3-aed1-cdcd089443c4" (UID: "113878d9-e1e4-4af3-aed1-cdcd089443c4"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 23:21:28.651653 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:21:28.651630 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/113878d9-e1e4-4af3-aed1-cdcd089443c4-kube-api-access-zrtwp" (OuterVolumeSpecName: "kube-api-access-zrtwp") pod "113878d9-e1e4-4af3-aed1-cdcd089443c4" (UID: "113878d9-e1e4-4af3-aed1-cdcd089443c4"). InnerVolumeSpecName "kube-api-access-zrtwp". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 23:21:28.657261 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:21:28.657234 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/113878d9-e1e4-4af3-aed1-cdcd089443c4-util" (OuterVolumeSpecName: "util") pod "113878d9-e1e4-4af3-aed1-cdcd089443c4" (UID: "113878d9-e1e4-4af3-aed1-cdcd089443c4"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 23:21:28.750688 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:21:28.750657 2570 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/113878d9-e1e4-4af3-aed1-cdcd089443c4-bundle\") on node \"ip-10-0-134-140.ec2.internal\" DevicePath \"\"" Jul 07 23:21:28.750688 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:21:28.750686 2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zrtwp\" (UniqueName: \"kubernetes.io/projected/113878d9-e1e4-4af3-aed1-cdcd089443c4-kube-api-access-zrtwp\") on node \"ip-10-0-134-140.ec2.internal\" DevicePath \"\"" Jul 07 23:21:28.750688 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:21:28.750695 2570 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/113878d9-e1e4-4af3-aed1-cdcd089443c4-util\") on node \"ip-10-0-134-140.ec2.internal\" DevicePath \"\"" Jul 07 23:21:29.377768 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:21:29.377689 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed735hg4d" Jul 07 23:21:29.378178 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:21:29.377688 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed735hg4d" event={"ID":"113878d9-e1e4-4af3-aed1-cdcd089443c4","Type":"ContainerDied","Data":"2d1e7be01c7b1a609e8e3a8d194b19fb4471f866af622cb15163a52d21d058b0"} Jul 07 23:21:29.378178 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:21:29.377819 2570 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2d1e7be01c7b1a609e8e3a8d194b19fb4471f866af622cb15163a52d21d058b0" Jul 07 23:21:35.797000 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:21:35.796961 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:21:36.786957 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:21:36.786915 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:21:36.883997 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:21:36.883960 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:21:36.982911 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:21:36.982880 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:21:37.440897 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:21:37.440864 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:21:37.503644 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:21:37.503608 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:21:39.785244 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:21:39.785210 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:21:42.273419 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:21:42.273383 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:21:43.650275 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:21:43.650236 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:21:45.002744 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:21:45.002704 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:21:47.202405 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:21:47.202372 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6b7b7c5b77-zl5n5"] Jul 07 23:21:47.312943 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:21:47.312910 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:21:47.497571 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:21:47.497458 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:21:47.671036 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:21:47.671000 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:21:47.776937 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:21:47.776907 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:21:50.039150 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:21:50.039114 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:21:52.026523 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:21:52.026457 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:21:52.127354 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:21:52.127317 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:21:54.113586 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:21:54.113543 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-8ptdh"] Jul 07 23:21:54.113951 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:21:54.113928 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="807f17fe-22a0-4a3b-82af-b242ca8e2013" containerName="util" Jul 07 23:21:54.113951 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:21:54.113941 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="807f17fe-22a0-4a3b-82af-b242ca8e2013" containerName="util" Jul 07 23:21:54.113951 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:21:54.113953 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ae1ba717-f216-48ff-9789-43126b0a9c1f" containerName="pull" Jul 07 23:21:54.114059 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:21:54.113958 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="ae1ba717-f216-48ff-9789-43126b0a9c1f" containerName="pull" Jul 07 23:21:54.114059 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:21:54.113964 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="113878d9-e1e4-4af3-aed1-cdcd089443c4" containerName="extract" Jul 07 23:21:54.114059 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:21:54.113970 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="113878d9-e1e4-4af3-aed1-cdcd089443c4" containerName="extract" Jul 07 23:21:54.114059 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:21:54.113981 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ae1ba717-f216-48ff-9789-43126b0a9c1f" containerName="extract" Jul 07 23:21:54.114059 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:21:54.113987 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="ae1ba717-f216-48ff-9789-43126b0a9c1f" containerName="extract" Jul 07 23:21:54.114059 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:21:54.113994 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="807f17fe-22a0-4a3b-82af-b242ca8e2013" containerName="pull" Jul 07 23:21:54.114059 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:21:54.113998 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="807f17fe-22a0-4a3b-82af-b242ca8e2013" containerName="pull" Jul 07 23:21:54.114059 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:21:54.114006 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ae1ba717-f216-48ff-9789-43126b0a9c1f" containerName="util" Jul 07 23:21:54.114059 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:21:54.114011 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="ae1ba717-f216-48ff-9789-43126b0a9c1f" containerName="util" Jul 07 23:21:54.114059 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:21:54.114018 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="113878d9-e1e4-4af3-aed1-cdcd089443c4" containerName="util" Jul 07 23:21:54.114059 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:21:54.114022 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="113878d9-e1e4-4af3-aed1-cdcd089443c4" containerName="util" Jul 07 23:21:54.114059 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:21:54.114028 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8ef9793b-9b49-43a9-9b83-cc09c9002479" containerName="extract" Jul 07 23:21:54.114059 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:21:54.114033 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="8ef9793b-9b49-43a9-9b83-cc09c9002479" containerName="extract" Jul 07 23:21:54.114059 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:21:54.114040 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="807f17fe-22a0-4a3b-82af-b242ca8e2013" containerName="extract" Jul 07 23:21:54.114059 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:21:54.114044 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="807f17fe-22a0-4a3b-82af-b242ca8e2013" containerName="extract" Jul 07 23:21:54.114059 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:21:54.114049 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="113878d9-e1e4-4af3-aed1-cdcd089443c4" containerName="pull" Jul 07 23:21:54.114059 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:21:54.114054 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="113878d9-e1e4-4af3-aed1-cdcd089443c4" containerName="pull" Jul 07 23:21:54.114059 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:21:54.114061 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8ef9793b-9b49-43a9-9b83-cc09c9002479" containerName="util" Jul 07 23:21:54.114059 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:21:54.114067 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="8ef9793b-9b49-43a9-9b83-cc09c9002479" containerName="util" Jul 07 23:21:54.114656 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:21:54.114079 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8ef9793b-9b49-43a9-9b83-cc09c9002479" containerName="pull" Jul 07 23:21:54.114656 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:21:54.114084 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="8ef9793b-9b49-43a9-9b83-cc09c9002479" containerName="pull" Jul 07 23:21:54.114656 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:21:54.114132 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="8ef9793b-9b49-43a9-9b83-cc09c9002479" containerName="extract" Jul 07 23:21:54.114656 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:21:54.114141 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="ae1ba717-f216-48ff-9789-43126b0a9c1f" containerName="extract" Jul 07 23:21:54.114656 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:21:54.114148 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="807f17fe-22a0-4a3b-82af-b242ca8e2013" containerName="extract" Jul 07 23:21:54.114656 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:21:54.114153 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="113878d9-e1e4-4af3-aed1-cdcd089443c4" containerName="extract" Jul 07 23:21:54.118708 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:21:54.118688 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-8ptdh" Jul 07 23:21:54.121795 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:21:54.121758 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:21:54.121908 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:21:54.121839 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\"" Jul 07 23:21:54.121908 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:21:54.121886 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\"" Jul 07 23:21:54.122006 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:21:54.121945 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 07 23:21:54.122680 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:21:54.122662 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 07 23:21:54.122680 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:21:54.122671 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-zdtnw\"" Jul 07 23:21:54.127155 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:21:54.127136 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-8ptdh"] Jul 07 23:21:54.172792 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:21:54.172759 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xlrlv\" (UniqueName: \"kubernetes.io/projected/cbc3faff-e9b2-48ad-a3c4-c0d77ec1af7a-kube-api-access-xlrlv\") pod \"kuadrant-console-plugin-66b5774985-8ptdh\" (UID: \"cbc3faff-e9b2-48ad-a3c4-c0d77ec1af7a\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-8ptdh" Jul 07 23:21:54.172938 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:21:54.172818 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/cbc3faff-e9b2-48ad-a3c4-c0d77ec1af7a-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-8ptdh\" (UID: \"cbc3faff-e9b2-48ad-a3c4-c0d77ec1af7a\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-8ptdh" Jul 07 23:21:54.172938 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:21:54.172860 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/cbc3faff-e9b2-48ad-a3c4-c0d77ec1af7a-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-8ptdh\" (UID: \"cbc3faff-e9b2-48ad-a3c4-c0d77ec1af7a\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-8ptdh" Jul 07 23:21:54.241269 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:21:54.241232 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:21:54.273925 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:21:54.273898 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/cbc3faff-e9b2-48ad-a3c4-c0d77ec1af7a-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-8ptdh\" (UID: \"cbc3faff-e9b2-48ad-a3c4-c0d77ec1af7a\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-8ptdh" Jul 07 23:21:54.274055 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:21:54.273932 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/cbc3faff-e9b2-48ad-a3c4-c0d77ec1af7a-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-8ptdh\" (UID: \"cbc3faff-e9b2-48ad-a3c4-c0d77ec1af7a\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-8ptdh" Jul 07 23:21:54.274055 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:21:54.273990 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xlrlv\" (UniqueName: \"kubernetes.io/projected/cbc3faff-e9b2-48ad-a3c4-c0d77ec1af7a-kube-api-access-xlrlv\") pod \"kuadrant-console-plugin-66b5774985-8ptdh\" (UID: \"cbc3faff-e9b2-48ad-a3c4-c0d77ec1af7a\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-8ptdh" Jul 07 23:21:54.274678 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:21:54.274659 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/cbc3faff-e9b2-48ad-a3c4-c0d77ec1af7a-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-8ptdh\" (UID: \"cbc3faff-e9b2-48ad-a3c4-c0d77ec1af7a\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-8ptdh" Jul 07 23:21:54.276387 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:21:54.276368 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/cbc3faff-e9b2-48ad-a3c4-c0d77ec1af7a-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-8ptdh\" (UID: \"cbc3faff-e9b2-48ad-a3c4-c0d77ec1af7a\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-8ptdh" Jul 07 23:21:54.282966 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:21:54.282941 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xlrlv\" (UniqueName: \"kubernetes.io/projected/cbc3faff-e9b2-48ad-a3c4-c0d77ec1af7a-kube-api-access-xlrlv\") pod \"kuadrant-console-plugin-66b5774985-8ptdh\" (UID: \"cbc3faff-e9b2-48ad-a3c4-c0d77ec1af7a\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-8ptdh" Jul 07 23:21:54.428660 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:21:54.428580 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-8ptdh" Jul 07 23:21:54.574174 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:21:54.574146 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-8ptdh"] Jul 07 23:21:54.574954 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:21:54.574930 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podcbc3faff_e9b2_48ad_a3c4_c0d77ec1af7a.slice/crio-25c891bc629da59fda65ac1a56cc7b455df5ddda14cd12b5b9120273f90f0cca WatchSource:0}: Error finding container 25c891bc629da59fda65ac1a56cc7b455df5ddda14cd12b5b9120273f90f0cca: Status 404 returned error can't find the container with id 25c891bc629da59fda65ac1a56cc7b455df5ddda14cd12b5b9120273f90f0cca Jul 07 23:21:55.511243 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:21:55.511154 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-8ptdh" event={"ID":"cbc3faff-e9b2-48ad-a3c4-c0d77ec1af7a","Type":"ContainerStarted","Data":"25c891bc629da59fda65ac1a56cc7b455df5ddda14cd12b5b9120273f90f0cca"} Jul 07 23:21:55.647492 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:21:55.647424 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:21:55.755715 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:21:55.755675 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:22:02.579080 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:22:02.579037 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:22:02.687993 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:22:02.687945 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:22:03.916678 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:22:03.916639 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:22:04.089025 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:22:04.088987 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:22:04.190535 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:22:04.190392 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:22:08.639910 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:22:08.639832 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:22:08.745502 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:22:08.745446 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:22:09.237886 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:22:09.237849 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:22:09.331839 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:22:09.331790 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:22:12.222537 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:22:12.222501 2570 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-6b7b7c5b77-zl5n5" podUID="a53651fd-186d-4c98-9823-bbea9451593f" containerName="console" containerID="cri-o://ccc1941070b5682188895f10ce6816a0000c7b21e38c6098740e0c62f2a997d7" gracePeriod=15 Jul 07 23:22:14.403487 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:22:14.403430 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:22:14.531968 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:22:14.531934 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:22:16.642339 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:22:16.642310 2570 generic.go:356] "Generic (PLEG): container finished" podID="a53651fd-186d-4c98-9823-bbea9451593f" containerID="ccc1941070b5682188895f10ce6816a0000c7b21e38c6098740e0c62f2a997d7" exitCode=0 Jul 07 23:22:16.642665 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:22:16.642375 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6b7b7c5b77-zl5n5" event={"ID":"a53651fd-186d-4c98-9823-bbea9451593f","Type":"ContainerDied","Data":"ccc1941070b5682188895f10ce6816a0000c7b21e38c6098740e0c62f2a997d7"} Jul 07 23:22:16.680133 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:22:16.680111 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6b7b7c5b77-zl5n5" Jul 07 23:22:16.782643 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:22:16.782611 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a53651fd-186d-4c98-9823-bbea9451593f-trusted-ca-bundle\") pod \"a53651fd-186d-4c98-9823-bbea9451593f\" (UID: \"a53651fd-186d-4c98-9823-bbea9451593f\") " Jul 07 23:22:16.782643 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:22:16.782643 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a53651fd-186d-4c98-9823-bbea9451593f-console-config\") pod \"a53651fd-186d-4c98-9823-bbea9451593f\" (UID: \"a53651fd-186d-4c98-9823-bbea9451593f\") " Jul 07 23:22:16.782862 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:22:16.782695 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a53651fd-186d-4c98-9823-bbea9451593f-service-ca\") pod \"a53651fd-186d-4c98-9823-bbea9451593f\" (UID: \"a53651fd-186d-4c98-9823-bbea9451593f\") " Jul 07 23:22:16.782862 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:22:16.782746 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a53651fd-186d-4c98-9823-bbea9451593f-console-serving-cert\") pod \"a53651fd-186d-4c98-9823-bbea9451593f\" (UID: \"a53651fd-186d-4c98-9823-bbea9451593f\") " Jul 07 23:22:16.782862 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:22:16.782781 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a53651fd-186d-4c98-9823-bbea9451593f-oauth-serving-cert\") pod \"a53651fd-186d-4c98-9823-bbea9451593f\" (UID: \"a53651fd-186d-4c98-9823-bbea9451593f\") " Jul 07 23:22:16.782862 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:22:16.782807 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nbnqj\" (UniqueName: \"kubernetes.io/projected/a53651fd-186d-4c98-9823-bbea9451593f-kube-api-access-nbnqj\") pod \"a53651fd-186d-4c98-9823-bbea9451593f\" (UID: \"a53651fd-186d-4c98-9823-bbea9451593f\") " Jul 07 23:22:16.782862 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:22:16.782845 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a53651fd-186d-4c98-9823-bbea9451593f-console-oauth-config\") pod \"a53651fd-186d-4c98-9823-bbea9451593f\" (UID: \"a53651fd-186d-4c98-9823-bbea9451593f\") " Jul 07 23:22:16.783174 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:22:16.783068 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a53651fd-186d-4c98-9823-bbea9451593f-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "a53651fd-186d-4c98-9823-bbea9451593f" (UID: "a53651fd-186d-4c98-9823-bbea9451593f"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 23:22:16.783243 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:22:16.783175 2570 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a53651fd-186d-4c98-9823-bbea9451593f-trusted-ca-bundle\") on node \"ip-10-0-134-140.ec2.internal\" DevicePath \"\"" Jul 07 23:22:16.783243 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:22:16.783193 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a53651fd-186d-4c98-9823-bbea9451593f-service-ca" (OuterVolumeSpecName: "service-ca") pod "a53651fd-186d-4c98-9823-bbea9451593f" (UID: "a53651fd-186d-4c98-9823-bbea9451593f"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 23:22:16.783243 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:22:16.783203 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a53651fd-186d-4c98-9823-bbea9451593f-console-config" (OuterVolumeSpecName: "console-config") pod "a53651fd-186d-4c98-9823-bbea9451593f" (UID: "a53651fd-186d-4c98-9823-bbea9451593f"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 23:22:16.783365 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:22:16.783258 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a53651fd-186d-4c98-9823-bbea9451593f-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "a53651fd-186d-4c98-9823-bbea9451593f" (UID: "a53651fd-186d-4c98-9823-bbea9451593f"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 23:22:16.784932 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:22:16.784906 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a53651fd-186d-4c98-9823-bbea9451593f-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "a53651fd-186d-4c98-9823-bbea9451593f" (UID: "a53651fd-186d-4c98-9823-bbea9451593f"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 23:22:16.785045 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:22:16.785022 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a53651fd-186d-4c98-9823-bbea9451593f-kube-api-access-nbnqj" (OuterVolumeSpecName: "kube-api-access-nbnqj") pod "a53651fd-186d-4c98-9823-bbea9451593f" (UID: "a53651fd-186d-4c98-9823-bbea9451593f"). InnerVolumeSpecName "kube-api-access-nbnqj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 23:22:16.785107 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:22:16.785068 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a53651fd-186d-4c98-9823-bbea9451593f-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "a53651fd-186d-4c98-9823-bbea9451593f" (UID: "a53651fd-186d-4c98-9823-bbea9451593f"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 23:22:16.884369 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:22:16.884287 2570 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a53651fd-186d-4c98-9823-bbea9451593f-console-config\") on node \"ip-10-0-134-140.ec2.internal\" DevicePath \"\"" Jul 07 23:22:16.884369 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:22:16.884314 2570 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a53651fd-186d-4c98-9823-bbea9451593f-service-ca\") on node \"ip-10-0-134-140.ec2.internal\" DevicePath \"\"" Jul 07 23:22:16.884369 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:22:16.884323 2570 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a53651fd-186d-4c98-9823-bbea9451593f-console-serving-cert\") on node \"ip-10-0-134-140.ec2.internal\" DevicePath \"\"" Jul 07 23:22:16.884369 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:22:16.884331 2570 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a53651fd-186d-4c98-9823-bbea9451593f-oauth-serving-cert\") on node \"ip-10-0-134-140.ec2.internal\" DevicePath \"\"" Jul 07 23:22:16.884369 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:22:16.884340 2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nbnqj\" (UniqueName: \"kubernetes.io/projected/a53651fd-186d-4c98-9823-bbea9451593f-kube-api-access-nbnqj\") on node \"ip-10-0-134-140.ec2.internal\" DevicePath \"\"" Jul 07 23:22:16.884369 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:22:16.884348 2570 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a53651fd-186d-4c98-9823-bbea9451593f-console-oauth-config\") on node \"ip-10-0-134-140.ec2.internal\" DevicePath \"\"" Jul 07 23:22:17.649338 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:22:17.649300 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-8ptdh" event={"ID":"cbc3faff-e9b2-48ad-a3c4-c0d77ec1af7a","Type":"ContainerStarted","Data":"da6a4578d09b51bf0bc6b72f4eea8251021e797bf28ee455d59dc26f9239dfa0"} Jul 07 23:22:17.650418 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:22:17.650398 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6b7b7c5b77-zl5n5" Jul 07 23:22:17.650418 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:22:17.650410 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6b7b7c5b77-zl5n5" event={"ID":"a53651fd-186d-4c98-9823-bbea9451593f","Type":"ContainerDied","Data":"1e34f5a96dc404915562c6784f36d94ffabfe3ee41b7300a1e9cfd7a04a57877"} Jul 07 23:22:17.650554 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:22:17.650460 2570 scope.go:117] "RemoveContainer" containerID="ccc1941070b5682188895f10ce6816a0000c7b21e38c6098740e0c62f2a997d7" Jul 07 23:22:17.671164 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:22:17.671119 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-8ptdh" podStartSLOduration=1.6313717300000001 podStartE2EDuration="23.671104293s" totalImagesPullingTime="22.039732563s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 23:21:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 23:22:17.669050357 +0000 UTC m=+555.066051873" watchObservedRunningTime="2026-07-07 23:22:17.671104293 +0000 UTC m=+555.068105806" Jul 07 23:22:17.687611 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:22:17.687585 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6b7b7c5b77-zl5n5"] Jul 07 23:22:17.689676 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:22:17.689649 2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-6b7b7c5b77-zl5n5"] Jul 07 23:22:18.210975 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:22:18.210939 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:22:18.306388 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:22:18.306344 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:22:19.097088 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:22:19.097057 2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a53651fd-186d-4c98-9823-bbea9451593f" path="/var/lib/kubelet/pods/a53651fd-186d-4c98-9823-bbea9451593f/volumes" Jul 07 23:22:25.612964 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:22:25.612925 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:22:25.713703 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:22:25.713667 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:22:30.426165 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:22:30.426128 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:22:30.528609 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:22:30.528572 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:22:39.661093 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:22:39.661048 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:22:40.185836 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:22:40.185788 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:22:41.402119 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:22:41.402079 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:22:41.507908 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:22:41.507810 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:22:41.548573 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:22:41.548538 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hs9gr"] Jul 07 23:22:41.548907 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:22:41.548895 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a53651fd-186d-4c98-9823-bbea9451593f" containerName="console" Jul 07 23:22:41.548951 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:22:41.548908 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="a53651fd-186d-4c98-9823-bbea9451593f" containerName="console" Jul 07 23:22:41.548986 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:22:41.548957 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="a53651fd-186d-4c98-9823-bbea9451593f" containerName="console" Jul 07 23:22:41.609172 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:22:41.609142 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:22:41.787148 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:22:41.787118 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hs9gr"] Jul 07 23:22:41.787307 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:22:41.787161 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hs9gr"] Jul 07 23:22:41.787307 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:22:41.787228 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-hs9gr" Jul 07 23:22:41.790318 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:22:41.790296 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" Jul 07 23:22:41.802946 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:22:41.802920 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/7798a566-a702-48eb-a51d-da784da4a2f2-config-file\") pod \"limitador-limitador-54d6d988d5-hs9gr\" (UID: \"7798a566-a702-48eb-a51d-da784da4a2f2\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-hs9gr" Jul 07 23:22:41.803043 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:22:41.802950 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-chnxt\" (UniqueName: \"kubernetes.io/projected/7798a566-a702-48eb-a51d-da784da4a2f2-kube-api-access-chnxt\") pod \"limitador-limitador-54d6d988d5-hs9gr\" (UID: \"7798a566-a702-48eb-a51d-da784da4a2f2\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-hs9gr" Jul 07 23:22:41.903942 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:22:41.903906 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/7798a566-a702-48eb-a51d-da784da4a2f2-config-file\") pod \"limitador-limitador-54d6d988d5-hs9gr\" (UID: \"7798a566-a702-48eb-a51d-da784da4a2f2\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-hs9gr" Jul 07 23:22:41.903942 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:22:41.903938 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-chnxt\" (UniqueName: \"kubernetes.io/projected/7798a566-a702-48eb-a51d-da784da4a2f2-kube-api-access-chnxt\") pod \"limitador-limitador-54d6d988d5-hs9gr\" (UID: \"7798a566-a702-48eb-a51d-da784da4a2f2\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-hs9gr" Jul 07 23:22:41.904604 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:22:41.904581 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/7798a566-a702-48eb-a51d-da784da4a2f2-config-file\") pod \"limitador-limitador-54d6d988d5-hs9gr\" (UID: \"7798a566-a702-48eb-a51d-da784da4a2f2\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-hs9gr" Jul 07 23:22:41.913515 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:22:41.913491 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-chnxt\" (UniqueName: \"kubernetes.io/projected/7798a566-a702-48eb-a51d-da784da4a2f2-kube-api-access-chnxt\") pod \"limitador-limitador-54d6d988d5-hs9gr\" (UID: \"7798a566-a702-48eb-a51d-da784da4a2f2\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-hs9gr" Jul 07 23:22:42.097518 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:22:42.097413 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-hs9gr" Jul 07 23:22:42.260752 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:22:42.260725 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hs9gr"] Jul 07 23:22:42.262373 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:22:42.262336 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7798a566_a702_48eb_a51d_da784da4a2f2.slice/crio-a533b44138a952df8b5237c9a25e1424b8ea4b72c683c262367b02e05e8daccb WatchSource:0}: Error finding container a533b44138a952df8b5237c9a25e1424b8ea4b72c683c262367b02e05e8daccb: Status 404 returned error can't find the container with id a533b44138a952df8b5237c9a25e1424b8ea4b72c683c262367b02e05e8daccb Jul 07 23:22:42.314508 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:22:42.314453 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:22:42.388662 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:22:42.388571 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:22:42.433727 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:22:42.433700 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-75d556bd77-z56lw"] Jul 07 23:22:42.436989 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:22:42.436970 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-z56lw" Jul 07 23:22:42.439177 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:22:42.439151 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-4xkc5\"" Jul 07 23:22:42.444104 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:22:42.444083 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-z56lw"] Jul 07 23:22:42.509647 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:22:42.509616 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jc65r\" (UniqueName: \"kubernetes.io/projected/95c5756d-cfec-4d50-be7d-dc25d46728ae-kube-api-access-jc65r\") pod \"authorino-75d556bd77-z56lw\" (UID: \"95c5756d-cfec-4d50-be7d-dc25d46728ae\") " pod="kuadrant-system/authorino-75d556bd77-z56lw" Jul 07 23:22:42.610425 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:22:42.610391 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jc65r\" (UniqueName: \"kubernetes.io/projected/95c5756d-cfec-4d50-be7d-dc25d46728ae-kube-api-access-jc65r\") pod \"authorino-75d556bd77-z56lw\" (UID: \"95c5756d-cfec-4d50-be7d-dc25d46728ae\") " pod="kuadrant-system/authorino-75d556bd77-z56lw" Jul 07 23:22:42.618940 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:22:42.618917 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jc65r\" (UniqueName: \"kubernetes.io/projected/95c5756d-cfec-4d50-be7d-dc25d46728ae-kube-api-access-jc65r\") pod \"authorino-75d556bd77-z56lw\" (UID: \"95c5756d-cfec-4d50-be7d-dc25d46728ae\") " pod="kuadrant-system/authorino-75d556bd77-z56lw" Jul 07 23:22:42.747311 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:22:42.747028 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-z56lw" Jul 07 23:22:42.786042 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:22:42.786007 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-hs9gr" event={"ID":"7798a566-a702-48eb-a51d-da784da4a2f2","Type":"ContainerStarted","Data":"a533b44138a952df8b5237c9a25e1424b8ea4b72c683c262367b02e05e8daccb"} Jul 07 23:22:42.942581 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:22:42.942270 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-z56lw"] Jul 07 23:22:42.945811 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:22:42.945779 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod95c5756d_cfec_4d50_be7d_dc25d46728ae.slice/crio-1c4bab78ea5e38c7e8e2c75b3d5830e43f8560d8e13e138740a1f7ee8af8cfa4 WatchSource:0}: Error finding container 1c4bab78ea5e38c7e8e2c75b3d5830e43f8560d8e13e138740a1f7ee8af8cfa4: Status 404 returned error can't find the container with id 1c4bab78ea5e38c7e8e2c75b3d5830e43f8560d8e13e138740a1f7ee8af8cfa4 Jul 07 23:22:43.796878 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:22:43.796811 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-z56lw" event={"ID":"95c5756d-cfec-4d50-be7d-dc25d46728ae","Type":"ContainerStarted","Data":"1c4bab78ea5e38c7e8e2c75b3d5830e43f8560d8e13e138740a1f7ee8af8cfa4"} Jul 07 23:22:46.817132 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:22:46.817092 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-hs9gr" event={"ID":"7798a566-a702-48eb-a51d-da784da4a2f2","Type":"ContainerStarted","Data":"c741034a9de9c37d5e0357c293d8969da2b7221efc8707730b5d88607405840c"} Jul 07 23:22:46.817643 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:22:46.817164 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-hs9gr" Jul 07 23:22:46.818640 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:22:46.818614 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-z56lw" event={"ID":"95c5756d-cfec-4d50-be7d-dc25d46728ae","Type":"ContainerStarted","Data":"209fc160eccc5e57f38a251a0d090e04932b975eeaa034de967143f0759ce6bd"} Jul 07 23:22:46.839185 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:22:46.839134 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-54d6d988d5-hs9gr" podStartSLOduration=1.3661911660000001 podStartE2EDuration="5.839117586s" totalImagesPullingTime="4.47292642s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 23:22:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 23:22:46.837664756 +0000 UTC m=+584.234666291" watchObservedRunningTime="2026-07-07 23:22:46.839117586 +0000 UTC m=+584.236119098" Jul 07 23:22:46.862301 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:22:46.862255 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-75d556bd77-z56lw" podStartSLOduration=1.115827158 podStartE2EDuration="4.862238622s" totalImagesPullingTime="3.746411464s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 23:22:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 23:22:46.860066507 +0000 UTC m=+584.257068019" watchObservedRunningTime="2026-07-07 23:22:46.862238622 +0000 UTC m=+584.259240148" Jul 07 23:22:47.690158 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:22:47.690116 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:22:47.765338 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:22:47.765304 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:22:57.824906 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:22:57.824857 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-hs9gr" Jul 07 23:23:02.576284 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:23:02.576242 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:23:02.676616 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:23:02.676583 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:23:03.041915 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:23:03.041881 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-8hvxr_faae2bd8-cbbd-4cfb-bce6-7954977b7ebc/console-operator/2.log" Jul 07 23:23:03.046252 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:23:03.046228 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-8hvxr_faae2bd8-cbbd-4cfb-bce6-7954977b7ebc/console-operator/2.log" Jul 07 23:23:03.067350 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:23:03.067189 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:23:14.901838 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:23:14.901788 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:23:15.022503 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:23:15.022448 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:23:15.321862 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:23:15.321832 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-76877994-lqqtt"] Jul 07 23:23:15.332199 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:23:15.332173 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-76877994-lqqtt" Jul 07 23:23:15.333266 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:23:15.333242 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-76877994-lqqtt"] Jul 07 23:23:15.334912 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:23:15.334889 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\"" Jul 07 23:23:15.356056 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:23:15.356021 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-76877994-lqqtt"] Jul 07 23:23:15.356285 ip-10-0-134-140 kubenswrapper[2570]: E0707 23:23:15.356266 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-x54fj tls-cert], unattached volumes=[], failed to process volumes=[kube-api-access-x54fj tls-cert]: context canceled" pod="kuadrant-system/authorino-76877994-lqqtt" podUID="5263f2fe-5375-4524-be18-02450a96b6ca" Jul 07 23:23:15.498958 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:23:15.498921 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x54fj\" (UniqueName: \"kubernetes.io/projected/5263f2fe-5375-4524-be18-02450a96b6ca-kube-api-access-x54fj\") pod \"authorino-76877994-lqqtt\" (UID: \"5263f2fe-5375-4524-be18-02450a96b6ca\") " pod="kuadrant-system/authorino-76877994-lqqtt" Jul 07 23:23:15.499119 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:23:15.498965 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/5263f2fe-5375-4524-be18-02450a96b6ca-tls-cert\") pod \"authorino-76877994-lqqtt\" (UID: \"5263f2fe-5375-4524-be18-02450a96b6ca\") " pod="kuadrant-system/authorino-76877994-lqqtt" Jul 07 23:23:15.599520 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:23:15.599417 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x54fj\" (UniqueName: \"kubernetes.io/projected/5263f2fe-5375-4524-be18-02450a96b6ca-kube-api-access-x54fj\") pod \"authorino-76877994-lqqtt\" (UID: \"5263f2fe-5375-4524-be18-02450a96b6ca\") " pod="kuadrant-system/authorino-76877994-lqqtt" Jul 07 23:23:15.599520 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:23:15.599460 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/5263f2fe-5375-4524-be18-02450a96b6ca-tls-cert\") pod \"authorino-76877994-lqqtt\" (UID: \"5263f2fe-5375-4524-be18-02450a96b6ca\") " pod="kuadrant-system/authorino-76877994-lqqtt" Jul 07 23:23:15.601965 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:23:15.601938 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/5263f2fe-5375-4524-be18-02450a96b6ca-tls-cert\") pod \"authorino-76877994-lqqtt\" (UID: \"5263f2fe-5375-4524-be18-02450a96b6ca\") " pod="kuadrant-system/authorino-76877994-lqqtt" Jul 07 23:23:15.607771 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:23:15.607742 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x54fj\" (UniqueName: \"kubernetes.io/projected/5263f2fe-5375-4524-be18-02450a96b6ca-kube-api-access-x54fj\") pod \"authorino-76877994-lqqtt\" (UID: \"5263f2fe-5375-4524-be18-02450a96b6ca\") " pod="kuadrant-system/authorino-76877994-lqqtt" Jul 07 23:23:15.973825 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:23:15.973745 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-76877994-lqqtt" Jul 07 23:23:16.038269 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:23:16.038242 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-76877994-lqqtt" Jul 07 23:23:16.203517 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:23:16.203483 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/5263f2fe-5375-4524-be18-02450a96b6ca-tls-cert\") pod \"5263f2fe-5375-4524-be18-02450a96b6ca\" (UID: \"5263f2fe-5375-4524-be18-02450a96b6ca\") " Jul 07 23:23:16.203517 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:23:16.203521 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x54fj\" (UniqueName: \"kubernetes.io/projected/5263f2fe-5375-4524-be18-02450a96b6ca-kube-api-access-x54fj\") pod \"5263f2fe-5375-4524-be18-02450a96b6ca\" (UID: \"5263f2fe-5375-4524-be18-02450a96b6ca\") " Jul 07 23:23:16.205520 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:23:16.205485 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5263f2fe-5375-4524-be18-02450a96b6ca-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "5263f2fe-5375-4524-be18-02450a96b6ca" (UID: "5263f2fe-5375-4524-be18-02450a96b6ca"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 23:23:16.205630 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:23:16.205534 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5263f2fe-5375-4524-be18-02450a96b6ca-kube-api-access-x54fj" (OuterVolumeSpecName: "kube-api-access-x54fj") pod "5263f2fe-5375-4524-be18-02450a96b6ca" (UID: "5263f2fe-5375-4524-be18-02450a96b6ca"). InnerVolumeSpecName "kube-api-access-x54fj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 23:23:16.305119 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:23:16.305089 2570 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/5263f2fe-5375-4524-be18-02450a96b6ca-tls-cert\") on node \"ip-10-0-134-140.ec2.internal\" DevicePath \"\"" Jul 07 23:23:16.305119 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:23:16.305117 2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x54fj\" (UniqueName: \"kubernetes.io/projected/5263f2fe-5375-4524-be18-02450a96b6ca-kube-api-access-x54fj\") on node \"ip-10-0-134-140.ec2.internal\" DevicePath \"\"" Jul 07 23:23:16.979182 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:23:16.979144 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-76877994-lqqtt" Jul 07 23:23:17.011093 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:23:17.011059 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-76877994-lqqtt"] Jul 07 23:23:17.015385 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:23:17.015362 2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-76877994-lqqtt"] Jul 07 23:23:17.096280 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:23:17.096248 2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5263f2fe-5375-4524-be18-02450a96b6ca" path="/var/lib/kubelet/pods/5263f2fe-5375-4524-be18-02450a96b6ca/volumes" Jul 07 23:23:17.573240 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:23:17.573207 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-z56lw"] Jul 07 23:23:17.573457 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:23:17.573417 2570 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-75d556bd77-z56lw" podUID="95c5756d-cfec-4d50-be7d-dc25d46728ae" containerName="authorino" containerID="cri-o://209fc160eccc5e57f38a251a0d090e04932b975eeaa034de967143f0759ce6bd" gracePeriod=30 Jul 07 23:23:17.848390 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:23:17.848357 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:23:17.855573 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:23:17.855547 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-z56lw" Jul 07 23:23:17.985103 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:23:17.985069 2570 generic.go:356] "Generic (PLEG): container finished" podID="95c5756d-cfec-4d50-be7d-dc25d46728ae" containerID="209fc160eccc5e57f38a251a0d090e04932b975eeaa034de967143f0759ce6bd" exitCode=0 Jul 07 23:23:17.985103 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:23:17.985104 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-z56lw" event={"ID":"95c5756d-cfec-4d50-be7d-dc25d46728ae","Type":"ContainerDied","Data":"209fc160eccc5e57f38a251a0d090e04932b975eeaa034de967143f0759ce6bd"} Jul 07 23:23:17.985648 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:23:17.985110 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-z56lw" Jul 07 23:23:17.985648 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:23:17.985131 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-z56lw" event={"ID":"95c5756d-cfec-4d50-be7d-dc25d46728ae","Type":"ContainerDied","Data":"1c4bab78ea5e38c7e8e2c75b3d5830e43f8560d8e13e138740a1f7ee8af8cfa4"} Jul 07 23:23:17.985648 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:23:17.985148 2570 scope.go:117] "RemoveContainer" containerID="209fc160eccc5e57f38a251a0d090e04932b975eeaa034de967143f0759ce6bd" Jul 07 23:23:17.993953 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:23:17.993925 2570 scope.go:117] "RemoveContainer" containerID="209fc160eccc5e57f38a251a0d090e04932b975eeaa034de967143f0759ce6bd" Jul 07 23:23:17.994187 ip-10-0-134-140 kubenswrapper[2570]: E0707 23:23:17.994169 2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"209fc160eccc5e57f38a251a0d090e04932b975eeaa034de967143f0759ce6bd\": container with ID starting with 209fc160eccc5e57f38a251a0d090e04932b975eeaa034de967143f0759ce6bd not found: ID does not exist" containerID="209fc160eccc5e57f38a251a0d090e04932b975eeaa034de967143f0759ce6bd" Jul 07 23:23:17.994252 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:23:17.994200 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"209fc160eccc5e57f38a251a0d090e04932b975eeaa034de967143f0759ce6bd"} err="failed to get container status \"209fc160eccc5e57f38a251a0d090e04932b975eeaa034de967143f0759ce6bd\": rpc error: code = NotFound desc = could not find container \"209fc160eccc5e57f38a251a0d090e04932b975eeaa034de967143f0759ce6bd\": container with ID starting with 209fc160eccc5e57f38a251a0d090e04932b975eeaa034de967143f0759ce6bd not found: ID does not exist" Jul 07 23:23:18.019564 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:23:18.019545 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jc65r\" (UniqueName: \"kubernetes.io/projected/95c5756d-cfec-4d50-be7d-dc25d46728ae-kube-api-access-jc65r\") pod \"95c5756d-cfec-4d50-be7d-dc25d46728ae\" (UID: \"95c5756d-cfec-4d50-be7d-dc25d46728ae\") " Jul 07 23:23:18.021395 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:23:18.021373 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/95c5756d-cfec-4d50-be7d-dc25d46728ae-kube-api-access-jc65r" (OuterVolumeSpecName: "kube-api-access-jc65r") pod "95c5756d-cfec-4d50-be7d-dc25d46728ae" (UID: "95c5756d-cfec-4d50-be7d-dc25d46728ae"). InnerVolumeSpecName "kube-api-access-jc65r". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 23:23:18.120673 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:23:18.120596 2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jc65r\" (UniqueName: \"kubernetes.io/projected/95c5756d-cfec-4d50-be7d-dc25d46728ae-kube-api-access-jc65r\") on node \"ip-10-0-134-140.ec2.internal\" DevicePath \"\"" Jul 07 23:23:18.311318 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:23:18.311285 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-z56lw"] Jul 07 23:23:18.315097 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:23:18.315074 2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-75d556bd77-z56lw"] Jul 07 23:23:19.097140 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:23:19.097106 2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="95c5756d-cfec-4d50-be7d-dc25d46728ae" path="/var/lib/kubelet/pods/95c5756d-cfec-4d50-be7d-dc25d46728ae/volumes" Jul 07 23:23:22.044381 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:23:22.044338 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:23:22.136682 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:23:22.136631 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:23:22.216365 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:23:22.216330 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:23:23.038412 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:23:23.038371 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:23:23.102175 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:23:23.102144 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:23:24.915114 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:23:24.915077 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:23:25.141551 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:23:25.141446 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:23:56.448283 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:23:56.448248 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:23:56.926426 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:23:56.926393 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:23:57.030030 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:23:57.029992 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:23:57.351638 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:23:57.351600 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:23:57.453310 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:23:57.453276 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:23:57.513405 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:23:57.513372 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hs9gr"] Jul 07 23:24:07.867266 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:24:07.867227 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:24:09.192062 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:24:09.192024 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:24:16.382478 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:24:16.382423 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:24:16.741322 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:24:16.741242 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hs9gr"] Jul 07 23:24:36.470569 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:24:36.470534 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:24:37.327569 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:24:37.327532 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hs9gr"] Jul 07 23:24:44.245854 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:24:44.245815 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:24:45.323814 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:24:45.323782 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hs9gr"] Jul 07 23:24:49.291255 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:24:49.291221 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:24:50.933725 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:24:50.933694 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hs9gr"] Jul 07 23:25:05.369982 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:25:05.369884 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:25:07.643233 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:25:07.643197 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hs9gr"] Jul 07 23:25:09.177732 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:25:09.177698 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-n9hpc"] Jul 07 23:25:09.178164 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:25:09.178067 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="95c5756d-cfec-4d50-be7d-dc25d46728ae" containerName="authorino" Jul 07 23:25:09.178164 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:25:09.178079 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="95c5756d-cfec-4d50-be7d-dc25d46728ae" containerName="authorino" Jul 07 23:25:09.178164 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:25:09.178151 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="95c5756d-cfec-4d50-be7d-dc25d46728ae" containerName="authorino" Jul 07 23:25:09.181116 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:25:09.181099 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-n9hpc" Jul 07 23:25:09.185592 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:25:09.185574 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 07 23:25:09.186582 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:25:09.186555 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 07 23:25:09.186703 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:25:09.186615 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\"" Jul 07 23:25:09.186703 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:25:09.186562 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-99wd9\"" Jul 07 23:25:09.199375 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:25:09.199348 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-n9hpc"] Jul 07 23:25:09.269659 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:25:09.269625 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7c3e7632-a72a-45bb-bb5c-0de9c48911d2-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-n9hpc\" (UID: \"7c3e7632-a72a-45bb-bb5c-0de9c48911d2\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-n9hpc" Jul 07 23:25:09.269845 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:25:09.269671 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gtb2k\" (UniqueName: \"kubernetes.io/projected/7c3e7632-a72a-45bb-bb5c-0de9c48911d2-kube-api-access-gtb2k\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-n9hpc\" (UID: \"7c3e7632-a72a-45bb-bb5c-0de9c48911d2\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-n9hpc" Jul 07 23:25:09.269845 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:25:09.269748 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7c3e7632-a72a-45bb-bb5c-0de9c48911d2-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-n9hpc\" (UID: \"7c3e7632-a72a-45bb-bb5c-0de9c48911d2\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-n9hpc" Jul 07 23:25:09.269845 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:25:09.269812 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7c3e7632-a72a-45bb-bb5c-0de9c48911d2-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-n9hpc\" (UID: \"7c3e7632-a72a-45bb-bb5c-0de9c48911d2\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-n9hpc" Jul 07 23:25:09.269994 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:25:09.269863 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7c3e7632-a72a-45bb-bb5c-0de9c48911d2-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-n9hpc\" (UID: \"7c3e7632-a72a-45bb-bb5c-0de9c48911d2\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-n9hpc" Jul 07 23:25:09.269994 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:25:09.269916 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7c3e7632-a72a-45bb-bb5c-0de9c48911d2-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-n9hpc\" (UID: \"7c3e7632-a72a-45bb-bb5c-0de9c48911d2\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-n9hpc" Jul 07 23:25:09.329996 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:25:09.329959 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:25:09.370815 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:25:09.370780 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7c3e7632-a72a-45bb-bb5c-0de9c48911d2-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-n9hpc\" (UID: \"7c3e7632-a72a-45bb-bb5c-0de9c48911d2\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-n9hpc" Jul 07 23:25:09.370988 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:25:09.370818 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gtb2k\" (UniqueName: \"kubernetes.io/projected/7c3e7632-a72a-45bb-bb5c-0de9c48911d2-kube-api-access-gtb2k\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-n9hpc\" (UID: \"7c3e7632-a72a-45bb-bb5c-0de9c48911d2\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-n9hpc" Jul 07 23:25:09.370988 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:25:09.370856 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7c3e7632-a72a-45bb-bb5c-0de9c48911d2-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-n9hpc\" (UID: \"7c3e7632-a72a-45bb-bb5c-0de9c48911d2\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-n9hpc" Jul 07 23:25:09.370988 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:25:09.370878 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7c3e7632-a72a-45bb-bb5c-0de9c48911d2-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-n9hpc\" (UID: \"7c3e7632-a72a-45bb-bb5c-0de9c48911d2\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-n9hpc" Jul 07 23:25:09.370988 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:25:09.370906 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7c3e7632-a72a-45bb-bb5c-0de9c48911d2-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-n9hpc\" (UID: \"7c3e7632-a72a-45bb-bb5c-0de9c48911d2\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-n9hpc" Jul 07 23:25:09.370988 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:25:09.370946 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7c3e7632-a72a-45bb-bb5c-0de9c48911d2-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-n9hpc\" (UID: \"7c3e7632-a72a-45bb-bb5c-0de9c48911d2\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-n9hpc" Jul 07 23:25:09.371317 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:25:09.371299 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7c3e7632-a72a-45bb-bb5c-0de9c48911d2-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-n9hpc\" (UID: \"7c3e7632-a72a-45bb-bb5c-0de9c48911d2\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-n9hpc" Jul 07 23:25:09.371386 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:25:09.371315 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7c3e7632-a72a-45bb-bb5c-0de9c48911d2-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-n9hpc\" (UID: \"7c3e7632-a72a-45bb-bb5c-0de9c48911d2\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-n9hpc" Jul 07 23:25:09.371386 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:25:09.371364 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7c3e7632-a72a-45bb-bb5c-0de9c48911d2-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-n9hpc\" (UID: \"7c3e7632-a72a-45bb-bb5c-0de9c48911d2\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-n9hpc" Jul 07 23:25:09.373055 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:25:09.373035 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7c3e7632-a72a-45bb-bb5c-0de9c48911d2-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-n9hpc\" (UID: \"7c3e7632-a72a-45bb-bb5c-0de9c48911d2\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-n9hpc" Jul 07 23:25:09.373319 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:25:09.373303 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7c3e7632-a72a-45bb-bb5c-0de9c48911d2-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-n9hpc\" (UID: \"7c3e7632-a72a-45bb-bb5c-0de9c48911d2\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-n9hpc" Jul 07 23:25:09.378841 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:25:09.378819 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gtb2k\" (UniqueName: \"kubernetes.io/projected/7c3e7632-a72a-45bb-bb5c-0de9c48911d2-kube-api-access-gtb2k\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-n9hpc\" (UID: \"7c3e7632-a72a-45bb-bb5c-0de9c48911d2\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-n9hpc" Jul 07 23:25:09.490118 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:25:09.490031 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-n9hpc" Jul 07 23:25:09.625171 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:25:09.625145 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-n9hpc"] Jul 07 23:25:09.627225 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:25:09.627200 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7c3e7632_a72a_45bb_bb5c_0de9c48911d2.slice/crio-015d31b6e59b6147960493abdbd968a2c4be46aa6ddb07ba72fba886f0416120 WatchSource:0}: Error finding container 015d31b6e59b6147960493abdbd968a2c4be46aa6ddb07ba72fba886f0416120: Status 404 returned error can't find the container with id 015d31b6e59b6147960493abdbd968a2c4be46aa6ddb07ba72fba886f0416120 Jul 07 23:25:09.628447 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:25:09.628431 2570 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 07 23:25:10.559540 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:25:10.559501 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-n9hpc" event={"ID":"7c3e7632-a72a-45bb-bb5c-0de9c48911d2","Type":"ContainerStarted","Data":"015d31b6e59b6147960493abdbd968a2c4be46aa6ddb07ba72fba886f0416120"} Jul 07 23:25:11.565636 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:25:11.565597 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-n9hpc" event={"ID":"7c3e7632-a72a-45bb-bb5c-0de9c48911d2","Type":"ContainerStarted","Data":"cdc09a7b60009c46e2911d87d7085cd05e1d63c47eb75d2f6afb1e96d12c0d9b"} Jul 07 23:25:11.585311 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:25:11.585256 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-n9hpc" podStartSLOduration=1.52908697 podStartE2EDuration="2.585241576s" totalImagesPullingTime="1.056154606s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 23:25:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 23:25:11.582722282 +0000 UTC m=+728.979723794" watchObservedRunningTime="2026-07-07 23:25:11.585241576 +0000 UTC m=+728.982243109" Jul 07 23:25:16.158955 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:25:16.158916 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hs9gr"] Jul 07 23:25:19.490965 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:25:19.490932 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-n9hpc" Jul 07 23:25:19.491499 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:25:19.491217 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-n9hpc" Jul 07 23:25:19.503401 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:25:19.503383 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-n9hpc" Jul 07 23:25:19.615564 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:25:19.615537 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-n9hpc" Jul 07 23:25:24.015722 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:25:24.015684 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:25:24.116767 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:25:24.116724 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:25:38.066967 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:25:38.066929 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:26:13.494228 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:26:13.494178 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-865bd97974-r7bjs"] Jul 07 23:26:13.497886 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:26:13.497861 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-865bd97974-r7bjs" Jul 07 23:26:13.500671 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:26:13.500650 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-4xkc5\"" Jul 07 23:26:13.500768 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:26:13.500714 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\"" Jul 07 23:26:13.507528 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:26:13.507504 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-865bd97974-r7bjs"] Jul 07 23:26:13.525763 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:26:13.525741 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/46d1cd8e-d71e-4aef-bd0e-9914a638a0a1-tls-cert\") pod \"authorino-865bd97974-r7bjs\" (UID: \"46d1cd8e-d71e-4aef-bd0e-9914a638a0a1\") " pod="kuadrant-system/authorino-865bd97974-r7bjs" Jul 07 23:26:13.525869 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:26:13.525777 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4dk8h\" (UniqueName: \"kubernetes.io/projected/46d1cd8e-d71e-4aef-bd0e-9914a638a0a1-kube-api-access-4dk8h\") pod \"authorino-865bd97974-r7bjs\" (UID: \"46d1cd8e-d71e-4aef-bd0e-9914a638a0a1\") " pod="kuadrant-system/authorino-865bd97974-r7bjs" Jul 07 23:26:13.626372 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:26:13.626343 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/46d1cd8e-d71e-4aef-bd0e-9914a638a0a1-tls-cert\") pod \"authorino-865bd97974-r7bjs\" (UID: \"46d1cd8e-d71e-4aef-bd0e-9914a638a0a1\") " pod="kuadrant-system/authorino-865bd97974-r7bjs" Jul 07 23:26:13.626531 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:26:13.626381 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4dk8h\" (UniqueName: \"kubernetes.io/projected/46d1cd8e-d71e-4aef-bd0e-9914a638a0a1-kube-api-access-4dk8h\") pod \"authorino-865bd97974-r7bjs\" (UID: \"46d1cd8e-d71e-4aef-bd0e-9914a638a0a1\") " pod="kuadrant-system/authorino-865bd97974-r7bjs" Jul 07 23:26:13.628825 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:26:13.628796 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/46d1cd8e-d71e-4aef-bd0e-9914a638a0a1-tls-cert\") pod \"authorino-865bd97974-r7bjs\" (UID: \"46d1cd8e-d71e-4aef-bd0e-9914a638a0a1\") " pod="kuadrant-system/authorino-865bd97974-r7bjs" Jul 07 23:26:13.635974 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:26:13.635948 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4dk8h\" (UniqueName: \"kubernetes.io/projected/46d1cd8e-d71e-4aef-bd0e-9914a638a0a1-kube-api-access-4dk8h\") pod \"authorino-865bd97974-r7bjs\" (UID: \"46d1cd8e-d71e-4aef-bd0e-9914a638a0a1\") " pod="kuadrant-system/authorino-865bd97974-r7bjs" Jul 07 23:26:13.783715 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:26:13.783681 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:26:13.808201 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:26:13.808173 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-865bd97974-r7bjs" Jul 07 23:26:13.885300 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:26:13.885269 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:26:13.943696 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:26:13.943669 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-865bd97974-r7bjs"] Jul 07 23:26:13.945672 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:26:13.945645 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod46d1cd8e_d71e_4aef_bd0e_9914a638a0a1.slice/crio-e7f752b928ea2c91699b256fb1bc64013eda5505763f013f2c3120d6475663e2 WatchSource:0}: Error finding container e7f752b928ea2c91699b256fb1bc64013eda5505763f013f2c3120d6475663e2: Status 404 returned error can't find the container with id e7f752b928ea2c91699b256fb1bc64013eda5505763f013f2c3120d6475663e2 Jul 07 23:26:14.891331 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:26:14.891284 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-865bd97974-r7bjs" event={"ID":"46d1cd8e-d71e-4aef-bd0e-9914a638a0a1","Type":"ContainerStarted","Data":"fa525d7d6f110163848dd9737f10ee8898a5653a55dd452759ed4219078cafa4"} Jul 07 23:26:14.891732 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:26:14.891341 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-865bd97974-r7bjs" event={"ID":"46d1cd8e-d71e-4aef-bd0e-9914a638a0a1","Type":"ContainerStarted","Data":"e7f752b928ea2c91699b256fb1bc64013eda5505763f013f2c3120d6475663e2"} Jul 07 23:26:14.903706 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:26:14.903670 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:26:14.913629 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:26:14.913121 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-865bd97974-r7bjs" podStartSLOduration=1.6300251289999999 podStartE2EDuration="1.913104885s" totalImagesPullingTime="283.079756ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 23:26:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 23:26:14.912285753 +0000 UTC m=+792.309287269" watchObservedRunningTime="2026-07-07 23:26:14.913104885 +0000 UTC m=+792.310106398" Jul 07 23:26:15.001418 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:26:15.000879 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:26:41.751515 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:26:41.751461 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hs9gr"] Jul 07 23:26:52.662886 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:26:52.662844 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hs9gr"] Jul 07 23:27:03.467790 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:27:03.467749 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hs9gr"] Jul 07 23:27:12.462090 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:27:12.462040 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hs9gr"] Jul 07 23:27:23.431955 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:27:23.431921 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hs9gr"] Jul 07 23:27:25.251673 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:27:25.251633 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:27:26.536613 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:27:26.536578 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:27:38.980712 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:27:38.980680 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:27:49.632982 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:27:49.632939 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hs9gr"] Jul 07 23:28:01.433388 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:28:01.433306 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hs9gr"] Jul 07 23:28:03.085038 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:28:03.085007 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-8hvxr_faae2bd8-cbbd-4cfb-bce6-7954977b7ebc/console-operator/2.log" Jul 07 23:28:03.086712 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:28:03.086692 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-8hvxr_faae2bd8-cbbd-4cfb-bce6-7954977b7ebc/console-operator/2.log" Jul 07 23:28:07.060302 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:28:07.060249 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:28:07.165298 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:28:07.165259 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:28:07.284240 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:28:07.284199 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:28:13.296082 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:28:13.296042 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:28:13.363669 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:28:13.363639 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:28:13.432875 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:28:13.432843 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:28:13.531445 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:28:13.531410 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:28:13.633880 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:28:13.633788 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:28:19.827948 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:28:19.827908 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:28:19.942363 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:28:19.942327 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:28:36.927505 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:28:36.927456 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hs9gr"] Jul 07 23:28:46.627159 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:28:46.627119 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hs9gr"] Jul 07 23:28:56.931175 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:28:56.931137 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hs9gr"] Jul 07 23:29:06.725045 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:29:06.725010 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hs9gr"] Jul 07 23:29:18.041456 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:29:18.041414 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hs9gr"] Jul 07 23:29:23.158945 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:29:23.158900 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:29:27.824443 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:29:27.824334 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hs9gr"] Jul 07 23:29:38.840126 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:29:38.840086 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hs9gr"] Jul 07 23:29:49.325212 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:29:49.325179 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hs9gr"] Jul 07 23:30:11.337704 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:30:11.337667 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hs9gr"] Jul 07 23:30:17.501538 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:30:17.501498 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:30:23.029753 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:30:23.029718 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hs9gr"] Jul 07 23:30:45.321669 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:30:45.321639 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hs9gr"] Jul 07 23:30:55.626992 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:30:55.626907 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hs9gr"] Jul 07 23:31:12.222006 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:31:12.221971 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hs9gr"] Jul 07 23:31:22.128896 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:31:22.128858 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hs9gr"] Jul 07 23:31:31.735362 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:31:31.735310 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hs9gr"] Jul 07 23:31:41.530302 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:31:41.530269 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hs9gr"] Jul 07 23:31:51.426561 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:31:51.426522 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hs9gr"] Jul 07 23:31:57.527837 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:31:57.527798 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:32:40.586936 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:32:40.586899 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:33:03.121246 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:33:03.121217 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-8hvxr_faae2bd8-cbbd-4cfb-bce6-7954977b7ebc/console-operator/2.log" Jul 07 23:33:03.122306 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:33:03.122286 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-8hvxr_faae2bd8-cbbd-4cfb-bce6-7954977b7ebc/console-operator/2.log" Jul 07 23:33:40.124717 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:33:40.124681 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hs9gr"] Jul 07 23:33:49.462315 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:33:49.462274 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hs9gr"] Jul 07 23:34:02.240155 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:34:02.240067 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hs9gr"] Jul 07 23:34:12.926811 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:34:12.926777 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hs9gr"] Jul 07 23:34:24.434022 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:34:24.433988 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hs9gr"] Jul 07 23:34:34.038911 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:34:34.038869 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hs9gr"] Jul 07 23:34:40.626403 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:34:40.626363 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hs9gr"] Jul 07 23:34:49.929740 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:34:49.929699 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hs9gr"] Jul 07 23:34:58.831874 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:34:58.831838 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hs9gr"] Jul 07 23:35:09.822498 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:35:09.822445 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hs9gr"] Jul 07 23:35:21.720934 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:35:21.720894 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hs9gr"] Jul 07 23:35:29.236855 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:35:29.236757 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hs9gr"] Jul 07 23:35:42.364944 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:35:42.364903 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:35:43.483315 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:35:43.483279 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:35:46.523493 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:35:46.523430 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hs9gr"] Jul 07 23:35:55.642160 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:35:55.642126 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hs9gr"] Jul 07 23:36:05.524253 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:36:05.524215 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hs9gr"] Jul 07 23:36:16.324147 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:36:16.324111 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hs9gr"] Jul 07 23:36:24.339169 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:36:24.339132 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hs9gr"] Jul 07 23:36:36.125886 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:36:36.125848 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hs9gr"] Jul 07 23:36:46.818948 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:36:46.818915 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hs9gr"] Jul 07 23:36:56.428230 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:36:56.428141 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hs9gr"] Jul 07 23:37:06.922398 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:37:06.922356 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hs9gr"] Jul 07 23:37:16.723861 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:37:16.723821 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hs9gr"] Jul 07 23:37:26.424871 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:37:26.424833 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hs9gr"] Jul 07 23:37:36.931525 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:37:36.931487 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hs9gr"] Jul 07 23:37:48.432565 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:37:48.432528 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hs9gr"] Jul 07 23:37:58.030339 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:37:58.030294 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hs9gr"] Jul 07 23:38:03.162275 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:38:03.162239 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-8hvxr_faae2bd8-cbbd-4cfb-bce6-7954977b7ebc/console-operator/2.log" Jul 07 23:38:03.163756 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:38:03.163729 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-8hvxr_faae2bd8-cbbd-4cfb-bce6-7954977b7ebc/console-operator/2.log" Jul 07 23:38:10.619669 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:38:10.619627 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hs9gr"] Jul 07 23:38:23.018805 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:38:23.018767 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hs9gr"] Jul 07 23:38:31.427678 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:38:31.427565 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hs9gr"] Jul 07 23:38:40.018360 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:38:40.018321 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hs9gr"] Jul 07 23:38:56.219022 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:38:56.218988 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hs9gr"] Jul 07 23:39:07.620344 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:39:07.620310 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hs9gr"] Jul 07 23:39:09.473198 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:39:09.473148 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:39:10.453416 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:39:10.453388 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:39:14.301334 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:39:14.301273 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:39:19.524965 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:39:19.524925 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hs9gr"] Jul 07 23:39:22.552065 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:39:22.552021 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:39:27.650832 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:39:27.650793 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:39:30.448005 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:39:30.447968 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:39:30.548208 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:39:30.548177 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:39:30.884693 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:39:30.884659 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:39:31.521158 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:39:31.521119 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hs9gr"] Jul 07 23:39:36.678527 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:39:36.678494 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:39:36.780412 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:39:36.780383 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:39:36.889018 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:39:36.888987 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:39:36.945671 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:39:36.945561 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:39:38.060781 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:39:38.060722 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:39:38.146712 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:39:38.146640 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:39:38.213507 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:39:38.213456 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:39:38.668657 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:39:38.668594 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:39:38.880718 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:39:38.880670 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:39:38.982919 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:39:38.982811 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:39:40.644354 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:39:40.644322 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:39:42.118508 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:39:42.118459 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hs9gr"] Jul 07 23:39:43.080651 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:39:43.080608 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:39:43.259020 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:39:43.258982 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:39:43.354567 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:39:43.354460 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:39:43.452917 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:39:43.452862 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:39:44.653124 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:39:44.653093 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:39:49.323364 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:39:49.323332 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:39:49.423912 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:39:49.423881 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:39:49.539479 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:39:49.539445 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:39:49.632853 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:39:49.632771 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:39:50.719082 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:39:50.719046 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:39:50.807079 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:39:50.807027 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:39:50.877803 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:39:50.877769 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:39:50.976411 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:39:50.976321 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:39:51.364721 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:39:51.364687 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:39:51.467565 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:39:51.467526 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:39:52.821387 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:39:52.821348 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hs9gr"] Jul 07 23:39:52.939349 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:39:52.939321 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:39:55.745417 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:39:55.745318 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:39:56.047426 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:39:56.047381 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:40:01.824379 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:40:01.824337 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:40:01.926819 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:40:01.926770 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:40:04.422129 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:40:04.422082 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:40:07.997943 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:40:07.997906 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:40:08.098505 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:40:08.098451 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:40:08.211852 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:40:08.211799 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:40:08.278704 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:40:08.278673 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:40:09.299258 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:40:09.299198 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:40:09.382873 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:40:09.382838 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:40:09.446863 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:40:09.446802 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:40:09.601001 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:40:09.600880 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:40:09.949004 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:40:09.948918 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:40:10.047449 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:40:10.047421 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:40:11.721435 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:40:11.721403 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:40:12.225225 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:40:12.225188 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hs9gr"] Jul 07 23:40:13.758237 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:40:13.758202 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:40:13.941693 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:40:13.941660 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:40:14.041825 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:40:14.041797 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:40:14.132312 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:40:14.132279 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:40:15.533766 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:40:15.533733 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:40:20.028931 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:40:20.028895 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:40:20.085397 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:40:20.085364 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:40:20.488115 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:40:20.488014 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:40:20.589015 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:40:20.588974 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:40:20.660659 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:40:20.660624 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:40:21.323881 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:40:21.323844 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hs9gr"] Jul 07 23:40:22.077722 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:40:22.077684 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:40:27.858813 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:40:27.858776 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:40:27.913011 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:40:27.912981 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:40:27.994006 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:40:27.993980 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:40:28.102582 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:40:28.102540 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:40:28.209763 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:40:28.209680 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:40:28.558657 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:40:28.558626 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:40:54.511174 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:40:54.511139 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:41:00.475030 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:41:00.474991 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:41:00.536374 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:41:00.536337 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:41:00.623527 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:41:00.623492 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:41:00.717399 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:41:00.717363 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:41:16.597642 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:41:16.597603 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:41:22.572112 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:41:22.572075 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:41:22.672860 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:41:22.672826 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:41:22.771229 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:41:22.771196 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:41:33.280118 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:41:33.280082 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:41:34.100133 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:41:34.100097 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:41:39.839374 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:41:39.839338 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:41:39.949073 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:41:39.949037 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:41:40.039595 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:41:40.039561 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:41:40.152972 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:41:40.152892 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:41:40.907597 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:41:40.907531 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:41:41.002249 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:41:41.002212 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:41:41.102050 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:41:41.102018 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:41:42.667094 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:41:42.667059 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:41:43.223747 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:41:43.223709 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hs9gr"] Jul 07 23:41:48.875419 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:41:48.875380 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:41:48.977220 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:41:48.977180 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:41:49.564858 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:41:49.564825 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:41:52.618313 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:41:52.618281 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hs9gr"] Jul 07 23:41:55.641780 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:41:55.641747 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:41:55.742038 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:41:55.742009 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:41:55.879642 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:41:55.879606 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:41:56.201275 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:41:56.201241 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:41:56.297780 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:41:56.297747 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:41:59.114453 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:41:59.114415 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:41:59.214936 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:41:59.214892 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:41:59.415361 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:41:59.415276 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:41:59.519476 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:41:59.519429 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:42:00.326638 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:42:00.326606 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:42:00.415932 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:42:00.415905 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:42:03.609417 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:42:03.609385 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:42:04.031909 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:42:04.031867 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hs9gr"] Jul 07 23:42:04.258769 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:42:04.258732 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:42:04.358184 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:42:04.358098 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:42:04.691053 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:42:04.690962 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:42:11.078483 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:42:11.078430 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:42:11.184247 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:42:11.184216 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:42:11.481844 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:42:11.481740 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:42:11.817206 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:42:11.817168 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:42:11.890387 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:42:11.890359 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:42:15.703008 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:42:15.702965 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hs9gr"] Jul 07 23:42:17.982480 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:42:17.982439 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:42:18.083255 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:42:18.083202 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:42:18.244696 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:42:18.244614 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:42:18.571725 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:42:18.571695 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:42:18.672846 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:42:18.672802 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:42:18.795326 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:42:18.795288 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:42:24.705550 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:42:24.705506 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:42:24.811137 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:42:24.811101 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:42:24.905179 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:42:24.905142 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:42:25.047964 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:42:25.047936 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:42:31.251437 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:42:31.251406 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:42:31.365531 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:42:31.365498 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:42:34.080916 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:42:34.080885 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:42:34.182376 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:42:34.182325 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:42:38.010531 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:42:38.010493 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:42:38.312365 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:42:38.312332 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:42:38.416904 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:42:38.416873 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:42:41.237018 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:42:41.236984 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:42:41.317351 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:42:41.317317 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:42:41.378152 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:42:41.378118 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:42:41.709632 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:42:41.709585 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:42:41.810106 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:42:41.810076 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:42:41.910874 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:42:41.910842 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:42:45.732872 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:42:45.732838 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:42:46.021611 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:42:46.021529 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hs9gr"] Jul 07 23:42:48.812919 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:42:48.812884 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:42:48.913388 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:42:48.913340 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:42:54.725210 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:42:54.725177 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-mt-dfd300c4/e2e-lifecycle-model-dfd300c4-kserve-59979cf795-cgzhm"] Jul 07 23:42:54.728823 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:42:54.728805 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-dfd300c4/e2e-lifecycle-model-dfd300c4-kserve-59979cf795-cgzhm" Jul 07 23:42:54.732242 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:42:54.732219 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-dfd300c4\"/\"kube-root-ca.crt\"" Jul 07 23:42:54.732365 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:42:54.732262 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-dfd300c4\"/\"e2e-lifecycle-model-dfd300c4-kserve-self-signed-certs\"" Jul 07 23:42:54.732365 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:42:54.732339 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-dfd300c4\"/\"openshift-service-ca.crt\"" Jul 07 23:42:54.732365 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:42:54.732353 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-dfd300c4\"/\"default-dockercfg-k7r24\"" Jul 07 23:42:54.739046 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:42:54.739024 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-dfd300c4/e2e-lifecycle-model-dfd300c4-kserve-59979cf795-cgzhm"] Jul 07 23:42:54.815916 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:42:54.815880 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7e4a0b2a-3913-43e9-a358-290ddfc2ed91-tmp-dir\") pod \"e2e-lifecycle-model-dfd300c4-kserve-59979cf795-cgzhm\" (UID: \"7e4a0b2a-3913-43e9-a358-290ddfc2ed91\") " pod="ai-tenant-e2e-mt-dfd300c4/e2e-lifecycle-model-dfd300c4-kserve-59979cf795-cgzhm" Jul 07 23:42:54.816096 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:42:54.815928 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hgc2p\" (UniqueName: \"kubernetes.io/projected/7e4a0b2a-3913-43e9-a358-290ddfc2ed91-kube-api-access-hgc2p\") pod \"e2e-lifecycle-model-dfd300c4-kserve-59979cf795-cgzhm\" (UID: \"7e4a0b2a-3913-43e9-a358-290ddfc2ed91\") " pod="ai-tenant-e2e-mt-dfd300c4/e2e-lifecycle-model-dfd300c4-kserve-59979cf795-cgzhm" Jul 07 23:42:54.816096 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:42:54.815991 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7e4a0b2a-3913-43e9-a358-290ddfc2ed91-tls-certs\") pod \"e2e-lifecycle-model-dfd300c4-kserve-59979cf795-cgzhm\" (UID: \"7e4a0b2a-3913-43e9-a358-290ddfc2ed91\") " pod="ai-tenant-e2e-mt-dfd300c4/e2e-lifecycle-model-dfd300c4-kserve-59979cf795-cgzhm" Jul 07 23:42:54.816096 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:42:54.816081 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7e4a0b2a-3913-43e9-a358-290ddfc2ed91-dshm\") pod \"e2e-lifecycle-model-dfd300c4-kserve-59979cf795-cgzhm\" (UID: \"7e4a0b2a-3913-43e9-a358-290ddfc2ed91\") " pod="ai-tenant-e2e-mt-dfd300c4/e2e-lifecycle-model-dfd300c4-kserve-59979cf795-cgzhm" Jul 07 23:42:54.816237 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:42:54.816110 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7e4a0b2a-3913-43e9-a358-290ddfc2ed91-model-cache\") pod \"e2e-lifecycle-model-dfd300c4-kserve-59979cf795-cgzhm\" (UID: \"7e4a0b2a-3913-43e9-a358-290ddfc2ed91\") " pod="ai-tenant-e2e-mt-dfd300c4/e2e-lifecycle-model-dfd300c4-kserve-59979cf795-cgzhm" Jul 07 23:42:54.816237 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:42:54.816158 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7e4a0b2a-3913-43e9-a358-290ddfc2ed91-home\") pod \"e2e-lifecycle-model-dfd300c4-kserve-59979cf795-cgzhm\" (UID: \"7e4a0b2a-3913-43e9-a358-290ddfc2ed91\") " pod="ai-tenant-e2e-mt-dfd300c4/e2e-lifecycle-model-dfd300c4-kserve-59979cf795-cgzhm" Jul 07 23:42:54.917152 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:42:54.917113 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7e4a0b2a-3913-43e9-a358-290ddfc2ed91-tmp-dir\") pod \"e2e-lifecycle-model-dfd300c4-kserve-59979cf795-cgzhm\" (UID: \"7e4a0b2a-3913-43e9-a358-290ddfc2ed91\") " pod="ai-tenant-e2e-mt-dfd300c4/e2e-lifecycle-model-dfd300c4-kserve-59979cf795-cgzhm" Jul 07 23:42:54.917314 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:42:54.917161 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hgc2p\" (UniqueName: \"kubernetes.io/projected/7e4a0b2a-3913-43e9-a358-290ddfc2ed91-kube-api-access-hgc2p\") pod \"e2e-lifecycle-model-dfd300c4-kserve-59979cf795-cgzhm\" (UID: \"7e4a0b2a-3913-43e9-a358-290ddfc2ed91\") " pod="ai-tenant-e2e-mt-dfd300c4/e2e-lifecycle-model-dfd300c4-kserve-59979cf795-cgzhm" Jul 07 23:42:54.917314 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:42:54.917199 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7e4a0b2a-3913-43e9-a358-290ddfc2ed91-tls-certs\") pod \"e2e-lifecycle-model-dfd300c4-kserve-59979cf795-cgzhm\" (UID: \"7e4a0b2a-3913-43e9-a358-290ddfc2ed91\") " pod="ai-tenant-e2e-mt-dfd300c4/e2e-lifecycle-model-dfd300c4-kserve-59979cf795-cgzhm" Jul 07 23:42:54.917314 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:42:54.917252 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7e4a0b2a-3913-43e9-a358-290ddfc2ed91-dshm\") pod \"e2e-lifecycle-model-dfd300c4-kserve-59979cf795-cgzhm\" (UID: \"7e4a0b2a-3913-43e9-a358-290ddfc2ed91\") " pod="ai-tenant-e2e-mt-dfd300c4/e2e-lifecycle-model-dfd300c4-kserve-59979cf795-cgzhm" Jul 07 23:42:54.917314 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:42:54.917275 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7e4a0b2a-3913-43e9-a358-290ddfc2ed91-model-cache\") pod \"e2e-lifecycle-model-dfd300c4-kserve-59979cf795-cgzhm\" (UID: \"7e4a0b2a-3913-43e9-a358-290ddfc2ed91\") " pod="ai-tenant-e2e-mt-dfd300c4/e2e-lifecycle-model-dfd300c4-kserve-59979cf795-cgzhm" Jul 07 23:42:54.917314 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:42:54.917306 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7e4a0b2a-3913-43e9-a358-290ddfc2ed91-home\") pod \"e2e-lifecycle-model-dfd300c4-kserve-59979cf795-cgzhm\" (UID: \"7e4a0b2a-3913-43e9-a358-290ddfc2ed91\") " pod="ai-tenant-e2e-mt-dfd300c4/e2e-lifecycle-model-dfd300c4-kserve-59979cf795-cgzhm" Jul 07 23:42:54.917608 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:42:54.917589 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7e4a0b2a-3913-43e9-a358-290ddfc2ed91-tmp-dir\") pod \"e2e-lifecycle-model-dfd300c4-kserve-59979cf795-cgzhm\" (UID: \"7e4a0b2a-3913-43e9-a358-290ddfc2ed91\") " pod="ai-tenant-e2e-mt-dfd300c4/e2e-lifecycle-model-dfd300c4-kserve-59979cf795-cgzhm" Jul 07 23:42:54.917710 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:42:54.917675 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7e4a0b2a-3913-43e9-a358-290ddfc2ed91-model-cache\") pod \"e2e-lifecycle-model-dfd300c4-kserve-59979cf795-cgzhm\" (UID: \"7e4a0b2a-3913-43e9-a358-290ddfc2ed91\") " pod="ai-tenant-e2e-mt-dfd300c4/e2e-lifecycle-model-dfd300c4-kserve-59979cf795-cgzhm" Jul 07 23:42:54.917710 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:42:54.917700 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7e4a0b2a-3913-43e9-a358-290ddfc2ed91-home\") pod \"e2e-lifecycle-model-dfd300c4-kserve-59979cf795-cgzhm\" (UID: \"7e4a0b2a-3913-43e9-a358-290ddfc2ed91\") " pod="ai-tenant-e2e-mt-dfd300c4/e2e-lifecycle-model-dfd300c4-kserve-59979cf795-cgzhm" Jul 07 23:42:54.919478 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:42:54.919445 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7e4a0b2a-3913-43e9-a358-290ddfc2ed91-dshm\") pod \"e2e-lifecycle-model-dfd300c4-kserve-59979cf795-cgzhm\" (UID: \"7e4a0b2a-3913-43e9-a358-290ddfc2ed91\") " pod="ai-tenant-e2e-mt-dfd300c4/e2e-lifecycle-model-dfd300c4-kserve-59979cf795-cgzhm" Jul 07 23:42:54.919753 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:42:54.919735 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7e4a0b2a-3913-43e9-a358-290ddfc2ed91-tls-certs\") pod \"e2e-lifecycle-model-dfd300c4-kserve-59979cf795-cgzhm\" (UID: \"7e4a0b2a-3913-43e9-a358-290ddfc2ed91\") " pod="ai-tenant-e2e-mt-dfd300c4/e2e-lifecycle-model-dfd300c4-kserve-59979cf795-cgzhm" Jul 07 23:42:54.928363 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:42:54.928341 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hgc2p\" (UniqueName: \"kubernetes.io/projected/7e4a0b2a-3913-43e9-a358-290ddfc2ed91-kube-api-access-hgc2p\") pod \"e2e-lifecycle-model-dfd300c4-kserve-59979cf795-cgzhm\" (UID: \"7e4a0b2a-3913-43e9-a358-290ddfc2ed91\") " pod="ai-tenant-e2e-mt-dfd300c4/e2e-lifecycle-model-dfd300c4-kserve-59979cf795-cgzhm" Jul 07 23:42:55.038320 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:42:55.038277 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-dfd300c4/e2e-lifecycle-model-dfd300c4-kserve-59979cf795-cgzhm" Jul 07 23:42:55.174557 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:42:55.174532 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-dfd300c4/e2e-lifecycle-model-dfd300c4-kserve-59979cf795-cgzhm"] Jul 07 23:42:55.175687 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:42:55.175646 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7e4a0b2a_3913_43e9_a358_290ddfc2ed91.slice/crio-f382659d9fe155dbb0be28369a43731f9c2336c0f13c516e9a376a9a6f38a4e2 WatchSource:0}: Error finding container f382659d9fe155dbb0be28369a43731f9c2336c0f13c516e9a376a9a6f38a4e2: Status 404 returned error can't find the container with id f382659d9fe155dbb0be28369a43731f9c2336c0f13c516e9a376a9a6f38a4e2 Jul 07 23:42:56.158735 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:42:56.158697 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-dfd300c4/e2e-lifecycle-model-dfd300c4-kserve-59979cf795-cgzhm" event={"ID":"7e4a0b2a-3913-43e9-a358-290ddfc2ed91","Type":"ContainerStarted","Data":"22160478135d111d5d77088c52a150aca5a88b88dcd25e85e603dffb7eb0d47a"} Jul 07 23:42:56.159129 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:42:56.158744 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-dfd300c4/e2e-lifecycle-model-dfd300c4-kserve-59979cf795-cgzhm" event={"ID":"7e4a0b2a-3913-43e9-a358-290ddfc2ed91","Type":"ContainerStarted","Data":"f382659d9fe155dbb0be28369a43731f9c2336c0f13c516e9a376a9a6f38a4e2"} Jul 07 23:42:56.176690 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:42:56.176643 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-mt-dfd300c4/e2e-lifecycle-model-dfd300c4-kserve-59979cf795-cgzhm" podStartSLOduration=2.176630012 podStartE2EDuration="2.176630012s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 23:42:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 23:42:56.174389551 +0000 UTC m=+1793.571391063" watchObservedRunningTime="2026-07-07 23:42:56.176630012 +0000 UTC m=+1793.573631523" Jul 07 23:43:02.626363 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:43:02.626332 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:43:02.728014 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:43:02.727981 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:43:03.197659 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:43:03.197631 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-8hvxr_faae2bd8-cbbd-4cfb-bce6-7954977b7ebc/console-operator/2.log" Jul 07 23:43:03.199292 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:43:03.199272 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-8hvxr_faae2bd8-cbbd-4cfb-bce6-7954977b7ebc/console-operator/2.log" Jul 07 23:43:05.038484 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:43:05.038419 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-dfd300c4/e2e-lifecycle-model-dfd300c4-kserve-59979cf795-cgzhm" Jul 07 23:43:05.038888 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:43:05.038501 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-mt-dfd300c4/e2e-lifecycle-model-dfd300c4-kserve-59979cf795-cgzhm" Jul 07 23:43:05.051114 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:43:05.051086 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-mt-dfd300c4/e2e-lifecycle-model-dfd300c4-kserve-59979cf795-cgzhm" Jul 07 23:43:05.215538 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:43:05.215510 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-dfd300c4/e2e-lifecycle-model-dfd300c4-kserve-59979cf795-cgzhm" Jul 07 23:43:14.541926 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:43:14.541892 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:43:17.629840 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:43:17.629803 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hs9gr"] Jul 07 23:43:28.266159 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:43:28.266119 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:43:28.365843 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:43:28.365810 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:43:28.521863 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:43:28.521779 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hs9gr"] Jul 07 23:43:28.738785 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:43:28.738753 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:43:34.448805 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:43:34.448772 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:43:39.132710 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:43:39.132675 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hs9gr"] Jul 07 23:43:39.433760 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:43:39.433676 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-dfd300c4/e2e-lifecycle-model-dfd300c4-kserve-59979cf795-cgzhm"] Jul 07 23:43:39.480810 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:43:39.480778 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:43:54.455131 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:43:54.455069 2570 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-mt-dfd300c4/e2e-lifecycle-model-dfd300c4-kserve-59979cf795-cgzhm" podUID="7e4a0b2a-3913-43e9-a358-290ddfc2ed91" containerName="main" containerID="cri-o://22160478135d111d5d77088c52a150aca5a88b88dcd25e85e603dffb7eb0d47a" gracePeriod=45 Jul 07 23:43:54.713817 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:43:54.713763 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-dfd300c4/e2e-lifecycle-model-dfd300c4-kserve-59979cf795-cgzhm" Jul 07 23:43:54.835387 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:43:54.835340 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7e4a0b2a-3913-43e9-a358-290ddfc2ed91-tls-certs\") pod \"7e4a0b2a-3913-43e9-a358-290ddfc2ed91\" (UID: \"7e4a0b2a-3913-43e9-a358-290ddfc2ed91\") " Jul 07 23:43:54.835587 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:43:54.835423 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7e4a0b2a-3913-43e9-a358-290ddfc2ed91-tmp-dir\") pod \"7e4a0b2a-3913-43e9-a358-290ddfc2ed91\" (UID: \"7e4a0b2a-3913-43e9-a358-290ddfc2ed91\") " Jul 07 23:43:54.835587 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:43:54.835489 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hgc2p\" (UniqueName: \"kubernetes.io/projected/7e4a0b2a-3913-43e9-a358-290ddfc2ed91-kube-api-access-hgc2p\") pod \"7e4a0b2a-3913-43e9-a358-290ddfc2ed91\" (UID: \"7e4a0b2a-3913-43e9-a358-290ddfc2ed91\") " Jul 07 23:43:54.835587 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:43:54.835537 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7e4a0b2a-3913-43e9-a358-290ddfc2ed91-home\") pod \"7e4a0b2a-3913-43e9-a358-290ddfc2ed91\" (UID: \"7e4a0b2a-3913-43e9-a358-290ddfc2ed91\") " Jul 07 23:43:54.835587 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:43:54.835582 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7e4a0b2a-3913-43e9-a358-290ddfc2ed91-dshm\") pod \"7e4a0b2a-3913-43e9-a358-290ddfc2ed91\" (UID: \"7e4a0b2a-3913-43e9-a358-290ddfc2ed91\") " Jul 07 23:43:54.835749 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:43:54.835615 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7e4a0b2a-3913-43e9-a358-290ddfc2ed91-model-cache\") pod \"7e4a0b2a-3913-43e9-a358-290ddfc2ed91\" (UID: \"7e4a0b2a-3913-43e9-a358-290ddfc2ed91\") " Jul 07 23:43:54.835749 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:43:54.835679 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7e4a0b2a-3913-43e9-a358-290ddfc2ed91-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "7e4a0b2a-3913-43e9-a358-290ddfc2ed91" (UID: "7e4a0b2a-3913-43e9-a358-290ddfc2ed91"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 23:43:54.835838 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:43:54.835791 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7e4a0b2a-3913-43e9-a358-290ddfc2ed91-home" (OuterVolumeSpecName: "home") pod "7e4a0b2a-3913-43e9-a358-290ddfc2ed91" (UID: "7e4a0b2a-3913-43e9-a358-290ddfc2ed91"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 23:43:54.835932 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:43:54.835910 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7e4a0b2a-3913-43e9-a358-290ddfc2ed91-model-cache" (OuterVolumeSpecName: "model-cache") pod "7e4a0b2a-3913-43e9-a358-290ddfc2ed91" (UID: "7e4a0b2a-3913-43e9-a358-290ddfc2ed91"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 23:43:54.835989 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:43:54.835932 2570 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7e4a0b2a-3913-43e9-a358-290ddfc2ed91-tmp-dir\") on node \"ip-10-0-134-140.ec2.internal\" DevicePath \"\"" Jul 07 23:43:54.835989 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:43:54.835946 2570 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7e4a0b2a-3913-43e9-a358-290ddfc2ed91-home\") on node \"ip-10-0-134-140.ec2.internal\" DevicePath \"\"" Jul 07 23:43:54.837809 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:43:54.837781 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7e4a0b2a-3913-43e9-a358-290ddfc2ed91-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "7e4a0b2a-3913-43e9-a358-290ddfc2ed91" (UID: "7e4a0b2a-3913-43e9-a358-290ddfc2ed91"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 23:43:54.837909 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:43:54.837806 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7e4a0b2a-3913-43e9-a358-290ddfc2ed91-dshm" (OuterVolumeSpecName: "dshm") pod "7e4a0b2a-3913-43e9-a358-290ddfc2ed91" (UID: "7e4a0b2a-3913-43e9-a358-290ddfc2ed91"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 23:43:54.837909 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:43:54.837823 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7e4a0b2a-3913-43e9-a358-290ddfc2ed91-kube-api-access-hgc2p" (OuterVolumeSpecName: "kube-api-access-hgc2p") pod "7e4a0b2a-3913-43e9-a358-290ddfc2ed91" (UID: "7e4a0b2a-3913-43e9-a358-290ddfc2ed91"). InnerVolumeSpecName "kube-api-access-hgc2p". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 23:43:54.937362 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:43:54.937312 2570 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7e4a0b2a-3913-43e9-a358-290ddfc2ed91-model-cache\") on node \"ip-10-0-134-140.ec2.internal\" DevicePath \"\"" Jul 07 23:43:54.937362 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:43:54.937355 2570 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7e4a0b2a-3913-43e9-a358-290ddfc2ed91-tls-certs\") on node \"ip-10-0-134-140.ec2.internal\" DevicePath \"\"" Jul 07 23:43:54.937362 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:43:54.937364 2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hgc2p\" (UniqueName: \"kubernetes.io/projected/7e4a0b2a-3913-43e9-a358-290ddfc2ed91-kube-api-access-hgc2p\") on node \"ip-10-0-134-140.ec2.internal\" DevicePath \"\"" Jul 07 23:43:54.937362 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:43:54.937373 2570 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7e4a0b2a-3913-43e9-a358-290ddfc2ed91-dshm\") on node \"ip-10-0-134-140.ec2.internal\" DevicePath \"\"" Jul 07 23:43:55.473164 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:43:55.473068 2570 generic.go:356] "Generic (PLEG): container finished" podID="7e4a0b2a-3913-43e9-a358-290ddfc2ed91" containerID="22160478135d111d5d77088c52a150aca5a88b88dcd25e85e603dffb7eb0d47a" exitCode=0 Jul 07 23:43:55.473164 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:43:55.473121 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-dfd300c4/e2e-lifecycle-model-dfd300c4-kserve-59979cf795-cgzhm" Jul 07 23:43:55.473164 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:43:55.473140 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-dfd300c4/e2e-lifecycle-model-dfd300c4-kserve-59979cf795-cgzhm" event={"ID":"7e4a0b2a-3913-43e9-a358-290ddfc2ed91","Type":"ContainerDied","Data":"22160478135d111d5d77088c52a150aca5a88b88dcd25e85e603dffb7eb0d47a"} Jul 07 23:43:55.473710 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:43:55.473176 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-dfd300c4/e2e-lifecycle-model-dfd300c4-kserve-59979cf795-cgzhm" event={"ID":"7e4a0b2a-3913-43e9-a358-290ddfc2ed91","Type":"ContainerDied","Data":"f382659d9fe155dbb0be28369a43731f9c2336c0f13c516e9a376a9a6f38a4e2"} Jul 07 23:43:55.473710 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:43:55.473196 2570 scope.go:117] "RemoveContainer" containerID="22160478135d111d5d77088c52a150aca5a88b88dcd25e85e603dffb7eb0d47a" Jul 07 23:43:55.482117 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:43:55.482100 2570 scope.go:117] "RemoveContainer" containerID="22160478135d111d5d77088c52a150aca5a88b88dcd25e85e603dffb7eb0d47a" Jul 07 23:43:55.482346 ip-10-0-134-140 kubenswrapper[2570]: E0707 23:43:55.482325 2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"22160478135d111d5d77088c52a150aca5a88b88dcd25e85e603dffb7eb0d47a\": container with ID starting with 22160478135d111d5d77088c52a150aca5a88b88dcd25e85e603dffb7eb0d47a not found: ID does not exist" containerID="22160478135d111d5d77088c52a150aca5a88b88dcd25e85e603dffb7eb0d47a" Jul 07 23:43:55.482394 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:43:55.482353 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"22160478135d111d5d77088c52a150aca5a88b88dcd25e85e603dffb7eb0d47a"} err="failed to get container status \"22160478135d111d5d77088c52a150aca5a88b88dcd25e85e603dffb7eb0d47a\": rpc error: code = NotFound desc = could not find container \"22160478135d111d5d77088c52a150aca5a88b88dcd25e85e603dffb7eb0d47a\": container with ID starting with 22160478135d111d5d77088c52a150aca5a88b88dcd25e85e603dffb7eb0d47a not found: ID does not exist" Jul 07 23:43:55.490030 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:43:55.490006 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-dfd300c4/e2e-lifecycle-model-dfd300c4-kserve-59979cf795-cgzhm"] Jul 07 23:43:55.495394 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:43:55.495368 2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-mt-dfd300c4/e2e-lifecycle-model-dfd300c4-kserve-59979cf795-cgzhm"] Jul 07 23:43:57.096636 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:43:57.096598 2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7e4a0b2a-3913-43e9-a358-290ddfc2ed91" path="/var/lib/kubelet/pods/7e4a0b2a-3913-43e9-a358-290ddfc2ed91/volumes" Jul 07 23:44:16.433860 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:44:16.433825 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:44:16.576538 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:44:16.576507 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:44:19.128973 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:44:19.128936 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hs9gr"] Jul 07 23:44:22.342143 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:44:22.342083 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:44:22.441985 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:44:22.441952 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:44:22.616843 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:44:22.616762 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:44:23.878852 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:44:23.878811 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:44:23.984771 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:44:23.984737 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:44:24.080658 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:44:24.080622 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:44:24.789874 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:44:24.789837 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:44:24.889658 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:44:24.889625 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:44:24.993754 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:44:24.993723 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:44:26.356679 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:44:26.356635 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:44:32.139614 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:44:32.139577 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hs9gr"] Jul 07 23:44:32.438789 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:44:32.438712 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:44:32.538949 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:44:32.538920 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:44:33.111830 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:44:33.111794 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:44:38.969935 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:44:38.969899 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:44:39.074732 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:44:39.074694 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:44:39.170704 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:44:39.170673 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:44:41.433518 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:44:41.433483 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hs9gr"] Jul 07 23:44:49.664060 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:44:49.664013 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:44:50.003596 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:44:50.003520 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:45:10.981561 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:45:10.981517 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:45:17.239284 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:45:17.239251 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:45:17.341907 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:45:17.341868 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:45:17.584355 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:45:17.584317 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:45:17.889355 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:45:17.889283 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:45:18.547785 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:45:18.547750 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:45:21.296842 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:45:21.296807 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:45:21.370903 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:45:21.370873 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:45:21.519925 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:45:21.519893 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:45:21.862041 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:45:21.862012 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:45:21.958746 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:45:21.958707 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:45:25.618302 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:45:25.618270 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:45:25.940840 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:45:25.940755 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:45:26.252901 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:45:26.252821 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:45:26.508780 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:45:26.508702 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:45:28.924194 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:45:28.924156 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hs9gr"] Jul 07 23:45:29.114189 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:45:29.114152 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:45:29.194155 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:45:29.194060 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:45:29.258795 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:45:29.258761 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:45:29.610032 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:45:29.609996 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:45:29.770933 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:45:29.770899 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:45:29.872857 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:45:29.872778 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:45:31.643915 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:45:31.643885 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:45:33.622975 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:45:33.622944 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:45:39.454811 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:45:39.454766 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hs9gr"] Jul 07 23:45:43.279780 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:45:43.279746 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:45:43.380286 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:45:43.380251 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:45:50.619112 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:45:50.619077 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:45:50.720655 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:45:50.720617 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:45:52.243651 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:45:52.243610 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-a-04f579/test-model-04f579-kserve-57d56ffb8c-8jglv"] Jul 07 23:45:52.246663 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:45:52.244614 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7e4a0b2a-3913-43e9-a358-290ddfc2ed91" containerName="main" Jul 07 23:45:52.246663 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:45:52.244652 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="7e4a0b2a-3913-43e9-a358-290ddfc2ed91" containerName="main" Jul 07 23:45:52.246663 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:45:52.244828 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="7e4a0b2a-3913-43e9-a358-290ddfc2ed91" containerName="main" Jul 07 23:45:52.249018 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:45:52.248989 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-04f579/test-model-04f579-kserve-57d56ffb8c-8jglv" Jul 07 23:45:52.252887 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:45:52.252843 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-04f579\"/\"openshift-service-ca.crt\"" Jul 07 23:45:52.253020 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:45:52.252930 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-04f579\"/\"kube-root-ca.crt\"" Jul 07 23:45:52.253130 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:45:52.253113 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-04f579\"/\"default-dockercfg-wspgf\"" Jul 07 23:45:52.253248 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:45:52.253231 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-04f579\"/\"test-model-04f579-kserve-self-signed-certs\"" Jul 07 23:45:52.255120 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:45:52.255099 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-04f579/test-model-04f579-kserve-57d56ffb8c-8jglv"] Jul 07 23:45:52.331011 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:45:52.330978 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:45:52.364833 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:45:52.364799 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5c0069c7-1e3a-498a-af64-1b8ac9e53e39-tmp-dir\") pod \"test-model-04f579-kserve-57d56ffb8c-8jglv\" (UID: \"5c0069c7-1e3a-498a-af64-1b8ac9e53e39\") " pod="ai-tenant-e2e-inf-a-04f579/test-model-04f579-kserve-57d56ffb8c-8jglv" Jul 07 23:45:52.364981 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:45:52.364906 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5c0069c7-1e3a-498a-af64-1b8ac9e53e39-tls-certs\") pod \"test-model-04f579-kserve-57d56ffb8c-8jglv\" (UID: \"5c0069c7-1e3a-498a-af64-1b8ac9e53e39\") " pod="ai-tenant-e2e-inf-a-04f579/test-model-04f579-kserve-57d56ffb8c-8jglv" Jul 07 23:45:52.364981 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:45:52.364938 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5c0069c7-1e3a-498a-af64-1b8ac9e53e39-dshm\") pod \"test-model-04f579-kserve-57d56ffb8c-8jglv\" (UID: \"5c0069c7-1e3a-498a-af64-1b8ac9e53e39\") " pod="ai-tenant-e2e-inf-a-04f579/test-model-04f579-kserve-57d56ffb8c-8jglv" Jul 07 23:45:52.364981 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:45:52.364959 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-psftt\" (UniqueName: \"kubernetes.io/projected/5c0069c7-1e3a-498a-af64-1b8ac9e53e39-kube-api-access-psftt\") pod \"test-model-04f579-kserve-57d56ffb8c-8jglv\" (UID: \"5c0069c7-1e3a-498a-af64-1b8ac9e53e39\") " pod="ai-tenant-e2e-inf-a-04f579/test-model-04f579-kserve-57d56ffb8c-8jglv" Jul 07 23:45:52.365108 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:45:52.364983 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5c0069c7-1e3a-498a-af64-1b8ac9e53e39-home\") pod \"test-model-04f579-kserve-57d56ffb8c-8jglv\" (UID: \"5c0069c7-1e3a-498a-af64-1b8ac9e53e39\") " pod="ai-tenant-e2e-inf-a-04f579/test-model-04f579-kserve-57d56ffb8c-8jglv" Jul 07 23:45:52.365108 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:45:52.364998 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5c0069c7-1e3a-498a-af64-1b8ac9e53e39-model-cache\") pod \"test-model-04f579-kserve-57d56ffb8c-8jglv\" (UID: \"5c0069c7-1e3a-498a-af64-1b8ac9e53e39\") " pod="ai-tenant-e2e-inf-a-04f579/test-model-04f579-kserve-57d56ffb8c-8jglv" Jul 07 23:45:52.465923 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:45:52.465887 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5c0069c7-1e3a-498a-af64-1b8ac9e53e39-tls-certs\") pod \"test-model-04f579-kserve-57d56ffb8c-8jglv\" (UID: \"5c0069c7-1e3a-498a-af64-1b8ac9e53e39\") " pod="ai-tenant-e2e-inf-a-04f579/test-model-04f579-kserve-57d56ffb8c-8jglv" Jul 07 23:45:52.466103 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:45:52.465929 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5c0069c7-1e3a-498a-af64-1b8ac9e53e39-dshm\") pod \"test-model-04f579-kserve-57d56ffb8c-8jglv\" (UID: \"5c0069c7-1e3a-498a-af64-1b8ac9e53e39\") " pod="ai-tenant-e2e-inf-a-04f579/test-model-04f579-kserve-57d56ffb8c-8jglv" Jul 07 23:45:52.466103 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:45:52.465954 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-psftt\" (UniqueName: \"kubernetes.io/projected/5c0069c7-1e3a-498a-af64-1b8ac9e53e39-kube-api-access-psftt\") pod \"test-model-04f579-kserve-57d56ffb8c-8jglv\" (UID: \"5c0069c7-1e3a-498a-af64-1b8ac9e53e39\") " pod="ai-tenant-e2e-inf-a-04f579/test-model-04f579-kserve-57d56ffb8c-8jglv" Jul 07 23:45:52.466103 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:45:52.465989 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5c0069c7-1e3a-498a-af64-1b8ac9e53e39-home\") pod \"test-model-04f579-kserve-57d56ffb8c-8jglv\" (UID: \"5c0069c7-1e3a-498a-af64-1b8ac9e53e39\") " pod="ai-tenant-e2e-inf-a-04f579/test-model-04f579-kserve-57d56ffb8c-8jglv" Jul 07 23:45:52.466103 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:45:52.466010 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5c0069c7-1e3a-498a-af64-1b8ac9e53e39-model-cache\") pod \"test-model-04f579-kserve-57d56ffb8c-8jglv\" (UID: \"5c0069c7-1e3a-498a-af64-1b8ac9e53e39\") " pod="ai-tenant-e2e-inf-a-04f579/test-model-04f579-kserve-57d56ffb8c-8jglv" Jul 07 23:45:52.466103 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:45:52.466049 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5c0069c7-1e3a-498a-af64-1b8ac9e53e39-tmp-dir\") pod \"test-model-04f579-kserve-57d56ffb8c-8jglv\" (UID: \"5c0069c7-1e3a-498a-af64-1b8ac9e53e39\") " pod="ai-tenant-e2e-inf-a-04f579/test-model-04f579-kserve-57d56ffb8c-8jglv" Jul 07 23:45:52.466423 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:45:52.466394 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5c0069c7-1e3a-498a-af64-1b8ac9e53e39-home\") pod \"test-model-04f579-kserve-57d56ffb8c-8jglv\" (UID: \"5c0069c7-1e3a-498a-af64-1b8ac9e53e39\") " pod="ai-tenant-e2e-inf-a-04f579/test-model-04f579-kserve-57d56ffb8c-8jglv" Jul 07 23:45:52.466529 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:45:52.466425 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5c0069c7-1e3a-498a-af64-1b8ac9e53e39-model-cache\") pod \"test-model-04f579-kserve-57d56ffb8c-8jglv\" (UID: \"5c0069c7-1e3a-498a-af64-1b8ac9e53e39\") " pod="ai-tenant-e2e-inf-a-04f579/test-model-04f579-kserve-57d56ffb8c-8jglv" Jul 07 23:45:52.466529 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:45:52.466455 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5c0069c7-1e3a-498a-af64-1b8ac9e53e39-tmp-dir\") pod \"test-model-04f579-kserve-57d56ffb8c-8jglv\" (UID: \"5c0069c7-1e3a-498a-af64-1b8ac9e53e39\") " pod="ai-tenant-e2e-inf-a-04f579/test-model-04f579-kserve-57d56ffb8c-8jglv" Jul 07 23:45:52.468135 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:45:52.468110 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5c0069c7-1e3a-498a-af64-1b8ac9e53e39-dshm\") pod \"test-model-04f579-kserve-57d56ffb8c-8jglv\" (UID: \"5c0069c7-1e3a-498a-af64-1b8ac9e53e39\") " pod="ai-tenant-e2e-inf-a-04f579/test-model-04f579-kserve-57d56ffb8c-8jglv" Jul 07 23:45:52.468413 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:45:52.468394 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5c0069c7-1e3a-498a-af64-1b8ac9e53e39-tls-certs\") pod \"test-model-04f579-kserve-57d56ffb8c-8jglv\" (UID: \"5c0069c7-1e3a-498a-af64-1b8ac9e53e39\") " pod="ai-tenant-e2e-inf-a-04f579/test-model-04f579-kserve-57d56ffb8c-8jglv" Jul 07 23:45:52.473687 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:45:52.473665 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-psftt\" (UniqueName: \"kubernetes.io/projected/5c0069c7-1e3a-498a-af64-1b8ac9e53e39-kube-api-access-psftt\") pod \"test-model-04f579-kserve-57d56ffb8c-8jglv\" (UID: \"5c0069c7-1e3a-498a-af64-1b8ac9e53e39\") " pod="ai-tenant-e2e-inf-a-04f579/test-model-04f579-kserve-57d56ffb8c-8jglv" Jul 07 23:45:52.560572 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:45:52.560540 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-04f579/test-model-04f579-kserve-57d56ffb8c-8jglv" Jul 07 23:45:52.694844 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:45:52.694802 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-04f579/test-model-04f579-kserve-57d56ffb8c-8jglv"] Jul 07 23:45:52.697228 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:45:52.697196 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5c0069c7_1e3a_498a_af64_1b8ac9e53e39.slice/crio-b367502302a9dc667771339bb2f9ee125e5e6dbd37d9325ac697776fc83d4bab WatchSource:0}: Error finding container b367502302a9dc667771339bb2f9ee125e5e6dbd37d9325ac697776fc83d4bab: Status 404 returned error can't find the container with id b367502302a9dc667771339bb2f9ee125e5e6dbd37d9325ac697776fc83d4bab Jul 07 23:45:53.080747 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:45:53.080705 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-04f579/test-model-04f579-kserve-57d56ffb8c-8jglv" event={"ID":"5c0069c7-1e3a-498a-af64-1b8ac9e53e39","Type":"ContainerStarted","Data":"9e43c2cbb21874c5573b72bb4a6deeee763a8845b1993a7c71bb0cb8ea361581"} Jul 07 23:45:53.080747 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:45:53.080748 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-04f579/test-model-04f579-kserve-57d56ffb8c-8jglv" event={"ID":"5c0069c7-1e3a-498a-af64-1b8ac9e53e39","Type":"ContainerStarted","Data":"b367502302a9dc667771339bb2f9ee125e5e6dbd37d9325ac697776fc83d4bab"} Jul 07 23:45:53.100635 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:45:53.100586 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-a-04f579/test-model-04f579-kserve-57d56ffb8c-8jglv" podStartSLOduration=1.10057204 podStartE2EDuration="1.10057204s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 23:45:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 23:45:53.099193345 +0000 UTC m=+1970.496194858" watchObservedRunningTime="2026-07-07 23:45:53.10057204 +0000 UTC m=+1970.497573593" Jul 07 23:45:56.019529 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:45:56.019425 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hs9gr"] Jul 07 23:46:02.561053 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:46:02.561014 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-a-04f579/test-model-04f579-kserve-57d56ffb8c-8jglv" Jul 07 23:46:02.561053 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:46:02.561066 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-a-04f579/test-model-04f579-kserve-57d56ffb8c-8jglv" Jul 07 23:46:02.573824 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:46:02.573796 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-a-04f579/test-model-04f579-kserve-57d56ffb8c-8jglv" Jul 07 23:46:03.145666 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:46:03.145638 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-a-04f579/test-model-04f579-kserve-57d56ffb8c-8jglv" Jul 07 23:46:06.930212 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:46:06.930179 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hs9gr"] Jul 07 23:46:17.638328 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:46:17.638287 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-b-c324fe/test-model-c324fe-kserve-776955fc6c-vsz7d"] Jul 07 23:46:17.641803 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:46:17.641787 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-c324fe/test-model-c324fe-kserve-776955fc6c-vsz7d" Jul 07 23:46:17.644946 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:46:17.644922 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-c324fe\"/\"kube-root-ca.crt\"" Jul 07 23:46:17.645088 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:46:17.644978 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-c324fe\"/\"openshift-service-ca.crt\"" Jul 07 23:46:17.645088 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:46:17.644930 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-c324fe\"/\"test-model-c324fe-kserve-self-signed-certs\"" Jul 07 23:46:17.645088 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:46:17.645042 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-c324fe\"/\"default-dockercfg-fqnnj\"" Jul 07 23:46:17.649866 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:46:17.649846 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-c324fe/test-model-c324fe-kserve-776955fc6c-vsz7d"] Jul 07 23:46:17.671924 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:46:17.671897 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ea42ca8c-ddd2-465d-aa2b-28eb480dfd7a-dshm\") pod \"test-model-c324fe-kserve-776955fc6c-vsz7d\" (UID: \"ea42ca8c-ddd2-465d-aa2b-28eb480dfd7a\") " pod="ai-tenant-e2e-inf-b-c324fe/test-model-c324fe-kserve-776955fc6c-vsz7d" Jul 07 23:46:17.672047 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:46:17.671928 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ea42ca8c-ddd2-465d-aa2b-28eb480dfd7a-tmp-dir\") pod \"test-model-c324fe-kserve-776955fc6c-vsz7d\" (UID: \"ea42ca8c-ddd2-465d-aa2b-28eb480dfd7a\") " pod="ai-tenant-e2e-inf-b-c324fe/test-model-c324fe-kserve-776955fc6c-vsz7d" Jul 07 23:46:17.672047 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:46:17.671954 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jc6dd\" (UniqueName: \"kubernetes.io/projected/ea42ca8c-ddd2-465d-aa2b-28eb480dfd7a-kube-api-access-jc6dd\") pod \"test-model-c324fe-kserve-776955fc6c-vsz7d\" (UID: \"ea42ca8c-ddd2-465d-aa2b-28eb480dfd7a\") " pod="ai-tenant-e2e-inf-b-c324fe/test-model-c324fe-kserve-776955fc6c-vsz7d" Jul 07 23:46:17.672047 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:46:17.672017 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ea42ca8c-ddd2-465d-aa2b-28eb480dfd7a-model-cache\") pod \"test-model-c324fe-kserve-776955fc6c-vsz7d\" (UID: \"ea42ca8c-ddd2-465d-aa2b-28eb480dfd7a\") " pod="ai-tenant-e2e-inf-b-c324fe/test-model-c324fe-kserve-776955fc6c-vsz7d" Jul 07 23:46:17.672047 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:46:17.672040 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ea42ca8c-ddd2-465d-aa2b-28eb480dfd7a-tls-certs\") pod \"test-model-c324fe-kserve-776955fc6c-vsz7d\" (UID: \"ea42ca8c-ddd2-465d-aa2b-28eb480dfd7a\") " pod="ai-tenant-e2e-inf-b-c324fe/test-model-c324fe-kserve-776955fc6c-vsz7d" Jul 07 23:46:17.672209 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:46:17.672061 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ea42ca8c-ddd2-465d-aa2b-28eb480dfd7a-home\") pod \"test-model-c324fe-kserve-776955fc6c-vsz7d\" (UID: \"ea42ca8c-ddd2-465d-aa2b-28eb480dfd7a\") " pod="ai-tenant-e2e-inf-b-c324fe/test-model-c324fe-kserve-776955fc6c-vsz7d" Jul 07 23:46:17.734581 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:46:17.734543 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:46:17.772622 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:46:17.772594 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ea42ca8c-ddd2-465d-aa2b-28eb480dfd7a-model-cache\") pod \"test-model-c324fe-kserve-776955fc6c-vsz7d\" (UID: \"ea42ca8c-ddd2-465d-aa2b-28eb480dfd7a\") " pod="ai-tenant-e2e-inf-b-c324fe/test-model-c324fe-kserve-776955fc6c-vsz7d" Jul 07 23:46:17.772622 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:46:17.772625 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ea42ca8c-ddd2-465d-aa2b-28eb480dfd7a-tls-certs\") pod \"test-model-c324fe-kserve-776955fc6c-vsz7d\" (UID: \"ea42ca8c-ddd2-465d-aa2b-28eb480dfd7a\") " pod="ai-tenant-e2e-inf-b-c324fe/test-model-c324fe-kserve-776955fc6c-vsz7d" Jul 07 23:46:17.772832 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:46:17.772649 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ea42ca8c-ddd2-465d-aa2b-28eb480dfd7a-home\") pod \"test-model-c324fe-kserve-776955fc6c-vsz7d\" (UID: \"ea42ca8c-ddd2-465d-aa2b-28eb480dfd7a\") " pod="ai-tenant-e2e-inf-b-c324fe/test-model-c324fe-kserve-776955fc6c-vsz7d" Jul 07 23:46:17.772832 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:46:17.772684 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ea42ca8c-ddd2-465d-aa2b-28eb480dfd7a-dshm\") pod \"test-model-c324fe-kserve-776955fc6c-vsz7d\" (UID: \"ea42ca8c-ddd2-465d-aa2b-28eb480dfd7a\") " pod="ai-tenant-e2e-inf-b-c324fe/test-model-c324fe-kserve-776955fc6c-vsz7d" Jul 07 23:46:17.772832 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:46:17.772702 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ea42ca8c-ddd2-465d-aa2b-28eb480dfd7a-tmp-dir\") pod \"test-model-c324fe-kserve-776955fc6c-vsz7d\" (UID: \"ea42ca8c-ddd2-465d-aa2b-28eb480dfd7a\") " pod="ai-tenant-e2e-inf-b-c324fe/test-model-c324fe-kserve-776955fc6c-vsz7d" Jul 07 23:46:17.772832 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:46:17.772719 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jc6dd\" (UniqueName: \"kubernetes.io/projected/ea42ca8c-ddd2-465d-aa2b-28eb480dfd7a-kube-api-access-jc6dd\") pod \"test-model-c324fe-kserve-776955fc6c-vsz7d\" (UID: \"ea42ca8c-ddd2-465d-aa2b-28eb480dfd7a\") " pod="ai-tenant-e2e-inf-b-c324fe/test-model-c324fe-kserve-776955fc6c-vsz7d" Jul 07 23:46:17.773043 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:46:17.773011 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ea42ca8c-ddd2-465d-aa2b-28eb480dfd7a-model-cache\") pod \"test-model-c324fe-kserve-776955fc6c-vsz7d\" (UID: \"ea42ca8c-ddd2-465d-aa2b-28eb480dfd7a\") " pod="ai-tenant-e2e-inf-b-c324fe/test-model-c324fe-kserve-776955fc6c-vsz7d" Jul 07 23:46:17.773119 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:46:17.773094 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ea42ca8c-ddd2-465d-aa2b-28eb480dfd7a-tmp-dir\") pod \"test-model-c324fe-kserve-776955fc6c-vsz7d\" (UID: \"ea42ca8c-ddd2-465d-aa2b-28eb480dfd7a\") " pod="ai-tenant-e2e-inf-b-c324fe/test-model-c324fe-kserve-776955fc6c-vsz7d" Jul 07 23:46:17.773202 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:46:17.773180 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ea42ca8c-ddd2-465d-aa2b-28eb480dfd7a-home\") pod \"test-model-c324fe-kserve-776955fc6c-vsz7d\" (UID: \"ea42ca8c-ddd2-465d-aa2b-28eb480dfd7a\") " pod="ai-tenant-e2e-inf-b-c324fe/test-model-c324fe-kserve-776955fc6c-vsz7d" Jul 07 23:46:17.774926 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:46:17.774905 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ea42ca8c-ddd2-465d-aa2b-28eb480dfd7a-dshm\") pod \"test-model-c324fe-kserve-776955fc6c-vsz7d\" (UID: \"ea42ca8c-ddd2-465d-aa2b-28eb480dfd7a\") " pod="ai-tenant-e2e-inf-b-c324fe/test-model-c324fe-kserve-776955fc6c-vsz7d" Jul 07 23:46:17.775185 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:46:17.775168 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ea42ca8c-ddd2-465d-aa2b-28eb480dfd7a-tls-certs\") pod \"test-model-c324fe-kserve-776955fc6c-vsz7d\" (UID: \"ea42ca8c-ddd2-465d-aa2b-28eb480dfd7a\") " pod="ai-tenant-e2e-inf-b-c324fe/test-model-c324fe-kserve-776955fc6c-vsz7d" Jul 07 23:46:17.782762 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:46:17.782740 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jc6dd\" (UniqueName: \"kubernetes.io/projected/ea42ca8c-ddd2-465d-aa2b-28eb480dfd7a-kube-api-access-jc6dd\") pod \"test-model-c324fe-kserve-776955fc6c-vsz7d\" (UID: \"ea42ca8c-ddd2-465d-aa2b-28eb480dfd7a\") " pod="ai-tenant-e2e-inf-b-c324fe/test-model-c324fe-kserve-776955fc6c-vsz7d" Jul 07 23:46:17.951803 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:46:17.951711 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-c324fe/test-model-c324fe-kserve-776955fc6c-vsz7d" Jul 07 23:46:18.083389 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:46:18.083362 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-c324fe/test-model-c324fe-kserve-776955fc6c-vsz7d"] Jul 07 23:46:18.085549 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:46:18.085524 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podea42ca8c_ddd2_465d_aa2b_28eb480dfd7a.slice/crio-b2bdf22212ce4275affbbe00158f06118b9cad3f1210e522fc426a2ce5010458 WatchSource:0}: Error finding container b2bdf22212ce4275affbbe00158f06118b9cad3f1210e522fc426a2ce5010458: Status 404 returned error can't find the container with id b2bdf22212ce4275affbbe00158f06118b9cad3f1210e522fc426a2ce5010458 Jul 07 23:46:18.211558 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:46:18.211456 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-c324fe/test-model-c324fe-kserve-776955fc6c-vsz7d" event={"ID":"ea42ca8c-ddd2-465d-aa2b-28eb480dfd7a","Type":"ContainerStarted","Data":"63a23c30ad73f5cf7a6af1a4969cb33c0885d9b61032af5e9a647ebd09fde305"} Jul 07 23:46:18.211558 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:46:18.211516 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-c324fe/test-model-c324fe-kserve-776955fc6c-vsz7d" event={"ID":"ea42ca8c-ddd2-465d-aa2b-28eb480dfd7a","Type":"ContainerStarted","Data":"b2bdf22212ce4275affbbe00158f06118b9cad3f1210e522fc426a2ce5010458"} Jul 07 23:46:18.228107 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:46:18.228026 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-b-c324fe/test-model-c324fe-kserve-776955fc6c-vsz7d" podStartSLOduration=1.227993566 podStartE2EDuration="1.227993566s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 23:46:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 23:46:18.227206678 +0000 UTC m=+1995.624208190" watchObservedRunningTime="2026-07-07 23:46:18.227993566 +0000 UTC m=+1995.624995079" Jul 07 23:46:22.623800 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:46:22.623759 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hs9gr"] Jul 07 23:46:27.952347 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:46:27.952310 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-b-c324fe/test-model-c324fe-kserve-776955fc6c-vsz7d" Jul 07 23:46:27.952347 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:46:27.952351 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-b-c324fe/test-model-c324fe-kserve-776955fc6c-vsz7d" Jul 07 23:46:27.964696 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:46:27.964670 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-b-c324fe/test-model-c324fe-kserve-776955fc6c-vsz7d" Jul 07 23:46:28.280946 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:46:28.280918 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-b-c324fe/test-model-c324fe-kserve-776955fc6c-vsz7d" Jul 07 23:46:36.228255 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:46:36.228221 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hs9gr"] Jul 07 23:46:37.228486 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:46:37.228439 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-04f579/test-model-04f579-kserve-57d56ffb8c-8jglv"] Jul 07 23:46:37.249368 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:46:37.249334 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:46:37.665996 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:46:37.665963 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:46:38.230583 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:46:38.230545 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-c324fe/test-model-c324fe-kserve-776955fc6c-vsz7d"] Jul 07 23:46:38.313317 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:46:38.313287 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:46:38.456555 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:46:38.456523 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:46:38.866124 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:46:38.866090 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:46:38.966555 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:46:38.966526 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:46:39.066742 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:46:39.066705 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:46:39.197950 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:46:39.197870 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:46:43.942195 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:46:43.942162 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:46:51.723287 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:46:51.723247 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hs9gr"] Jul 07 23:46:52.250938 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:46:52.250892 2570 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-inf-a-04f579/test-model-04f579-kserve-57d56ffb8c-8jglv" podUID="5c0069c7-1e3a-498a-af64-1b8ac9e53e39" containerName="main" containerID="cri-o://9e43c2cbb21874c5573b72bb4a6deeee763a8845b1993a7c71bb0cb8ea361581" gracePeriod=45 Jul 07 23:46:52.396535 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:46:52.396495 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-04f579/test-model-04f579-kserve-57d56ffb8c-8jglv" event={"ID":"5c0069c7-1e3a-498a-af64-1b8ac9e53e39","Type":"ContainerDied","Data":"9e43c2cbb21874c5573b72bb4a6deeee763a8845b1993a7c71bb0cb8ea361581"} Jul 07 23:46:52.396729 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:46:52.396457 2570 generic.go:356] "Generic (PLEG): container finished" podID="5c0069c7-1e3a-498a-af64-1b8ac9e53e39" containerID="9e43c2cbb21874c5573b72bb4a6deeee763a8845b1993a7c71bb0cb8ea361581" exitCode=0 Jul 07 23:46:52.495975 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:46:52.495952 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-04f579/test-model-04f579-kserve-57d56ffb8c-8jglv" Jul 07 23:46:52.589397 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:46:52.589370 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5c0069c7-1e3a-498a-af64-1b8ac9e53e39-dshm\") pod \"5c0069c7-1e3a-498a-af64-1b8ac9e53e39\" (UID: \"5c0069c7-1e3a-498a-af64-1b8ac9e53e39\") " Jul 07 23:46:52.589594 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:46:52.589404 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-psftt\" (UniqueName: \"kubernetes.io/projected/5c0069c7-1e3a-498a-af64-1b8ac9e53e39-kube-api-access-psftt\") pod \"5c0069c7-1e3a-498a-af64-1b8ac9e53e39\" (UID: \"5c0069c7-1e3a-498a-af64-1b8ac9e53e39\") " Jul 07 23:46:52.589594 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:46:52.589433 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5c0069c7-1e3a-498a-af64-1b8ac9e53e39-home\") pod \"5c0069c7-1e3a-498a-af64-1b8ac9e53e39\" (UID: \"5c0069c7-1e3a-498a-af64-1b8ac9e53e39\") " Jul 07 23:46:52.589594 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:46:52.589506 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5c0069c7-1e3a-498a-af64-1b8ac9e53e39-model-cache\") pod \"5c0069c7-1e3a-498a-af64-1b8ac9e53e39\" (UID: \"5c0069c7-1e3a-498a-af64-1b8ac9e53e39\") " Jul 07 23:46:52.589594 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:46:52.589545 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5c0069c7-1e3a-498a-af64-1b8ac9e53e39-tmp-dir\") pod \"5c0069c7-1e3a-498a-af64-1b8ac9e53e39\" (UID: \"5c0069c7-1e3a-498a-af64-1b8ac9e53e39\") " Jul 07 23:46:52.589594 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:46:52.589562 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5c0069c7-1e3a-498a-af64-1b8ac9e53e39-tls-certs\") pod \"5c0069c7-1e3a-498a-af64-1b8ac9e53e39\" (UID: \"5c0069c7-1e3a-498a-af64-1b8ac9e53e39\") " Jul 07 23:46:52.589865 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:46:52.589765 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5c0069c7-1e3a-498a-af64-1b8ac9e53e39-home" (OuterVolumeSpecName: "home") pod "5c0069c7-1e3a-498a-af64-1b8ac9e53e39" (UID: "5c0069c7-1e3a-498a-af64-1b8ac9e53e39"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 23:46:52.589865 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:46:52.589829 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5c0069c7-1e3a-498a-af64-1b8ac9e53e39-model-cache" (OuterVolumeSpecName: "model-cache") pod "5c0069c7-1e3a-498a-af64-1b8ac9e53e39" (UID: "5c0069c7-1e3a-498a-af64-1b8ac9e53e39"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 23:46:52.589865 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:46:52.589823 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5c0069c7-1e3a-498a-af64-1b8ac9e53e39-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "5c0069c7-1e3a-498a-af64-1b8ac9e53e39" (UID: "5c0069c7-1e3a-498a-af64-1b8ac9e53e39"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 23:46:52.591685 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:46:52.591663 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5c0069c7-1e3a-498a-af64-1b8ac9e53e39-dshm" (OuterVolumeSpecName: "dshm") pod "5c0069c7-1e3a-498a-af64-1b8ac9e53e39" (UID: "5c0069c7-1e3a-498a-af64-1b8ac9e53e39"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 23:46:52.591801 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:46:52.591689 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5c0069c7-1e3a-498a-af64-1b8ac9e53e39-kube-api-access-psftt" (OuterVolumeSpecName: "kube-api-access-psftt") pod "5c0069c7-1e3a-498a-af64-1b8ac9e53e39" (UID: "5c0069c7-1e3a-498a-af64-1b8ac9e53e39"). InnerVolumeSpecName "kube-api-access-psftt". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 23:46:52.591801 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:46:52.591702 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5c0069c7-1e3a-498a-af64-1b8ac9e53e39-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "5c0069c7-1e3a-498a-af64-1b8ac9e53e39" (UID: "5c0069c7-1e3a-498a-af64-1b8ac9e53e39"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 23:46:52.690982 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:46:52.690951 2570 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5c0069c7-1e3a-498a-af64-1b8ac9e53e39-dshm\") on node \"ip-10-0-134-140.ec2.internal\" DevicePath \"\"" Jul 07 23:46:52.690982 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:46:52.690980 2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-psftt\" (UniqueName: \"kubernetes.io/projected/5c0069c7-1e3a-498a-af64-1b8ac9e53e39-kube-api-access-psftt\") on node \"ip-10-0-134-140.ec2.internal\" DevicePath \"\"" Jul 07 23:46:52.691184 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:46:52.690994 2570 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5c0069c7-1e3a-498a-af64-1b8ac9e53e39-home\") on node \"ip-10-0-134-140.ec2.internal\" DevicePath \"\"" Jul 07 23:46:52.691184 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:46:52.691006 2570 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5c0069c7-1e3a-498a-af64-1b8ac9e53e39-model-cache\") on node \"ip-10-0-134-140.ec2.internal\" DevicePath \"\"" Jul 07 23:46:52.691184 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:46:52.691018 2570 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5c0069c7-1e3a-498a-af64-1b8ac9e53e39-tmp-dir\") on node \"ip-10-0-134-140.ec2.internal\" DevicePath \"\"" Jul 07 23:46:52.691184 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:46:52.691029 2570 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5c0069c7-1e3a-498a-af64-1b8ac9e53e39-tls-certs\") on node \"ip-10-0-134-140.ec2.internal\" DevicePath \"\"" Jul 07 23:46:53.250610 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:46:53.250574 2570 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-inf-b-c324fe/test-model-c324fe-kserve-776955fc6c-vsz7d" podUID="ea42ca8c-ddd2-465d-aa2b-28eb480dfd7a" containerName="main" containerID="cri-o://63a23c30ad73f5cf7a6af1a4969cb33c0885d9b61032af5e9a647ebd09fde305" gracePeriod=45 Jul 07 23:46:53.282303 ip-10-0-134-140 kubenswrapper[2570]: E0707 23:46:53.282273 2570 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podea42ca8c_ddd2_465d_aa2b_28eb480dfd7a.slice/crio-conmon-63a23c30ad73f5cf7a6af1a4969cb33c0885d9b61032af5e9a647ebd09fde305.scope\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podea42ca8c_ddd2_465d_aa2b_28eb480dfd7a.slice/crio-63a23c30ad73f5cf7a6af1a4969cb33c0885d9b61032af5e9a647ebd09fde305.scope\": RecentStats: unable to find data in memory cache]" Jul 07 23:46:53.403329 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:46:53.403296 2570 generic.go:356] "Generic (PLEG): container finished" podID="ea42ca8c-ddd2-465d-aa2b-28eb480dfd7a" containerID="63a23c30ad73f5cf7a6af1a4969cb33c0885d9b61032af5e9a647ebd09fde305" exitCode=0 Jul 07 23:46:53.403523 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:46:53.403389 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-c324fe/test-model-c324fe-kserve-776955fc6c-vsz7d" event={"ID":"ea42ca8c-ddd2-465d-aa2b-28eb480dfd7a","Type":"ContainerDied","Data":"63a23c30ad73f5cf7a6af1a4969cb33c0885d9b61032af5e9a647ebd09fde305"} Jul 07 23:46:53.404864 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:46:53.404837 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-04f579/test-model-04f579-kserve-57d56ffb8c-8jglv" Jul 07 23:46:53.404864 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:46:53.404847 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-04f579/test-model-04f579-kserve-57d56ffb8c-8jglv" event={"ID":"5c0069c7-1e3a-498a-af64-1b8ac9e53e39","Type":"ContainerDied","Data":"b367502302a9dc667771339bb2f9ee125e5e6dbd37d9325ac697776fc83d4bab"} Jul 07 23:46:53.405038 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:46:53.404895 2570 scope.go:117] "RemoveContainer" containerID="9e43c2cbb21874c5573b72bb4a6deeee763a8845b1993a7c71bb0cb8ea361581" Jul 07 23:46:53.423970 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:46:53.423898 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-04f579/test-model-04f579-kserve-57d56ffb8c-8jglv"] Jul 07 23:46:53.426674 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:46:53.426653 2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-a-04f579/test-model-04f579-kserve-57d56ffb8c-8jglv"] Jul 07 23:46:53.504685 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:46:53.504656 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-c324fe/test-model-c324fe-kserve-776955fc6c-vsz7d" Jul 07 23:46:53.596964 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:46:53.596940 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ea42ca8c-ddd2-465d-aa2b-28eb480dfd7a-dshm\") pod \"ea42ca8c-ddd2-465d-aa2b-28eb480dfd7a\" (UID: \"ea42ca8c-ddd2-465d-aa2b-28eb480dfd7a\") " Jul 07 23:46:53.596964 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:46:53.596967 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ea42ca8c-ddd2-465d-aa2b-28eb480dfd7a-home\") pod \"ea42ca8c-ddd2-465d-aa2b-28eb480dfd7a\" (UID: \"ea42ca8c-ddd2-465d-aa2b-28eb480dfd7a\") " Jul 07 23:46:53.597182 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:46:53.596990 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ea42ca8c-ddd2-465d-aa2b-28eb480dfd7a-tls-certs\") pod \"ea42ca8c-ddd2-465d-aa2b-28eb480dfd7a\" (UID: \"ea42ca8c-ddd2-465d-aa2b-28eb480dfd7a\") " Jul 07 23:46:53.597182 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:46:53.597023 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jc6dd\" (UniqueName: \"kubernetes.io/projected/ea42ca8c-ddd2-465d-aa2b-28eb480dfd7a-kube-api-access-jc6dd\") pod \"ea42ca8c-ddd2-465d-aa2b-28eb480dfd7a\" (UID: \"ea42ca8c-ddd2-465d-aa2b-28eb480dfd7a\") " Jul 07 23:46:53.597182 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:46:53.597066 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ea42ca8c-ddd2-465d-aa2b-28eb480dfd7a-tmp-dir\") pod \"ea42ca8c-ddd2-465d-aa2b-28eb480dfd7a\" (UID: \"ea42ca8c-ddd2-465d-aa2b-28eb480dfd7a\") " Jul 07 23:46:53.597182 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:46:53.597147 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ea42ca8c-ddd2-465d-aa2b-28eb480dfd7a-model-cache\") pod \"ea42ca8c-ddd2-465d-aa2b-28eb480dfd7a\" (UID: \"ea42ca8c-ddd2-465d-aa2b-28eb480dfd7a\") " Jul 07 23:46:53.597406 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:46:53.597279 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ea42ca8c-ddd2-465d-aa2b-28eb480dfd7a-home" (OuterVolumeSpecName: "home") pod "ea42ca8c-ddd2-465d-aa2b-28eb480dfd7a" (UID: "ea42ca8c-ddd2-465d-aa2b-28eb480dfd7a"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 23:46:53.597406 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:46:53.597394 2570 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ea42ca8c-ddd2-465d-aa2b-28eb480dfd7a-home\") on node \"ip-10-0-134-140.ec2.internal\" DevicePath \"\"" Jul 07 23:46:53.597573 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:46:53.597552 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ea42ca8c-ddd2-465d-aa2b-28eb480dfd7a-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "ea42ca8c-ddd2-465d-aa2b-28eb480dfd7a" (UID: "ea42ca8c-ddd2-465d-aa2b-28eb480dfd7a"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 23:46:53.597617 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:46:53.597593 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ea42ca8c-ddd2-465d-aa2b-28eb480dfd7a-model-cache" (OuterVolumeSpecName: "model-cache") pod "ea42ca8c-ddd2-465d-aa2b-28eb480dfd7a" (UID: "ea42ca8c-ddd2-465d-aa2b-28eb480dfd7a"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 23:46:53.599321 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:46:53.599286 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ea42ca8c-ddd2-465d-aa2b-28eb480dfd7a-dshm" (OuterVolumeSpecName: "dshm") pod "ea42ca8c-ddd2-465d-aa2b-28eb480dfd7a" (UID: "ea42ca8c-ddd2-465d-aa2b-28eb480dfd7a"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 23:46:53.599418 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:46:53.599318 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ea42ca8c-ddd2-465d-aa2b-28eb480dfd7a-kube-api-access-jc6dd" (OuterVolumeSpecName: "kube-api-access-jc6dd") pod "ea42ca8c-ddd2-465d-aa2b-28eb480dfd7a" (UID: "ea42ca8c-ddd2-465d-aa2b-28eb480dfd7a"). InnerVolumeSpecName "kube-api-access-jc6dd". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 23:46:53.599418 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:46:53.599403 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ea42ca8c-ddd2-465d-aa2b-28eb480dfd7a-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "ea42ca8c-ddd2-465d-aa2b-28eb480dfd7a" (UID: "ea42ca8c-ddd2-465d-aa2b-28eb480dfd7a"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 23:46:53.698628 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:46:53.698555 2570 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ea42ca8c-ddd2-465d-aa2b-28eb480dfd7a-dshm\") on node \"ip-10-0-134-140.ec2.internal\" DevicePath \"\"" Jul 07 23:46:53.698628 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:46:53.698581 2570 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ea42ca8c-ddd2-465d-aa2b-28eb480dfd7a-tls-certs\") on node \"ip-10-0-134-140.ec2.internal\" DevicePath \"\"" Jul 07 23:46:53.698628 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:46:53.698592 2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jc6dd\" (UniqueName: \"kubernetes.io/projected/ea42ca8c-ddd2-465d-aa2b-28eb480dfd7a-kube-api-access-jc6dd\") on node \"ip-10-0-134-140.ec2.internal\" DevicePath \"\"" Jul 07 23:46:53.698628 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:46:53.698622 2570 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ea42ca8c-ddd2-465d-aa2b-28eb480dfd7a-tmp-dir\") on node \"ip-10-0-134-140.ec2.internal\" DevicePath \"\"" Jul 07 23:46:53.698628 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:46:53.698631 2570 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ea42ca8c-ddd2-465d-aa2b-28eb480dfd7a-model-cache\") on node \"ip-10-0-134-140.ec2.internal\" DevicePath \"\"" Jul 07 23:46:54.411860 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:46:54.411824 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-c324fe/test-model-c324fe-kserve-776955fc6c-vsz7d" Jul 07 23:46:54.411860 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:46:54.411844 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-c324fe/test-model-c324fe-kserve-776955fc6c-vsz7d" event={"ID":"ea42ca8c-ddd2-465d-aa2b-28eb480dfd7a","Type":"ContainerDied","Data":"b2bdf22212ce4275affbbe00158f06118b9cad3f1210e522fc426a2ce5010458"} Jul 07 23:46:54.412393 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:46:54.411924 2570 scope.go:117] "RemoveContainer" containerID="63a23c30ad73f5cf7a6af1a4969cb33c0885d9b61032af5e9a647ebd09fde305" Jul 07 23:46:54.434478 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:46:54.434434 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-c324fe/test-model-c324fe-kserve-776955fc6c-vsz7d"] Jul 07 23:46:54.438015 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:46:54.437988 2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-b-c324fe/test-model-c324fe-kserve-776955fc6c-vsz7d"] Jul 07 23:46:55.097201 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:46:55.097169 2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5c0069c7-1e3a-498a-af64-1b8ac9e53e39" path="/var/lib/kubelet/pods/5c0069c7-1e3a-498a-af64-1b8ac9e53e39/volumes" Jul 07 23:46:55.097609 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:46:55.097596 2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ea42ca8c-ddd2-465d-aa2b-28eb480dfd7a" path="/var/lib/kubelet/pods/ea42ca8c-ddd2-465d-aa2b-28eb480dfd7a/volumes" Jul 07 23:47:20.645459 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:47:20.645418 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:47:20.927099 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:47:20.927014 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:47:21.336973 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:47:21.336940 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:47:21.754538 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:47:21.754438 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:47:21.855613 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:47:21.855580 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:47:21.956174 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:47:21.956135 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:47:22.123427 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:47:22.123396 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:47:25.547397 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:47:25.547311 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hs9gr"] Jul 07 23:47:28.191595 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:47:28.191560 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:47:28.256384 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:47:28.256352 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:47:28.456128 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:47:28.456049 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:47:29.049878 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:47:29.049839 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:47:29.358439 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:47:29.358329 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:47:29.841945 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:47:29.841910 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:47:31.825060 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:47:31.825027 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:47:31.917571 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:47:31.917540 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:47:32.034514 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:47:32.034483 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:47:32.419762 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:47:32.419727 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:47:32.521873 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:47:32.521843 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:47:32.620110 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:47:32.620075 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:47:36.307402 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:47:36.307365 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:47:36.758078 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:47:36.757969 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:47:37.051021 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:47:37.050986 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:47:37.147871 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:47:37.147834 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:47:39.883366 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:47:39.883317 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:47:39.964996 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:47:39.964958 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:47:40.031811 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:47:40.031754 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:47:40.052857 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:47:40.052811 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hs9gr"] Jul 07 23:47:40.444387 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:47:40.444348 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:47:40.540624 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:47:40.540575 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:47:40.641033 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:47:40.640990 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:47:44.408569 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:47:44.408534 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:47:49.481928 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:47:49.481889 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:47:49.584873 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:47:49.584835 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:47:52.503653 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:47:52.503620 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:47:52.889963 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:47:52.889929 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:47:53.017070 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:47:53.017034 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:47:53.119672 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:47:53.119643 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:47:53.296145 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:47:53.296116 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:47:56.225137 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:47:56.225107 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hs9gr"] Jul 07 23:47:58.861161 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:47:58.861111 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:47:58.916768 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:47:58.916732 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:47:59.133864 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:47:59.133781 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:47:59.561675 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:47:59.561622 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:47:59.993001 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:47:59.992901 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:48:00.183055 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:48:00.183021 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:48:00.281940 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:48:00.281909 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:48:00.516269 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:48:00.516230 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:48:03.232517 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:48:03.232485 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-8hvxr_faae2bd8-cbbd-4cfb-bce6-7954977b7ebc/console-operator/2.log" Jul 07 23:48:03.235685 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:48:03.235664 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-8hvxr_faae2bd8-cbbd-4cfb-bce6-7954977b7ebc/console-operator/2.log" Jul 07 23:48:06.155392 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:48:06.155366 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:48:06.214233 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:48:06.214196 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:48:06.426700 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:48:06.426611 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:48:06.985342 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:48:06.985303 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:48:07.293476 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:48:07.293443 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:48:07.933947 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:48:07.933891 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:48:09.323073 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:48:09.323033 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hs9gr"] Jul 07 23:48:10.014925 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:48:10.014868 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:48:10.090873 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:48:10.090832 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:48:10.146432 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:48:10.146394 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:48:10.352359 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:48:10.352320 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:48:10.676553 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:48:10.676446 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:48:10.778020 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:48:10.777993 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:48:14.501533 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:48:14.501499 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:48:14.850159 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:48:14.850120 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:48:15.141055 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:48:15.140966 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:48:15.340861 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:48:15.340821 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:48:18.094282 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:48:18.094235 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:48:18.210959 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:48:18.210927 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:48:18.271922 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:48:18.271886 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:48:18.528757 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:48:18.528722 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:48:18.771175 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:48:18.771136 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:48:18.871244 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:48:18.871160 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:48:19.635507 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:48:19.635474 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hs9gr"] Jul 07 23:48:22.582291 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:48:22.582256 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:48:28.048277 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:48:28.048240 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hs9gr"] Jul 07 23:48:31.364949 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:48:31.364914 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:48:31.536760 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:48:31.536722 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:48:39.593717 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:48:39.593680 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:48:39.694370 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:48:39.694334 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:50:09.490360 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:50:09.490297 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:51:07.908224 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:51:07.908189 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:51:25.125603 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:51:25.125565 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:51:27.508312 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:51:27.508278 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:51:27.921978 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:51:27.921942 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:51:28.030858 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:51:28.030823 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:51:28.122311 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:51:28.122282 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:51:28.208057 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:51:28.207979 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:51:31.528953 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:51:31.528921 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hs9gr"] Jul 07 23:51:33.676061 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:51:33.676028 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:51:33.739106 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:51:33.739076 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:51:33.939003 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:51:33.938916 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:51:34.306737 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:51:34.306697 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:51:34.708873 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:51:34.708789 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:51:34.818216 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:51:34.818170 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:51:34.917143 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:51:34.917114 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:51:35.080918 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:51:35.080883 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:51:40.622061 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:51:40.622026 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:51:40.721090 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:51:40.721059 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:51:40.902144 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:51:40.902054 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:51:41.695024 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:51:41.694994 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-865bd97974-r7bjs_46d1cd8e-d71e-4aef-bd0e-9914a638a0a1/authorino/0.log" Jul 07 23:51:43.222925 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:51:43.222892 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-hs9gr"] Jul 07 23:51:46.856818 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:51:46.856786 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7c9859966d-7bxpt_33b45f14-3cc1-4712-bf73-206fa4041802/copy-manifests/0.log" Jul 07 23:51:46.869070 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:51:46.869044 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7c9859966d-7bxpt_33b45f14-3cc1-4712-bf73-206fa4041802/manager/0.log" Jul 07 23:51:47.901767 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:51:47.901734 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759wgjps_8ef9793b-9b49-43a9-9b83-cc09c9002479/pull/0.log" Jul 07 23:51:47.908963 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:51:47.908942 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759wgjps_8ef9793b-9b49-43a9-9b83-cc09c9002479/extract/0.log" Jul 07 23:51:47.915846 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:51:47.915832 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759wgjps_8ef9793b-9b49-43a9-9b83-cc09c9002479/util/0.log" Jul 07 23:51:48.025381 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:51:48.025354 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e067bwg_ae1ba717-f216-48ff-9789-43126b0a9c1f/extract/0.log" Jul 07 23:51:48.031748 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:51:48.031727 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e067bwg_ae1ba717-f216-48ff-9789-43126b0a9c1f/util/0.log" Jul 07 23:51:48.038641 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:51:48.038617 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e067bwg_ae1ba717-f216-48ff-9789-43126b0a9c1f/pull/0.log" Jul 07 23:51:48.147310 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:51:48.147285 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed735hg4d_113878d9-e1e4-4af3-aed1-cdcd089443c4/util/0.log" Jul 07 23:51:48.154478 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:51:48.154406 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed735hg4d_113878d9-e1e4-4af3-aed1-cdcd089443c4/pull/0.log" Jul 07 23:51:48.161374 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:51:48.161352 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed735hg4d_113878d9-e1e4-4af3-aed1-cdcd089443c4/extract/0.log" Jul 07 23:51:48.289096 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:51:48.289068 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1stzl9_807f17fe-22a0-4a3b-82af-b242ca8e2013/util/0.log" Jul 07 23:51:48.307165 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:51:48.307138 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1stzl9_807f17fe-22a0-4a3b-82af-b242ca8e2013/pull/0.log" Jul 07 23:51:48.322355 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:51:48.322330 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1stzl9_807f17fe-22a0-4a3b-82af-b242ca8e2013/extract/0.log" Jul 07 23:51:48.444335 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:51:48.444257 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-865bd97974-r7bjs_46d1cd8e-d71e-4aef-bd0e-9914a638a0a1/authorino/0.log" Jul 07 23:51:48.789889 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:51:48.789862 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-8ptdh_cbc3faff-e9b2-48ad-a3c4-c0d77ec1af7a/kuadrant-console-plugin/0.log" Jul 07 23:51:49.144849 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:51:49.144763 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-hs9gr_7798a566-a702-48eb-a51d-da784da4a2f2/limitador/0.log" Jul 07 23:51:51.013858 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:51:51.013789 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7b7668c465-j5zks_5157b9da-e2ac-44a1-8ab2-c5f3c787e896/kube-auth-proxy/0.log" Jul 07 23:51:51.521116 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:51:51.521079 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-65d5dfb6cf-84llt_40bbfb8b-2c2b-43c5-8419-3dba3777f888/router/0.log" Jul 07 23:51:52.419809 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:51:52.419783 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-5fc6fb456f-n9hpc_7c3e7632-a72a-45bb-bb5c-0de9c48911d2/main/0.log" Jul 07 23:51:56.859599 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:51:56.859520 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-dx64r/must-gather-v4c6l"] Jul 07 23:51:56.859963 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:51:56.859879 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ea42ca8c-ddd2-465d-aa2b-28eb480dfd7a" containerName="main" Jul 07 23:51:56.859963 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:51:56.859890 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="ea42ca8c-ddd2-465d-aa2b-28eb480dfd7a" containerName="main" Jul 07 23:51:56.859963 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:51:56.859904 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5c0069c7-1e3a-498a-af64-1b8ac9e53e39" containerName="main" Jul 07 23:51:56.859963 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:51:56.859909 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="5c0069c7-1e3a-498a-af64-1b8ac9e53e39" containerName="main" Jul 07 23:51:56.860098 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:51:56.859973 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="5c0069c7-1e3a-498a-af64-1b8ac9e53e39" containerName="main" Jul 07 23:51:56.860098 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:51:56.859984 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="ea42ca8c-ddd2-465d-aa2b-28eb480dfd7a" containerName="main" Jul 07 23:51:56.863281 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:51:56.863264 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-dx64r/must-gather-v4c6l" Jul 07 23:51:56.865586 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:51:56.865559 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-dx64r\"/\"openshift-service-ca.crt\"" Jul 07 23:51:56.865738 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:51:56.865562 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-dx64r\"/\"kube-root-ca.crt\"" Jul 07 23:51:56.866227 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:51:56.866212 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-dx64r\"/\"default-dockercfg-52fsr\"" Jul 07 23:51:56.871183 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:51:56.871160 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-dx64r/must-gather-v4c6l"] Jul 07 23:51:56.872877 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:51:56.872854 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:51:56.974025 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:51:56.973979 2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" limit=1024 Jul 07 23:51:56.986577 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:51:56.986557 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pph2b\" (UniqueName: \"kubernetes.io/projected/4438945e-1da3-40b7-8848-988a1f40a231-kube-api-access-pph2b\") pod \"must-gather-v4c6l\" (UID: \"4438945e-1da3-40b7-8848-988a1f40a231\") " pod="openshift-must-gather-dx64r/must-gather-v4c6l" Jul 07 23:51:56.986692 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:51:56.986604 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/4438945e-1da3-40b7-8848-988a1f40a231-must-gather-output\") pod \"must-gather-v4c6l\" (UID: \"4438945e-1da3-40b7-8848-988a1f40a231\") " pod="openshift-must-gather-dx64r/must-gather-v4c6l" Jul 07 23:51:57.093498 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:51:57.087763 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/4438945e-1da3-40b7-8848-988a1f40a231-must-gather-output\") pod \"must-gather-v4c6l\" (UID: \"4438945e-1da3-40b7-8848-988a1f40a231\") " pod="openshift-must-gather-dx64r/must-gather-v4c6l" Jul 07 23:51:57.093498 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:51:57.087950 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pph2b\" (UniqueName: \"kubernetes.io/projected/4438945e-1da3-40b7-8848-988a1f40a231-kube-api-access-pph2b\") pod \"must-gather-v4c6l\" (UID: \"4438945e-1da3-40b7-8848-988a1f40a231\") " pod="openshift-must-gather-dx64r/must-gather-v4c6l" Jul 07 23:51:57.093498 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:51:57.088737 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/4438945e-1da3-40b7-8848-988a1f40a231-must-gather-output\") pod \"must-gather-v4c6l\" (UID: \"4438945e-1da3-40b7-8848-988a1f40a231\") " pod="openshift-must-gather-dx64r/must-gather-v4c6l" Jul 07 23:51:57.097851 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:51:57.097829 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pph2b\" (UniqueName: \"kubernetes.io/projected/4438945e-1da3-40b7-8848-988a1f40a231-kube-api-access-pph2b\") pod \"must-gather-v4c6l\" (UID: \"4438945e-1da3-40b7-8848-988a1f40a231\") " pod="openshift-must-gather-dx64r/must-gather-v4c6l" Jul 07 23:51:57.172969 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:51:57.172905 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-dx64r/must-gather-v4c6l" Jul 07 23:51:57.305098 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:51:57.305074 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-dx64r/must-gather-v4c6l"] Jul 07 23:51:57.307205 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:51:57.307176 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4438945e_1da3_40b7_8848_988a1f40a231.slice/crio-b5e4c94bee5158fae5fd3426d1cf1986637f838cfc284cad0b78e5a1fbbe38d4 WatchSource:0}: Error finding container b5e4c94bee5158fae5fd3426d1cf1986637f838cfc284cad0b78e5a1fbbe38d4: Status 404 returned error can't find the container with id b5e4c94bee5158fae5fd3426d1cf1986637f838cfc284cad0b78e5a1fbbe38d4 Jul 07 23:51:57.309432 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:51:57.309417 2570 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 07 23:51:57.992140 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:51:57.992088 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-dx64r/must-gather-v4c6l" event={"ID":"4438945e-1da3-40b7-8848-988a1f40a231","Type":"ContainerStarted","Data":"b5e4c94bee5158fae5fd3426d1cf1986637f838cfc284cad0b78e5a1fbbe38d4"} Jul 07 23:51:59.000028 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:51:58.999988 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-dx64r/must-gather-v4c6l" event={"ID":"4438945e-1da3-40b7-8848-988a1f40a231","Type":"ContainerStarted","Data":"dcf72d85a38fa42dbd9befd288cd36125b78de48c14f168044770dfdac51a220"} Jul 07 23:51:59.000422 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:51:59.000039 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-dx64r/must-gather-v4c6l" event={"ID":"4438945e-1da3-40b7-8848-988a1f40a231","Type":"ContainerStarted","Data":"0af6cb627393d20251ced0647c8a7613c059b0abca2f8f9e4707c2ed85d6b517"} Jul 07 23:51:59.019088 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:51:59.019032 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-dx64r/must-gather-v4c6l" podStartSLOduration=2.175613559 podStartE2EDuration="3.019016353s" totalImagesPullingTime="843.402794ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 23:51:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 23:51:59.016872296 +0000 UTC m=+2336.413873810" watchObservedRunningTime="2026-07-07 23:51:59.019016353 +0000 UTC m=+2336.416017861" Jul 07 23:51:59.803241 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:51:59.803210 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-84hdc_1496ff7f-366b-4cd2-8396-2cdbfed841fa/global-pull-secret-syncer/0.log" Jul 07 23:51:59.906627 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:51:59.906597 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-wjn84_0bb01852-570a-4456-9aca-52c99f9f83e5/konnectivity-agent/0.log" Jul 07 23:52:00.036264 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:52:00.036238 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-134-140.ec2.internal_7257f873d4401d71530f0192e065f67c/haproxy/0.log" Jul 07 23:52:04.611294 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:52:04.611256 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759wgjps_8ef9793b-9b49-43a9-9b83-cc09c9002479/extract/0.log" Jul 07 23:52:04.658606 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:52:04.658507 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759wgjps_8ef9793b-9b49-43a9-9b83-cc09c9002479/util/0.log" Jul 07 23:52:04.701832 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:52:04.701803 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759wgjps_8ef9793b-9b49-43a9-9b83-cc09c9002479/pull/0.log" Jul 07 23:52:04.751814 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:52:04.751771 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e067bwg_ae1ba717-f216-48ff-9789-43126b0a9c1f/extract/0.log" Jul 07 23:52:04.787492 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:52:04.787437 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e067bwg_ae1ba717-f216-48ff-9789-43126b0a9c1f/util/0.log" Jul 07 23:52:04.834545 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:52:04.834514 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e067bwg_ae1ba717-f216-48ff-9789-43126b0a9c1f/pull/0.log" Jul 07 23:52:04.875383 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:52:04.875357 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed735hg4d_113878d9-e1e4-4af3-aed1-cdcd089443c4/extract/0.log" Jul 07 23:52:04.924501 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:52:04.924404 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed735hg4d_113878d9-e1e4-4af3-aed1-cdcd089443c4/util/0.log" Jul 07 23:52:04.978094 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:52:04.978066 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed735hg4d_113878d9-e1e4-4af3-aed1-cdcd089443c4/pull/0.log" Jul 07 23:52:05.020939 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:52:05.020905 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1stzl9_807f17fe-22a0-4a3b-82af-b242ca8e2013/extract/0.log" Jul 07 23:52:05.061211 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:52:05.061170 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1stzl9_807f17fe-22a0-4a3b-82af-b242ca8e2013/util/0.log" Jul 07 23:52:05.109145 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:52:05.109090 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1stzl9_807f17fe-22a0-4a3b-82af-b242ca8e2013/pull/0.log" Jul 07 23:52:05.525493 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:52:05.524926 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-865bd97974-r7bjs_46d1cd8e-d71e-4aef-bd0e-9914a638a0a1/authorino/0.log" Jul 07 23:52:05.877602 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:52:05.877518 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-8ptdh_cbc3faff-e9b2-48ad-a3c4-c0d77ec1af7a/kuadrant-console-plugin/0.log" Jul 07 23:52:06.095345 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:52:06.095312 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-hs9gr_7798a566-a702-48eb-a51d-da784da4a2f2/limitador/0.log" Jul 07 23:52:07.596172 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:52:07.596142 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_ae3fe82c-0bfa-4507-a1db-ade9479e1c26/alertmanager/0.log" Jul 07 23:52:07.621920 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:52:07.621888 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_ae3fe82c-0bfa-4507-a1db-ade9479e1c26/config-reloader/0.log" Jul 07 23:52:07.645709 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:52:07.645675 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_ae3fe82c-0bfa-4507-a1db-ade9479e1c26/kube-rbac-proxy-web/0.log" Jul 07 23:52:07.668139 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:52:07.668074 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_ae3fe82c-0bfa-4507-a1db-ade9479e1c26/kube-rbac-proxy/0.log" Jul 07 23:52:07.691307 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:52:07.691274 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_ae3fe82c-0bfa-4507-a1db-ade9479e1c26/kube-rbac-proxy-metric/0.log" Jul 07 23:52:07.716621 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:52:07.716594 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_ae3fe82c-0bfa-4507-a1db-ade9479e1c26/prom-label-proxy/0.log" Jul 07 23:52:07.741969 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:52:07.741897 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_ae3fe82c-0bfa-4507-a1db-ade9479e1c26/init-config-reloader/0.log" Jul 07 23:52:07.785797 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:52:07.785763 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-64786d9cbb-4qp2l_dbb27ecc-1295-4648-851e-0daefb42af23/cluster-monitoring-operator/0.log" Jul 07 23:52:07.817629 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:52:07.817596 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-7c9dbb8b7d-fkvsb_93aecf5b-2135-4f91-8ed2-3828b00f4b2f/kube-state-metrics/0.log" Jul 07 23:52:07.841893 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:52:07.841867 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-7c9dbb8b7d-fkvsb_93aecf5b-2135-4f91-8ed2-3828b00f4b2f/kube-rbac-proxy-main/0.log" Jul 07 23:52:07.869829 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:52:07.869806 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-7c9dbb8b7d-fkvsb_93aecf5b-2135-4f91-8ed2-3828b00f4b2f/kube-rbac-proxy-self/0.log" Jul 07 23:52:08.124787 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:52:08.124706 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-xctgs_58a67519-a641-49b9-ba37-aac45e387515/node-exporter/0.log" Jul 07 23:52:08.146033 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:52:08.146001 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-xctgs_58a67519-a641-49b9-ba37-aac45e387515/kube-rbac-proxy/0.log" Jul 07 23:52:08.167133 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:52:08.167104 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-xctgs_58a67519-a641-49b9-ba37-aac45e387515/init-textfile/0.log" Jul 07 23:52:08.192959 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:52:08.192921 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-54cf96dfcf-j8m6n_0e28cfdd-d891-4c6e-b4a3-808c02e3df64/kube-rbac-proxy-main/0.log" Jul 07 23:52:08.219042 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:52:08.219006 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-54cf96dfcf-j8m6n_0e28cfdd-d891-4c6e-b4a3-808c02e3df64/kube-rbac-proxy-self/0.log" Jul 07 23:52:08.242525 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:52:08.242453 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-54cf96dfcf-j8m6n_0e28cfdd-d891-4c6e-b4a3-808c02e3df64/openshift-state-metrics/0.log" Jul 07 23:52:08.513872 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:52:08.513789 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-d559d66d5-7lwcs_1b5943d1-f098-4857-ae9f-158dcbcb7b79/prometheus-operator-admission-webhook/0.log" Jul 07 23:52:08.543641 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:52:08.543587 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-9f6c8d65b-pxb5q_073840f8-716e-426b-8516-494fae621efb/telemeter-client/0.log" Jul 07 23:52:08.565036 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:52:08.565008 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-9f6c8d65b-pxb5q_073840f8-716e-426b-8516-494fae621efb/reload/0.log" Jul 07 23:52:08.595598 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:52:08.595569 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-9f6c8d65b-pxb5q_073840f8-716e-426b-8516-494fae621efb/kube-rbac-proxy/0.log" Jul 07 23:52:09.868737 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:52:09.868709 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-6bffdf9778-qqf65_51966fcb-7a64-464e-ab66-1d049570bcd3/networking-console-plugin/0.log" Jul 07 23:52:09.979032 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:52:09.978991 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-dx64r/perf-node-gather-daemonset-xrlts"] Jul 07 23:52:09.985617 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:52:09.985590 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-dx64r/perf-node-gather-daemonset-xrlts" Jul 07 23:52:09.993433 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:52:09.993406 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-dx64r/perf-node-gather-daemonset-xrlts"] Jul 07 23:52:10.133002 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:52:10.132916 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5710774d-6459-4b92-8cd0-73f95cb26bae-sys\") pod \"perf-node-gather-daemonset-xrlts\" (UID: \"5710774d-6459-4b92-8cd0-73f95cb26bae\") " pod="openshift-must-gather-dx64r/perf-node-gather-daemonset-xrlts" Jul 07 23:52:10.133179 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:52:10.133053 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/5710774d-6459-4b92-8cd0-73f95cb26bae-podres\") pod \"perf-node-gather-daemonset-xrlts\" (UID: \"5710774d-6459-4b92-8cd0-73f95cb26bae\") " pod="openshift-must-gather-dx64r/perf-node-gather-daemonset-xrlts" Jul 07 23:52:10.133179 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:52:10.133085 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/5710774d-6459-4b92-8cd0-73f95cb26bae-lib-modules\") pod \"perf-node-gather-daemonset-xrlts\" (UID: \"5710774d-6459-4b92-8cd0-73f95cb26bae\") " pod="openshift-must-gather-dx64r/perf-node-gather-daemonset-xrlts" Jul 07 23:52:10.133179 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:52:10.133131 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/5710774d-6459-4b92-8cd0-73f95cb26bae-proc\") pod \"perf-node-gather-daemonset-xrlts\" (UID: \"5710774d-6459-4b92-8cd0-73f95cb26bae\") " pod="openshift-must-gather-dx64r/perf-node-gather-daemonset-xrlts" Jul 07 23:52:10.133352 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:52:10.133252 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2vqts\" (UniqueName: \"kubernetes.io/projected/5710774d-6459-4b92-8cd0-73f95cb26bae-kube-api-access-2vqts\") pod \"perf-node-gather-daemonset-xrlts\" (UID: \"5710774d-6459-4b92-8cd0-73f95cb26bae\") " pod="openshift-must-gather-dx64r/perf-node-gather-daemonset-xrlts" Jul 07 23:52:10.234582 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:52:10.234542 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/5710774d-6459-4b92-8cd0-73f95cb26bae-podres\") pod \"perf-node-gather-daemonset-xrlts\" (UID: \"5710774d-6459-4b92-8cd0-73f95cb26bae\") " pod="openshift-must-gather-dx64r/perf-node-gather-daemonset-xrlts" Jul 07 23:52:10.234811 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:52:10.234765 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/5710774d-6459-4b92-8cd0-73f95cb26bae-podres\") pod \"perf-node-gather-daemonset-xrlts\" (UID: \"5710774d-6459-4b92-8cd0-73f95cb26bae\") " pod="openshift-must-gather-dx64r/perf-node-gather-daemonset-xrlts" Jul 07 23:52:10.234934 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:52:10.234916 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/5710774d-6459-4b92-8cd0-73f95cb26bae-lib-modules\") pod \"perf-node-gather-daemonset-xrlts\" (UID: \"5710774d-6459-4b92-8cd0-73f95cb26bae\") " pod="openshift-must-gather-dx64r/perf-node-gather-daemonset-xrlts" Jul 07 23:52:10.235065 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:52:10.235046 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/5710774d-6459-4b92-8cd0-73f95cb26bae-lib-modules\") pod \"perf-node-gather-daemonset-xrlts\" (UID: \"5710774d-6459-4b92-8cd0-73f95cb26bae\") " pod="openshift-must-gather-dx64r/perf-node-gather-daemonset-xrlts" Jul 07 23:52:10.235239 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:52:10.235205 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/5710774d-6459-4b92-8cd0-73f95cb26bae-proc\") pod \"perf-node-gather-daemonset-xrlts\" (UID: \"5710774d-6459-4b92-8cd0-73f95cb26bae\") " pod="openshift-must-gather-dx64r/perf-node-gather-daemonset-xrlts" Jul 07 23:52:10.235348 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:52:10.235333 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/5710774d-6459-4b92-8cd0-73f95cb26bae-proc\") pod \"perf-node-gather-daemonset-xrlts\" (UID: \"5710774d-6459-4b92-8cd0-73f95cb26bae\") " pod="openshift-must-gather-dx64r/perf-node-gather-daemonset-xrlts" Jul 07 23:52:10.235571 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:52:10.235553 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2vqts\" (UniqueName: \"kubernetes.io/projected/5710774d-6459-4b92-8cd0-73f95cb26bae-kube-api-access-2vqts\") pod \"perf-node-gather-daemonset-xrlts\" (UID: \"5710774d-6459-4b92-8cd0-73f95cb26bae\") " pod="openshift-must-gather-dx64r/perf-node-gather-daemonset-xrlts" Jul 07 23:52:10.236325 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:52:10.236299 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5710774d-6459-4b92-8cd0-73f95cb26bae-sys\") pod \"perf-node-gather-daemonset-xrlts\" (UID: \"5710774d-6459-4b92-8cd0-73f95cb26bae\") " pod="openshift-must-gather-dx64r/perf-node-gather-daemonset-xrlts" Jul 07 23:52:10.236776 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:52:10.236753 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5710774d-6459-4b92-8cd0-73f95cb26bae-sys\") pod \"perf-node-gather-daemonset-xrlts\" (UID: \"5710774d-6459-4b92-8cd0-73f95cb26bae\") " pod="openshift-must-gather-dx64r/perf-node-gather-daemonset-xrlts" Jul 07 23:52:10.244804 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:52:10.244776 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2vqts\" (UniqueName: \"kubernetes.io/projected/5710774d-6459-4b92-8cd0-73f95cb26bae-kube-api-access-2vqts\") pod \"perf-node-gather-daemonset-xrlts\" (UID: \"5710774d-6459-4b92-8cd0-73f95cb26bae\") " pod="openshift-must-gather-dx64r/perf-node-gather-daemonset-xrlts" Jul 07 23:52:10.300436 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:52:10.300391 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-dx64r/perf-node-gather-daemonset-xrlts" Jul 07 23:52:10.466179 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:52:10.466139 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-8hvxr_faae2bd8-cbbd-4cfb-bce6-7954977b7ebc/console-operator/2.log" Jul 07 23:52:10.466975 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:52:10.466950 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-dx64r/perf-node-gather-daemonset-xrlts"] Jul 07 23:52:10.470977 ip-10-0-134-140 kubenswrapper[2570]: W0707 23:52:10.470917 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod5710774d_6459_4b92_8cd0_73f95cb26bae.slice/crio-f662891cb5288c6b39b229d54dfafcf69d2373fc29a26f81421f72415a34d18f WatchSource:0}: Error finding container f662891cb5288c6b39b229d54dfafcf69d2373fc29a26f81421f72415a34d18f: Status 404 returned error can't find the container with id f662891cb5288c6b39b229d54dfafcf69d2373fc29a26f81421f72415a34d18f Jul 07 23:52:10.473012 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:52:10.472935 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-8hvxr_faae2bd8-cbbd-4cfb-bce6-7954977b7ebc/console-operator/3.log" Jul 07 23:52:11.098298 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:52:11.098260 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-dx64r/perf-node-gather-daemonset-xrlts" event={"ID":"5710774d-6459-4b92-8cd0-73f95cb26bae","Type":"ContainerStarted","Data":"c1e191957b69287a0f67a8c684c38736fb1dafc0c85d945b48e46024ac8251ca"} Jul 07 23:52:11.098714 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:52:11.098325 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-dx64r/perf-node-gather-daemonset-xrlts" event={"ID":"5710774d-6459-4b92-8cd0-73f95cb26bae","Type":"ContainerStarted","Data":"f662891cb5288c6b39b229d54dfafcf69d2373fc29a26f81421f72415a34d18f"} Jul 07 23:52:11.098714 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:52:11.098360 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-dx64r/perf-node-gather-daemonset-xrlts" Jul 07 23:52:11.119162 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:52:11.119111 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-dx64r/perf-node-gather-daemonset-xrlts" podStartSLOduration=2.119092238 podStartE2EDuration="2.119092238s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 23:52:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 23:52:11.117144498 +0000 UTC m=+2348.514146012" watchObservedRunningTime="2026-07-07 23:52:11.119092238 +0000 UTC m=+2348.516093749" Jul 07 23:52:11.581476 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:52:11.581440 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-5d456fdbcd-rwtl7_01587fde-e08c-441e-a73e-644b4d50b608/volume-data-source-validator/0.log" Jul 07 23:52:12.589672 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:52:12.589648 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-4th55_bd5aafbd-ed3c-46fa-bc87-661f74ba728a/dns/0.log" Jul 07 23:52:12.611277 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:52:12.611249 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-4th55_bd5aafbd-ed3c-46fa-bc87-661f74ba728a/kube-rbac-proxy/0.log" Jul 07 23:52:12.765827 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:52:12.765790 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-qgcpr_c02c0747-b1ae-452c-81aa-5101d7125fc9/dns-node-resolver/0.log" Jul 07 23:52:13.372902 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:52:13.372872 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-7z6hc_df04fe1d-eb1a-4711-80ea-72d68116bbca/node-ca/0.log" Jul 07 23:52:14.580964 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:52:14.580921 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7b7668c465-j5zks_5157b9da-e2ac-44a1-8ab2-c5f3c787e896/kube-auth-proxy/0.log" Jul 07 23:52:14.821130 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:52:14.821077 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-65d5dfb6cf-84llt_40bbfb8b-2c2b-43c5-8419-3dba3777f888/router/0.log" Jul 07 23:52:15.511133 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:52:15.511103 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-gbf74_143dd7b8-6ed7-489a-893a-e288926cc1f7/serve-healthcheck-canary/0.log" Jul 07 23:52:16.039479 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:52:16.039424 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5c847dcfc4-r6r4j_12b98da0-429d-4232-a0d6-b5c2699ec31a/insights-operator/0.log" Jul 07 23:52:16.040068 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:52:16.040043 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5c847dcfc4-r6r4j_12b98da0-429d-4232-a0d6-b5c2699ec31a/insights-operator/1.log" Jul 07 23:52:16.133074 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:52:16.133044 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-n9t7q_ccb9c548-3ffa-4ec0-8871-a7b956ff502f/kube-rbac-proxy/0.log" Jul 07 23:52:16.155789 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:52:16.155759 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-n9t7q_ccb9c548-3ffa-4ec0-8871-a7b956ff502f/exporter/0.log" Jul 07 23:52:16.179004 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:52:16.178975 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-n9t7q_ccb9c548-3ffa-4ec0-8871-a7b956ff502f/extractor/0.log" Jul 07 23:52:17.115243 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:52:17.115218 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-dx64r/perf-node-gather-daemonset-xrlts" Jul 07 23:52:19.030328 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:52:19.030289 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7c9859966d-7bxpt_33b45f14-3cc1-4712-bf73-206fa4041802/manager/0.log" Jul 07 23:52:19.047849 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:52:19.047818 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7c9859966d-7bxpt_33b45f14-3cc1-4712-bf73-206fa4041802/copy-manifests/0.log" Jul 07 23:52:20.104644 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:52:20.104571 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7b7668c465-j5zks_5157b9da-e2ac-44a1-8ab2-c5f3c787e896/kube-auth-proxy/0.log" Jul 07 23:52:20.307981 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:52:20.307950 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-65d5dfb6cf-84llt_40bbfb8b-2c2b-43c5-8419-3dba3777f888/router/0.log" Jul 07 23:52:20.536246 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:52:20.536216 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-7895d9b5-ct8kv_4fa028e3-4f6a-4053-a9d6-ca798750c5eb/manager/0.log" Jul 07 23:52:20.587950 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:52:20.587917 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-8jnrg_68a7079f-266e-4430-aa48-048b11c59f0a/openshift-lws-operator/0.log" Jul 07 23:52:21.705326 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:52:21.705302 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7b7668c465-j5zks_5157b9da-e2ac-44a1-8ab2-c5f3c787e896/kube-auth-proxy/0.log" Jul 07 23:52:21.935908 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:52:21.935879 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-65d5dfb6cf-84llt_40bbfb8b-2c2b-43c5-8419-3dba3777f888/router/0.log" Jul 07 23:52:26.843584 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:52:26.843552 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-4xzds_48b46894-bcf9-4176-82a7-bab0ee169c67/kube-multus/0.log" Jul 07 23:52:27.199883 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:52:27.199794 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-vztph_573e90ac-1d7d-4584-b231-6635ac4e12e2/kube-multus-additional-cni-plugins/0.log" Jul 07 23:52:27.220942 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:52:27.220914 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-vztph_573e90ac-1d7d-4584-b231-6635ac4e12e2/egress-router-binary-copy/0.log" Jul 07 23:52:27.245657 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:52:27.245629 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-vztph_573e90ac-1d7d-4584-b231-6635ac4e12e2/cni-plugins/0.log" Jul 07 23:52:27.266791 ip-10-0-134-140 kubenswrapper[2570]: I0707 23:52:27.266757 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-vztph_573e90ac-1d7d-4584-b231-6635ac4e12e2/bond-cni-plugin/0.log"