Jul 14 20:20:04.699192 ip-10-0-141-61 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory Jul 14 20:20:04.699206 ip-10-0-141-61 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory Jul 14 20:20:04.699216 ip-10-0-141-61 systemd[1]: kubelet.service: Failed with result 'resources'. Jul 14 20:20:04.699583 ip-10-0-141-61 systemd[1]: Failed to start Kubernetes Kubelet. Jul 14 20:20:14.946902 ip-10-0-141-61 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found. Jul 14 20:20:14.946923 ip-10-0-141-61 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot cba691b3aef3459cbd917e6ba25f5e8b -- Jul 14 20:22:19.848507 ip-10-0-141-61 systemd[1]: Starting Kubernetes Kubelet... Jul 14 20:22:20.221562 ip-10-0-141-61 kubenswrapper[2563]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 14 20:22:20.221562 ip-10-0-141-61 kubenswrapper[2563]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 14 20:22:20.221562 ip-10-0-141-61 kubenswrapper[2563]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 14 20:22:20.221562 ip-10-0-141-61 kubenswrapper[2563]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 14 20:22:20.221562 ip-10-0-141-61 kubenswrapper[2563]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 14 20:22:20.222507 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:20.222416 2563 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 14 20:22:20.228320 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.228303 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 14 20:22:20.228320 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.228320 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 14 20:22:20.228387 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.228324 2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 14 20:22:20.228387 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.228327 2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 14 20:22:20.228387 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.228330 2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 14 20:22:20.228387 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.228333 2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 14 20:22:20.228387 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.228335 2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 14 20:22:20.228387 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.228338 2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 14 20:22:20.228387 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.228341 2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 14 20:22:20.228387 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.228343 2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 14 20:22:20.228387 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.228346 2563 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 14 20:22:20.228387 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.228349 2563 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 14 20:22:20.228387 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.228351 2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 14 20:22:20.228387 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.228354 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 14 20:22:20.228387 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.228356 2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 14 20:22:20.228387 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.228367 2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 14 20:22:20.228387 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.228370 2563 feature_gate.go:328] unrecognized feature gate: Example Jul 14 20:22:20.228387 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.228375 2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 14 20:22:20.228387 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.228379 2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 14 20:22:20.228387 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.228382 2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 14 20:22:20.228387 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.228384 2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 14 20:22:20.228387 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.228387 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 14 20:22:20.228897 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.228391 2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 14 20:22:20.228897 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.228394 2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 14 20:22:20.228897 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.228396 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 14 20:22:20.228897 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.228399 2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 14 20:22:20.228897 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.228402 2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 14 20:22:20.228897 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.228405 2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 14 20:22:20.228897 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.228408 2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 14 20:22:20.228897 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.228410 2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 14 20:22:20.228897 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.228413 2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 14 20:22:20.228897 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.228415 2563 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 14 20:22:20.228897 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.228418 2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 14 20:22:20.228897 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.228421 2563 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 14 20:22:20.228897 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.228423 2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 14 20:22:20.228897 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.228426 2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 14 20:22:20.228897 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.228429 2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 14 20:22:20.228897 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.228431 2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 14 20:22:20.228897 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.228433 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 14 20:22:20.228897 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.228436 2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 14 20:22:20.228897 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.228438 2563 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 14 20:22:20.228897 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.228441 2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 14 20:22:20.229405 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.228443 2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 14 20:22:20.229405 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.228446 2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 14 20:22:20.229405 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.228448 2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 14 20:22:20.229405 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.228452 2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 14 20:22:20.229405 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.228455 2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 14 20:22:20.229405 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.228459 2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 14 20:22:20.229405 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.228462 2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 14 20:22:20.229405 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.228464 2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 14 20:22:20.229405 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.228467 2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 14 20:22:20.229405 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.228469 2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 14 20:22:20.229405 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.228472 2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 14 20:22:20.229405 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.228474 2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 14 20:22:20.229405 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.228477 2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 14 20:22:20.229405 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.228480 2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 14 20:22:20.229405 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.228483 2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 14 20:22:20.229405 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.228485 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 14 20:22:20.229405 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.228488 2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 14 20:22:20.229405 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.228491 2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 14 20:22:20.229405 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.228493 2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 14 20:22:20.229944 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.228496 2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 14 20:22:20.229944 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.228499 2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 14 20:22:20.229944 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.228501 2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 14 20:22:20.229944 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.228504 2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 14 20:22:20.229944 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.228506 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 14 20:22:20.229944 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.228509 2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 14 20:22:20.229944 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.228512 2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 14 20:22:20.229944 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.228514 2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 14 20:22:20.229944 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.228517 2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 14 20:22:20.229944 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.228519 2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 14 20:22:20.229944 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.228522 2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 14 20:22:20.229944 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.228526 2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 14 20:22:20.229944 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.228529 2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 14 20:22:20.229944 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.228532 2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 14 20:22:20.229944 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.228535 2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 14 20:22:20.229944 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.228537 2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 14 20:22:20.229944 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.228540 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 14 20:22:20.229944 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.228542 2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 14 20:22:20.229944 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.228545 2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 14 20:22:20.230462 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.228548 2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 14 20:22:20.230462 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.228550 2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 14 20:22:20.230462 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.228553 2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 14 20:22:20.230462 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.228555 2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 14 20:22:20.230462 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.228559 2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 14 20:22:20.230462 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.228561 2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 14 20:22:20.230462 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.228565 2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 14 20:22:20.230462 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.228568 2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 14 20:22:20.230462 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.228571 2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 14 20:22:20.230462 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.228574 2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 14 20:22:20.230462 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.228577 2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 14 20:22:20.230462 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.228579 2563 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 14 20:22:20.230462 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.228582 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 14 20:22:20.230462 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.228584 2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 14 20:22:20.230462 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.228587 2563 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 14 20:22:20.230462 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.228589 2563 feature_gate.go:328] unrecognized feature gate: Example2 Jul 14 20:22:20.230462 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.228592 2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 14 20:22:20.230462 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.228594 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 14 20:22:20.230462 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.228597 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 14 20:22:20.230462 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.229370 2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 14 20:22:20.231160 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.229377 2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 14 20:22:20.231160 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.229380 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 14 20:22:20.231160 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.229383 2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 14 20:22:20.231160 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.229386 2563 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 14 20:22:20.231160 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.229389 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 14 20:22:20.231160 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.229392 2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 14 20:22:20.231160 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.229395 2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 14 20:22:20.231160 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.229397 2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 14 20:22:20.231160 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.229400 2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 14 20:22:20.231160 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.229402 2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 14 20:22:20.231160 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.229405 2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 14 20:22:20.231160 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.229408 2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 14 20:22:20.231160 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.229411 2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 14 20:22:20.231160 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.229414 2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 14 20:22:20.231160 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.229417 2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 14 20:22:20.231160 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.229420 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 14 20:22:20.231160 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.229422 2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 14 20:22:20.231160 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.229426 2563 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 14 20:22:20.231160 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.229428 2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 14 20:22:20.231160 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.229432 2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 14 20:22:20.231796 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.229435 2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 14 20:22:20.231796 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.229437 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 14 20:22:20.231796 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.229440 2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 14 20:22:20.231796 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.229442 2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 14 20:22:20.231796 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.229445 2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 14 20:22:20.231796 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.229447 2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 14 20:22:20.231796 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.229451 2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 14 20:22:20.231796 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.229453 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 14 20:22:20.231796 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.229456 2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 14 20:22:20.231796 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.229458 2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 14 20:22:20.231796 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.229461 2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 14 20:22:20.231796 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.229464 2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 14 20:22:20.231796 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.229466 2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 14 20:22:20.231796 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.229469 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 14 20:22:20.231796 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.229471 2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 14 20:22:20.231796 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.229474 2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 14 20:22:20.231796 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.229476 2563 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 14 20:22:20.231796 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.229479 2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 14 20:22:20.231796 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.229481 2563 feature_gate.go:328] unrecognized feature gate: Example2 Jul 14 20:22:20.231796 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.229484 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 14 20:22:20.232322 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.229486 2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 14 20:22:20.232322 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.229489 2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 14 20:22:20.232322 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.229493 2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 14 20:22:20.232322 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.229497 2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 14 20:22:20.232322 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.229501 2563 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 14 20:22:20.232322 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.229505 2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 14 20:22:20.232322 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.229508 2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 14 20:22:20.232322 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.229512 2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 14 20:22:20.232322 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.229515 2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 14 20:22:20.232322 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.229519 2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 14 20:22:20.232322 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.229522 2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 14 20:22:20.232322 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.229525 2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 14 20:22:20.232322 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.229528 2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 14 20:22:20.232322 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.229530 2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 14 20:22:20.232322 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.229533 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 14 20:22:20.232322 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.229536 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 14 20:22:20.232322 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.229538 2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 14 20:22:20.232322 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.229541 2563 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 14 20:22:20.232322 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.229544 2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 14 20:22:20.232813 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.229546 2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 14 20:22:20.232813 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.229549 2563 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 14 20:22:20.232813 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.229551 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 14 20:22:20.232813 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.229554 2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 14 20:22:20.232813 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.229556 2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 14 20:22:20.232813 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.229559 2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 14 20:22:20.232813 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.229561 2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 14 20:22:20.232813 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.229564 2563 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 14 20:22:20.232813 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.229567 2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 14 20:22:20.232813 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.229570 2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 14 20:22:20.232813 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.229572 2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 14 20:22:20.232813 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.229575 2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 14 20:22:20.232813 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.229578 2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 14 20:22:20.232813 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.229582 2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 14 20:22:20.232813 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.229585 2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 14 20:22:20.232813 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.229588 2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 14 20:22:20.232813 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.229590 2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 14 20:22:20.232813 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.229593 2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 14 20:22:20.232813 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.229596 2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 14 20:22:20.233299 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.229599 2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 14 20:22:20.233299 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.229602 2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 14 20:22:20.233299 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.229605 2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 14 20:22:20.233299 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.229608 2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 14 20:22:20.233299 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.229610 2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 14 20:22:20.233299 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.229613 2563 feature_gate.go:328] unrecognized feature gate: Example Jul 14 20:22:20.233299 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.229616 2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 14 20:22:20.233299 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.229618 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 14 20:22:20.233299 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.229621 2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 14 20:22:20.233299 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.229623 2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 14 20:22:20.233299 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.229626 2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 14 20:22:20.233299 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.229628 2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 14 20:22:20.233299 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.229631 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 14 20:22:20.233299 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.229633 2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 14 20:22:20.233299 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.229635 2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 14 20:22:20.233299 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.229638 2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 14 20:22:20.233299 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.229641 2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 14 20:22:20.233299 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.229643 2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 14 20:22:20.233299 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.229646 2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 14 20:22:20.233299 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.229649 2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 14 20:22:20.233777 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.230087 2563 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 14 20:22:20.233777 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.230093 2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 14 20:22:20.233777 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.230097 2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 14 20:22:20.233777 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.230100 2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 14 20:22:20.233777 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.230103 2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 14 20:22:20.233777 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.230106 2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 14 20:22:20.233777 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.230108 2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 14 20:22:20.233777 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.230112 2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 14 20:22:20.233777 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.230115 2563 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 14 20:22:20.233777 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.230117 2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 14 20:22:20.233777 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.230120 2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 14 20:22:20.233777 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.230123 2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 14 20:22:20.233777 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.230126 2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 14 20:22:20.233777 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.230129 2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 14 20:22:20.233777 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.230132 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 14 20:22:20.233777 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.230136 2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 14 20:22:20.233777 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.230139 2563 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 14 20:22:20.233777 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.230142 2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 14 20:22:20.233777 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.230145 2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 14 20:22:20.233777 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.230147 2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 14 20:22:20.234280 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.230150 2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 14 20:22:20.234280 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.230152 2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 14 20:22:20.234280 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.230155 2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 14 20:22:20.234280 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.230158 2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 14 20:22:20.234280 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.230160 2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 14 20:22:20.234280 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.230163 2563 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 14 20:22:20.234280 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.230165 2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 14 20:22:20.234280 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.230168 2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 14 20:22:20.234280 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.230170 2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 14 20:22:20.234280 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.230173 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 14 20:22:20.234280 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.230175 2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 14 20:22:20.234280 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.230178 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 14 20:22:20.234280 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.230181 2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 14 20:22:20.234280 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.230184 2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 14 20:22:20.234280 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.230187 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 14 20:22:20.234280 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.230189 2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 14 20:22:20.234280 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.230212 2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 14 20:22:20.234280 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.230216 2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 14 20:22:20.234280 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.230218 2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 14 20:22:20.234280 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.230221 2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 14 20:22:20.234762 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.230224 2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 14 20:22:20.234762 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.230227 2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 14 20:22:20.234762 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.230229 2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 14 20:22:20.234762 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.230232 2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 14 20:22:20.234762 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.230235 2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 14 20:22:20.234762 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.230237 2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 14 20:22:20.234762 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.230240 2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 14 20:22:20.234762 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.230243 2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 14 20:22:20.234762 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.230246 2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 14 20:22:20.234762 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.230249 2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 14 20:22:20.234762 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.230251 2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 14 20:22:20.234762 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.230254 2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 14 20:22:20.234762 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.230256 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 14 20:22:20.234762 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.230259 2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 14 20:22:20.234762 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.230261 2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 14 20:22:20.234762 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.230264 2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 14 20:22:20.234762 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.230266 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 14 20:22:20.234762 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.230269 2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 14 20:22:20.234762 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.230271 2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 14 20:22:20.234762 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.230274 2563 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 14 20:22:20.235262 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.230276 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 14 20:22:20.235262 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.230278 2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 14 20:22:20.235262 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.230281 2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 14 20:22:20.235262 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.230284 2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 14 20:22:20.235262 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.230286 2563 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 14 20:22:20.235262 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.230289 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 14 20:22:20.235262 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.230292 2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 14 20:22:20.235262 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.230297 2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 14 20:22:20.235262 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.230300 2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 14 20:22:20.235262 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.230303 2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 14 20:22:20.235262 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.230306 2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 14 20:22:20.235262 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.230309 2563 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 14 20:22:20.235262 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.230312 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 14 20:22:20.235262 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.230315 2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 14 20:22:20.235262 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.230318 2563 feature_gate.go:328] unrecognized feature gate: Example Jul 14 20:22:20.235262 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.230320 2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 14 20:22:20.235262 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.230323 2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 14 20:22:20.235262 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.230326 2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 14 20:22:20.235262 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.230328 2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 14 20:22:20.235722 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.230331 2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 14 20:22:20.235722 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.230335 2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 14 20:22:20.235722 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.230337 2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 14 20:22:20.235722 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.230339 2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 14 20:22:20.235722 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.230342 2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 14 20:22:20.235722 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.230346 2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 14 20:22:20.235722 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.230348 2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 14 20:22:20.235722 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.230350 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 14 20:22:20.235722 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.230353 2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 14 20:22:20.235722 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.230355 2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 14 20:22:20.235722 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.230358 2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 14 20:22:20.235722 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.230360 2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 14 20:22:20.235722 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.230363 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 14 20:22:20.235722 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.230365 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 14 20:22:20.235722 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.230368 2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 14 20:22:20.235722 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.230370 2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 14 20:22:20.235722 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.230374 2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 14 20:22:20.235722 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.230380 2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 14 20:22:20.235722 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.230385 2563 feature_gate.go:328] unrecognized feature gate: Example2 Jul 14 20:22:20.236180 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.230388 2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 14 20:22:20.236180 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:20.230458 2563 flags.go:64] FLAG: --address="0.0.0.0" Jul 14 20:22:20.236180 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:20.230465 2563 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 14 20:22:20.236180 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:20.230471 2563 flags.go:64] FLAG: --anonymous-auth="true" Jul 14 20:22:20.236180 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:20.230476 2563 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 14 20:22:20.236180 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:20.230481 2563 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 14 20:22:20.236180 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:20.230484 2563 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 14 20:22:20.236180 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:20.230488 2563 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 14 20:22:20.236180 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:20.230493 2563 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 14 20:22:20.236180 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:20.230496 2563 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 14 20:22:20.236180 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:20.230499 2563 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 14 20:22:20.236180 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:20.230503 2563 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 14 20:22:20.236180 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:20.230506 2563 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 14 20:22:20.236180 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:20.230510 2563 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 14 20:22:20.236180 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:20.230512 2563 flags.go:64] FLAG: --cgroup-root="" Jul 14 20:22:20.236180 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:20.230558 2563 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 14 20:22:20.236180 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:20.230611 2563 flags.go:64] FLAG: --client-ca-file="" Jul 14 20:22:20.236180 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:20.230615 2563 flags.go:64] FLAG: --cloud-provider="external" Jul 14 20:22:20.236180 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:20.230619 2563 flags.go:64] FLAG: --cluster-dns="[]" Jul 14 20:22:20.236180 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:20.230625 2563 flags.go:64] FLAG: --cluster-domain="" Jul 14 20:22:20.236180 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:20.230628 2563 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 14 20:22:20.236180 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:20.230632 2563 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 14 20:22:20.236180 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:20.230636 2563 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 14 20:22:20.236738 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:20.230639 2563 flags.go:64] FLAG: --container-log-max-files="5" Jul 14 20:22:20.236738 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:20.230644 2563 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 14 20:22:20.236738 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:20.230655 2563 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 14 20:22:20.236738 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:20.230660 2563 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 14 20:22:20.236738 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:20.230665 2563 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 14 20:22:20.236738 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:20.230668 2563 flags.go:64] FLAG: --contention-profiling="false" Jul 14 20:22:20.236738 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:20.230672 2563 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 14 20:22:20.236738 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:20.230679 2563 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 14 20:22:20.236738 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:20.231633 2563 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 14 20:22:20.236738 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:20.231658 2563 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 14 20:22:20.236738 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:20.231790 2563 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 14 20:22:20.236738 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:20.231795 2563 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 14 20:22:20.236738 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:20.231799 2563 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 14 20:22:20.236738 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:20.231802 2563 flags.go:64] FLAG: --enable-load-reader="false" Jul 14 20:22:20.236738 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:20.231806 2563 flags.go:64] FLAG: --enable-server="true" Jul 14 20:22:20.236738 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:20.231809 2563 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 14 20:22:20.236738 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:20.231816 2563 flags.go:64] FLAG: --event-burst="100" Jul 14 20:22:20.236738 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:20.231820 2563 flags.go:64] FLAG: --event-qps="50" Jul 14 20:22:20.236738 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:20.231823 2563 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 14 20:22:20.236738 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:20.231827 2563 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 14 20:22:20.236738 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:20.231830 2563 flags.go:64] FLAG: --eviction-hard="" Jul 14 20:22:20.236738 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:20.231834 2563 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 14 20:22:20.236738 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:20.231837 2563 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 14 20:22:20.236738 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:20.231839 2563 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 14 20:22:20.236738 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:20.231843 2563 flags.go:64] FLAG: --eviction-soft="" Jul 14 20:22:20.237356 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:20.231846 2563 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 14 20:22:20.237356 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:20.231849 2563 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 14 20:22:20.237356 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:20.231852 2563 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 14 20:22:20.237356 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:20.231855 2563 flags.go:64] FLAG: --experimental-mounter-path="" Jul 14 20:22:20.237356 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:20.231859 2563 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 14 20:22:20.237356 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:20.231862 2563 flags.go:64] FLAG: --fail-swap-on="true" Jul 14 20:22:20.237356 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:20.231865 2563 flags.go:64] FLAG: --feature-gates="" Jul 14 20:22:20.237356 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:20.231869 2563 flags.go:64] FLAG: --file-check-frequency="20s" Jul 14 20:22:20.237356 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:20.231872 2563 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 14 20:22:20.237356 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:20.231875 2563 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 14 20:22:20.237356 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:20.231878 2563 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 14 20:22:20.237356 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:20.231882 2563 flags.go:64] FLAG: --healthz-port="10248" Jul 14 20:22:20.237356 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:20.231885 2563 flags.go:64] FLAG: --help="false" Jul 14 20:22:20.237356 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:20.231888 2563 flags.go:64] FLAG: --hostname-override="ip-10-0-141-61.ec2.internal" Jul 14 20:22:20.237356 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:20.231891 2563 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 14 20:22:20.237356 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:20.231900 2563 flags.go:64] FLAG: --http-check-frequency="20s" Jul 14 20:22:20.237356 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:20.231903 2563 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 14 20:22:20.237356 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:20.231907 2563 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 14 20:22:20.237356 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:20.231911 2563 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 14 20:22:20.237356 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:20.231914 2563 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 14 20:22:20.237356 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:20.231917 2563 flags.go:64] FLAG: --image-service-endpoint="" Jul 14 20:22:20.237356 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:20.231920 2563 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 14 20:22:20.237356 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:20.231923 2563 flags.go:64] FLAG: --kube-api-burst="100" Jul 14 20:22:20.237958 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:20.231926 2563 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 14 20:22:20.237958 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:20.231929 2563 flags.go:64] FLAG: --kube-api-qps="50" Jul 14 20:22:20.237958 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:20.231932 2563 flags.go:64] FLAG: --kube-reserved="" Jul 14 20:22:20.237958 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:20.231935 2563 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 14 20:22:20.237958 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:20.231938 2563 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 14 20:22:20.237958 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:20.231941 2563 flags.go:64] FLAG: --kubelet-cgroups="" Jul 14 20:22:20.237958 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:20.231944 2563 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 14 20:22:20.237958 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:20.231947 2563 flags.go:64] FLAG: --lock-file="" Jul 14 20:22:20.237958 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:20.231950 2563 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 14 20:22:20.237958 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:20.231953 2563 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 14 20:22:20.237958 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:20.231956 2563 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 14 20:22:20.237958 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:20.231961 2563 flags.go:64] FLAG: --log-json-split-stream="false" Jul 14 20:22:20.237958 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:20.231964 2563 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 14 20:22:20.237958 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:20.231967 2563 flags.go:64] FLAG: --log-text-split-stream="false" Jul 14 20:22:20.237958 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:20.231970 2563 flags.go:64] FLAG: --logging-format="text" Jul 14 20:22:20.237958 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:20.231973 2563 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 14 20:22:20.237958 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:20.231976 2563 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 14 20:22:20.237958 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:20.231979 2563 flags.go:64] FLAG: --manifest-url="" Jul 14 20:22:20.237958 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:20.231981 2563 flags.go:64] FLAG: --manifest-url-header="" Jul 14 20:22:20.237958 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:20.231987 2563 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 14 20:22:20.237958 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:20.231990 2563 flags.go:64] FLAG: --max-open-files="1000000" Jul 14 20:22:20.237958 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:20.231994 2563 flags.go:64] FLAG: --max-pods="110" Jul 14 20:22:20.237958 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:20.231997 2563 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 14 20:22:20.237958 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:20.232001 2563 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 14 20:22:20.237958 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:20.232007 2563 flags.go:64] FLAG: --memory-manager-policy="None" Jul 14 20:22:20.238744 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:20.232010 2563 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 14 20:22:20.238744 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:20.232013 2563 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 14 20:22:20.238744 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:20.232018 2563 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 14 20:22:20.238744 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:20.232021 2563 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 14 20:22:20.238744 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:20.232038 2563 flags.go:64] FLAG: --node-status-max-images="50" Jul 14 20:22:20.238744 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:20.232042 2563 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 14 20:22:20.238744 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:20.232045 2563 flags.go:64] FLAG: --oom-score-adj="-999" Jul 14 20:22:20.238744 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:20.232048 2563 flags.go:64] FLAG: --pod-cidr="" Jul 14 20:22:20.238744 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:20.232051 2563 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b04c0b1f73af4d021dd8f0b51bfeb302c24482c093de5f2c43654ab56c506926" Jul 14 20:22:20.238744 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:20.232057 2563 flags.go:64] FLAG: --pod-manifest-path="" Jul 14 20:22:20.238744 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:20.232059 2563 flags.go:64] FLAG: --pod-max-pids="-1" Jul 14 20:22:20.238744 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:20.232062 2563 flags.go:64] FLAG: --pods-per-core="0" Jul 14 20:22:20.238744 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:20.232065 2563 flags.go:64] FLAG: --port="10250" Jul 14 20:22:20.238744 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:20.232069 2563 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 14 20:22:20.238744 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:20.232072 2563 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-080ac1c3f1e0b2707" Jul 14 20:22:20.238744 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:20.232075 2563 flags.go:64] FLAG: --qos-reserved="" Jul 14 20:22:20.238744 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:20.232078 2563 flags.go:64] FLAG: --read-only-port="10255" Jul 14 20:22:20.238744 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:20.232081 2563 flags.go:64] FLAG: --register-node="true" Jul 14 20:22:20.238744 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:20.232084 2563 flags.go:64] FLAG: --register-with-taints="" Jul 14 20:22:20.238744 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:20.232088 2563 flags.go:64] FLAG: --registry-burst="10" Jul 14 20:22:20.238744 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:20.232091 2563 flags.go:64] FLAG: --registry-qps="5" Jul 14 20:22:20.238744 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:20.232093 2563 flags.go:64] FLAG: --reserved-cpus="" Jul 14 20:22:20.238744 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:20.232096 2563 flags.go:64] FLAG: --reserved-memory="" Jul 14 20:22:20.238744 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:20.232100 2563 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 14 20:22:20.238744 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:20.232103 2563 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 14 20:22:20.239807 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:20.232105 2563 flags.go:64] FLAG: --rotate-certificates="false" Jul 14 20:22:20.239807 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:20.232109 2563 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 14 20:22:20.239807 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:20.232112 2563 flags.go:64] FLAG: --runonce="false" Jul 14 20:22:20.239807 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:20.232115 2563 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 14 20:22:20.239807 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:20.232118 2563 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 14 20:22:20.239807 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:20.232121 2563 flags.go:64] FLAG: --seccomp-default="false" Jul 14 20:22:20.239807 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:20.232125 2563 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 14 20:22:20.239807 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:20.232129 2563 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 14 20:22:20.239807 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:20.232132 2563 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 14 20:22:20.239807 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:20.232135 2563 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 14 20:22:20.239807 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:20.232139 2563 flags.go:64] FLAG: --storage-driver-password="root" Jul 14 20:22:20.239807 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:20.232142 2563 flags.go:64] FLAG: --storage-driver-secure="false" Jul 14 20:22:20.239807 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:20.232145 2563 flags.go:64] FLAG: --storage-driver-table="stats" Jul 14 20:22:20.239807 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:20.232148 2563 flags.go:64] FLAG: --storage-driver-user="root" Jul 14 20:22:20.239807 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:20.232151 2563 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 14 20:22:20.239807 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:20.232154 2563 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 14 20:22:20.239807 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:20.232156 2563 flags.go:64] FLAG: --system-cgroups="" Jul 14 20:22:20.239807 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:20.232159 2563 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 14 20:22:20.239807 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:20.232165 2563 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 14 20:22:20.239807 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:20.232168 2563 flags.go:64] FLAG: --tls-cert-file="" Jul 14 20:22:20.239807 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:20.232171 2563 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 14 20:22:20.239807 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:20.232176 2563 flags.go:64] FLAG: --tls-min-version="" Jul 14 20:22:20.239807 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:20.232179 2563 flags.go:64] FLAG: --tls-private-key-file="" Jul 14 20:22:20.239807 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:20.232182 2563 flags.go:64] FLAG: --topology-manager-policy="none" Jul 14 20:22:20.239807 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:20.232185 2563 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 14 20:22:20.240880 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:20.232188 2563 flags.go:64] FLAG: --topology-manager-scope="container" Jul 14 20:22:20.240880 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:20.232206 2563 flags.go:64] FLAG: --v="2" Jul 14 20:22:20.240880 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:20.232211 2563 flags.go:64] FLAG: --version="false" Jul 14 20:22:20.240880 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:20.232215 2563 flags.go:64] FLAG: --vmodule="" Jul 14 20:22:20.240880 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:20.232219 2563 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 14 20:22:20.240880 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:20.232222 2563 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 14 20:22:20.240880 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.232349 2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 14 20:22:20.240880 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.232354 2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 14 20:22:20.240880 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.232358 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 14 20:22:20.240880 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.232361 2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 14 20:22:20.240880 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.232365 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 14 20:22:20.240880 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.232368 2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 14 20:22:20.240880 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.232370 2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 14 20:22:20.240880 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.232374 2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 14 20:22:20.240880 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.232377 2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 14 20:22:20.240880 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.232380 2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 14 20:22:20.240880 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.232383 2563 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 14 20:22:20.240880 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.232385 2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 14 20:22:20.240880 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.232389 2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 14 20:22:20.240880 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.232392 2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 14 20:22:20.240880 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.232394 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 14 20:22:20.241828 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.232397 2563 feature_gate.go:328] unrecognized feature gate: Example2 Jul 14 20:22:20.241828 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.232401 2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 14 20:22:20.241828 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.232404 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 14 20:22:20.241828 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.232407 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 14 20:22:20.241828 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.232410 2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 14 20:22:20.241828 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.232414 2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 14 20:22:20.241828 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.232418 2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 14 20:22:20.241828 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.232422 2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 14 20:22:20.241828 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.232425 2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 14 20:22:20.241828 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.232427 2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 14 20:22:20.241828 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.232430 2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 14 20:22:20.241828 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.232433 2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 14 20:22:20.241828 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.232436 2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 14 20:22:20.241828 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.232439 2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 14 20:22:20.241828 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.232441 2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 14 20:22:20.241828 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.232444 2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 14 20:22:20.241828 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.232446 2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 14 20:22:20.241828 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.232449 2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 14 20:22:20.241828 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.232451 2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 14 20:22:20.242572 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.232453 2563 feature_gate.go:328] unrecognized feature gate: Example Jul 14 20:22:20.242572 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.232456 2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 14 20:22:20.242572 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.232459 2563 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 14 20:22:20.242572 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.232461 2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 14 20:22:20.242572 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.232464 2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 14 20:22:20.242572 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.232466 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 14 20:22:20.242572 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.232469 2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 14 20:22:20.242572 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.232472 2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 14 20:22:20.242572 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.232474 2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 14 20:22:20.242572 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.232477 2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 14 20:22:20.242572 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.232479 2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 14 20:22:20.242572 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.232482 2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 14 20:22:20.242572 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.232485 2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 14 20:22:20.242572 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.232487 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 14 20:22:20.242572 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.232490 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 14 20:22:20.242572 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.232492 2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 14 20:22:20.242572 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.232495 2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 14 20:22:20.242572 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.232497 2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 14 20:22:20.242572 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.232500 2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 14 20:22:20.242572 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.232502 2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 14 20:22:20.243088 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.232505 2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 14 20:22:20.243088 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.232508 2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 14 20:22:20.243088 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.232512 2563 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 14 20:22:20.243088 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.232515 2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 14 20:22:20.243088 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.232518 2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 14 20:22:20.243088 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.232521 2563 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 14 20:22:20.243088 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.232523 2563 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 14 20:22:20.243088 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.232526 2563 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 14 20:22:20.243088 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.232528 2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 14 20:22:20.243088 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.232531 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 14 20:22:20.243088 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.232534 2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 14 20:22:20.243088 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.232536 2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 14 20:22:20.243088 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.232539 2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 14 20:22:20.243088 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.232541 2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 14 20:22:20.243088 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.232544 2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 14 20:22:20.243088 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.232546 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 14 20:22:20.243088 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.232549 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 14 20:22:20.243088 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.232552 2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 14 20:22:20.243088 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.232554 2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 14 20:22:20.243088 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.232557 2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 14 20:22:20.243955 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.232559 2563 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 14 20:22:20.243955 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.232562 2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 14 20:22:20.243955 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.232564 2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 14 20:22:20.243955 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.232567 2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 14 20:22:20.243955 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.232571 2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 14 20:22:20.243955 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.232574 2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 14 20:22:20.243955 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.232576 2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 14 20:22:20.243955 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.232579 2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 14 20:22:20.243955 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.232582 2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 14 20:22:20.243955 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.232584 2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 14 20:22:20.243955 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.232586 2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 14 20:22:20.243955 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.232589 2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 14 20:22:20.243955 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.232592 2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 14 20:22:20.243955 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.232594 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 14 20:22:20.243955 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.232597 2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 14 20:22:20.243955 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.232599 2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 14 20:22:20.243955 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.232602 2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 14 20:22:20.243955 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.232604 2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 14 20:22:20.243955 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.232607 2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 14 20:22:20.243955 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.232610 2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 14 20:22:20.244772 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.232612 2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 14 20:22:20.244772 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.232615 2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 14 20:22:20.244772 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.232617 2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 14 20:22:20.244772 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.232620 2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 14 20:22:20.244772 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.232623 2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 14 20:22:20.244772 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:20.233290 2563 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 14 20:22:20.244772 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:20.240804 2563 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Jul 14 20:22:20.244772 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:20.240826 2563 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 14 20:22:20.244772 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:20.240854 2563 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 14 20:22:20.244772 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:20.240863 2563 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 14 20:22:20.244772 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.240955 2563 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 14 20:22:20.244772 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.240964 2563 feature_gate.go:328] unrecognized feature gate: Example Jul 14 20:22:20.244772 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.240970 2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 14 20:22:20.244772 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.240975 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 14 20:22:20.244772 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.240980 2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 14 20:22:20.245143 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.240984 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 14 20:22:20.245143 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.240989 2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 14 20:22:20.245143 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.240993 2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 14 20:22:20.245143 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.240998 2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 14 20:22:20.245143 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.241002 2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 14 20:22:20.245143 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.241006 2563 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 14 20:22:20.245143 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.241012 2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 14 20:22:20.245143 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.241017 2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 14 20:22:20.245143 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.241022 2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 14 20:22:20.245143 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.241027 2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 14 20:22:20.245143 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.241032 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 14 20:22:20.245143 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.241037 2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 14 20:22:20.245143 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.241041 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 14 20:22:20.245143 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.241046 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 14 20:22:20.245143 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.241051 2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 14 20:22:20.245143 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.241055 2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 14 20:22:20.245143 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.241060 2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 14 20:22:20.245143 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.241064 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 14 20:22:20.245143 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.241068 2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 14 20:22:20.245816 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.241073 2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 14 20:22:20.245816 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.241077 2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 14 20:22:20.245816 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.241081 2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 14 20:22:20.245816 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.241085 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 14 20:22:20.245816 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.241089 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 14 20:22:20.245816 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.241096 2563 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 14 20:22:20.245816 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.241100 2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 14 20:22:20.245816 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.241104 2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 14 20:22:20.245816 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.241109 2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 14 20:22:20.245816 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.241113 2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 14 20:22:20.245816 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.241117 2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 14 20:22:20.245816 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.241121 2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 14 20:22:20.245816 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.241125 2563 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 14 20:22:20.245816 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.241130 2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 14 20:22:20.245816 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.241134 2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 14 20:22:20.245816 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.241138 2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 14 20:22:20.245816 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.241143 2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 14 20:22:20.245816 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.241147 2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 14 20:22:20.245816 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.241151 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 14 20:22:20.245816 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.241155 2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 14 20:22:20.246349 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.241159 2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 14 20:22:20.246349 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.241164 2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 14 20:22:20.246349 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.241169 2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 14 20:22:20.246349 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.241173 2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 14 20:22:20.246349 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.241177 2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 14 20:22:20.246349 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.241184 2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 14 20:22:20.246349 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.241210 2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 14 20:22:20.246349 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.241215 2563 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 14 20:22:20.246349 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.241219 2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 14 20:22:20.246349 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.241223 2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 14 20:22:20.246349 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.241227 2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 14 20:22:20.246349 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.241231 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 14 20:22:20.246349 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.241235 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 14 20:22:20.246349 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.241239 2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 14 20:22:20.246349 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.241243 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 14 20:22:20.246349 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.241247 2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 14 20:22:20.246349 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.241251 2563 feature_gate.go:328] unrecognized feature gate: Example2 Jul 14 20:22:20.246349 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.241256 2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 14 20:22:20.246349 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.241260 2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 14 20:22:20.246814 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.241265 2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 14 20:22:20.246814 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.241270 2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 14 20:22:20.246814 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.241274 2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 14 20:22:20.246814 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.241278 2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 14 20:22:20.246814 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.241282 2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 14 20:22:20.246814 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.241286 2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 14 20:22:20.246814 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.241290 2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 14 20:22:20.246814 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.241294 2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 14 20:22:20.246814 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.241298 2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 14 20:22:20.246814 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.241302 2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 14 20:22:20.246814 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.241306 2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 14 20:22:20.246814 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.241311 2563 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 14 20:22:20.246814 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.241315 2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 14 20:22:20.246814 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.241319 2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 14 20:22:20.246814 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.241323 2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 14 20:22:20.246814 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.241327 2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 14 20:22:20.246814 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.241332 2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 14 20:22:20.246814 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.241337 2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 14 20:22:20.246814 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.241341 2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 14 20:22:20.246814 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.241345 2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 14 20:22:20.247394 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.241349 2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 14 20:22:20.247394 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.241353 2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 14 20:22:20.247394 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.241360 2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 14 20:22:20.247394 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.241366 2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 14 20:22:20.247394 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.241371 2563 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 14 20:22:20.247394 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.241375 2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 14 20:22:20.247394 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.241379 2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 14 20:22:20.247394 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.241387 2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 14 20:22:20.247394 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.241393 2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 14 20:22:20.247394 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.241397 2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 14 20:22:20.247394 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.241402 2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 14 20:22:20.247394 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.241407 2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 14 20:22:20.247394 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.241412 2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 14 20:22:20.247394 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.241416 2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 14 20:22:20.247394 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.241421 2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 14 20:22:20.247394 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.241425 2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 14 20:22:20.247861 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:20.241433 2563 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 14 20:22:20.247861 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.241633 2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 14 20:22:20.247861 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.241667 2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 14 20:22:20.247861 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.241673 2563 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 14 20:22:20.247861 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.241678 2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 14 20:22:20.247861 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.241682 2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 14 20:22:20.247861 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.241686 2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 14 20:22:20.247861 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.241691 2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 14 20:22:20.247861 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.241695 2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 14 20:22:20.247861 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.241699 2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 14 20:22:20.247861 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.241704 2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 14 20:22:20.247861 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.241709 2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 14 20:22:20.247861 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.241713 2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 14 20:22:20.247861 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.241718 2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 14 20:22:20.247861 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.241722 2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 14 20:22:20.248278 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.241727 2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 14 20:22:20.248278 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.241731 2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 14 20:22:20.248278 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.241736 2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 14 20:22:20.248278 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.241740 2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 14 20:22:20.248278 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.241745 2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 14 20:22:20.248278 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.241748 2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 14 20:22:20.248278 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.241755 2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 14 20:22:20.248278 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.241761 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 14 20:22:20.248278 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.241766 2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 14 20:22:20.248278 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.241771 2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 14 20:22:20.248278 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.241775 2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 14 20:22:20.248278 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.241782 2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 14 20:22:20.248278 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.241787 2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 14 20:22:20.248278 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.241791 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 14 20:22:20.248278 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.241797 2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 14 20:22:20.248278 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.241801 2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 14 20:22:20.248278 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.241806 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 14 20:22:20.248278 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.241810 2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 14 20:22:20.248278 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.241814 2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 14 20:22:20.248764 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.241818 2563 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 14 20:22:20.248764 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.241822 2563 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 14 20:22:20.248764 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.241826 2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 14 20:22:20.248764 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.241830 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 14 20:22:20.248764 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.241834 2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 14 20:22:20.248764 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.241837 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 14 20:22:20.248764 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.241841 2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 14 20:22:20.248764 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.241845 2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 14 20:22:20.248764 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.241849 2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 14 20:22:20.248764 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.241853 2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 14 20:22:20.248764 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.241858 2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 14 20:22:20.248764 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.241862 2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 14 20:22:20.248764 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.241867 2563 feature_gate.go:328] unrecognized feature gate: Example2 Jul 14 20:22:20.248764 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.241871 2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 14 20:22:20.248764 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.241875 2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 14 20:22:20.248764 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.241879 2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 14 20:22:20.248764 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.241883 2563 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 14 20:22:20.248764 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.241887 2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 14 20:22:20.248764 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.241892 2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 14 20:22:20.248764 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.241896 2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 14 20:22:20.249273 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.241899 2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 14 20:22:20.249273 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.241904 2563 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 14 20:22:20.249273 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.241909 2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 14 20:22:20.249273 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.241914 2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 14 20:22:20.249273 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.241920 2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 14 20:22:20.249273 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.241925 2563 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 14 20:22:20.249273 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.241930 2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 14 20:22:20.249273 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.241935 2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 14 20:22:20.249273 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.241941 2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 14 20:22:20.249273 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.241945 2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 14 20:22:20.249273 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.241950 2563 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 14 20:22:20.249273 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.241955 2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 14 20:22:20.249273 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.241960 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 14 20:22:20.249273 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.241964 2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 14 20:22:20.249273 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.241968 2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 14 20:22:20.249273 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.241972 2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 14 20:22:20.249273 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.241977 2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 14 20:22:20.249273 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.241981 2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 14 20:22:20.249273 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.241985 2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 14 20:22:20.249273 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.241989 2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 14 20:22:20.250122 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.241993 2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 14 20:22:20.250122 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.241998 2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 14 20:22:20.250122 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.242003 2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 14 20:22:20.250122 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.242007 2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 14 20:22:20.250122 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.242011 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 14 20:22:20.250122 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.242016 2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 14 20:22:20.250122 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.242020 2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 14 20:22:20.250122 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.242024 2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 14 20:22:20.250122 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.242029 2563 feature_gate.go:328] unrecognized feature gate: Example Jul 14 20:22:20.250122 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.242033 2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 14 20:22:20.250122 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.242037 2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 14 20:22:20.250122 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.242041 2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 14 20:22:20.250122 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.242045 2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 14 20:22:20.250122 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.242050 2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 14 20:22:20.250122 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.242054 2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 14 20:22:20.250122 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.242058 2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 14 20:22:20.250122 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.242062 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 14 20:22:20.250122 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.242066 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 14 20:22:20.250122 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.242071 2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 14 20:22:20.250122 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.242075 2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 14 20:22:20.250931 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.242079 2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 14 20:22:20.250931 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.242084 2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 14 20:22:20.250931 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.242089 2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 14 20:22:20.250931 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.242093 2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 14 20:22:20.250931 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.242098 2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 14 20:22:20.250931 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:20.242102 2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 14 20:22:20.250931 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:20.242110 2563 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 14 20:22:20.250931 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:20.242305 2563 server.go:962] "Client rotation is on, will bootstrap in background" Jul 14 20:22:20.250931 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:20.245325 2563 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 14 20:22:20.250931 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:20.247027 2563 server.go:1019] "Starting client certificate rotation" Jul 14 20:22:20.250931 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:20.247133 2563 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 14 20:22:20.250931 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:20.248018 2563 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 14 20:22:20.250931 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:20.249943 2563 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 14 20:22:20.251790 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:20.251772 2563 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 14 20:22:20.261917 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:20.261898 2563 log.go:25] "Validated CRI v1 runtime API" Jul 14 20:22:20.265924 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:20.265902 2563 log.go:25] "Validated CRI v1 image API" Jul 14 20:22:20.266793 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:20.266776 2563 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 14 20:22:20.271273 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:20.271247 2563 fs.go:135] Filesystem UUIDs: map[4b73ed7d-24c5-4e30-8e10-61da251327bc:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 7e60be5e-53f6-48e2-85d7-47abef1a5b4c:/dev/nvme0n1p3] Jul 14 20:22:20.271351 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:20.271270 2563 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 14 20:22:20.275057 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:20.275031 2563 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 14 20:22:20.277029 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:20.276918 2563 manager.go:216] Machine: {Timestamp:2026-07-14 20:22:20.275060282 +0000 UTC m=+0.314154141 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3111504 MemoryCapacity:32812163072 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec26e0d6a14214e842e0965adb2c14de SystemUUID:ec26e0d6-a142-14e8-42e0-965adb2c14de BootID:cba691b3-aef3-459c-bd91-7e6ba25f5e8b Filesystems:[{Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406081536 Type:vfs Inodes:4005391 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562435072 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406081536 Type:vfs Inodes:1048576 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:47:f7:69:a9:ff Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:47:f7:69:a9:ff Speed:0 Mtu:9001} {Name:ovs-system MacAddress:3a:02:33:09:bd:84 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812163072 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 14 20:22:20.277029 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:20.277023 2563 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 14 20:22:20.277144 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:20.277112 2563 manager.go:232] Version: {KernelVersion:5.14.0-570.126.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260702-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 14 20:22:20.280678 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:20.280652 2563 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 14 20:22:20.280821 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:20.280682 2563 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-141-61.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 14 20:22:20.281468 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:20.281457 2563 topology_manager.go:138] "Creating topology manager with none policy" Jul 14 20:22:20.281508 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:20.281470 2563 container_manager_linux.go:309] "Creating device plugin manager" Jul 14 20:22:20.281508 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:20.281479 2563 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 14 20:22:20.282243 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:20.282232 2563 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 14 20:22:20.283012 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:20.283001 2563 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 14 20:22:20.285638 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:20.285625 2563 state_mem.go:36] "Initialized new in-memory state store" Jul 14 20:22:20.285757 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:20.285748 2563 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 14 20:22:20.285790 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:20.285777 2563 kubelet.go:486] "Attempting to sync node with API server" Jul 14 20:22:20.285790 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:20.285789 2563 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 14 20:22:20.285850 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:20.285802 2563 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 14 20:22:20.285850 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:20.285813 2563 kubelet.go:398] "Adding apiserver pod source" Jul 14 20:22:20.285850 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:20.285822 2563 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 14 20:22:20.286211 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:20.286179 2563 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 14 20:22:20.286258 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:20.286214 2563 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 14 20:22:20.289297 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:20.289278 2563 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1" Jul 14 20:22:20.291120 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:20.291106 2563 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 14 20:22:20.291173 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:20.291137 2563 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 14 20:22:20.291797 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:20.291777 2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 14 20:22:20.291797 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:20.291794 2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 14 20:22:20.291864 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:20.291800 2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 14 20:22:20.291864 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:20.291805 2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 14 20:22:20.291864 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:20.291814 2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 14 20:22:20.291864 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:20.291819 2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 14 20:22:20.291864 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:20.291826 2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 14 20:22:20.291864 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:20.291831 2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 14 20:22:20.291864 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:20.291838 2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 14 20:22:20.291864 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:20.291844 2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 14 20:22:20.291864 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:20.291852 2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 14 20:22:20.291864 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:20.291857 2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 14 20:22:20.292459 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:20.292449 2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 14 20:22:20.292459 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:20.292459 2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 14 20:22:20.295681 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:20.295665 2563 server.go:1268] "Started kubelet" Jul 14 20:22:20.295804 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:20.295775 2563 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 14 20:22:20.295854 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:20.295828 2563 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 14 20:22:20.296073 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:20.296048 2563 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 14 20:22:20.296655 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:20.296588 2563 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-2b2md" Jul 14 20:22:20.296777 ip-10-0-141-61 systemd[1]: Started Kubernetes Kubelet. Jul 14 20:22:20.297453 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:20.297377 2563 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 14 20:22:20.297558 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:20.297528 2563 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-141-61.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 14 20:22:20.297628 ip-10-0-141-61 kubenswrapper[2563]: E0714 20:22:20.297552 2563 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 14 20:22:20.297798 ip-10-0-141-61 kubenswrapper[2563]: E0714 20:22:20.297775 2563 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-141-61.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 14 20:22:20.297854 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:20.297788 2563 server.go:310] "Adding debug handlers to kubelet server" Jul 14 20:22:20.302666 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:20.302224 2563 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-2b2md" Jul 14 20:22:20.304816 ip-10-0-141-61 kubenswrapper[2563]: E0714 20:22:20.304795 2563 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 14 20:22:20.305658 ip-10-0-141-61 kubenswrapper[2563]: E0714 20:22:20.303612 2563 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-141-61.ec2.internal.18c2419bea1d058a default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-141-61.ec2.internal,UID:ip-10-0-141-61.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-141-61.ec2.internal,},FirstTimestamp:2026-07-14 20:22:20.295644554 +0000 UTC m=+0.334738411,LastTimestamp:2026-07-14 20:22:20.295644554 +0000 UTC m=+0.334738411,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-141-61.ec2.internal,}" Jul 14 20:22:20.306482 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:20.306466 2563 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 14 20:22:20.306557 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:20.306498 2563 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 14 20:22:20.306641 ip-10-0-141-61 kubenswrapper[2563]: E0714 20:22:20.306623 2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-61.ec2.internal\" not found" Jul 14 20:22:20.306972 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:20.306960 2563 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 14 20:22:20.306972 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:20.306963 2563 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 14 20:22:20.307083 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:20.306980 2563 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 14 20:22:20.307134 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:20.307118 2563 reconstruct.go:97] "Volume reconstruction finished" Jul 14 20:22:20.307134 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:20.307126 2563 reconciler.go:29] "Reconciler: start to sync state" Jul 14 20:22:20.307808 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:20.307786 2563 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 14 20:22:20.307808 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:20.307808 2563 factory.go:55] Registering systemd factory Jul 14 20:22:20.307942 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:20.307818 2563 factory.go:223] Registration of the systemd container factory successfully Jul 14 20:22:20.308061 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:20.308047 2563 factory.go:153] Registering CRI-O factory Jul 14 20:22:20.308061 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:20.308065 2563 factory.go:223] Registration of the crio container factory successfully Jul 14 20:22:20.308182 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:20.308087 2563 factory.go:103] Registering Raw factory Jul 14 20:22:20.308182 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:20.308101 2563 manager.go:1196] Started watching for new ooms in manager Jul 14 20:22:20.308874 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:20.308854 2563 manager.go:349] Starting recovery of all containers Jul 14 20:22:20.311042 ip-10-0-141-61 kubenswrapper[2563]: E0714 20:22:20.311011 2563 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-141-61.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 14 20:22:20.311140 ip-10-0-141-61 kubenswrapper[2563]: E0714 20:22:20.311070 2563 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 14 20:22:20.321347 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:20.321141 2563 manager.go:354] Recovery completed Jul 14 20:22:20.325725 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:20.325706 2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 14 20:22:20.327769 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:20.327749 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-61.ec2.internal" event="NodeHasSufficientMemory" Jul 14 20:22:20.327858 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:20.327783 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-61.ec2.internal" event="NodeHasNoDiskPressure" Jul 14 20:22:20.327858 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:20.327795 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-61.ec2.internal" event="NodeHasSufficientPID" Jul 14 20:22:20.328393 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:20.328376 2563 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 14 20:22:20.328393 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:20.328391 2563 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 14 20:22:20.328511 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:20.328411 2563 state_mem.go:36] "Initialized new in-memory state store" Jul 14 20:22:20.328511 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:20.328412 2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 14 20:22:20.329483 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:20.329470 2563 policy_none.go:49] "None policy: Start" Jul 14 20:22:20.329548 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:20.329489 2563 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 14 20:22:20.329548 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:20.329503 2563 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 14 20:22:20.329694 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:20.329546 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-61.ec2.internal" event="NodeHasSufficientMemory" Jul 14 20:22:20.329694 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:20.329591 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-61.ec2.internal" event="NodeHasNoDiskPressure" Jul 14 20:22:20.329694 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:20.329607 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-61.ec2.internal" event="NodeHasSufficientPID" Jul 14 20:22:20.330622 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:20.330609 2563 policy_none.go:47] "Start" Jul 14 20:22:20.373871 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:20.373852 2563 manager.go:337] "Starting Device Plugin manager" Jul 14 20:22:20.374010 ip-10-0-141-61 kubenswrapper[2563]: E0714 20:22:20.373901 2563 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 14 20:22:20.374010 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:20.373915 2563 server.go:87] "Starting device plugin registration server" Jul 14 20:22:20.374248 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:20.374236 2563 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 14 20:22:20.374297 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:20.374250 2563 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 14 20:22:20.374382 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:20.374361 2563 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 14 20:22:20.374475 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:20.374458 2563 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 14 20:22:20.374475 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:20.374472 2563 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 14 20:22:20.375055 ip-10-0-141-61 kubenswrapper[2563]: E0714 20:22:20.375034 2563 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 14 20:22:20.375118 ip-10-0-141-61 kubenswrapper[2563]: E0714 20:22:20.375094 2563 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-141-61.ec2.internal\" not found" Jul 14 20:22:20.474686 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:20.474587 2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 14 20:22:20.475949 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:20.475614 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-61.ec2.internal" event="NodeHasSufficientMemory" Jul 14 20:22:20.475949 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:20.475647 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-61.ec2.internal" event="NodeHasNoDiskPressure" Jul 14 20:22:20.475949 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:20.475662 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-61.ec2.internal" event="NodeHasSufficientPID" Jul 14 20:22:20.475949 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:20.475696 2563 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-141-61.ec2.internal" Jul 14 20:22:20.480356 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:20.480327 2563 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 14 20:22:20.481714 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:20.481695 2563 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 14 20:22:20.481800 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:20.481751 2563 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 14 20:22:20.481800 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:20.481779 2563 kubelet.go:2448] "Starting kubelet main sync loop" Jul 14 20:22:20.481892 ip-10-0-141-61 kubenswrapper[2563]: E0714 20:22:20.481822 2563 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 14 20:22:20.484370 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:20.484347 2563 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-141-61.ec2.internal" Jul 14 20:22:20.484370 ip-10-0-141-61 kubenswrapper[2563]: E0714 20:22:20.484370 2563 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-141-61.ec2.internal\": node \"ip-10-0-141-61.ec2.internal\" not found" Jul 14 20:22:20.484599 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:20.484584 2563 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 14 20:22:20.554382 ip-10-0-141-61 kubenswrapper[2563]: E0714 20:22:20.554339 2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-61.ec2.internal\" not found" Jul 14 20:22:20.581959 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:20.581923 2563 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-61.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-141-61.ec2.internal"] Jul 14 20:22:20.582040 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:20.582029 2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 14 20:22:20.583470 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:20.583456 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-61.ec2.internal" event="NodeHasSufficientMemory" Jul 14 20:22:20.583526 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:20.583487 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-61.ec2.internal" event="NodeHasNoDiskPressure" Jul 14 20:22:20.583526 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:20.583497 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-61.ec2.internal" event="NodeHasSufficientPID" Jul 14 20:22:20.585041 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:20.585027 2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 14 20:22:20.585153 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:20.585135 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-61.ec2.internal" Jul 14 20:22:20.585221 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:20.585170 2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 14 20:22:20.586516 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:20.586500 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-61.ec2.internal" event="NodeHasSufficientMemory" Jul 14 20:22:20.586598 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:20.586527 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-61.ec2.internal" event="NodeHasNoDiskPressure" Jul 14 20:22:20.586598 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:20.586537 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-61.ec2.internal" event="NodeHasSufficientPID" Jul 14 20:22:20.586598 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:20.586501 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-61.ec2.internal" event="NodeHasSufficientMemory" Jul 14 20:22:20.586686 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:20.586603 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-61.ec2.internal" event="NodeHasNoDiskPressure" Jul 14 20:22:20.586686 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:20.586614 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-61.ec2.internal" event="NodeHasSufficientPID" Jul 14 20:22:20.587919 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:20.587902 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-61.ec2.internal" Jul 14 20:22:20.588012 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:20.587936 2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 14 20:22:20.588724 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:20.588708 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-61.ec2.internal" event="NodeHasSufficientMemory" Jul 14 20:22:20.588804 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:20.588732 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-61.ec2.internal" event="NodeHasNoDiskPressure" Jul 14 20:22:20.588804 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:20.588741 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-61.ec2.internal" event="NodeHasSufficientPID" Jul 14 20:22:20.608812 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:20.608788 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/9a7e15c41e1da145327771a9c8cb3e39-config\") pod \"kube-apiserver-proxy-ip-10-0-141-61.ec2.internal\" (UID: \"9a7e15c41e1da145327771a9c8cb3e39\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-141-61.ec2.internal" Jul 14 20:22:20.608914 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:20.608817 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/0110f0e9eedb7f728fd5fbcff19c03aa-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-141-61.ec2.internal\" (UID: \"0110f0e9eedb7f728fd5fbcff19c03aa\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-61.ec2.internal" Jul 14 20:22:20.608914 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:20.608846 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0110f0e9eedb7f728fd5fbcff19c03aa-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-141-61.ec2.internal\" (UID: \"0110f0e9eedb7f728fd5fbcff19c03aa\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-61.ec2.internal" Jul 14 20:22:20.620003 ip-10-0-141-61 kubenswrapper[2563]: E0714 20:22:20.619979 2563 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-141-61.ec2.internal\" not found" node="ip-10-0-141-61.ec2.internal" Jul 14 20:22:20.623453 ip-10-0-141-61 kubenswrapper[2563]: E0714 20:22:20.623437 2563 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-141-61.ec2.internal\" not found" node="ip-10-0-141-61.ec2.internal" Jul 14 20:22:20.655469 ip-10-0-141-61 kubenswrapper[2563]: E0714 20:22:20.655442 2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-61.ec2.internal\" not found" Jul 14 20:22:20.709045 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:20.709014 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0110f0e9eedb7f728fd5fbcff19c03aa-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-141-61.ec2.internal\" (UID: \"0110f0e9eedb7f728fd5fbcff19c03aa\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-61.ec2.internal" Jul 14 20:22:20.709045 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:20.709048 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/9a7e15c41e1da145327771a9c8cb3e39-config\") pod \"kube-apiserver-proxy-ip-10-0-141-61.ec2.internal\" (UID: \"9a7e15c41e1da145327771a9c8cb3e39\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-141-61.ec2.internal" Jul 14 20:22:20.709262 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:20.709065 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/0110f0e9eedb7f728fd5fbcff19c03aa-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-141-61.ec2.internal\" (UID: \"0110f0e9eedb7f728fd5fbcff19c03aa\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-61.ec2.internal" Jul 14 20:22:20.709262 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:20.709108 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/0110f0e9eedb7f728fd5fbcff19c03aa-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-141-61.ec2.internal\" (UID: \"0110f0e9eedb7f728fd5fbcff19c03aa\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-61.ec2.internal" Jul 14 20:22:20.709262 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:20.709139 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/9a7e15c41e1da145327771a9c8cb3e39-config\") pod \"kube-apiserver-proxy-ip-10-0-141-61.ec2.internal\" (UID: \"9a7e15c41e1da145327771a9c8cb3e39\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-141-61.ec2.internal" Jul 14 20:22:20.709262 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:20.709115 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0110f0e9eedb7f728fd5fbcff19c03aa-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-141-61.ec2.internal\" (UID: \"0110f0e9eedb7f728fd5fbcff19c03aa\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-61.ec2.internal" Jul 14 20:22:20.756347 ip-10-0-141-61 kubenswrapper[2563]: E0714 20:22:20.756265 2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-61.ec2.internal\" not found" Jul 14 20:22:20.856960 ip-10-0-141-61 kubenswrapper[2563]: E0714 20:22:20.856920 2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-61.ec2.internal\" not found" Jul 14 20:22:20.922424 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:20.922397 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-61.ec2.internal" Jul 14 20:22:20.926215 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:20.926028 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-61.ec2.internal" Jul 14 20:22:20.957718 ip-10-0-141-61 kubenswrapper[2563]: E0714 20:22:20.957684 2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-61.ec2.internal\" not found" Jul 14 20:22:22.176228 ip-10-0-141-61 kubenswrapper[2563]: E0714 20:22:22.176194 2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-61.ec2.internal\" not found" Jul 14 20:22:22.276700 ip-10-0-141-61 kubenswrapper[2563]: E0714 20:22:22.276665 2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-61.ec2.internal\" not found" Jul 14 20:22:22.364419 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:22.364375 2563 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 14 20:22:22.364956 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:22.364537 2563 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 14 20:22:22.377674 ip-10-0-141-61 kubenswrapper[2563]: E0714 20:22:22.377649 2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-61.ec2.internal\" not found" Jul 14 20:22:22.427473 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:22.427385 2563 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-13 20:17:21 +0000 UTC" deadline="2028-04-15 05:21:54.687877774 +0000 UTC" Jul 14 20:22:22.427473 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:22.427422 2563 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15368h59m32.260460636s" Jul 14 20:22:22.478069 ip-10-0-141-61 kubenswrapper[2563]: E0714 20:22:22.478035 2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-61.ec2.internal\" not found" Jul 14 20:22:22.564645 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:22.564618 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0110f0e9eedb7f728fd5fbcff19c03aa.slice/crio-3091b3c06f7f52adcdcf7f813cfcd25ab3ca07bb4d69c69ac5b3d10ab10561ae WatchSource:0}: Error finding container 3091b3c06f7f52adcdcf7f813cfcd25ab3ca07bb4d69c69ac5b3d10ab10561ae: Status 404 returned error can't find the container with id 3091b3c06f7f52adcdcf7f813cfcd25ab3ca07bb4d69c69ac5b3d10ab10561ae Jul 14 20:22:22.565061 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:22.565040 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9a7e15c41e1da145327771a9c8cb3e39.slice/crio-c49cb5e5811ac0794db411ee435b08b5620bbf759c090f7cd451976268267ab4 WatchSource:0}: Error finding container c49cb5e5811ac0794db411ee435b08b5620bbf759c090f7cd451976268267ab4: Status 404 returned error can't find the container with id c49cb5e5811ac0794db411ee435b08b5620bbf759c090f7cd451976268267ab4 Jul 14 20:22:22.565175 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:22.565103 2563 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 14 20:22:22.571105 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:22.571088 2563 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 14 20:22:22.603075 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:22.603018 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-61.ec2.internal" event={"ID":"0110f0e9eedb7f728fd5fbcff19c03aa","Type":"ContainerStarted","Data":"3091b3c06f7f52adcdcf7f813cfcd25ab3ca07bb4d69c69ac5b3d10ab10561ae"} Jul 14 20:22:22.604065 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:22.604037 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-61.ec2.internal" event={"ID":"9a7e15c41e1da145327771a9c8cb3e39","Type":"ContainerStarted","Data":"c49cb5e5811ac0794db411ee435b08b5620bbf759c090f7cd451976268267ab4"} Jul 14 20:22:22.625297 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:22.625276 2563 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-61.ec2.internal" Jul 14 20:22:22.639556 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:22.639525 2563 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 14 20:22:22.641158 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:22.641143 2563 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-61.ec2.internal" Jul 14 20:22:22.648254 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:22.648240 2563 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 14 20:22:22.792608 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:22.792524 2563 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 14 20:22:22.798712 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:22.798694 2563 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 14 20:22:23.405212 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:23.405185 2563 apiserver.go:52] "Watching apiserver" Jul 14 20:22:23.414117 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:23.414087 2563 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 14 20:22:23.414963 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:23.414938 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-target-2r82p","kube-system/konnectivity-agent-dwzbf","kube-system/kube-apiserver-proxy-ip-10-0-141-61.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-g77ql","openshift-cluster-node-tuning-operator/tuned-zltwm","openshift-image-registry/node-ca-88985","openshift-multus/multus-bbzpp","openshift-network-operator/iptables-alerter-9bnrv","openshift-ovn-kubernetes/ovnkube-node-hzbx8","kube-system/global-pull-secret-syncer-sgvdx","openshift-dns/node-resolver-dznvt","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-61.ec2.internal","openshift-multus/multus-additional-cni-plugins-jzpsb","openshift-multus/network-metrics-daemon-rvhpg"] Jul 14 20:22:23.416509 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:23.416485 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-bbzpp" Jul 14 20:22:23.417556 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:23.417515 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-jzpsb" Jul 14 20:22:23.418533 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:23.418511 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rvhpg" Jul 14 20:22:23.418647 ip-10-0-141-61 kubenswrapper[2563]: E0714 20:22:23.418585 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rvhpg" podUID="92cbd473-9b2e-4faa-a232-810627648cc9" Jul 14 20:22:23.418712 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:23.418623 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 14 20:22:23.418859 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:23.418843 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 14 20:22:23.418933 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:23.418918 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 14 20:22:23.419220 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:23.419202 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 14 20:22:23.419281 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:23.419225 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 14 20:22:23.419522 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:23.419498 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-qmld8\"" Jul 14 20:22:23.419590 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:23.419508 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-dt9kg\"" Jul 14 20:22:23.419721 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:23.419700 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 14 20:22:23.419936 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:23.419916 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-g77ql" Jul 14 20:22:23.420819 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:23.420800 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-zltwm" Jul 14 20:22:23.421832 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:23.421807 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 14 20:22:23.422078 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:23.422056 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-88985" Jul 14 20:22:23.422205 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:23.422167 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-kzwtc\"" Jul 14 20:22:23.423155 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:23.422339 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 14 20:22:23.423155 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:23.422610 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 14 20:22:23.423155 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:23.422840 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 14 20:22:23.423155 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:23.422841 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-7c8jc\"" Jul 14 20:22:23.423155 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:23.422969 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 14 20:22:23.423709 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:23.423690 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-hzbx8" Jul 14 20:22:23.423793 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:23.423726 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 14 20:22:23.424263 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:23.424226 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 14 20:22:23.424466 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:23.424451 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 14 20:22:23.424536 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:23.424508 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-ptfwh\"" Jul 14 20:22:23.425954 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:23.425848 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2r82p" Jul 14 20:22:23.425954 ip-10-0-141-61 kubenswrapper[2563]: E0714 20:22:23.425906 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2r82p" podUID="92f9c302-6d80-4679-9c71-1d37255dc852" Jul 14 20:22:23.426161 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:23.426115 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 14 20:22:23.426270 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:23.426213 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 14 20:22:23.426589 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:23.426388 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-tcb2j\"" Jul 14 20:22:23.426589 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:23.426568 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 14 20:22:23.426734 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:23.426386 2563 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 14 20:22:23.426789 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:23.426768 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 14 20:22:23.426916 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:23.426898 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 14 20:22:23.426992 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:23.426971 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-dwzbf" Jul 14 20:22:23.426992 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:23.426978 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 14 20:22:23.427834 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:23.427818 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sgvdx" Jul 14 20:22:23.427922 ip-10-0-141-61 kubenswrapper[2563]: E0714 20:22:23.427881 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sgvdx" podUID="05045fa2-6bd3-4ee8-8f2a-e9056c8a316f" Jul 14 20:22:23.428877 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:23.428858 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-d6g7v\"" Jul 14 20:22:23.429051 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:23.429029 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 14 20:22:23.429246 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:23.429228 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 14 20:22:23.429471 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:23.429457 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-dznvt" Jul 14 20:22:23.431114 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:23.431057 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 14 20:22:23.431284 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:23.431268 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-hjbt2\"" Jul 14 20:22:23.431479 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:23.431459 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-9bnrv" Jul 14 20:22:23.431911 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:23.431891 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 14 20:22:23.433425 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:23.433400 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-twxlb\"" Jul 14 20:22:23.434318 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:23.433967 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 14 20:22:23.434318 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:23.434071 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 14 20:22:23.434318 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:23.434201 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 14 20:22:23.439368 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:23.439344 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ztchf\" (UniqueName: \"kubernetes.io/projected/035331bf-20d9-4df2-99f6-b0e0db2e05f0-kube-api-access-ztchf\") pod \"multus-additional-cni-plugins-jzpsb\" (UID: \"035331bf-20d9-4df2-99f6-b0e0db2e05f0\") " pod="openshift-multus/multus-additional-cni-plugins-jzpsb" Jul 14 20:22:23.439476 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:23.439376 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/5835902d-ae0d-4395-9843-0ef04ad24976-multus-daemon-config\") pod \"multus-bbzpp\" (UID: \"5835902d-ae0d-4395-9843-0ef04ad24976\") " pod="openshift-multus/multus-bbzpp" Jul 14 20:22:23.439476 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:23.439398 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/035331bf-20d9-4df2-99f6-b0e0db2e05f0-system-cni-dir\") pod \"multus-additional-cni-plugins-jzpsb\" (UID: \"035331bf-20d9-4df2-99f6-b0e0db2e05f0\") " pod="openshift-multus/multus-additional-cni-plugins-jzpsb" Jul 14 20:22:23.439476 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:23.439420 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/07899815-131f-4211-a7f0-6ce06c8c9ea1-etc-selinux\") pod \"aws-ebs-csi-driver-node-g77ql\" (UID: \"07899815-131f-4211-a7f0-6ce06c8c9ea1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-g77ql" Jul 14 20:22:23.439476 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:23.439469 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/b36e05db-0a22-4de6-bed8-73b483a3a2a2-etc-modprobe-d\") pod \"tuned-zltwm\" (UID: \"b36e05db-0a22-4de6-bed8-73b483a3a2a2\") " pod="openshift-cluster-node-tuning-operator/tuned-zltwm" Jul 14 20:22:23.439666 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:23.439488 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/b36e05db-0a22-4de6-bed8-73b483a3a2a2-etc-sysctl-d\") pod \"tuned-zltwm\" (UID: \"b36e05db-0a22-4de6-bed8-73b483a3a2a2\") " pod="openshift-cluster-node-tuning-operator/tuned-zltwm" Jul 14 20:22:23.439666 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:23.439503 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/b36e05db-0a22-4de6-bed8-73b483a3a2a2-etc-sysctl-conf\") pod \"tuned-zltwm\" (UID: \"b36e05db-0a22-4de6-bed8-73b483a3a2a2\") " pod="openshift-cluster-node-tuning-operator/tuned-zltwm" Jul 14 20:22:23.439666 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:23.439520 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/b36e05db-0a22-4de6-bed8-73b483a3a2a2-run\") pod \"tuned-zltwm\" (UID: \"b36e05db-0a22-4de6-bed8-73b483a3a2a2\") " pod="openshift-cluster-node-tuning-operator/tuned-zltwm" Jul 14 20:22:23.439666 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:23.439543 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5835902d-ae0d-4395-9843-0ef04ad24976-system-cni-dir\") pod \"multus-bbzpp\" (UID: \"5835902d-ae0d-4395-9843-0ef04ad24976\") " pod="openshift-multus/multus-bbzpp" Jul 14 20:22:23.439666 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:23.439561 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/07899815-131f-4211-a7f0-6ce06c8c9ea1-sys-fs\") pod \"aws-ebs-csi-driver-node-g77ql\" (UID: \"07899815-131f-4211-a7f0-6ce06c8c9ea1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-g77ql" Jul 14 20:22:23.439666 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:23.439580 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b36e05db-0a22-4de6-bed8-73b483a3a2a2-lib-modules\") pod \"tuned-zltwm\" (UID: \"b36e05db-0a22-4de6-bed8-73b483a3a2a2\") " pod="openshift-cluster-node-tuning-operator/tuned-zltwm" Jul 14 20:22:23.439666 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:23.439642 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/afbb09bb-badd-4dc0-8082-581c48dda65b-host\") pod \"node-ca-88985\" (UID: \"afbb09bb-badd-4dc0-8082-581c48dda65b\") " pod="openshift-image-registry/node-ca-88985" Jul 14 20:22:23.439666 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:23.439661 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cbwhw\" (UniqueName: \"kubernetes.io/projected/afbb09bb-badd-4dc0-8082-581c48dda65b-kube-api-access-cbwhw\") pod \"node-ca-88985\" (UID: \"afbb09bb-badd-4dc0-8082-581c48dda65b\") " pod="openshift-image-registry/node-ca-88985" Jul 14 20:22:23.440003 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:23.439679 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/9588ab98-33e8-4c89-807d-a805240f1b81-run-systemd\") pod \"ovnkube-node-hzbx8\" (UID: \"9588ab98-33e8-4c89-807d-a805240f1b81\") " pod="openshift-ovn-kubernetes/ovnkube-node-hzbx8" Jul 14 20:22:23.440003 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:23.439720 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9588ab98-33e8-4c89-807d-a805240f1b81-var-lib-openvswitch\") pod \"ovnkube-node-hzbx8\" (UID: \"9588ab98-33e8-4c89-807d-a805240f1b81\") " pod="openshift-ovn-kubernetes/ovnkube-node-hzbx8" Jul 14 20:22:23.440003 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:23.439737 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/9588ab98-33e8-4c89-807d-a805240f1b81-env-overrides\") pod \"ovnkube-node-hzbx8\" (UID: \"9588ab98-33e8-4c89-807d-a805240f1b81\") " pod="openshift-ovn-kubernetes/ovnkube-node-hzbx8" Jul 14 20:22:23.440003 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:23.439753 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/5835902d-ae0d-4395-9843-0ef04ad24976-host-run-multus-certs\") pod \"multus-bbzpp\" (UID: \"5835902d-ae0d-4395-9843-0ef04ad24976\") " pod="openshift-multus/multus-bbzpp" Jul 14 20:22:23.440003 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:23.439833 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/b36e05db-0a22-4de6-bed8-73b483a3a2a2-etc-sysconfig\") pod \"tuned-zltwm\" (UID: \"b36e05db-0a22-4de6-bed8-73b483a3a2a2\") " pod="openshift-cluster-node-tuning-operator/tuned-zltwm" Jul 14 20:22:23.440003 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:23.439851 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/9588ab98-33e8-4c89-807d-a805240f1b81-host-slash\") pod \"ovnkube-node-hzbx8\" (UID: \"9588ab98-33e8-4c89-807d-a805240f1b81\") " pod="openshift-ovn-kubernetes/ovnkube-node-hzbx8" Jul 14 20:22:23.440003 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:23.439886 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/5835902d-ae0d-4395-9843-0ef04ad24976-cni-binary-copy\") pod \"multus-bbzpp\" (UID: \"5835902d-ae0d-4395-9843-0ef04ad24976\") " pod="openshift-multus/multus-bbzpp" Jul 14 20:22:23.440003 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:23.439912 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/035331bf-20d9-4df2-99f6-b0e0db2e05f0-cnibin\") pod \"multus-additional-cni-plugins-jzpsb\" (UID: \"035331bf-20d9-4df2-99f6-b0e0db2e05f0\") " pod="openshift-multus/multus-additional-cni-plugins-jzpsb" Jul 14 20:22:23.440003 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:23.439931 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/92cbd473-9b2e-4faa-a232-810627648cc9-metrics-certs\") pod \"network-metrics-daemon-rvhpg\" (UID: \"92cbd473-9b2e-4faa-a232-810627648cc9\") " pod="openshift-multus/network-metrics-daemon-rvhpg" Jul 14 20:22:23.440003 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:23.439952 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/07899815-131f-4211-a7f0-6ce06c8c9ea1-registration-dir\") pod \"aws-ebs-csi-driver-node-g77ql\" (UID: \"07899815-131f-4211-a7f0-6ce06c8c9ea1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-g77ql" Jul 14 20:22:23.440003 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:23.439971 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/9588ab98-33e8-4c89-807d-a805240f1b81-ovnkube-script-lib\") pod \"ovnkube-node-hzbx8\" (UID: \"9588ab98-33e8-4c89-807d-a805240f1b81\") " pod="openshift-ovn-kubernetes/ovnkube-node-hzbx8" Jul 14 20:22:23.440507 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:23.439996 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rv8g8\" (UniqueName: \"kubernetes.io/projected/92cbd473-9b2e-4faa-a232-810627648cc9-kube-api-access-rv8g8\") pod \"network-metrics-daemon-rvhpg\" (UID: \"92cbd473-9b2e-4faa-a232-810627648cc9\") " pod="openshift-multus/network-metrics-daemon-rvhpg" Jul 14 20:22:23.440507 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:23.440171 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-26qpq\" (UniqueName: \"kubernetes.io/projected/07899815-131f-4211-a7f0-6ce06c8c9ea1-kube-api-access-26qpq\") pod \"aws-ebs-csi-driver-node-g77ql\" (UID: \"07899815-131f-4211-a7f0-6ce06c8c9ea1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-g77ql" Jul 14 20:22:23.440507 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:23.440187 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b36e05db-0a22-4de6-bed8-73b483a3a2a2-tmp\") pod \"tuned-zltwm\" (UID: \"b36e05db-0a22-4de6-bed8-73b483a3a2a2\") " pod="openshift-cluster-node-tuning-operator/tuned-zltwm" Jul 14 20:22:23.440507 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:23.440205 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/9588ab98-33e8-4c89-807d-a805240f1b81-ovn-node-metrics-cert\") pod \"ovnkube-node-hzbx8\" (UID: \"9588ab98-33e8-4c89-807d-a805240f1b81\") " pod="openshift-ovn-kubernetes/ovnkube-node-hzbx8" Jul 14 20:22:23.440507 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:23.440223 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5835902d-ae0d-4395-9843-0ef04ad24976-multus-cni-dir\") pod \"multus-bbzpp\" (UID: \"5835902d-ae0d-4395-9843-0ef04ad24976\") " pod="openshift-multus/multus-bbzpp" Jul 14 20:22:23.440507 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:23.440241 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/5835902d-ae0d-4395-9843-0ef04ad24976-cnibin\") pod \"multus-bbzpp\" (UID: \"5835902d-ae0d-4395-9843-0ef04ad24976\") " pod="openshift-multus/multus-bbzpp" Jul 14 20:22:23.440507 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:23.440258 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/5835902d-ae0d-4395-9843-0ef04ad24976-os-release\") pod \"multus-bbzpp\" (UID: \"5835902d-ae0d-4395-9843-0ef04ad24976\") " pod="openshift-multus/multus-bbzpp" Jul 14 20:22:23.440507 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:23.440255 2563 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 14 20:22:23.440507 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:23.440289 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/035331bf-20d9-4df2-99f6-b0e0db2e05f0-os-release\") pod \"multus-additional-cni-plugins-jzpsb\" (UID: \"035331bf-20d9-4df2-99f6-b0e0db2e05f0\") " pod="openshift-multus/multus-additional-cni-plugins-jzpsb" Jul 14 20:22:23.440507 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:23.440311 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/07899815-131f-4211-a7f0-6ce06c8c9ea1-socket-dir\") pod \"aws-ebs-csi-driver-node-g77ql\" (UID: \"07899815-131f-4211-a7f0-6ce06c8c9ea1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-g77ql" Jul 14 20:22:23.440507 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:23.440335 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/5835902d-ae0d-4395-9843-0ef04ad24976-host-run-netns\") pod \"multus-bbzpp\" (UID: \"5835902d-ae0d-4395-9843-0ef04ad24976\") " pod="openshift-multus/multus-bbzpp" Jul 14 20:22:23.440507 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:23.440396 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/07899815-131f-4211-a7f0-6ce06c8c9ea1-device-dir\") pod \"aws-ebs-csi-driver-node-g77ql\" (UID: \"07899815-131f-4211-a7f0-6ce06c8c9ea1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-g77ql" Jul 14 20:22:23.440507 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:23.440424 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/9588ab98-33e8-4c89-807d-a805240f1b81-ovnkube-config\") pod \"ovnkube-node-hzbx8\" (UID: \"9588ab98-33e8-4c89-807d-a805240f1b81\") " pod="openshift-ovn-kubernetes/ovnkube-node-hzbx8" Jul 14 20:22:23.440507 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:23.440459 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/035331bf-20d9-4df2-99f6-b0e0db2e05f0-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-jzpsb\" (UID: \"035331bf-20d9-4df2-99f6-b0e0db2e05f0\") " pod="openshift-multus/multus-additional-cni-plugins-jzpsb" Jul 14 20:22:23.441146 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:23.440480 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9588ab98-33e8-4c89-807d-a805240f1b81-run-openvswitch\") pod \"ovnkube-node-hzbx8\" (UID: \"9588ab98-33e8-4c89-807d-a805240f1b81\") " pod="openshift-ovn-kubernetes/ovnkube-node-hzbx8" Jul 14 20:22:23.441146 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:23.440614 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/9588ab98-33e8-4c89-807d-a805240f1b81-node-log\") pod \"ovnkube-node-hzbx8\" (UID: \"9588ab98-33e8-4c89-807d-a805240f1b81\") " pod="openshift-ovn-kubernetes/ovnkube-node-hzbx8" Jul 14 20:22:23.441146 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:23.440633 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/9588ab98-33e8-4c89-807d-a805240f1b81-log-socket\") pod \"ovnkube-node-hzbx8\" (UID: \"9588ab98-33e8-4c89-807d-a805240f1b81\") " pod="openshift-ovn-kubernetes/ovnkube-node-hzbx8" Jul 14 20:22:23.441146 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:23.440652 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/5835902d-ae0d-4395-9843-0ef04ad24976-host-run-k8s-cni-cncf-io\") pod \"multus-bbzpp\" (UID: \"5835902d-ae0d-4395-9843-0ef04ad24976\") " pod="openshift-multus/multus-bbzpp" Jul 14 20:22:23.441146 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:23.440756 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/5835902d-ae0d-4395-9843-0ef04ad24976-host-var-lib-cni-bin\") pod \"multus-bbzpp\" (UID: \"5835902d-ae0d-4395-9843-0ef04ad24976\") " pod="openshift-multus/multus-bbzpp" Jul 14 20:22:23.441146 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:23.440784 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b36e05db-0a22-4de6-bed8-73b483a3a2a2-sys\") pod \"tuned-zltwm\" (UID: \"b36e05db-0a22-4de6-bed8-73b483a3a2a2\") " pod="openshift-cluster-node-tuning-operator/tuned-zltwm" Jul 14 20:22:23.441146 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:23.440812 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/3d2553b9-0627-4fd5-b973-76e17eabf098-konnectivity-ca\") pod \"konnectivity-agent-dwzbf\" (UID: \"3d2553b9-0627-4fd5-b973-76e17eabf098\") " pod="kube-system/konnectivity-agent-dwzbf" Jul 14 20:22:23.441146 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:23.440832 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/035331bf-20d9-4df2-99f6-b0e0db2e05f0-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-jzpsb\" (UID: \"035331bf-20d9-4df2-99f6-b0e0db2e05f0\") " pod="openshift-multus/multus-additional-cni-plugins-jzpsb" Jul 14 20:22:23.441146 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:23.440852 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/035331bf-20d9-4df2-99f6-b0e0db2e05f0-tuning-conf-dir\") pod \"multus-additional-cni-plugins-jzpsb\" (UID: \"035331bf-20d9-4df2-99f6-b0e0db2e05f0\") " pod="openshift-multus/multus-additional-cni-plugins-jzpsb" Jul 14 20:22:23.441146 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:23.440870 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/afbb09bb-badd-4dc0-8082-581c48dda65b-serviceca\") pod \"node-ca-88985\" (UID: \"afbb09bb-badd-4dc0-8082-581c48dda65b\") " pod="openshift-image-registry/node-ca-88985" Jul 14 20:22:23.441146 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:23.440886 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pkcsg\" (UniqueName: \"kubernetes.io/projected/9588ab98-33e8-4c89-807d-a805240f1b81-kube-api-access-pkcsg\") pod \"ovnkube-node-hzbx8\" (UID: \"9588ab98-33e8-4c89-807d-a805240f1b81\") " pod="openshift-ovn-kubernetes/ovnkube-node-hzbx8" Jul 14 20:22:23.441146 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:23.440906 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b36e05db-0a22-4de6-bed8-73b483a3a2a2-var-lib-kubelet\") pod \"tuned-zltwm\" (UID: \"b36e05db-0a22-4de6-bed8-73b483a3a2a2\") " pod="openshift-cluster-node-tuning-operator/tuned-zltwm" Jul 14 20:22:23.441146 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:23.440934 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b36e05db-0a22-4de6-bed8-73b483a3a2a2-host\") pod \"tuned-zltwm\" (UID: \"b36e05db-0a22-4de6-bed8-73b483a3a2a2\") " pod="openshift-cluster-node-tuning-operator/tuned-zltwm" Jul 14 20:22:23.441146 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:23.440953 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/9588ab98-33e8-4c89-807d-a805240f1b81-host-kubelet\") pod \"ovnkube-node-hzbx8\" (UID: \"9588ab98-33e8-4c89-807d-a805240f1b81\") " pod="openshift-ovn-kubernetes/ovnkube-node-hzbx8" Jul 14 20:22:23.441146 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:23.440976 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/9588ab98-33e8-4c89-807d-a805240f1b81-host-run-netns\") pod \"ovnkube-node-hzbx8\" (UID: \"9588ab98-33e8-4c89-807d-a805240f1b81\") " pod="openshift-ovn-kubernetes/ovnkube-node-hzbx8" Jul 14 20:22:23.441146 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:23.440996 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9588ab98-33e8-4c89-807d-a805240f1b81-etc-openvswitch\") pod \"ovnkube-node-hzbx8\" (UID: \"9588ab98-33e8-4c89-807d-a805240f1b81\") " pod="openshift-ovn-kubernetes/ovnkube-node-hzbx8" Jul 14 20:22:23.441838 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:23.441014 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/5835902d-ae0d-4395-9843-0ef04ad24976-multus-socket-dir-parent\") pod \"multus-bbzpp\" (UID: \"5835902d-ae0d-4395-9843-0ef04ad24976\") " pod="openshift-multus/multus-bbzpp" Jul 14 20:22:23.441838 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:23.441032 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/5835902d-ae0d-4395-9843-0ef04ad24976-host-var-lib-cni-multus\") pod \"multus-bbzpp\" (UID: \"5835902d-ae0d-4395-9843-0ef04ad24976\") " pod="openshift-multus/multus-bbzpp" Jul 14 20:22:23.441838 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:23.441050 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5835902d-ae0d-4395-9843-0ef04ad24976-host-var-lib-kubelet\") pod \"multus-bbzpp\" (UID: \"5835902d-ae0d-4395-9843-0ef04ad24976\") " pod="openshift-multus/multus-bbzpp" Jul 14 20:22:23.441838 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:23.441072 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/b36e05db-0a22-4de6-bed8-73b483a3a2a2-etc-tuned\") pod \"tuned-zltwm\" (UID: \"b36e05db-0a22-4de6-bed8-73b483a3a2a2\") " pod="openshift-cluster-node-tuning-operator/tuned-zltwm" Jul 14 20:22:23.441838 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:23.441090 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/5835902d-ae0d-4395-9843-0ef04ad24976-multus-conf-dir\") pod \"multus-bbzpp\" (UID: \"5835902d-ae0d-4395-9843-0ef04ad24976\") " pod="openshift-multus/multus-bbzpp" Jul 14 20:22:23.441838 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:23.441111 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dhk5d\" (UniqueName: \"kubernetes.io/projected/5835902d-ae0d-4395-9843-0ef04ad24976-kube-api-access-dhk5d\") pod \"multus-bbzpp\" (UID: \"5835902d-ae0d-4395-9843-0ef04ad24976\") " pod="openshift-multus/multus-bbzpp" Jul 14 20:22:23.441838 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:23.441151 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b36e05db-0a22-4de6-bed8-73b483a3a2a2-etc-kubernetes\") pod \"tuned-zltwm\" (UID: \"b36e05db-0a22-4de6-bed8-73b483a3a2a2\") " pod="openshift-cluster-node-tuning-operator/tuned-zltwm" Jul 14 20:22:23.441838 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:23.441170 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/b36e05db-0a22-4de6-bed8-73b483a3a2a2-etc-systemd\") pod \"tuned-zltwm\" (UID: \"b36e05db-0a22-4de6-bed8-73b483a3a2a2\") " pod="openshift-cluster-node-tuning-operator/tuned-zltwm" Jul 14 20:22:23.441838 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:23.441189 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9588ab98-33e8-4c89-807d-a805240f1b81-host-run-ovn-kubernetes\") pod \"ovnkube-node-hzbx8\" (UID: \"9588ab98-33e8-4c89-807d-a805240f1b81\") " pod="openshift-ovn-kubernetes/ovnkube-node-hzbx8" Jul 14 20:22:23.441838 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:23.441211 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/9588ab98-33e8-4c89-807d-a805240f1b81-host-cni-netd\") pod \"ovnkube-node-hzbx8\" (UID: \"9588ab98-33e8-4c89-807d-a805240f1b81\") " pod="openshift-ovn-kubernetes/ovnkube-node-hzbx8" Jul 14 20:22:23.441838 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:23.441234 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r8lsz\" (UniqueName: \"kubernetes.io/projected/92f9c302-6d80-4679-9c71-1d37255dc852-kube-api-access-r8lsz\") pod \"network-check-target-2r82p\" (UID: \"92f9c302-6d80-4679-9c71-1d37255dc852\") " pod="openshift-network-diagnostics/network-check-target-2r82p" Jul 14 20:22:23.442924 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:23.442898 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/5835902d-ae0d-4395-9843-0ef04ad24976-hostroot\") pod \"multus-bbzpp\" (UID: \"5835902d-ae0d-4395-9843-0ef04ad24976\") " pod="openshift-multus/multus-bbzpp" Jul 14 20:22:23.443010 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:23.442968 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/07899815-131f-4211-a7f0-6ce06c8c9ea1-kubelet-dir\") pod \"aws-ebs-csi-driver-node-g77ql\" (UID: \"07899815-131f-4211-a7f0-6ce06c8c9ea1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-g77ql" Jul 14 20:22:23.443154 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:23.443062 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/9588ab98-33e8-4c89-807d-a805240f1b81-systemd-units\") pod \"ovnkube-node-hzbx8\" (UID: \"9588ab98-33e8-4c89-807d-a805240f1b81\") " pod="openshift-ovn-kubernetes/ovnkube-node-hzbx8" Jul 14 20:22:23.443154 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:23.443111 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/9588ab98-33e8-4c89-807d-a805240f1b81-run-ovn\") pod \"ovnkube-node-hzbx8\" (UID: \"9588ab98-33e8-4c89-807d-a805240f1b81\") " pod="openshift-ovn-kubernetes/ovnkube-node-hzbx8" Jul 14 20:22:23.443313 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:23.443157 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/9588ab98-33e8-4c89-807d-a805240f1b81-host-cni-bin\") pod \"ovnkube-node-hzbx8\" (UID: \"9588ab98-33e8-4c89-807d-a805240f1b81\") " pod="openshift-ovn-kubernetes/ovnkube-node-hzbx8" Jul 14 20:22:23.443313 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:23.443182 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9588ab98-33e8-4c89-807d-a805240f1b81-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-hzbx8\" (UID: \"9588ab98-33e8-4c89-807d-a805240f1b81\") " pod="openshift-ovn-kubernetes/ovnkube-node-hzbx8" Jul 14 20:22:23.443313 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:23.443254 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/3d2553b9-0627-4fd5-b973-76e17eabf098-agent-certs\") pod \"konnectivity-agent-dwzbf\" (UID: \"3d2553b9-0627-4fd5-b973-76e17eabf098\") " pod="kube-system/konnectivity-agent-dwzbf" Jul 14 20:22:23.443313 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:23.443298 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5835902d-ae0d-4395-9843-0ef04ad24976-etc-kubernetes\") pod \"multus-bbzpp\" (UID: \"5835902d-ae0d-4395-9843-0ef04ad24976\") " pod="openshift-multus/multus-bbzpp" Jul 14 20:22:23.443484 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:23.443332 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/035331bf-20d9-4df2-99f6-b0e0db2e05f0-cni-binary-copy\") pod \"multus-additional-cni-plugins-jzpsb\" (UID: \"035331bf-20d9-4df2-99f6-b0e0db2e05f0\") " pod="openshift-multus/multus-additional-cni-plugins-jzpsb" Jul 14 20:22:23.443484 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:23.443367 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cxsdd\" (UniqueName: \"kubernetes.io/projected/b36e05db-0a22-4de6-bed8-73b483a3a2a2-kube-api-access-cxsdd\") pod \"tuned-zltwm\" (UID: \"b36e05db-0a22-4de6-bed8-73b483a3a2a2\") " pod="openshift-cluster-node-tuning-operator/tuned-zltwm" Jul 14 20:22:23.458800 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:23.458776 2563 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-bnz54" Jul 14 20:22:23.467070 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:23.467047 2563 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-bnz54" Jul 14 20:22:23.526466 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:23.526436 2563 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 14 20:22:23.543535 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:23.543501 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dhk5d\" (UniqueName: \"kubernetes.io/projected/5835902d-ae0d-4395-9843-0ef04ad24976-kube-api-access-dhk5d\") pod \"multus-bbzpp\" (UID: \"5835902d-ae0d-4395-9843-0ef04ad24976\") " pod="openshift-multus/multus-bbzpp" Jul 14 20:22:23.543707 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:23.543543 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b36e05db-0a22-4de6-bed8-73b483a3a2a2-etc-kubernetes\") pod \"tuned-zltwm\" (UID: \"b36e05db-0a22-4de6-bed8-73b483a3a2a2\") " pod="openshift-cluster-node-tuning-operator/tuned-zltwm" Jul 14 20:22:23.543707 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:23.543563 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/b36e05db-0a22-4de6-bed8-73b483a3a2a2-etc-systemd\") pod \"tuned-zltwm\" (UID: \"b36e05db-0a22-4de6-bed8-73b483a3a2a2\") " pod="openshift-cluster-node-tuning-operator/tuned-zltwm" Jul 14 20:22:23.543707 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:23.543579 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9588ab98-33e8-4c89-807d-a805240f1b81-host-run-ovn-kubernetes\") pod \"ovnkube-node-hzbx8\" (UID: \"9588ab98-33e8-4c89-807d-a805240f1b81\") " pod="openshift-ovn-kubernetes/ovnkube-node-hzbx8" Jul 14 20:22:23.543707 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:23.543596 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/9588ab98-33e8-4c89-807d-a805240f1b81-host-cni-netd\") pod \"ovnkube-node-hzbx8\" (UID: \"9588ab98-33e8-4c89-807d-a805240f1b81\") " pod="openshift-ovn-kubernetes/ovnkube-node-hzbx8" Jul 14 20:22:23.543707 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:23.543662 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/9588ab98-33e8-4c89-807d-a805240f1b81-host-cni-netd\") pod \"ovnkube-node-hzbx8\" (UID: \"9588ab98-33e8-4c89-807d-a805240f1b81\") " pod="openshift-ovn-kubernetes/ovnkube-node-hzbx8" Jul 14 20:22:23.543707 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:23.543662 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b36e05db-0a22-4de6-bed8-73b483a3a2a2-etc-kubernetes\") pod \"tuned-zltwm\" (UID: \"b36e05db-0a22-4de6-bed8-73b483a3a2a2\") " pod="openshift-cluster-node-tuning-operator/tuned-zltwm" Jul 14 20:22:23.543707 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:23.543676 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/b36e05db-0a22-4de6-bed8-73b483a3a2a2-etc-systemd\") pod \"tuned-zltwm\" (UID: \"b36e05db-0a22-4de6-bed8-73b483a3a2a2\") " pod="openshift-cluster-node-tuning-operator/tuned-zltwm" Jul 14 20:22:23.543707 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:23.543666 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9588ab98-33e8-4c89-807d-a805240f1b81-host-run-ovn-kubernetes\") pod \"ovnkube-node-hzbx8\" (UID: \"9588ab98-33e8-4c89-807d-a805240f1b81\") " pod="openshift-ovn-kubernetes/ovnkube-node-hzbx8" Jul 14 20:22:23.543707 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:23.543697 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r8lsz\" (UniqueName: \"kubernetes.io/projected/92f9c302-6d80-4679-9c71-1d37255dc852-kube-api-access-r8lsz\") pod \"network-check-target-2r82p\" (UID: \"92f9c302-6d80-4679-9c71-1d37255dc852\") " pod="openshift-network-diagnostics/network-check-target-2r82p" Jul 14 20:22:23.544071 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:23.543741 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/5835902d-ae0d-4395-9843-0ef04ad24976-hostroot\") pod \"multus-bbzpp\" (UID: \"5835902d-ae0d-4395-9843-0ef04ad24976\") " pod="openshift-multus/multus-bbzpp" Jul 14 20:22:23.544071 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:23.543771 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/07899815-131f-4211-a7f0-6ce06c8c9ea1-kubelet-dir\") pod \"aws-ebs-csi-driver-node-g77ql\" (UID: \"07899815-131f-4211-a7f0-6ce06c8c9ea1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-g77ql" Jul 14 20:22:23.544071 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:23.543795 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/9588ab98-33e8-4c89-807d-a805240f1b81-systemd-units\") pod \"ovnkube-node-hzbx8\" (UID: \"9588ab98-33e8-4c89-807d-a805240f1b81\") " pod="openshift-ovn-kubernetes/ovnkube-node-hzbx8" Jul 14 20:22:23.544071 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:23.543797 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/5835902d-ae0d-4395-9843-0ef04ad24976-hostroot\") pod \"multus-bbzpp\" (UID: \"5835902d-ae0d-4395-9843-0ef04ad24976\") " pod="openshift-multus/multus-bbzpp" Jul 14 20:22:23.544071 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:23.543819 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/9588ab98-33e8-4c89-807d-a805240f1b81-run-ovn\") pod \"ovnkube-node-hzbx8\" (UID: \"9588ab98-33e8-4c89-807d-a805240f1b81\") " pod="openshift-ovn-kubernetes/ovnkube-node-hzbx8" Jul 14 20:22:23.544071 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:23.543859 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/9588ab98-33e8-4c89-807d-a805240f1b81-run-ovn\") pod \"ovnkube-node-hzbx8\" (UID: \"9588ab98-33e8-4c89-807d-a805240f1b81\") " pod="openshift-ovn-kubernetes/ovnkube-node-hzbx8" Jul 14 20:22:23.544071 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:23.543860 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/9588ab98-33e8-4c89-807d-a805240f1b81-systemd-units\") pod \"ovnkube-node-hzbx8\" (UID: \"9588ab98-33e8-4c89-807d-a805240f1b81\") " pod="openshift-ovn-kubernetes/ovnkube-node-hzbx8" Jul 14 20:22:23.544071 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:23.543892 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/07899815-131f-4211-a7f0-6ce06c8c9ea1-kubelet-dir\") pod \"aws-ebs-csi-driver-node-g77ql\" (UID: \"07899815-131f-4211-a7f0-6ce06c8c9ea1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-g77ql" Jul 14 20:22:23.544071 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:23.543921 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/9588ab98-33e8-4c89-807d-a805240f1b81-host-cni-bin\") pod \"ovnkube-node-hzbx8\" (UID: \"9588ab98-33e8-4c89-807d-a805240f1b81\") " pod="openshift-ovn-kubernetes/ovnkube-node-hzbx8" Jul 14 20:22:23.544071 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:23.543963 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9588ab98-33e8-4c89-807d-a805240f1b81-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-hzbx8\" (UID: \"9588ab98-33e8-4c89-807d-a805240f1b81\") " pod="openshift-ovn-kubernetes/ovnkube-node-hzbx8" Jul 14 20:22:23.544071 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:23.543984 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/3d2553b9-0627-4fd5-b973-76e17eabf098-agent-certs\") pod \"konnectivity-agent-dwzbf\" (UID: \"3d2553b9-0627-4fd5-b973-76e17eabf098\") " pod="kube-system/konnectivity-agent-dwzbf" Jul 14 20:22:23.544071 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:23.544002 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/05045fa2-6bd3-4ee8-8f2a-e9056c8a316f-dbus\") pod \"global-pull-secret-syncer-sgvdx\" (UID: \"05045fa2-6bd3-4ee8-8f2a-e9056c8a316f\") " pod="kube-system/global-pull-secret-syncer-sgvdx" Jul 14 20:22:23.544071 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:23.544022 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5835902d-ae0d-4395-9843-0ef04ad24976-etc-kubernetes\") pod \"multus-bbzpp\" (UID: \"5835902d-ae0d-4395-9843-0ef04ad24976\") " pod="openshift-multus/multus-bbzpp" Jul 14 20:22:23.544071 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:23.544027 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9588ab98-33e8-4c89-807d-a805240f1b81-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-hzbx8\" (UID: \"9588ab98-33e8-4c89-807d-a805240f1b81\") " pod="openshift-ovn-kubernetes/ovnkube-node-hzbx8" Jul 14 20:22:23.544071 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:23.543967 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/9588ab98-33e8-4c89-807d-a805240f1b81-host-cni-bin\") pod \"ovnkube-node-hzbx8\" (UID: \"9588ab98-33e8-4c89-807d-a805240f1b81\") " pod="openshift-ovn-kubernetes/ovnkube-node-hzbx8" Jul 14 20:22:23.544071 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:23.544045 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/035331bf-20d9-4df2-99f6-b0e0db2e05f0-cni-binary-copy\") pod \"multus-additional-cni-plugins-jzpsb\" (UID: \"035331bf-20d9-4df2-99f6-b0e0db2e05f0\") " pod="openshift-multus/multus-additional-cni-plugins-jzpsb" Jul 14 20:22:23.544597 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:23.544071 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5835902d-ae0d-4395-9843-0ef04ad24976-etc-kubernetes\") pod \"multus-bbzpp\" (UID: \"5835902d-ae0d-4395-9843-0ef04ad24976\") " pod="openshift-multus/multus-bbzpp" Jul 14 20:22:23.544597 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:23.544118 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cxsdd\" (UniqueName: \"kubernetes.io/projected/b36e05db-0a22-4de6-bed8-73b483a3a2a2-kube-api-access-cxsdd\") pod \"tuned-zltwm\" (UID: \"b36e05db-0a22-4de6-bed8-73b483a3a2a2\") " pod="openshift-cluster-node-tuning-operator/tuned-zltwm" Jul 14 20:22:23.544597 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:23.544172 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fhfr4\" (UniqueName: \"kubernetes.io/projected/9d9cd7af-00ce-4f0a-8205-faf1e29a50e7-kube-api-access-fhfr4\") pod \"node-resolver-dznvt\" (UID: \"9d9cd7af-00ce-4f0a-8205-faf1e29a50e7\") " pod="openshift-dns/node-resolver-dznvt" Jul 14 20:22:23.544597 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:23.544196 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d4588597-416b-427f-85d1-d01e54549a84-host-slash\") pod \"iptables-alerter-9bnrv\" (UID: \"d4588597-416b-427f-85d1-d01e54549a84\") " pod="openshift-network-operator/iptables-alerter-9bnrv" Jul 14 20:22:23.544597 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:23.544223 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ztchf\" (UniqueName: \"kubernetes.io/projected/035331bf-20d9-4df2-99f6-b0e0db2e05f0-kube-api-access-ztchf\") pod \"multus-additional-cni-plugins-jzpsb\" (UID: \"035331bf-20d9-4df2-99f6-b0e0db2e05f0\") " pod="openshift-multus/multus-additional-cni-plugins-jzpsb" Jul 14 20:22:23.544597 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:23.544247 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/5835902d-ae0d-4395-9843-0ef04ad24976-multus-daemon-config\") pod \"multus-bbzpp\" (UID: \"5835902d-ae0d-4395-9843-0ef04ad24976\") " pod="openshift-multus/multus-bbzpp" Jul 14 20:22:23.544597 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:23.544269 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/035331bf-20d9-4df2-99f6-b0e0db2e05f0-system-cni-dir\") pod \"multus-additional-cni-plugins-jzpsb\" (UID: \"035331bf-20d9-4df2-99f6-b0e0db2e05f0\") " pod="openshift-multus/multus-additional-cni-plugins-jzpsb" Jul 14 20:22:23.544597 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:23.544291 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/07899815-131f-4211-a7f0-6ce06c8c9ea1-etc-selinux\") pod \"aws-ebs-csi-driver-node-g77ql\" (UID: \"07899815-131f-4211-a7f0-6ce06c8c9ea1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-g77ql" Jul 14 20:22:23.544597 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:23.544313 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/b36e05db-0a22-4de6-bed8-73b483a3a2a2-etc-modprobe-d\") pod \"tuned-zltwm\" (UID: \"b36e05db-0a22-4de6-bed8-73b483a3a2a2\") " pod="openshift-cluster-node-tuning-operator/tuned-zltwm" Jul 14 20:22:23.544597 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:23.544347 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/035331bf-20d9-4df2-99f6-b0e0db2e05f0-system-cni-dir\") pod \"multus-additional-cni-plugins-jzpsb\" (UID: \"035331bf-20d9-4df2-99f6-b0e0db2e05f0\") " pod="openshift-multus/multus-additional-cni-plugins-jzpsb" Jul 14 20:22:23.544597 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:23.544370 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/b36e05db-0a22-4de6-bed8-73b483a3a2a2-etc-sysctl-d\") pod \"tuned-zltwm\" (UID: \"b36e05db-0a22-4de6-bed8-73b483a3a2a2\") " pod="openshift-cluster-node-tuning-operator/tuned-zltwm" Jul 14 20:22:23.544597 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:23.544391 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/b36e05db-0a22-4de6-bed8-73b483a3a2a2-etc-sysctl-conf\") pod \"tuned-zltwm\" (UID: \"b36e05db-0a22-4de6-bed8-73b483a3a2a2\") " pod="openshift-cluster-node-tuning-operator/tuned-zltwm" Jul 14 20:22:23.544597 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:23.544410 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/b36e05db-0a22-4de6-bed8-73b483a3a2a2-run\") pod \"tuned-zltwm\" (UID: \"b36e05db-0a22-4de6-bed8-73b483a3a2a2\") " pod="openshift-cluster-node-tuning-operator/tuned-zltwm" Jul 14 20:22:23.544597 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:23.544432 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5835902d-ae0d-4395-9843-0ef04ad24976-system-cni-dir\") pod \"multus-bbzpp\" (UID: \"5835902d-ae0d-4395-9843-0ef04ad24976\") " pod="openshift-multus/multus-bbzpp" Jul 14 20:22:23.544597 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:23.544410 2563 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 14 20:22:23.544597 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:23.544455 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/07899815-131f-4211-a7f0-6ce06c8c9ea1-sys-fs\") pod \"aws-ebs-csi-driver-node-g77ql\" (UID: \"07899815-131f-4211-a7f0-6ce06c8c9ea1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-g77ql" Jul 14 20:22:23.544597 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:23.544463 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/07899815-131f-4211-a7f0-6ce06c8c9ea1-etc-selinux\") pod \"aws-ebs-csi-driver-node-g77ql\" (UID: \"07899815-131f-4211-a7f0-6ce06c8c9ea1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-g77ql" Jul 14 20:22:23.545287 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:23.544479 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b36e05db-0a22-4de6-bed8-73b483a3a2a2-lib-modules\") pod \"tuned-zltwm\" (UID: \"b36e05db-0a22-4de6-bed8-73b483a3a2a2\") " pod="openshift-cluster-node-tuning-operator/tuned-zltwm" Jul 14 20:22:23.545287 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:23.544502 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/afbb09bb-badd-4dc0-8082-581c48dda65b-host\") pod \"node-ca-88985\" (UID: \"afbb09bb-badd-4dc0-8082-581c48dda65b\") " pod="openshift-image-registry/node-ca-88985" Jul 14 20:22:23.545287 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:23.544508 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/b36e05db-0a22-4de6-bed8-73b483a3a2a2-run\") pod \"tuned-zltwm\" (UID: \"b36e05db-0a22-4de6-bed8-73b483a3a2a2\") " pod="openshift-cluster-node-tuning-operator/tuned-zltwm" Jul 14 20:22:23.545287 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:23.544523 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/b36e05db-0a22-4de6-bed8-73b483a3a2a2-etc-sysctl-d\") pod \"tuned-zltwm\" (UID: \"b36e05db-0a22-4de6-bed8-73b483a3a2a2\") " pod="openshift-cluster-node-tuning-operator/tuned-zltwm" Jul 14 20:22:23.545287 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:23.544570 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5835902d-ae0d-4395-9843-0ef04ad24976-system-cni-dir\") pod \"multus-bbzpp\" (UID: \"5835902d-ae0d-4395-9843-0ef04ad24976\") " pod="openshift-multus/multus-bbzpp" Jul 14 20:22:23.545287 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:23.544613 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/07899815-131f-4211-a7f0-6ce06c8c9ea1-sys-fs\") pod \"aws-ebs-csi-driver-node-g77ql\" (UID: \"07899815-131f-4211-a7f0-6ce06c8c9ea1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-g77ql" Jul 14 20:22:23.545287 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:23.544673 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/035331bf-20d9-4df2-99f6-b0e0db2e05f0-cni-binary-copy\") pod \"multus-additional-cni-plugins-jzpsb\" (UID: \"035331bf-20d9-4df2-99f6-b0e0db2e05f0\") " pod="openshift-multus/multus-additional-cni-plugins-jzpsb" Jul 14 20:22:23.545287 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:23.544701 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b36e05db-0a22-4de6-bed8-73b483a3a2a2-lib-modules\") pod \"tuned-zltwm\" (UID: \"b36e05db-0a22-4de6-bed8-73b483a3a2a2\") " pod="openshift-cluster-node-tuning-operator/tuned-zltwm" Jul 14 20:22:23.545287 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:23.544526 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cbwhw\" (UniqueName: \"kubernetes.io/projected/afbb09bb-badd-4dc0-8082-581c48dda65b-kube-api-access-cbwhw\") pod \"node-ca-88985\" (UID: \"afbb09bb-badd-4dc0-8082-581c48dda65b\") " pod="openshift-image-registry/node-ca-88985" Jul 14 20:22:23.545287 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:23.544740 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/b36e05db-0a22-4de6-bed8-73b483a3a2a2-etc-sysctl-conf\") pod \"tuned-zltwm\" (UID: \"b36e05db-0a22-4de6-bed8-73b483a3a2a2\") " pod="openshift-cluster-node-tuning-operator/tuned-zltwm" Jul 14 20:22:23.545287 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:23.544742 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/9588ab98-33e8-4c89-807d-a805240f1b81-run-systemd\") pod \"ovnkube-node-hzbx8\" (UID: \"9588ab98-33e8-4c89-807d-a805240f1b81\") " pod="openshift-ovn-kubernetes/ovnkube-node-hzbx8" Jul 14 20:22:23.545287 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:23.544759 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/afbb09bb-badd-4dc0-8082-581c48dda65b-host\") pod \"node-ca-88985\" (UID: \"afbb09bb-badd-4dc0-8082-581c48dda65b\") " pod="openshift-image-registry/node-ca-88985" Jul 14 20:22:23.545287 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:23.544771 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/b36e05db-0a22-4de6-bed8-73b483a3a2a2-etc-modprobe-d\") pod \"tuned-zltwm\" (UID: \"b36e05db-0a22-4de6-bed8-73b483a3a2a2\") " pod="openshift-cluster-node-tuning-operator/tuned-zltwm" Jul 14 20:22:23.545287 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:23.544779 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/9588ab98-33e8-4c89-807d-a805240f1b81-run-systemd\") pod \"ovnkube-node-hzbx8\" (UID: \"9588ab98-33e8-4c89-807d-a805240f1b81\") " pod="openshift-ovn-kubernetes/ovnkube-node-hzbx8" Jul 14 20:22:23.545287 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:23.544799 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9588ab98-33e8-4c89-807d-a805240f1b81-var-lib-openvswitch\") pod \"ovnkube-node-hzbx8\" (UID: \"9588ab98-33e8-4c89-807d-a805240f1b81\") " pod="openshift-ovn-kubernetes/ovnkube-node-hzbx8" Jul 14 20:22:23.545287 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:23.544820 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/9588ab98-33e8-4c89-807d-a805240f1b81-env-overrides\") pod \"ovnkube-node-hzbx8\" (UID: \"9588ab98-33e8-4c89-807d-a805240f1b81\") " pod="openshift-ovn-kubernetes/ovnkube-node-hzbx8" Jul 14 20:22:23.545287 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:23.544834 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/5835902d-ae0d-4395-9843-0ef04ad24976-host-run-multus-certs\") pod \"multus-bbzpp\" (UID: \"5835902d-ae0d-4395-9843-0ef04ad24976\") " pod="openshift-multus/multus-bbzpp" Jul 14 20:22:23.545287 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:23.544854 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/b36e05db-0a22-4de6-bed8-73b483a3a2a2-etc-sysconfig\") pod \"tuned-zltwm\" (UID: \"b36e05db-0a22-4de6-bed8-73b483a3a2a2\") " pod="openshift-cluster-node-tuning-operator/tuned-zltwm" Jul 14 20:22:23.546222 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:23.544860 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9588ab98-33e8-4c89-807d-a805240f1b81-var-lib-openvswitch\") pod \"ovnkube-node-hzbx8\" (UID: \"9588ab98-33e8-4c89-807d-a805240f1b81\") " pod="openshift-ovn-kubernetes/ovnkube-node-hzbx8" Jul 14 20:22:23.546222 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:23.544892 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/5835902d-ae0d-4395-9843-0ef04ad24976-host-run-multus-certs\") pod \"multus-bbzpp\" (UID: \"5835902d-ae0d-4395-9843-0ef04ad24976\") " pod="openshift-multus/multus-bbzpp" Jul 14 20:22:23.546222 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:23.544875 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/9588ab98-33e8-4c89-807d-a805240f1b81-host-slash\") pod \"ovnkube-node-hzbx8\" (UID: \"9588ab98-33e8-4c89-807d-a805240f1b81\") " pod="openshift-ovn-kubernetes/ovnkube-node-hzbx8" Jul 14 20:22:23.546222 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:23.544917 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/b36e05db-0a22-4de6-bed8-73b483a3a2a2-etc-sysconfig\") pod \"tuned-zltwm\" (UID: \"b36e05db-0a22-4de6-bed8-73b483a3a2a2\") " pod="openshift-cluster-node-tuning-operator/tuned-zltwm" Jul 14 20:22:23.546222 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:23.544920 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/5835902d-ae0d-4395-9843-0ef04ad24976-cni-binary-copy\") pod \"multus-bbzpp\" (UID: \"5835902d-ae0d-4395-9843-0ef04ad24976\") " pod="openshift-multus/multus-bbzpp" Jul 14 20:22:23.546222 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:23.544950 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/035331bf-20d9-4df2-99f6-b0e0db2e05f0-cnibin\") pod \"multus-additional-cni-plugins-jzpsb\" (UID: \"035331bf-20d9-4df2-99f6-b0e0db2e05f0\") " pod="openshift-multus/multus-additional-cni-plugins-jzpsb" Jul 14 20:22:23.546222 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:23.544973 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/9588ab98-33e8-4c89-807d-a805240f1b81-host-slash\") pod \"ovnkube-node-hzbx8\" (UID: \"9588ab98-33e8-4c89-807d-a805240f1b81\") " pod="openshift-ovn-kubernetes/ovnkube-node-hzbx8" Jul 14 20:22:23.546222 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:23.544977 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/92cbd473-9b2e-4faa-a232-810627648cc9-metrics-certs\") pod \"network-metrics-daemon-rvhpg\" (UID: \"92cbd473-9b2e-4faa-a232-810627648cc9\") " pod="openshift-multus/network-metrics-daemon-rvhpg" Jul 14 20:22:23.546222 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:23.545003 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/07899815-131f-4211-a7f0-6ce06c8c9ea1-registration-dir\") pod \"aws-ebs-csi-driver-node-g77ql\" (UID: \"07899815-131f-4211-a7f0-6ce06c8c9ea1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-g77ql" Jul 14 20:22:23.546222 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:23.545011 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/035331bf-20d9-4df2-99f6-b0e0db2e05f0-cnibin\") pod \"multus-additional-cni-plugins-jzpsb\" (UID: \"035331bf-20d9-4df2-99f6-b0e0db2e05f0\") " pod="openshift-multus/multus-additional-cni-plugins-jzpsb" Jul 14 20:22:23.546222 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:23.545027 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/9588ab98-33e8-4c89-807d-a805240f1b81-ovnkube-script-lib\") pod \"ovnkube-node-hzbx8\" (UID: \"9588ab98-33e8-4c89-807d-a805240f1b81\") " pod="openshift-ovn-kubernetes/ovnkube-node-hzbx8" Jul 14 20:22:23.546222 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:23.545056 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/05045fa2-6bd3-4ee8-8f2a-e9056c8a316f-kubelet-config\") pod \"global-pull-secret-syncer-sgvdx\" (UID: \"05045fa2-6bd3-4ee8-8f2a-e9056c8a316f\") " pod="kube-system/global-pull-secret-syncer-sgvdx" Jul 14 20:22:23.546222 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:23.545067 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/07899815-131f-4211-a7f0-6ce06c8c9ea1-registration-dir\") pod \"aws-ebs-csi-driver-node-g77ql\" (UID: \"07899815-131f-4211-a7f0-6ce06c8c9ea1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-g77ql" Jul 14 20:22:23.546222 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:23.545082 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rv8g8\" (UniqueName: \"kubernetes.io/projected/92cbd473-9b2e-4faa-a232-810627648cc9-kube-api-access-rv8g8\") pod \"network-metrics-daemon-rvhpg\" (UID: \"92cbd473-9b2e-4faa-a232-810627648cc9\") " pod="openshift-multus/network-metrics-daemon-rvhpg" Jul 14 20:22:23.546222 ip-10-0-141-61 kubenswrapper[2563]: E0714 20:22:23.545090 2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 14 20:22:23.546222 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:23.545108 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-26qpq\" (UniqueName: \"kubernetes.io/projected/07899815-131f-4211-a7f0-6ce06c8c9ea1-kube-api-access-26qpq\") pod \"aws-ebs-csi-driver-node-g77ql\" (UID: \"07899815-131f-4211-a7f0-6ce06c8c9ea1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-g77ql" Jul 14 20:22:23.546222 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:23.545155 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b36e05db-0a22-4de6-bed8-73b483a3a2a2-tmp\") pod \"tuned-zltwm\" (UID: \"b36e05db-0a22-4de6-bed8-73b483a3a2a2\") " pod="openshift-cluster-node-tuning-operator/tuned-zltwm" Jul 14 20:22:23.546994 ip-10-0-141-61 kubenswrapper[2563]: E0714 20:22:23.545232 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/92cbd473-9b2e-4faa-a232-810627648cc9-metrics-certs podName:92cbd473-9b2e-4faa-a232-810627648cc9 nodeName:}" failed. No retries permitted until 2026-07-14 20:22:24.045155464 +0000 UTC m=+2.966318767 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/92cbd473-9b2e-4faa-a232-810627648cc9-metrics-certs") pod "network-metrics-daemon-rvhpg" (UID: "92cbd473-9b2e-4faa-a232-810627648cc9") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 14 20:22:23.546994 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:23.545260 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/9588ab98-33e8-4c89-807d-a805240f1b81-ovn-node-metrics-cert\") pod \"ovnkube-node-hzbx8\" (UID: \"9588ab98-33e8-4c89-807d-a805240f1b81\") " pod="openshift-ovn-kubernetes/ovnkube-node-hzbx8" Jul 14 20:22:23.546994 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:23.545285 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9d9cd7af-00ce-4f0a-8205-faf1e29a50e7-tmp-dir\") pod \"node-resolver-dznvt\" (UID: \"9d9cd7af-00ce-4f0a-8205-faf1e29a50e7\") " pod="openshift-dns/node-resolver-dznvt" Jul 14 20:22:23.546994 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:23.545311 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5835902d-ae0d-4395-9843-0ef04ad24976-multus-cni-dir\") pod \"multus-bbzpp\" (UID: \"5835902d-ae0d-4395-9843-0ef04ad24976\") " pod="openshift-multus/multus-bbzpp" Jul 14 20:22:23.546994 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:23.545342 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/9588ab98-33e8-4c89-807d-a805240f1b81-env-overrides\") pod \"ovnkube-node-hzbx8\" (UID: \"9588ab98-33e8-4c89-807d-a805240f1b81\") " pod="openshift-ovn-kubernetes/ovnkube-node-hzbx8" Jul 14 20:22:23.546994 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:23.545336 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/5835902d-ae0d-4395-9843-0ef04ad24976-cnibin\") pod \"multus-bbzpp\" (UID: \"5835902d-ae0d-4395-9843-0ef04ad24976\") " pod="openshift-multus/multus-bbzpp" Jul 14 20:22:23.546994 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:23.545378 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/5835902d-ae0d-4395-9843-0ef04ad24976-os-release\") pod \"multus-bbzpp\" (UID: \"5835902d-ae0d-4395-9843-0ef04ad24976\") " pod="openshift-multus/multus-bbzpp" Jul 14 20:22:23.546994 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:23.545402 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/035331bf-20d9-4df2-99f6-b0e0db2e05f0-os-release\") pod \"multus-additional-cni-plugins-jzpsb\" (UID: \"035331bf-20d9-4df2-99f6-b0e0db2e05f0\") " pod="openshift-multus/multus-additional-cni-plugins-jzpsb" Jul 14 20:22:23.546994 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:23.545403 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/5835902d-ae0d-4395-9843-0ef04ad24976-multus-daemon-config\") pod \"multus-bbzpp\" (UID: \"5835902d-ae0d-4395-9843-0ef04ad24976\") " pod="openshift-multus/multus-bbzpp" Jul 14 20:22:23.546994 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:23.545438 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/5835902d-ae0d-4395-9843-0ef04ad24976-cni-binary-copy\") pod \"multus-bbzpp\" (UID: \"5835902d-ae0d-4395-9843-0ef04ad24976\") " pod="openshift-multus/multus-bbzpp" Jul 14 20:22:23.546994 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:23.545429 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/07899815-131f-4211-a7f0-6ce06c8c9ea1-socket-dir\") pod \"aws-ebs-csi-driver-node-g77ql\" (UID: \"07899815-131f-4211-a7f0-6ce06c8c9ea1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-g77ql" Jul 14 20:22:23.546994 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:23.545488 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/5835902d-ae0d-4395-9843-0ef04ad24976-host-run-netns\") pod \"multus-bbzpp\" (UID: \"5835902d-ae0d-4395-9843-0ef04ad24976\") " pod="openshift-multus/multus-bbzpp" Jul 14 20:22:23.546994 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:23.545509 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5835902d-ae0d-4395-9843-0ef04ad24976-multus-cni-dir\") pod \"multus-bbzpp\" (UID: \"5835902d-ae0d-4395-9843-0ef04ad24976\") " pod="openshift-multus/multus-bbzpp" Jul 14 20:22:23.546994 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:23.545515 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/07899815-131f-4211-a7f0-6ce06c8c9ea1-device-dir\") pod \"aws-ebs-csi-driver-node-g77ql\" (UID: \"07899815-131f-4211-a7f0-6ce06c8c9ea1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-g77ql" Jul 14 20:22:23.546994 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:23.545543 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/9588ab98-33e8-4c89-807d-a805240f1b81-ovnkube-config\") pod \"ovnkube-node-hzbx8\" (UID: \"9588ab98-33e8-4c89-807d-a805240f1b81\") " pod="openshift-ovn-kubernetes/ovnkube-node-hzbx8" Jul 14 20:22:23.546994 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:23.545544 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/5835902d-ae0d-4395-9843-0ef04ad24976-cnibin\") pod \"multus-bbzpp\" (UID: \"5835902d-ae0d-4395-9843-0ef04ad24976\") " pod="openshift-multus/multus-bbzpp" Jul 14 20:22:23.546994 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:23.545512 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/07899815-131f-4211-a7f0-6ce06c8c9ea1-socket-dir\") pod \"aws-ebs-csi-driver-node-g77ql\" (UID: \"07899815-131f-4211-a7f0-6ce06c8c9ea1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-g77ql" Jul 14 20:22:23.547890 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:23.545569 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/9588ab98-33e8-4c89-807d-a805240f1b81-ovnkube-script-lib\") pod \"ovnkube-node-hzbx8\" (UID: \"9588ab98-33e8-4c89-807d-a805240f1b81\") " pod="openshift-ovn-kubernetes/ovnkube-node-hzbx8" Jul 14 20:22:23.547890 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:23.545582 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/035331bf-20d9-4df2-99f6-b0e0db2e05f0-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-jzpsb\" (UID: \"035331bf-20d9-4df2-99f6-b0e0db2e05f0\") " pod="openshift-multus/multus-additional-cni-plugins-jzpsb" Jul 14 20:22:23.547890 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:23.545597 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/07899815-131f-4211-a7f0-6ce06c8c9ea1-device-dir\") pod \"aws-ebs-csi-driver-node-g77ql\" (UID: \"07899815-131f-4211-a7f0-6ce06c8c9ea1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-g77ql" Jul 14 20:22:23.547890 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:23.545627 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9588ab98-33e8-4c89-807d-a805240f1b81-run-openvswitch\") pod \"ovnkube-node-hzbx8\" (UID: \"9588ab98-33e8-4c89-807d-a805240f1b81\") " pod="openshift-ovn-kubernetes/ovnkube-node-hzbx8" Jul 14 20:22:23.547890 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:23.545715 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9588ab98-33e8-4c89-807d-a805240f1b81-run-openvswitch\") pod \"ovnkube-node-hzbx8\" (UID: \"9588ab98-33e8-4c89-807d-a805240f1b81\") " pod="openshift-ovn-kubernetes/ovnkube-node-hzbx8" Jul 14 20:22:23.547890 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:23.545718 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/9588ab98-33e8-4c89-807d-a805240f1b81-node-log\") pod \"ovnkube-node-hzbx8\" (UID: \"9588ab98-33e8-4c89-807d-a805240f1b81\") " pod="openshift-ovn-kubernetes/ovnkube-node-hzbx8" Jul 14 20:22:23.547890 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:23.545739 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/035331bf-20d9-4df2-99f6-b0e0db2e05f0-os-release\") pod \"multus-additional-cni-plugins-jzpsb\" (UID: \"035331bf-20d9-4df2-99f6-b0e0db2e05f0\") " pod="openshift-multus/multus-additional-cni-plugins-jzpsb" Jul 14 20:22:23.547890 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:23.545754 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/9588ab98-33e8-4c89-807d-a805240f1b81-node-log\") pod \"ovnkube-node-hzbx8\" (UID: \"9588ab98-33e8-4c89-807d-a805240f1b81\") " pod="openshift-ovn-kubernetes/ovnkube-node-hzbx8" Jul 14 20:22:23.547890 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:23.545759 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/9588ab98-33e8-4c89-807d-a805240f1b81-log-socket\") pod \"ovnkube-node-hzbx8\" (UID: \"9588ab98-33e8-4c89-807d-a805240f1b81\") " pod="openshift-ovn-kubernetes/ovnkube-node-hzbx8" Jul 14 20:22:23.547890 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:23.545777 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/5835902d-ae0d-4395-9843-0ef04ad24976-host-run-netns\") pod \"multus-bbzpp\" (UID: \"5835902d-ae0d-4395-9843-0ef04ad24976\") " pod="openshift-multus/multus-bbzpp" Jul 14 20:22:23.547890 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:23.545786 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/5835902d-ae0d-4395-9843-0ef04ad24976-host-run-k8s-cni-cncf-io\") pod \"multus-bbzpp\" (UID: \"5835902d-ae0d-4395-9843-0ef04ad24976\") " pod="openshift-multus/multus-bbzpp" Jul 14 20:22:23.547890 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:23.545805 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/5835902d-ae0d-4395-9843-0ef04ad24976-host-var-lib-cni-bin\") pod \"multus-bbzpp\" (UID: \"5835902d-ae0d-4395-9843-0ef04ad24976\") " pod="openshift-multus/multus-bbzpp" Jul 14 20:22:23.547890 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:23.545820 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b36e05db-0a22-4de6-bed8-73b483a3a2a2-sys\") pod \"tuned-zltwm\" (UID: \"b36e05db-0a22-4de6-bed8-73b483a3a2a2\") " pod="openshift-cluster-node-tuning-operator/tuned-zltwm" Jul 14 20:22:23.547890 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:23.545841 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/3d2553b9-0627-4fd5-b973-76e17eabf098-konnectivity-ca\") pod \"konnectivity-agent-dwzbf\" (UID: \"3d2553b9-0627-4fd5-b973-76e17eabf098\") " pod="kube-system/konnectivity-agent-dwzbf" Jul 14 20:22:23.547890 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:23.545851 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/5835902d-ae0d-4395-9843-0ef04ad24976-host-run-k8s-cni-cncf-io\") pod \"multus-bbzpp\" (UID: \"5835902d-ae0d-4395-9843-0ef04ad24976\") " pod="openshift-multus/multus-bbzpp" Jul 14 20:22:23.547890 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:23.545860 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/5835902d-ae0d-4395-9843-0ef04ad24976-host-var-lib-cni-bin\") pod \"multus-bbzpp\" (UID: \"5835902d-ae0d-4395-9843-0ef04ad24976\") " pod="openshift-multus/multus-bbzpp" Jul 14 20:22:23.547890 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:23.545868 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/05045fa2-6bd3-4ee8-8f2a-e9056c8a316f-original-pull-secret\") pod \"global-pull-secret-syncer-sgvdx\" (UID: \"05045fa2-6bd3-4ee8-8f2a-e9056c8a316f\") " pod="kube-system/global-pull-secret-syncer-sgvdx" Jul 14 20:22:23.548623 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:23.545896 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/035331bf-20d9-4df2-99f6-b0e0db2e05f0-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-jzpsb\" (UID: \"035331bf-20d9-4df2-99f6-b0e0db2e05f0\") " pod="openshift-multus/multus-additional-cni-plugins-jzpsb" Jul 14 20:22:23.548623 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:23.545899 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/9588ab98-33e8-4c89-807d-a805240f1b81-log-socket\") pod \"ovnkube-node-hzbx8\" (UID: \"9588ab98-33e8-4c89-807d-a805240f1b81\") " pod="openshift-ovn-kubernetes/ovnkube-node-hzbx8" Jul 14 20:22:23.548623 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:23.545909 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b36e05db-0a22-4de6-bed8-73b483a3a2a2-sys\") pod \"tuned-zltwm\" (UID: \"b36e05db-0a22-4de6-bed8-73b483a3a2a2\") " pod="openshift-cluster-node-tuning-operator/tuned-zltwm" Jul 14 20:22:23.548623 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:23.545934 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/035331bf-20d9-4df2-99f6-b0e0db2e05f0-tuning-conf-dir\") pod \"multus-additional-cni-plugins-jzpsb\" (UID: \"035331bf-20d9-4df2-99f6-b0e0db2e05f0\") " pod="openshift-multus/multus-additional-cni-plugins-jzpsb" Jul 14 20:22:23.548623 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:23.545972 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/afbb09bb-badd-4dc0-8082-581c48dda65b-serviceca\") pod \"node-ca-88985\" (UID: \"afbb09bb-badd-4dc0-8082-581c48dda65b\") " pod="openshift-image-registry/node-ca-88985" Jul 14 20:22:23.548623 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:23.545990 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/5835902d-ae0d-4395-9843-0ef04ad24976-os-release\") pod \"multus-bbzpp\" (UID: \"5835902d-ae0d-4395-9843-0ef04ad24976\") " pod="openshift-multus/multus-bbzpp" Jul 14 20:22:23.548623 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:23.545998 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pkcsg\" (UniqueName: \"kubernetes.io/projected/9588ab98-33e8-4c89-807d-a805240f1b81-kube-api-access-pkcsg\") pod \"ovnkube-node-hzbx8\" (UID: \"9588ab98-33e8-4c89-807d-a805240f1b81\") " pod="openshift-ovn-kubernetes/ovnkube-node-hzbx8" Jul 14 20:22:23.548623 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:23.546025 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/9d9cd7af-00ce-4f0a-8205-faf1e29a50e7-hosts-file\") pod \"node-resolver-dznvt\" (UID: \"9d9cd7af-00ce-4f0a-8205-faf1e29a50e7\") " pod="openshift-dns/node-resolver-dznvt" Jul 14 20:22:23.548623 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:23.546025 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/035331bf-20d9-4df2-99f6-b0e0db2e05f0-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-jzpsb\" (UID: \"035331bf-20d9-4df2-99f6-b0e0db2e05f0\") " pod="openshift-multus/multus-additional-cni-plugins-jzpsb" Jul 14 20:22:23.548623 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:23.546049 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-76hvp\" (UniqueName: \"kubernetes.io/projected/d4588597-416b-427f-85d1-d01e54549a84-kube-api-access-76hvp\") pod \"iptables-alerter-9bnrv\" (UID: \"d4588597-416b-427f-85d1-d01e54549a84\") " pod="openshift-network-operator/iptables-alerter-9bnrv" Jul 14 20:22:23.548623 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:23.546075 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b36e05db-0a22-4de6-bed8-73b483a3a2a2-var-lib-kubelet\") pod \"tuned-zltwm\" (UID: \"b36e05db-0a22-4de6-bed8-73b483a3a2a2\") " pod="openshift-cluster-node-tuning-operator/tuned-zltwm" Jul 14 20:22:23.548623 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:23.546097 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b36e05db-0a22-4de6-bed8-73b483a3a2a2-host\") pod \"tuned-zltwm\" (UID: \"b36e05db-0a22-4de6-bed8-73b483a3a2a2\") " pod="openshift-cluster-node-tuning-operator/tuned-zltwm" Jul 14 20:22:23.548623 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:23.546140 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/9588ab98-33e8-4c89-807d-a805240f1b81-host-kubelet\") pod \"ovnkube-node-hzbx8\" (UID: \"9588ab98-33e8-4c89-807d-a805240f1b81\") " pod="openshift-ovn-kubernetes/ovnkube-node-hzbx8" Jul 14 20:22:23.548623 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:23.546163 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/9588ab98-33e8-4c89-807d-a805240f1b81-host-run-netns\") pod \"ovnkube-node-hzbx8\" (UID: \"9588ab98-33e8-4c89-807d-a805240f1b81\") " pod="openshift-ovn-kubernetes/ovnkube-node-hzbx8" Jul 14 20:22:23.548623 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:23.546193 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9588ab98-33e8-4c89-807d-a805240f1b81-etc-openvswitch\") pod \"ovnkube-node-hzbx8\" (UID: \"9588ab98-33e8-4c89-807d-a805240f1b81\") " pod="openshift-ovn-kubernetes/ovnkube-node-hzbx8" Jul 14 20:22:23.548623 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:23.546219 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/d4588597-416b-427f-85d1-d01e54549a84-iptables-alerter-script\") pod \"iptables-alerter-9bnrv\" (UID: \"d4588597-416b-427f-85d1-d01e54549a84\") " pod="openshift-network-operator/iptables-alerter-9bnrv" Jul 14 20:22:23.548623 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:23.546244 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/5835902d-ae0d-4395-9843-0ef04ad24976-multus-socket-dir-parent\") pod \"multus-bbzpp\" (UID: \"5835902d-ae0d-4395-9843-0ef04ad24976\") " pod="openshift-multus/multus-bbzpp" Jul 14 20:22:23.549079 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:23.546268 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/5835902d-ae0d-4395-9843-0ef04ad24976-host-var-lib-cni-multus\") pod \"multus-bbzpp\" (UID: \"5835902d-ae0d-4395-9843-0ef04ad24976\") " pod="openshift-multus/multus-bbzpp" Jul 14 20:22:23.549079 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:23.546289 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5835902d-ae0d-4395-9843-0ef04ad24976-host-var-lib-kubelet\") pod \"multus-bbzpp\" (UID: \"5835902d-ae0d-4395-9843-0ef04ad24976\") " pod="openshift-multus/multus-bbzpp" Jul 14 20:22:23.549079 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:23.546314 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/b36e05db-0a22-4de6-bed8-73b483a3a2a2-etc-tuned\") pod \"tuned-zltwm\" (UID: \"b36e05db-0a22-4de6-bed8-73b483a3a2a2\") " pod="openshift-cluster-node-tuning-operator/tuned-zltwm" Jul 14 20:22:23.549079 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:23.546340 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/5835902d-ae0d-4395-9843-0ef04ad24976-multus-conf-dir\") pod \"multus-bbzpp\" (UID: \"5835902d-ae0d-4395-9843-0ef04ad24976\") " pod="openshift-multus/multus-bbzpp" Jul 14 20:22:23.549079 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:23.546419 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/5835902d-ae0d-4395-9843-0ef04ad24976-multus-conf-dir\") pod \"multus-bbzpp\" (UID: \"5835902d-ae0d-4395-9843-0ef04ad24976\") " pod="openshift-multus/multus-bbzpp" Jul 14 20:22:23.549079 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:23.546467 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/afbb09bb-badd-4dc0-8082-581c48dda65b-serviceca\") pod \"node-ca-88985\" (UID: \"afbb09bb-badd-4dc0-8082-581c48dda65b\") " pod="openshift-image-registry/node-ca-88985" Jul 14 20:22:23.549079 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:23.546484 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/3d2553b9-0627-4fd5-b973-76e17eabf098-konnectivity-ca\") pod \"konnectivity-agent-dwzbf\" (UID: \"3d2553b9-0627-4fd5-b973-76e17eabf098\") " pod="kube-system/konnectivity-agent-dwzbf" Jul 14 20:22:23.549079 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:23.546493 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/035331bf-20d9-4df2-99f6-b0e0db2e05f0-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-jzpsb\" (UID: \"035331bf-20d9-4df2-99f6-b0e0db2e05f0\") " pod="openshift-multus/multus-additional-cni-plugins-jzpsb" Jul 14 20:22:23.549079 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:23.546554 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b36e05db-0a22-4de6-bed8-73b483a3a2a2-var-lib-kubelet\") pod \"tuned-zltwm\" (UID: \"b36e05db-0a22-4de6-bed8-73b483a3a2a2\") " pod="openshift-cluster-node-tuning-operator/tuned-zltwm" Jul 14 20:22:23.549079 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:23.546556 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/9588ab98-33e8-4c89-807d-a805240f1b81-host-kubelet\") pod \"ovnkube-node-hzbx8\" (UID: \"9588ab98-33e8-4c89-807d-a805240f1b81\") " pod="openshift-ovn-kubernetes/ovnkube-node-hzbx8" Jul 14 20:22:23.549079 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:23.546589 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/035331bf-20d9-4df2-99f6-b0e0db2e05f0-tuning-conf-dir\") pod \"multus-additional-cni-plugins-jzpsb\" (UID: \"035331bf-20d9-4df2-99f6-b0e0db2e05f0\") " pod="openshift-multus/multus-additional-cni-plugins-jzpsb" Jul 14 20:22:23.549079 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:23.546595 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b36e05db-0a22-4de6-bed8-73b483a3a2a2-host\") pod \"tuned-zltwm\" (UID: \"b36e05db-0a22-4de6-bed8-73b483a3a2a2\") " pod="openshift-cluster-node-tuning-operator/tuned-zltwm" Jul 14 20:22:23.549079 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:23.546623 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/9588ab98-33e8-4c89-807d-a805240f1b81-host-run-netns\") pod \"ovnkube-node-hzbx8\" (UID: \"9588ab98-33e8-4c89-807d-a805240f1b81\") " pod="openshift-ovn-kubernetes/ovnkube-node-hzbx8" Jul 14 20:22:23.549079 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:23.546637 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9588ab98-33e8-4c89-807d-a805240f1b81-etc-openvswitch\") pod \"ovnkube-node-hzbx8\" (UID: \"9588ab98-33e8-4c89-807d-a805240f1b81\") " pod="openshift-ovn-kubernetes/ovnkube-node-hzbx8" Jul 14 20:22:23.549079 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:23.546644 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5835902d-ae0d-4395-9843-0ef04ad24976-host-var-lib-kubelet\") pod \"multus-bbzpp\" (UID: \"5835902d-ae0d-4395-9843-0ef04ad24976\") " pod="openshift-multus/multus-bbzpp" Jul 14 20:22:23.549079 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:23.546650 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/5835902d-ae0d-4395-9843-0ef04ad24976-multus-socket-dir-parent\") pod \"multus-bbzpp\" (UID: \"5835902d-ae0d-4395-9843-0ef04ad24976\") " pod="openshift-multus/multus-bbzpp" Jul 14 20:22:23.549079 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:23.546675 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/5835902d-ae0d-4395-9843-0ef04ad24976-host-var-lib-cni-multus\") pod \"multus-bbzpp\" (UID: \"5835902d-ae0d-4395-9843-0ef04ad24976\") " pod="openshift-multus/multus-bbzpp" Jul 14 20:22:23.549079 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:23.547339 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/9588ab98-33e8-4c89-807d-a805240f1b81-ovnkube-config\") pod \"ovnkube-node-hzbx8\" (UID: \"9588ab98-33e8-4c89-807d-a805240f1b81\") " pod="openshift-ovn-kubernetes/ovnkube-node-hzbx8" Jul 14 20:22:23.549699 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:23.548195 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b36e05db-0a22-4de6-bed8-73b483a3a2a2-tmp\") pod \"tuned-zltwm\" (UID: \"b36e05db-0a22-4de6-bed8-73b483a3a2a2\") " pod="openshift-cluster-node-tuning-operator/tuned-zltwm" Jul 14 20:22:23.549699 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:23.548453 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/9588ab98-33e8-4c89-807d-a805240f1b81-ovn-node-metrics-cert\") pod \"ovnkube-node-hzbx8\" (UID: \"9588ab98-33e8-4c89-807d-a805240f1b81\") " pod="openshift-ovn-kubernetes/ovnkube-node-hzbx8" Jul 14 20:22:23.549699 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:23.548500 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/3d2553b9-0627-4fd5-b973-76e17eabf098-agent-certs\") pod \"konnectivity-agent-dwzbf\" (UID: \"3d2553b9-0627-4fd5-b973-76e17eabf098\") " pod="kube-system/konnectivity-agent-dwzbf" Jul 14 20:22:23.549699 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:23.548645 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/b36e05db-0a22-4de6-bed8-73b483a3a2a2-etc-tuned\") pod \"tuned-zltwm\" (UID: \"b36e05db-0a22-4de6-bed8-73b483a3a2a2\") " pod="openshift-cluster-node-tuning-operator/tuned-zltwm" Jul 14 20:22:23.554781 ip-10-0-141-61 kubenswrapper[2563]: E0714 20:22:23.554743 2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 14 20:22:23.554781 ip-10-0-141-61 kubenswrapper[2563]: E0714 20:22:23.554773 2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 14 20:22:23.554951 ip-10-0-141-61 kubenswrapper[2563]: E0714 20:22:23.554789 2563 projected.go:196] Error preparing data for projected volume kube-api-access-r8lsz for pod openshift-network-diagnostics/network-check-target-2r82p: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 14 20:22:23.554951 ip-10-0-141-61 kubenswrapper[2563]: E0714 20:22:23.554858 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/92f9c302-6d80-4679-9c71-1d37255dc852-kube-api-access-r8lsz podName:92f9c302-6d80-4679-9c71-1d37255dc852 nodeName:}" failed. No retries permitted until 2026-07-14 20:22:24.054838086 +0000 UTC m=+2.976001399 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-r8lsz" (UniqueName: "kubernetes.io/projected/92f9c302-6d80-4679-9c71-1d37255dc852-kube-api-access-r8lsz") pod "network-check-target-2r82p" (UID: "92f9c302-6d80-4679-9c71-1d37255dc852") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 14 20:22:23.558147 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:23.558096 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cxsdd\" (UniqueName: \"kubernetes.io/projected/b36e05db-0a22-4de6-bed8-73b483a3a2a2-kube-api-access-cxsdd\") pod \"tuned-zltwm\" (UID: \"b36e05db-0a22-4de6-bed8-73b483a3a2a2\") " pod="openshift-cluster-node-tuning-operator/tuned-zltwm" Jul 14 20:22:23.558260 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:23.558233 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dhk5d\" (UniqueName: \"kubernetes.io/projected/5835902d-ae0d-4395-9843-0ef04ad24976-kube-api-access-dhk5d\") pod \"multus-bbzpp\" (UID: \"5835902d-ae0d-4395-9843-0ef04ad24976\") " pod="openshift-multus/multus-bbzpp" Jul 14 20:22:23.558329 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:23.558259 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ztchf\" (UniqueName: \"kubernetes.io/projected/035331bf-20d9-4df2-99f6-b0e0db2e05f0-kube-api-access-ztchf\") pod \"multus-additional-cni-plugins-jzpsb\" (UID: \"035331bf-20d9-4df2-99f6-b0e0db2e05f0\") " pod="openshift-multus/multus-additional-cni-plugins-jzpsb" Jul 14 20:22:23.558661 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:23.558640 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cbwhw\" (UniqueName: \"kubernetes.io/projected/afbb09bb-badd-4dc0-8082-581c48dda65b-kube-api-access-cbwhw\") pod \"node-ca-88985\" (UID: \"afbb09bb-badd-4dc0-8082-581c48dda65b\") " pod="openshift-image-registry/node-ca-88985" Jul 14 20:22:23.558970 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:23.558949 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rv8g8\" (UniqueName: \"kubernetes.io/projected/92cbd473-9b2e-4faa-a232-810627648cc9-kube-api-access-rv8g8\") pod \"network-metrics-daemon-rvhpg\" (UID: \"92cbd473-9b2e-4faa-a232-810627648cc9\") " pod="openshift-multus/network-metrics-daemon-rvhpg" Jul 14 20:22:23.559031 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:23.559022 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-26qpq\" (UniqueName: \"kubernetes.io/projected/07899815-131f-4211-a7f0-6ce06c8c9ea1-kube-api-access-26qpq\") pod \"aws-ebs-csi-driver-node-g77ql\" (UID: \"07899815-131f-4211-a7f0-6ce06c8c9ea1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-g77ql" Jul 14 20:22:23.560697 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:23.560681 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pkcsg\" (UniqueName: \"kubernetes.io/projected/9588ab98-33e8-4c89-807d-a805240f1b81-kube-api-access-pkcsg\") pod \"ovnkube-node-hzbx8\" (UID: \"9588ab98-33e8-4c89-807d-a805240f1b81\") " pod="openshift-ovn-kubernetes/ovnkube-node-hzbx8" Jul 14 20:22:23.647262 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:23.647217 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/05045fa2-6bd3-4ee8-8f2a-e9056c8a316f-kubelet-config\") pod \"global-pull-secret-syncer-sgvdx\" (UID: \"05045fa2-6bd3-4ee8-8f2a-e9056c8a316f\") " pod="kube-system/global-pull-secret-syncer-sgvdx" Jul 14 20:22:23.647450 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:23.647273 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9d9cd7af-00ce-4f0a-8205-faf1e29a50e7-tmp-dir\") pod \"node-resolver-dznvt\" (UID: \"9d9cd7af-00ce-4f0a-8205-faf1e29a50e7\") " pod="openshift-dns/node-resolver-dznvt" Jul 14 20:22:23.647450 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:23.647308 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/05045fa2-6bd3-4ee8-8f2a-e9056c8a316f-original-pull-secret\") pod \"global-pull-secret-syncer-sgvdx\" (UID: \"05045fa2-6bd3-4ee8-8f2a-e9056c8a316f\") " pod="kube-system/global-pull-secret-syncer-sgvdx" Jul 14 20:22:23.647450 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:23.647329 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/9d9cd7af-00ce-4f0a-8205-faf1e29a50e7-hosts-file\") pod \"node-resolver-dznvt\" (UID: \"9d9cd7af-00ce-4f0a-8205-faf1e29a50e7\") " pod="openshift-dns/node-resolver-dznvt" Jul 14 20:22:23.647450 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:23.647327 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/05045fa2-6bd3-4ee8-8f2a-e9056c8a316f-kubelet-config\") pod \"global-pull-secret-syncer-sgvdx\" (UID: \"05045fa2-6bd3-4ee8-8f2a-e9056c8a316f\") " pod="kube-system/global-pull-secret-syncer-sgvdx" Jul 14 20:22:23.647450 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:23.647351 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-76hvp\" (UniqueName: \"kubernetes.io/projected/d4588597-416b-427f-85d1-d01e54549a84-kube-api-access-76hvp\") pod \"iptables-alerter-9bnrv\" (UID: \"d4588597-416b-427f-85d1-d01e54549a84\") " pod="openshift-network-operator/iptables-alerter-9bnrv" Jul 14 20:22:23.647450 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:23.647396 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/d4588597-416b-427f-85d1-d01e54549a84-iptables-alerter-script\") pod \"iptables-alerter-9bnrv\" (UID: \"d4588597-416b-427f-85d1-d01e54549a84\") " pod="openshift-network-operator/iptables-alerter-9bnrv" Jul 14 20:22:23.647450 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:23.647436 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/05045fa2-6bd3-4ee8-8f2a-e9056c8a316f-dbus\") pod \"global-pull-secret-syncer-sgvdx\" (UID: \"05045fa2-6bd3-4ee8-8f2a-e9056c8a316f\") " pod="kube-system/global-pull-secret-syncer-sgvdx" Jul 14 20:22:23.647450 ip-10-0-141-61 kubenswrapper[2563]: E0714 20:22:23.647445 2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 14 20:22:23.647817 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:23.647455 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fhfr4\" (UniqueName: \"kubernetes.io/projected/9d9cd7af-00ce-4f0a-8205-faf1e29a50e7-kube-api-access-fhfr4\") pod \"node-resolver-dznvt\" (UID: \"9d9cd7af-00ce-4f0a-8205-faf1e29a50e7\") " pod="openshift-dns/node-resolver-dznvt" Jul 14 20:22:23.647817 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:23.647480 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d4588597-416b-427f-85d1-d01e54549a84-host-slash\") pod \"iptables-alerter-9bnrv\" (UID: \"d4588597-416b-427f-85d1-d01e54549a84\") " pod="openshift-network-operator/iptables-alerter-9bnrv" Jul 14 20:22:23.647817 ip-10-0-141-61 kubenswrapper[2563]: E0714 20:22:23.647508 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/05045fa2-6bd3-4ee8-8f2a-e9056c8a316f-original-pull-secret podName:05045fa2-6bd3-4ee8-8f2a-e9056c8a316f nodeName:}" failed. No retries permitted until 2026-07-14 20:22:24.147487318 +0000 UTC m=+3.068650619 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/05045fa2-6bd3-4ee8-8f2a-e9056c8a316f-original-pull-secret") pod "global-pull-secret-syncer-sgvdx" (UID: "05045fa2-6bd3-4ee8-8f2a-e9056c8a316f") : object "kube-system"/"original-pull-secret" not registered Jul 14 20:22:23.647817 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:23.647521 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d4588597-416b-427f-85d1-d01e54549a84-host-slash\") pod \"iptables-alerter-9bnrv\" (UID: \"d4588597-416b-427f-85d1-d01e54549a84\") " pod="openshift-network-operator/iptables-alerter-9bnrv" Jul 14 20:22:23.647817 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:23.647595 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/9d9cd7af-00ce-4f0a-8205-faf1e29a50e7-hosts-file\") pod \"node-resolver-dznvt\" (UID: \"9d9cd7af-00ce-4f0a-8205-faf1e29a50e7\") " pod="openshift-dns/node-resolver-dznvt" Jul 14 20:22:23.647817 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:23.647617 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9d9cd7af-00ce-4f0a-8205-faf1e29a50e7-tmp-dir\") pod \"node-resolver-dznvt\" (UID: \"9d9cd7af-00ce-4f0a-8205-faf1e29a50e7\") " pod="openshift-dns/node-resolver-dznvt" Jul 14 20:22:23.648114 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:23.647875 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/05045fa2-6bd3-4ee8-8f2a-e9056c8a316f-dbus\") pod \"global-pull-secret-syncer-sgvdx\" (UID: \"05045fa2-6bd3-4ee8-8f2a-e9056c8a316f\") " pod="kube-system/global-pull-secret-syncer-sgvdx" Jul 14 20:22:23.648266 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:23.648246 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/d4588597-416b-427f-85d1-d01e54549a84-iptables-alerter-script\") pod \"iptables-alerter-9bnrv\" (UID: \"d4588597-416b-427f-85d1-d01e54549a84\") " pod="openshift-network-operator/iptables-alerter-9bnrv" Jul 14 20:22:23.656711 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:23.656656 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fhfr4\" (UniqueName: \"kubernetes.io/projected/9d9cd7af-00ce-4f0a-8205-faf1e29a50e7-kube-api-access-fhfr4\") pod \"node-resolver-dznvt\" (UID: \"9d9cd7af-00ce-4f0a-8205-faf1e29a50e7\") " pod="openshift-dns/node-resolver-dznvt" Jul 14 20:22:23.657229 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:23.657205 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-76hvp\" (UniqueName: \"kubernetes.io/projected/d4588597-416b-427f-85d1-d01e54549a84-kube-api-access-76hvp\") pod \"iptables-alerter-9bnrv\" (UID: \"d4588597-416b-427f-85d1-d01e54549a84\") " pod="openshift-network-operator/iptables-alerter-9bnrv" Jul 14 20:22:23.731503 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:23.731469 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-bbzpp" Jul 14 20:22:23.739408 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:23.739380 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-jzpsb" Jul 14 20:22:23.748340 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:23.748309 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-g77ql" Jul 14 20:22:23.755095 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:23.755070 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-zltwm" Jul 14 20:22:23.770904 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:23.770875 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-88985" Jul 14 20:22:23.778636 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:23.778605 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-hzbx8" Jul 14 20:22:23.785475 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:23.785443 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-dwzbf" Jul 14 20:22:23.792371 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:23.792348 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-dznvt" Jul 14 20:22:23.798920 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:23.798895 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-9bnrv" Jul 14 20:22:23.835607 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:23.835577 2563 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 14 20:22:24.051474 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:24.051433 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/92cbd473-9b2e-4faa-a232-810627648cc9-metrics-certs\") pod \"network-metrics-daemon-rvhpg\" (UID: \"92cbd473-9b2e-4faa-a232-810627648cc9\") " pod="openshift-multus/network-metrics-daemon-rvhpg" Jul 14 20:22:24.051664 ip-10-0-141-61 kubenswrapper[2563]: E0714 20:22:24.051598 2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 14 20:22:24.051664 ip-10-0-141-61 kubenswrapper[2563]: E0714 20:22:24.051661 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/92cbd473-9b2e-4faa-a232-810627648cc9-metrics-certs podName:92cbd473-9b2e-4faa-a232-810627648cc9 nodeName:}" failed. No retries permitted until 2026-07-14 20:22:25.051642894 +0000 UTC m=+3.972806184 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/92cbd473-9b2e-4faa-a232-810627648cc9-metrics-certs") pod "network-metrics-daemon-rvhpg" (UID: "92cbd473-9b2e-4faa-a232-810627648cc9") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 14 20:22:24.152638 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:24.152597 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/05045fa2-6bd3-4ee8-8f2a-e9056c8a316f-original-pull-secret\") pod \"global-pull-secret-syncer-sgvdx\" (UID: \"05045fa2-6bd3-4ee8-8f2a-e9056c8a316f\") " pod="kube-system/global-pull-secret-syncer-sgvdx" Jul 14 20:22:24.152842 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:24.152661 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r8lsz\" (UniqueName: \"kubernetes.io/projected/92f9c302-6d80-4679-9c71-1d37255dc852-kube-api-access-r8lsz\") pod \"network-check-target-2r82p\" (UID: \"92f9c302-6d80-4679-9c71-1d37255dc852\") " pod="openshift-network-diagnostics/network-check-target-2r82p" Jul 14 20:22:24.152842 ip-10-0-141-61 kubenswrapper[2563]: E0714 20:22:24.152766 2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 14 20:22:24.152842 ip-10-0-141-61 kubenswrapper[2563]: E0714 20:22:24.152830 2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 14 20:22:24.152842 ip-10-0-141-61 kubenswrapper[2563]: E0714 20:22:24.152847 2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 14 20:22:24.153069 ip-10-0-141-61 kubenswrapper[2563]: E0714 20:22:24.152856 2563 projected.go:196] Error preparing data for projected volume kube-api-access-r8lsz for pod openshift-network-diagnostics/network-check-target-2r82p: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 14 20:22:24.153069 ip-10-0-141-61 kubenswrapper[2563]: E0714 20:22:24.152837 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/05045fa2-6bd3-4ee8-8f2a-e9056c8a316f-original-pull-secret podName:05045fa2-6bd3-4ee8-8f2a-e9056c8a316f nodeName:}" failed. No retries permitted until 2026-07-14 20:22:25.152821752 +0000 UTC m=+4.073985045 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/05045fa2-6bd3-4ee8-8f2a-e9056c8a316f-original-pull-secret") pod "global-pull-secret-syncer-sgvdx" (UID: "05045fa2-6bd3-4ee8-8f2a-e9056c8a316f") : object "kube-system"/"original-pull-secret" not registered Jul 14 20:22:24.153069 ip-10-0-141-61 kubenswrapper[2563]: E0714 20:22:24.152912 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/92f9c302-6d80-4679-9c71-1d37255dc852-kube-api-access-r8lsz podName:92f9c302-6d80-4679-9c71-1d37255dc852 nodeName:}" failed. No retries permitted until 2026-07-14 20:22:25.152896718 +0000 UTC m=+4.074060009 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-r8lsz" (UniqueName: "kubernetes.io/projected/92f9c302-6d80-4679-9c71-1d37255dc852-kube-api-access-r8lsz") pod "network-check-target-2r82p" (UID: "92f9c302-6d80-4679-9c71-1d37255dc852") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 14 20:22:24.469523 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:24.469490 2563 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-13 20:17:23 +0000 UTC" deadline="2028-04-05 03:01:08.796036364 +0000 UTC" Jul 14 20:22:24.469523 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:24.469524 2563 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15126h38m44.326517575s" Jul 14 20:22:24.480151 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:24.480053 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod035331bf_20d9_4df2_99f6_b0e0db2e05f0.slice/crio-2b507b04cac4e78cdacdd017e9d4a00bcc189cf1f09355239bb6de54cae58745 WatchSource:0}: Error finding container 2b507b04cac4e78cdacdd017e9d4a00bcc189cf1f09355239bb6de54cae58745: Status 404 returned error can't find the container with id 2b507b04cac4e78cdacdd017e9d4a00bcc189cf1f09355239bb6de54cae58745 Jul 14 20:22:24.482451 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:24.482412 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5835902d_ae0d_4395_9843_0ef04ad24976.slice/crio-939210e33540a1ec94b61de447614bc762298da0de7e22b72088a5b4d31c7724 WatchSource:0}: Error finding container 939210e33540a1ec94b61de447614bc762298da0de7e22b72088a5b4d31c7724: Status 404 returned error can't find the container with id 939210e33540a1ec94b61de447614bc762298da0de7e22b72088a5b4d31c7724 Jul 14 20:22:24.483238 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:24.483211 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb36e05db_0a22_4de6_bed8_73b483a3a2a2.slice/crio-13954007542d06d8445793efcd829ea7bbd1e2dd59cb2dec0025141c1dd8c340 WatchSource:0}: Error finding container 13954007542d06d8445793efcd829ea7bbd1e2dd59cb2dec0025141c1dd8c340: Status 404 returned error can't find the container with id 13954007542d06d8445793efcd829ea7bbd1e2dd59cb2dec0025141c1dd8c340 Jul 14 20:22:24.484572 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:24.484550 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podafbb09bb_badd_4dc0_8082_581c48dda65b.slice/crio-a2045bae273f6e3c17c8c9d4c37a9359d65a29fc96902b00a41cbe2448942ebf WatchSource:0}: Error finding container a2045bae273f6e3c17c8c9d4c37a9359d65a29fc96902b00a41cbe2448942ebf: Status 404 returned error can't find the container with id a2045bae273f6e3c17c8c9d4c37a9359d65a29fc96902b00a41cbe2448942ebf Jul 14 20:22:24.485762 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:24.485741 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9d9cd7af_00ce_4f0a_8205_faf1e29a50e7.slice/crio-a5a6bc39938f4f62d1f430cc03683c56cb1acedbf3049ad487984a5f42bcfd75 WatchSource:0}: Error finding container a5a6bc39938f4f62d1f430cc03683c56cb1acedbf3049ad487984a5f42bcfd75: Status 404 returned error can't find the container with id a5a6bc39938f4f62d1f430cc03683c56cb1acedbf3049ad487984a5f42bcfd75 Jul 14 20:22:24.485986 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:24.485950 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3d2553b9_0627_4fd5_b973_76e17eabf098.slice/crio-f15f864f2702dc2648593c815ddd181fd13193c04f486831b66bb26de594a2fd WatchSource:0}: Error finding container f15f864f2702dc2648593c815ddd181fd13193c04f486831b66bb26de594a2fd: Status 404 returned error can't find the container with id f15f864f2702dc2648593c815ddd181fd13193c04f486831b66bb26de594a2fd Jul 14 20:22:24.487636 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:24.487275 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod07899815_131f_4211_a7f0_6ce06c8c9ea1.slice/crio-05fa22705da42afafd2d15c710e2fddc4772b5b1870f90018325075305d6562d WatchSource:0}: Error finding container 05fa22705da42afafd2d15c710e2fddc4772b5b1870f90018325075305d6562d: Status 404 returned error can't find the container with id 05fa22705da42afafd2d15c710e2fddc4772b5b1870f90018325075305d6562d Jul 14 20:22:24.488805 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:24.488788 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9588ab98_33e8_4c89_807d_a805240f1b81.slice/crio-2806019598c0c3a0e072faa2a64af1ac71bfebb351ffb6563aea92b0c6c81a1f WatchSource:0}: Error finding container 2806019598c0c3a0e072faa2a64af1ac71bfebb351ffb6563aea92b0c6c81a1f: Status 404 returned error can't find the container with id 2806019598c0c3a0e072faa2a64af1ac71bfebb351ffb6563aea92b0c6c81a1f Jul 14 20:22:24.493864 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:24.492480 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd4588597_416b_427f_85d1_d01e54549a84.slice/crio-cb6c78bc5087275c7bf9aeb58502db6d897079e5c5c892f1cc083550425de0ea WatchSource:0}: Error finding container cb6c78bc5087275c7bf9aeb58502db6d897079e5c5c892f1cc083550425de0ea: Status 404 returned error can't find the container with id cb6c78bc5087275c7bf9aeb58502db6d897079e5c5c892f1cc083550425de0ea Jul 14 20:22:24.613575 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:24.613402 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-61.ec2.internal" event={"ID":"9a7e15c41e1da145327771a9c8cb3e39","Type":"ContainerStarted","Data":"b29bb3c443564ea6b1ef3e4419bb99489518de724c68ce578f46676e0a81b61b"} Jul 14 20:22:24.614950 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:24.614855 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-hzbx8" event={"ID":"9588ab98-33e8-4c89-807d-a805240f1b81","Type":"ContainerStarted","Data":"2806019598c0c3a0e072faa2a64af1ac71bfebb351ffb6563aea92b0c6c81a1f"} Jul 14 20:22:24.616056 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:24.616024 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-dwzbf" event={"ID":"3d2553b9-0627-4fd5-b973-76e17eabf098","Type":"ContainerStarted","Data":"f15f864f2702dc2648593c815ddd181fd13193c04f486831b66bb26de594a2fd"} Jul 14 20:22:24.617169 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:24.617149 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-dznvt" event={"ID":"9d9cd7af-00ce-4f0a-8205-faf1e29a50e7","Type":"ContainerStarted","Data":"a5a6bc39938f4f62d1f430cc03683c56cb1acedbf3049ad487984a5f42bcfd75"} Jul 14 20:22:24.618240 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:24.618216 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-zltwm" event={"ID":"b36e05db-0a22-4de6-bed8-73b483a3a2a2","Type":"ContainerStarted","Data":"13954007542d06d8445793efcd829ea7bbd1e2dd59cb2dec0025141c1dd8c340"} Jul 14 20:22:24.619306 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:24.619285 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-bbzpp" event={"ID":"5835902d-ae0d-4395-9843-0ef04ad24976","Type":"ContainerStarted","Data":"939210e33540a1ec94b61de447614bc762298da0de7e22b72088a5b4d31c7724"} Jul 14 20:22:24.620358 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:24.620321 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-jzpsb" event={"ID":"035331bf-20d9-4df2-99f6-b0e0db2e05f0","Type":"ContainerStarted","Data":"2b507b04cac4e78cdacdd017e9d4a00bcc189cf1f09355239bb6de54cae58745"} Jul 14 20:22:24.621280 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:24.621260 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-9bnrv" event={"ID":"d4588597-416b-427f-85d1-d01e54549a84","Type":"ContainerStarted","Data":"cb6c78bc5087275c7bf9aeb58502db6d897079e5c5c892f1cc083550425de0ea"} Jul 14 20:22:24.622320 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:24.622297 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-g77ql" event={"ID":"07899815-131f-4211-a7f0-6ce06c8c9ea1","Type":"ContainerStarted","Data":"05fa22705da42afafd2d15c710e2fddc4772b5b1870f90018325075305d6562d"} Jul 14 20:22:24.623368 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:24.623350 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-88985" event={"ID":"afbb09bb-badd-4dc0-8082-581c48dda65b","Type":"ContainerStarted","Data":"a2045bae273f6e3c17c8c9d4c37a9359d65a29fc96902b00a41cbe2448942ebf"} Jul 14 20:22:24.627067 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:24.627021 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-61.ec2.internal" podStartSLOduration=2.627005948 podStartE2EDuration="2.627005948s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 20:22:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 20:22:24.626274512 +0000 UTC m=+3.547437822" watchObservedRunningTime="2026-07-14 20:22:24.627005948 +0000 UTC m=+3.548169340" Jul 14 20:22:25.061474 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:25.061310 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/92cbd473-9b2e-4faa-a232-810627648cc9-metrics-certs\") pod \"network-metrics-daemon-rvhpg\" (UID: \"92cbd473-9b2e-4faa-a232-810627648cc9\") " pod="openshift-multus/network-metrics-daemon-rvhpg" Jul 14 20:22:25.061776 ip-10-0-141-61 kubenswrapper[2563]: E0714 20:22:25.061513 2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 14 20:22:25.061776 ip-10-0-141-61 kubenswrapper[2563]: E0714 20:22:25.061577 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/92cbd473-9b2e-4faa-a232-810627648cc9-metrics-certs podName:92cbd473-9b2e-4faa-a232-810627648cc9 nodeName:}" failed. No retries permitted until 2026-07-14 20:22:27.061558062 +0000 UTC m=+5.982721352 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/92cbd473-9b2e-4faa-a232-810627648cc9-metrics-certs") pod "network-metrics-daemon-rvhpg" (UID: "92cbd473-9b2e-4faa-a232-810627648cc9") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 14 20:22:25.162568 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:25.162526 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r8lsz\" (UniqueName: \"kubernetes.io/projected/92f9c302-6d80-4679-9c71-1d37255dc852-kube-api-access-r8lsz\") pod \"network-check-target-2r82p\" (UID: \"92f9c302-6d80-4679-9c71-1d37255dc852\") " pod="openshift-network-diagnostics/network-check-target-2r82p" Jul 14 20:22:25.162749 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:25.162638 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/05045fa2-6bd3-4ee8-8f2a-e9056c8a316f-original-pull-secret\") pod \"global-pull-secret-syncer-sgvdx\" (UID: \"05045fa2-6bd3-4ee8-8f2a-e9056c8a316f\") " pod="kube-system/global-pull-secret-syncer-sgvdx" Jul 14 20:22:25.162808 ip-10-0-141-61 kubenswrapper[2563]: E0714 20:22:25.162764 2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 14 20:22:25.162857 ip-10-0-141-61 kubenswrapper[2563]: E0714 20:22:25.162827 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/05045fa2-6bd3-4ee8-8f2a-e9056c8a316f-original-pull-secret podName:05045fa2-6bd3-4ee8-8f2a-e9056c8a316f nodeName:}" failed. No retries permitted until 2026-07-14 20:22:27.162809436 +0000 UTC m=+6.083972731 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/05045fa2-6bd3-4ee8-8f2a-e9056c8a316f-original-pull-secret") pod "global-pull-secret-syncer-sgvdx" (UID: "05045fa2-6bd3-4ee8-8f2a-e9056c8a316f") : object "kube-system"/"original-pull-secret" not registered Jul 14 20:22:25.163397 ip-10-0-141-61 kubenswrapper[2563]: E0714 20:22:25.163253 2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 14 20:22:25.163397 ip-10-0-141-61 kubenswrapper[2563]: E0714 20:22:25.163274 2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 14 20:22:25.163397 ip-10-0-141-61 kubenswrapper[2563]: E0714 20:22:25.163286 2563 projected.go:196] Error preparing data for projected volume kube-api-access-r8lsz for pod openshift-network-diagnostics/network-check-target-2r82p: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 14 20:22:25.163397 ip-10-0-141-61 kubenswrapper[2563]: E0714 20:22:25.163338 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/92f9c302-6d80-4679-9c71-1d37255dc852-kube-api-access-r8lsz podName:92f9c302-6d80-4679-9c71-1d37255dc852 nodeName:}" failed. No retries permitted until 2026-07-14 20:22:27.163317835 +0000 UTC m=+6.084481126 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-r8lsz" (UniqueName: "kubernetes.io/projected/92f9c302-6d80-4679-9c71-1d37255dc852-kube-api-access-r8lsz") pod "network-check-target-2r82p" (UID: "92f9c302-6d80-4679-9c71-1d37255dc852") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 14 20:22:25.470583 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:25.470540 2563 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-13 20:17:23 +0000 UTC" deadline="2028-02-23 07:44:44.629574625 +0000 UTC" Jul 14 20:22:25.470583 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:25.470582 2563 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14123h22m19.158997241s" Jul 14 20:22:25.610467 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:25.610269 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sgvdx" Jul 14 20:22:25.610467 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:25.610328 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2r82p" Jul 14 20:22:25.610467 ip-10-0-141-61 kubenswrapper[2563]: E0714 20:22:25.610459 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sgvdx" podUID="05045fa2-6bd3-4ee8-8f2a-e9056c8a316f" Jul 14 20:22:25.610743 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:25.610273 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rvhpg" Jul 14 20:22:25.610743 ip-10-0-141-61 kubenswrapper[2563]: E0714 20:22:25.610637 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rvhpg" podUID="92cbd473-9b2e-4faa-a232-810627648cc9" Jul 14 20:22:25.610842 ip-10-0-141-61 kubenswrapper[2563]: E0714 20:22:25.610741 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2r82p" podUID="92f9c302-6d80-4679-9c71-1d37255dc852" Jul 14 20:22:25.647629 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:25.647561 2563 generic.go:356] "Generic (PLEG): container finished" podID="0110f0e9eedb7f728fd5fbcff19c03aa" containerID="46369147ba150d2b92135809f51d17ea16cededd43bd0882136dbb396b19d3ad" exitCode=0 Jul 14 20:22:25.647813 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:25.647692 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-61.ec2.internal" event={"ID":"0110f0e9eedb7f728fd5fbcff19c03aa","Type":"ContainerDied","Data":"46369147ba150d2b92135809f51d17ea16cededd43bd0882136dbb396b19d3ad"} Jul 14 20:22:26.661245 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:26.661206 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-61.ec2.internal" event={"ID":"0110f0e9eedb7f728fd5fbcff19c03aa","Type":"ContainerStarted","Data":"14cbe6d18139b61432560168644790b00ae72731a935671ecd77212e8483f37c"} Jul 14 20:22:26.676772 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:26.676062 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-61.ec2.internal" podStartSLOduration=4.676040409 podStartE2EDuration="4.676040409s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 20:22:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 20:22:26.675242825 +0000 UTC m=+5.596406135" watchObservedRunningTime="2026-07-14 20:22:26.676040409 +0000 UTC m=+5.597203718" Jul 14 20:22:27.078279 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:27.078182 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/92cbd473-9b2e-4faa-a232-810627648cc9-metrics-certs\") pod \"network-metrics-daemon-rvhpg\" (UID: \"92cbd473-9b2e-4faa-a232-810627648cc9\") " pod="openshift-multus/network-metrics-daemon-rvhpg" Jul 14 20:22:27.078446 ip-10-0-141-61 kubenswrapper[2563]: E0714 20:22:27.078398 2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 14 20:22:27.078504 ip-10-0-141-61 kubenswrapper[2563]: E0714 20:22:27.078462 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/92cbd473-9b2e-4faa-a232-810627648cc9-metrics-certs podName:92cbd473-9b2e-4faa-a232-810627648cc9 nodeName:}" failed. No retries permitted until 2026-07-14 20:22:31.078442349 +0000 UTC m=+9.999605638 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/92cbd473-9b2e-4faa-a232-810627648cc9-metrics-certs") pod "network-metrics-daemon-rvhpg" (UID: "92cbd473-9b2e-4faa-a232-810627648cc9") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 14 20:22:27.180447 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:27.180407 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r8lsz\" (UniqueName: \"kubernetes.io/projected/92f9c302-6d80-4679-9c71-1d37255dc852-kube-api-access-r8lsz\") pod \"network-check-target-2r82p\" (UID: \"92f9c302-6d80-4679-9c71-1d37255dc852\") " pod="openshift-network-diagnostics/network-check-target-2r82p" Jul 14 20:22:27.180612 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:27.180492 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/05045fa2-6bd3-4ee8-8f2a-e9056c8a316f-original-pull-secret\") pod \"global-pull-secret-syncer-sgvdx\" (UID: \"05045fa2-6bd3-4ee8-8f2a-e9056c8a316f\") " pod="kube-system/global-pull-secret-syncer-sgvdx" Jul 14 20:22:27.180691 ip-10-0-141-61 kubenswrapper[2563]: E0714 20:22:27.180610 2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 14 20:22:27.180691 ip-10-0-141-61 kubenswrapper[2563]: E0714 20:22:27.180673 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/05045fa2-6bd3-4ee8-8f2a-e9056c8a316f-original-pull-secret podName:05045fa2-6bd3-4ee8-8f2a-e9056c8a316f nodeName:}" failed. No retries permitted until 2026-07-14 20:22:31.180653353 +0000 UTC m=+10.101816654 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/05045fa2-6bd3-4ee8-8f2a-e9056c8a316f-original-pull-secret") pod "global-pull-secret-syncer-sgvdx" (UID: "05045fa2-6bd3-4ee8-8f2a-e9056c8a316f") : object "kube-system"/"original-pull-secret" not registered Jul 14 20:22:27.181080 ip-10-0-141-61 kubenswrapper[2563]: E0714 20:22:27.181059 2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 14 20:22:27.181080 ip-10-0-141-61 kubenswrapper[2563]: E0714 20:22:27.181083 2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 14 20:22:27.181277 ip-10-0-141-61 kubenswrapper[2563]: E0714 20:22:27.181095 2563 projected.go:196] Error preparing data for projected volume kube-api-access-r8lsz for pod openshift-network-diagnostics/network-check-target-2r82p: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 14 20:22:27.181277 ip-10-0-141-61 kubenswrapper[2563]: E0714 20:22:27.181160 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/92f9c302-6d80-4679-9c71-1d37255dc852-kube-api-access-r8lsz podName:92f9c302-6d80-4679-9c71-1d37255dc852 nodeName:}" failed. No retries permitted until 2026-07-14 20:22:31.181143918 +0000 UTC m=+10.102307220 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-r8lsz" (UniqueName: "kubernetes.io/projected/92f9c302-6d80-4679-9c71-1d37255dc852-kube-api-access-r8lsz") pod "network-check-target-2r82p" (UID: "92f9c302-6d80-4679-9c71-1d37255dc852") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 14 20:22:27.600848 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:27.600215 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2r82p" Jul 14 20:22:27.600848 ip-10-0-141-61 kubenswrapper[2563]: E0714 20:22:27.600342 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2r82p" podUID="92f9c302-6d80-4679-9c71-1d37255dc852" Jul 14 20:22:27.600848 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:27.600708 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sgvdx" Jul 14 20:22:27.600848 ip-10-0-141-61 kubenswrapper[2563]: E0714 20:22:27.600798 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sgvdx" podUID="05045fa2-6bd3-4ee8-8f2a-e9056c8a316f" Jul 14 20:22:27.603661 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:27.601212 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rvhpg" Jul 14 20:22:27.603661 ip-10-0-141-61 kubenswrapper[2563]: E0714 20:22:27.601380 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rvhpg" podUID="92cbd473-9b2e-4faa-a232-810627648cc9" Jul 14 20:22:29.600865 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:29.600777 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rvhpg" Jul 14 20:22:29.600865 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:29.600828 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sgvdx" Jul 14 20:22:29.601492 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:29.600779 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2r82p" Jul 14 20:22:29.601492 ip-10-0-141-61 kubenswrapper[2563]: E0714 20:22:29.600914 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rvhpg" podUID="92cbd473-9b2e-4faa-a232-810627648cc9" Jul 14 20:22:29.601492 ip-10-0-141-61 kubenswrapper[2563]: E0714 20:22:29.601054 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2r82p" podUID="92f9c302-6d80-4679-9c71-1d37255dc852" Jul 14 20:22:29.601492 ip-10-0-141-61 kubenswrapper[2563]: E0714 20:22:29.601155 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sgvdx" podUID="05045fa2-6bd3-4ee8-8f2a-e9056c8a316f" Jul 14 20:22:31.119278 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:31.119221 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/92cbd473-9b2e-4faa-a232-810627648cc9-metrics-certs\") pod \"network-metrics-daemon-rvhpg\" (UID: \"92cbd473-9b2e-4faa-a232-810627648cc9\") " pod="openshift-multus/network-metrics-daemon-rvhpg" Jul 14 20:22:31.119724 ip-10-0-141-61 kubenswrapper[2563]: E0714 20:22:31.119376 2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 14 20:22:31.119724 ip-10-0-141-61 kubenswrapper[2563]: E0714 20:22:31.119465 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/92cbd473-9b2e-4faa-a232-810627648cc9-metrics-certs podName:92cbd473-9b2e-4faa-a232-810627648cc9 nodeName:}" failed. No retries permitted until 2026-07-14 20:22:39.119438059 +0000 UTC m=+18.040601351 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/92cbd473-9b2e-4faa-a232-810627648cc9-metrics-certs") pod "network-metrics-daemon-rvhpg" (UID: "92cbd473-9b2e-4faa-a232-810627648cc9") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 14 20:22:31.220277 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:31.220233 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r8lsz\" (UniqueName: \"kubernetes.io/projected/92f9c302-6d80-4679-9c71-1d37255dc852-kube-api-access-r8lsz\") pod \"network-check-target-2r82p\" (UID: \"92f9c302-6d80-4679-9c71-1d37255dc852\") " pod="openshift-network-diagnostics/network-check-target-2r82p" Jul 14 20:22:31.220438 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:31.220329 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/05045fa2-6bd3-4ee8-8f2a-e9056c8a316f-original-pull-secret\") pod \"global-pull-secret-syncer-sgvdx\" (UID: \"05045fa2-6bd3-4ee8-8f2a-e9056c8a316f\") " pod="kube-system/global-pull-secret-syncer-sgvdx" Jul 14 20:22:31.220520 ip-10-0-141-61 kubenswrapper[2563]: E0714 20:22:31.220452 2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 14 20:22:31.220520 ip-10-0-141-61 kubenswrapper[2563]: E0714 20:22:31.220513 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/05045fa2-6bd3-4ee8-8f2a-e9056c8a316f-original-pull-secret podName:05045fa2-6bd3-4ee8-8f2a-e9056c8a316f nodeName:}" failed. No retries permitted until 2026-07-14 20:22:39.220495104 +0000 UTC m=+18.141658404 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/05045fa2-6bd3-4ee8-8f2a-e9056c8a316f-original-pull-secret") pod "global-pull-secret-syncer-sgvdx" (UID: "05045fa2-6bd3-4ee8-8f2a-e9056c8a316f") : object "kube-system"/"original-pull-secret" not registered Jul 14 20:22:31.220956 ip-10-0-141-61 kubenswrapper[2563]: E0714 20:22:31.220937 2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 14 20:22:31.221066 ip-10-0-141-61 kubenswrapper[2563]: E0714 20:22:31.220963 2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 14 20:22:31.221066 ip-10-0-141-61 kubenswrapper[2563]: E0714 20:22:31.220975 2563 projected.go:196] Error preparing data for projected volume kube-api-access-r8lsz for pod openshift-network-diagnostics/network-check-target-2r82p: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 14 20:22:31.221066 ip-10-0-141-61 kubenswrapper[2563]: E0714 20:22:31.221022 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/92f9c302-6d80-4679-9c71-1d37255dc852-kube-api-access-r8lsz podName:92f9c302-6d80-4679-9c71-1d37255dc852 nodeName:}" failed. No retries permitted until 2026-07-14 20:22:39.221006704 +0000 UTC m=+18.142170006 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-r8lsz" (UniqueName: "kubernetes.io/projected/92f9c302-6d80-4679-9c71-1d37255dc852-kube-api-access-r8lsz") pod "network-check-target-2r82p" (UID: "92f9c302-6d80-4679-9c71-1d37255dc852") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 14 20:22:31.601705 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:31.601669 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rvhpg" Jul 14 20:22:31.601897 ip-10-0-141-61 kubenswrapper[2563]: E0714 20:22:31.601801 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rvhpg" podUID="92cbd473-9b2e-4faa-a232-810627648cc9" Jul 14 20:22:31.602258 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:31.602235 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2r82p" Jul 14 20:22:31.602398 ip-10-0-141-61 kubenswrapper[2563]: E0714 20:22:31.602366 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2r82p" podUID="92f9c302-6d80-4679-9c71-1d37255dc852" Jul 14 20:22:31.602560 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:31.602514 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sgvdx" Jul 14 20:22:31.602621 ip-10-0-141-61 kubenswrapper[2563]: E0714 20:22:31.602602 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sgvdx" podUID="05045fa2-6bd3-4ee8-8f2a-e9056c8a316f" Jul 14 20:22:33.601680 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:33.601266 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2r82p" Jul 14 20:22:33.601680 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:33.601302 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sgvdx" Jul 14 20:22:33.601680 ip-10-0-141-61 kubenswrapper[2563]: E0714 20:22:33.601384 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2r82p" podUID="92f9c302-6d80-4679-9c71-1d37255dc852" Jul 14 20:22:33.601680 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:33.601449 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rvhpg" Jul 14 20:22:33.601680 ip-10-0-141-61 kubenswrapper[2563]: E0714 20:22:33.601554 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rvhpg" podUID="92cbd473-9b2e-4faa-a232-810627648cc9" Jul 14 20:22:33.601680 ip-10-0-141-61 kubenswrapper[2563]: E0714 20:22:33.601638 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sgvdx" podUID="05045fa2-6bd3-4ee8-8f2a-e9056c8a316f" Jul 14 20:22:35.600921 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:35.600883 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2r82p" Jul 14 20:22:35.600921 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:35.600905 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sgvdx" Jul 14 20:22:35.601428 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:35.600905 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rvhpg" Jul 14 20:22:35.601428 ip-10-0-141-61 kubenswrapper[2563]: E0714 20:22:35.600985 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2r82p" podUID="92f9c302-6d80-4679-9c71-1d37255dc852" Jul 14 20:22:35.601428 ip-10-0-141-61 kubenswrapper[2563]: E0714 20:22:35.601083 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sgvdx" podUID="05045fa2-6bd3-4ee8-8f2a-e9056c8a316f" Jul 14 20:22:35.601428 ip-10-0-141-61 kubenswrapper[2563]: E0714 20:22:35.601180 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rvhpg" podUID="92cbd473-9b2e-4faa-a232-810627648cc9" Jul 14 20:22:37.600446 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:37.600407 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sgvdx" Jul 14 20:22:37.600884 ip-10-0-141-61 kubenswrapper[2563]: E0714 20:22:37.600534 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sgvdx" podUID="05045fa2-6bd3-4ee8-8f2a-e9056c8a316f" Jul 14 20:22:37.600884 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:37.600407 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2r82p" Jul 14 20:22:37.600884 ip-10-0-141-61 kubenswrapper[2563]: E0714 20:22:37.600645 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2r82p" podUID="92f9c302-6d80-4679-9c71-1d37255dc852" Jul 14 20:22:37.600884 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:37.600407 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rvhpg" Jul 14 20:22:37.600884 ip-10-0-141-61 kubenswrapper[2563]: E0714 20:22:37.600708 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rvhpg" podUID="92cbd473-9b2e-4faa-a232-810627648cc9" Jul 14 20:22:39.183281 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:39.183240 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/92cbd473-9b2e-4faa-a232-810627648cc9-metrics-certs\") pod \"network-metrics-daemon-rvhpg\" (UID: \"92cbd473-9b2e-4faa-a232-810627648cc9\") " pod="openshift-multus/network-metrics-daemon-rvhpg" Jul 14 20:22:39.183809 ip-10-0-141-61 kubenswrapper[2563]: E0714 20:22:39.183423 2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 14 20:22:39.183809 ip-10-0-141-61 kubenswrapper[2563]: E0714 20:22:39.183506 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/92cbd473-9b2e-4faa-a232-810627648cc9-metrics-certs podName:92cbd473-9b2e-4faa-a232-810627648cc9 nodeName:}" failed. No retries permitted until 2026-07-14 20:22:55.183483637 +0000 UTC m=+34.104646924 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/92cbd473-9b2e-4faa-a232-810627648cc9-metrics-certs") pod "network-metrics-daemon-rvhpg" (UID: "92cbd473-9b2e-4faa-a232-810627648cc9") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 14 20:22:39.284660 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:39.284617 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r8lsz\" (UniqueName: \"kubernetes.io/projected/92f9c302-6d80-4679-9c71-1d37255dc852-kube-api-access-r8lsz\") pod \"network-check-target-2r82p\" (UID: \"92f9c302-6d80-4679-9c71-1d37255dc852\") " pod="openshift-network-diagnostics/network-check-target-2r82p" Jul 14 20:22:39.284858 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:39.284725 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/05045fa2-6bd3-4ee8-8f2a-e9056c8a316f-original-pull-secret\") pod \"global-pull-secret-syncer-sgvdx\" (UID: \"05045fa2-6bd3-4ee8-8f2a-e9056c8a316f\") " pod="kube-system/global-pull-secret-syncer-sgvdx" Jul 14 20:22:39.284858 ip-10-0-141-61 kubenswrapper[2563]: E0714 20:22:39.284842 2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 14 20:22:39.284858 ip-10-0-141-61 kubenswrapper[2563]: E0714 20:22:39.284847 2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 14 20:22:39.285004 ip-10-0-141-61 kubenswrapper[2563]: E0714 20:22:39.284873 2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 14 20:22:39.285004 ip-10-0-141-61 kubenswrapper[2563]: E0714 20:22:39.284887 2563 projected.go:196] Error preparing data for projected volume kube-api-access-r8lsz for pod openshift-network-diagnostics/network-check-target-2r82p: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 14 20:22:39.285004 ip-10-0-141-61 kubenswrapper[2563]: E0714 20:22:39.284904 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/05045fa2-6bd3-4ee8-8f2a-e9056c8a316f-original-pull-secret podName:05045fa2-6bd3-4ee8-8f2a-e9056c8a316f nodeName:}" failed. No retries permitted until 2026-07-14 20:22:55.284884531 +0000 UTC m=+34.206047864 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/05045fa2-6bd3-4ee8-8f2a-e9056c8a316f-original-pull-secret") pod "global-pull-secret-syncer-sgvdx" (UID: "05045fa2-6bd3-4ee8-8f2a-e9056c8a316f") : object "kube-system"/"original-pull-secret" not registered Jul 14 20:22:39.285004 ip-10-0-141-61 kubenswrapper[2563]: E0714 20:22:39.284942 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/92f9c302-6d80-4679-9c71-1d37255dc852-kube-api-access-r8lsz podName:92f9c302-6d80-4679-9c71-1d37255dc852 nodeName:}" failed. No retries permitted until 2026-07-14 20:22:55.28492537 +0000 UTC m=+34.206088674 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-r8lsz" (UniqueName: "kubernetes.io/projected/92f9c302-6d80-4679-9c71-1d37255dc852-kube-api-access-r8lsz") pod "network-check-target-2r82p" (UID: "92f9c302-6d80-4679-9c71-1d37255dc852") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 14 20:22:39.604179 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:39.604081 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sgvdx" Jul 14 20:22:39.604179 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:39.604154 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rvhpg" Jul 14 20:22:39.604389 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:39.604154 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2r82p" Jul 14 20:22:39.604389 ip-10-0-141-61 kubenswrapper[2563]: E0714 20:22:39.604232 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sgvdx" podUID="05045fa2-6bd3-4ee8-8f2a-e9056c8a316f" Jul 14 20:22:39.604389 ip-10-0-141-61 kubenswrapper[2563]: E0714 20:22:39.604341 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rvhpg" podUID="92cbd473-9b2e-4faa-a232-810627648cc9" Jul 14 20:22:39.604389 ip-10-0-141-61 kubenswrapper[2563]: E0714 20:22:39.604382 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2r82p" podUID="92f9c302-6d80-4679-9c71-1d37255dc852" Jul 14 20:22:41.600852 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:41.600821 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sgvdx" Jul 14 20:22:41.601355 ip-10-0-141-61 kubenswrapper[2563]: E0714 20:22:41.600923 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sgvdx" podUID="05045fa2-6bd3-4ee8-8f2a-e9056c8a316f" Jul 14 20:22:41.601355 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:41.600983 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rvhpg" Jul 14 20:22:41.601355 ip-10-0-141-61 kubenswrapper[2563]: E0714 20:22:41.601072 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rvhpg" podUID="92cbd473-9b2e-4faa-a232-810627648cc9" Jul 14 20:22:41.601355 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:41.601093 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2r82p" Jul 14 20:22:41.601355 ip-10-0-141-61 kubenswrapper[2563]: E0714 20:22:41.601222 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2r82p" podUID="92f9c302-6d80-4679-9c71-1d37255dc852" Jul 14 20:22:43.602746 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:43.602708 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rvhpg" Jul 14 20:22:43.603230 ip-10-0-141-61 kubenswrapper[2563]: E0714 20:22:43.602853 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rvhpg" podUID="92cbd473-9b2e-4faa-a232-810627648cc9" Jul 14 20:22:43.603230 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:43.602930 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2r82p" Jul 14 20:22:43.603230 ip-10-0-141-61 kubenswrapper[2563]: E0714 20:22:43.602993 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2r82p" podUID="92f9c302-6d80-4679-9c71-1d37255dc852" Jul 14 20:22:43.603230 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:43.603012 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sgvdx" Jul 14 20:22:43.603230 ip-10-0-141-61 kubenswrapper[2563]: E0714 20:22:43.603076 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sgvdx" podUID="05045fa2-6bd3-4ee8-8f2a-e9056c8a316f" Jul 14 20:22:44.703396 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:44.702935 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-zltwm" event={"ID":"b36e05db-0a22-4de6-bed8-73b483a3a2a2","Type":"ContainerStarted","Data":"ec0a0232c5dd0597f0a502d5808194bcf9d9ffd1df1747f18a5c730c99028d3b"} Jul 14 20:22:44.704514 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:44.704490 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-bbzpp" event={"ID":"5835902d-ae0d-4395-9843-0ef04ad24976","Type":"ContainerStarted","Data":"b3d7662cba810ade266cc04198dfba7810936479eae946fb6446a9c297268c87"} Jul 14 20:22:44.706172 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:44.706150 2563 generic.go:356] "Generic (PLEG): container finished" podID="035331bf-20d9-4df2-99f6-b0e0db2e05f0" containerID="d3679526ff913399608d57b6099022c52558b4f765ca953b563419dd220a830b" exitCode=0 Jul 14 20:22:44.706286 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:44.706226 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-jzpsb" event={"ID":"035331bf-20d9-4df2-99f6-b0e0db2e05f0","Type":"ContainerDied","Data":"d3679526ff913399608d57b6099022c52558b4f765ca953b563419dd220a830b"} Jul 14 20:22:44.707916 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:44.707875 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-g77ql" event={"ID":"07899815-131f-4211-a7f0-6ce06c8c9ea1","Type":"ContainerStarted","Data":"a1e02db7cb9afc0883e3df9412462b0890b096ec44116c8a4d44e37061e56eb5"} Jul 14 20:22:44.709494 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:44.709452 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-88985" event={"ID":"afbb09bb-badd-4dc0-8082-581c48dda65b","Type":"ContainerStarted","Data":"1250e18a5e14c3390f92de34952baaf0e5427aed4205330069031ab3e6915edd"} Jul 14 20:22:44.712621 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:44.712598 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hzbx8_9588ab98-33e8-4c89-807d-a805240f1b81/ovn-acl-logging/0.log" Jul 14 20:22:44.712961 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:44.712941 2563 generic.go:356] "Generic (PLEG): container finished" podID="9588ab98-33e8-4c89-807d-a805240f1b81" containerID="0f752d1843af91f1b54670bcee7d503e02a6a977c271e42dc6bb08ecd6198cdf" exitCode=1 Jul 14 20:22:44.713052 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:44.713007 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-hzbx8" event={"ID":"9588ab98-33e8-4c89-807d-a805240f1b81","Type":"ContainerStarted","Data":"357fcf1b4cd91ce2299c4d04f15f0dd06be61daf70eb41c839def99e7192f81c"} Jul 14 20:22:44.713095 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:44.713072 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-hzbx8" event={"ID":"9588ab98-33e8-4c89-807d-a805240f1b81","Type":"ContainerStarted","Data":"9161f2b2103656b47392395524463e3edc7cf61ed0989599acbd18c9aa8fca8b"} Jul 14 20:22:44.713158 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:44.713140 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-hzbx8" event={"ID":"9588ab98-33e8-4c89-807d-a805240f1b81","Type":"ContainerStarted","Data":"b1fe9401955b3f457a61d2576696f04b230fdacc0d27233360d39eb19ec07549"} Jul 14 20:22:44.713203 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:44.713184 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-hzbx8" event={"ID":"9588ab98-33e8-4c89-807d-a805240f1b81","Type":"ContainerStarted","Data":"3c1d588fdbe8ce61926f30d86442207af3cfc3c0fb80339bd7f9f180742118a9"} Jul 14 20:22:44.713249 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:44.713229 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-hzbx8" event={"ID":"9588ab98-33e8-4c89-807d-a805240f1b81","Type":"ContainerDied","Data":"0f752d1843af91f1b54670bcee7d503e02a6a977c271e42dc6bb08ecd6198cdf"} Jul 14 20:22:44.713283 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:44.713276 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-hzbx8" event={"ID":"9588ab98-33e8-4c89-807d-a805240f1b81","Type":"ContainerStarted","Data":"a80fb9dd1f3d3fded60d4081edc4b423b51c4481fe5c23ddcb25db662f00384e"} Jul 14 20:22:44.714486 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:44.714453 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-dwzbf" event={"ID":"3d2553b9-0627-4fd5-b973-76e17eabf098","Type":"ContainerStarted","Data":"7f068d70aa7d76656b8b33125c35da254282d36788a498f069b46c9c197974b2"} Jul 14 20:22:44.715986 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:44.715966 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-dznvt" event={"ID":"9d9cd7af-00ce-4f0a-8205-faf1e29a50e7","Type":"ContainerStarted","Data":"dd204b693ae6c41c1d7ec4dc935cd45f22ea15b4ef1ad82a706a977856036b39"} Jul 14 20:22:44.717902 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:44.717859 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-zltwm" podStartSLOduration=4.580542013 podStartE2EDuration="23.717845403s" totalImagesPullingTime="19.13730339s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 20:22:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 20:22:44.717418761 +0000 UTC m=+23.638582071" watchObservedRunningTime="2026-07-14 20:22:44.717845403 +0000 UTC m=+23.639008712" Jul 14 20:22:44.729967 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:44.729908 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-dznvt" podStartSLOduration=4.607294456 podStartE2EDuration="23.7298899s" totalImagesPullingTime="19.122595444s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 20:22:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 20:22:44.729646499 +0000 UTC m=+23.650809808" watchObservedRunningTime="2026-07-14 20:22:44.7298899 +0000 UTC m=+23.651053213" Jul 14 20:22:44.743116 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:44.743050 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-88985" podStartSLOduration=4.628036936 podStartE2EDuration="23.743028882s" totalImagesPullingTime="19.114991946s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 20:22:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 20:22:44.742893665 +0000 UTC m=+23.664056985" watchObservedRunningTime="2026-07-14 20:22:44.743028882 +0000 UTC m=+23.664192191" Jul 14 20:22:44.761491 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:44.761438 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-dwzbf" podStartSLOduration=4.677369994 podStartE2EDuration="23.761422075s" totalImagesPullingTime="19.084052081s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 20:22:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 20:22:44.761335535 +0000 UTC m=+23.682498844" watchObservedRunningTime="2026-07-14 20:22:44.761422075 +0000 UTC m=+23.682585383" Jul 14 20:22:44.794768 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:44.794719 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-bbzpp" podStartSLOduration=4.61600133 podStartE2EDuration="23.794702884s" totalImagesPullingTime="19.178701554s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 20:22:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 20:22:44.794620325 +0000 UTC m=+23.715783645" watchObservedRunningTime="2026-07-14 20:22:44.794702884 +0000 UTC m=+23.715866193" Jul 14 20:22:45.318575 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:45.318523 2563 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 14 20:22:45.502910 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:45.502754 2563 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-14T20:22:45.318552613Z","UUID":"5a2ca229-2c2a-48ee-8f54-d18967a164d6","Handler":null,"Name":"","Endpoint":""} Jul 14 20:22:45.504601 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:45.504529 2563 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 14 20:22:45.504601 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:45.504573 2563 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 14 20:22:45.604364 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:45.604333 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2r82p" Jul 14 20:22:45.604364 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:45.604366 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rvhpg" Jul 14 20:22:45.604596 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:45.604367 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sgvdx" Jul 14 20:22:45.604596 ip-10-0-141-61 kubenswrapper[2563]: E0714 20:22:45.604455 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2r82p" podUID="92f9c302-6d80-4679-9c71-1d37255dc852" Jul 14 20:22:45.604596 ip-10-0-141-61 kubenswrapper[2563]: E0714 20:22:45.604585 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rvhpg" podUID="92cbd473-9b2e-4faa-a232-810627648cc9" Jul 14 20:22:45.604747 ip-10-0-141-61 kubenswrapper[2563]: E0714 20:22:45.604675 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sgvdx" podUID="05045fa2-6bd3-4ee8-8f2a-e9056c8a316f" Jul 14 20:22:45.720068 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:45.720026 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-9bnrv" event={"ID":"d4588597-416b-427f-85d1-d01e54549a84","Type":"ContainerStarted","Data":"d7fa5620dc6590ca10354c4f279c5b80478e8745f1fca1035e5754815660f0dc"} Jul 14 20:22:45.722588 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:45.722557 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-g77ql" event={"ID":"07899815-131f-4211-a7f0-6ce06c8c9ea1","Type":"ContainerStarted","Data":"e72ec5871dd7cb452f8c48b78aeb9bcfe74512c15081ea3d7a09d0b850e6c96b"} Jul 14 20:22:45.733728 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:45.733676 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-9bnrv" podStartSLOduration=5.621798266 podStartE2EDuration="24.733658638s" totalImagesPullingTime="19.111860372s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 20:22:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 20:22:45.733583018 +0000 UTC m=+24.654746317" watchObservedRunningTime="2026-07-14 20:22:45.733658638 +0000 UTC m=+24.654821946" Jul 14 20:22:47.604087 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:47.603879 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sgvdx" Jul 14 20:22:47.604648 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:47.603928 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rvhpg" Jul 14 20:22:47.604648 ip-10-0-141-61 kubenswrapper[2563]: E0714 20:22:47.604186 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sgvdx" podUID="05045fa2-6bd3-4ee8-8f2a-e9056c8a316f" Jul 14 20:22:47.604648 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:47.603928 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2r82p" Jul 14 20:22:47.604648 ip-10-0-141-61 kubenswrapper[2563]: E0714 20:22:47.604268 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rvhpg" podUID="92cbd473-9b2e-4faa-a232-810627648cc9" Jul 14 20:22:47.604648 ip-10-0-141-61 kubenswrapper[2563]: E0714 20:22:47.604327 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2r82p" podUID="92f9c302-6d80-4679-9c71-1d37255dc852" Jul 14 20:22:47.716067 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:47.716024 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-dwzbf" Jul 14 20:22:47.716714 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:47.716693 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-dwzbf" Jul 14 20:22:47.729938 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:47.729913 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hzbx8_9588ab98-33e8-4c89-807d-a805240f1b81/ovn-acl-logging/0.log" Jul 14 20:22:47.730355 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:47.730325 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-hzbx8" event={"ID":"9588ab98-33e8-4c89-807d-a805240f1b81","Type":"ContainerStarted","Data":"5d463207696123229bfce5b275f6f08db4e96fd2b4e3619000b9fc1f7e4c1414"} Jul 14 20:22:47.732756 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:47.732728 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-g77ql" event={"ID":"07899815-131f-4211-a7f0-6ce06c8c9ea1","Type":"ContainerStarted","Data":"c967b2416d93a17d97ac33058a7ccd7e0ed2b7991c1ff77814484b606c2db84c"} Jul 14 20:22:47.733167 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:47.732979 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-dwzbf" Jul 14 20:22:47.733506 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:47.733491 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-dwzbf" Jul 14 20:22:47.765671 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:47.765602 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-g77ql" podStartSLOduration=4.901364358 podStartE2EDuration="26.765580717s" totalImagesPullingTime="21.864216359s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 20:22:21 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 20:22:47.749339305 +0000 UTC m=+26.670502616" watchObservedRunningTime="2026-07-14 20:22:47.765580717 +0000 UTC m=+26.686744029" Jul 14 20:22:49.603650 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:49.603390 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sgvdx" Jul 14 20:22:49.604398 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:49.603390 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rvhpg" Jul 14 20:22:49.604398 ip-10-0-141-61 kubenswrapper[2563]: E0714 20:22:49.603763 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rvhpg" podUID="92cbd473-9b2e-4faa-a232-810627648cc9" Jul 14 20:22:49.604398 ip-10-0-141-61 kubenswrapper[2563]: E0714 20:22:49.603649 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sgvdx" podUID="05045fa2-6bd3-4ee8-8f2a-e9056c8a316f" Jul 14 20:22:49.604398 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:49.603395 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2r82p" Jul 14 20:22:49.604398 ip-10-0-141-61 kubenswrapper[2563]: E0714 20:22:49.603853 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2r82p" podUID="92f9c302-6d80-4679-9c71-1d37255dc852" Jul 14 20:22:49.741062 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:49.741034 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hzbx8_9588ab98-33e8-4c89-807d-a805240f1b81/ovn-acl-logging/0.log" Jul 14 20:22:49.741422 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:49.741397 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-hzbx8" event={"ID":"9588ab98-33e8-4c89-807d-a805240f1b81","Type":"ContainerStarted","Data":"ea722396ba2c17f72be76df88693cbe9f0738d29a536ab16e75e536ece5d95f3"} Jul 14 20:22:49.741688 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:49.741639 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-hzbx8" Jul 14 20:22:49.741920 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:49.741904 2563 scope.go:117] "RemoveContainer" containerID="0f752d1843af91f1b54670bcee7d503e02a6a977c271e42dc6bb08ecd6198cdf" Jul 14 20:22:49.743424 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:49.743397 2563 generic.go:356] "Generic (PLEG): container finished" podID="035331bf-20d9-4df2-99f6-b0e0db2e05f0" containerID="d4093021b19b204167c2fb01a9e8ab5dc2f5b6b379107ab1ec94a8b2fc72ceb1" exitCode=0 Jul 14 20:22:49.743515 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:49.743462 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-jzpsb" event={"ID":"035331bf-20d9-4df2-99f6-b0e0db2e05f0","Type":"ContainerDied","Data":"d4093021b19b204167c2fb01a9e8ab5dc2f5b6b379107ab1ec94a8b2fc72ceb1"} Jul 14 20:22:49.758640 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:49.758618 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-hzbx8" Jul 14 20:22:50.751291 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:50.751266 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hzbx8_9588ab98-33e8-4c89-807d-a805240f1b81/ovn-acl-logging/0.log" Jul 14 20:22:50.751732 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:50.751666 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-hzbx8" event={"ID":"9588ab98-33e8-4c89-807d-a805240f1b81","Type":"ContainerStarted","Data":"149f658d7950ef8e0ce63d16d399891b9cf78b96400df96ed0b3566a243a1085"} Jul 14 20:22:50.751919 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:50.751897 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-hzbx8" Jul 14 20:22:50.751993 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:50.751966 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-hzbx8" Jul 14 20:22:50.768182 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:50.768155 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-hzbx8" Jul 14 20:22:50.783022 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:50.782969 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-hzbx8" podStartSLOduration=10.580686959 podStartE2EDuration="29.782950389s" totalImagesPullingTime="19.20226343s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 20:22:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 20:22:50.782378125 +0000 UTC m=+29.703541437" watchObservedRunningTime="2026-07-14 20:22:50.782950389 +0000 UTC m=+29.704113698" Jul 14 20:22:50.921827 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:50.921800 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-rvhpg"] Jul 14 20:22:50.921982 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:50.921918 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rvhpg" Jul 14 20:22:50.922037 ip-10-0-141-61 kubenswrapper[2563]: E0714 20:22:50.922019 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rvhpg" podUID="92cbd473-9b2e-4faa-a232-810627648cc9" Jul 14 20:22:50.925172 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:50.925116 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-sgvdx"] Jul 14 20:22:50.925317 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:50.925249 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sgvdx" Jul 14 20:22:50.925398 ip-10-0-141-61 kubenswrapper[2563]: E0714 20:22:50.925335 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sgvdx" podUID="05045fa2-6bd3-4ee8-8f2a-e9056c8a316f" Jul 14 20:22:50.925817 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:50.925780 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-2r82p"] Jul 14 20:22:50.925919 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:50.925882 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2r82p" Jul 14 20:22:50.925991 ip-10-0-141-61 kubenswrapper[2563]: E0714 20:22:50.925972 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2r82p" podUID="92f9c302-6d80-4679-9c71-1d37255dc852" Jul 14 20:22:51.756087 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:51.755845 2563 generic.go:356] "Generic (PLEG): container finished" podID="035331bf-20d9-4df2-99f6-b0e0db2e05f0" containerID="be65749932d8026003091845f33851c066daee171e63a25a6a8e9f4a4f2b242b" exitCode=0 Jul 14 20:22:51.756502 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:51.755933 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-jzpsb" event={"ID":"035331bf-20d9-4df2-99f6-b0e0db2e05f0","Type":"ContainerDied","Data":"be65749932d8026003091845f33851c066daee171e63a25a6a8e9f4a4f2b242b"} Jul 14 20:22:52.376321 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:52.376268 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:22:52.478775 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:52.478741 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:22:52.600520 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:52.600481 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rvhpg" Jul 14 20:22:52.600520 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:52.600517 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2r82p" Jul 14 20:22:52.600768 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:52.600600 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sgvdx" Jul 14 20:22:52.600768 ip-10-0-141-61 kubenswrapper[2563]: E0714 20:22:52.600616 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rvhpg" podUID="92cbd473-9b2e-4faa-a232-810627648cc9" Jul 14 20:22:52.600768 ip-10-0-141-61 kubenswrapper[2563]: E0714 20:22:52.600671 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sgvdx" podUID="05045fa2-6bd3-4ee8-8f2a-e9056c8a316f" Jul 14 20:22:52.600768 ip-10-0-141-61 kubenswrapper[2563]: E0714 20:22:52.600723 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2r82p" podUID="92f9c302-6d80-4679-9c71-1d37255dc852" Jul 14 20:22:53.440062 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:53.440035 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:22:53.492280 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:53.492248 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:22:53.688734 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:53.688704 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:22:53.752140 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:53.752043 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:22:53.763081 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:53.763045 2563 generic.go:356] "Generic (PLEG): container finished" podID="035331bf-20d9-4df2-99f6-b0e0db2e05f0" containerID="7d1220033dfc673b2df8c3ca4f94003393f6cc27493a246db94ee8ca71e4b5a8" exitCode=0 Jul 14 20:22:53.763267 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:53.763141 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-jzpsb" event={"ID":"035331bf-20d9-4df2-99f6-b0e0db2e05f0","Type":"ContainerDied","Data":"7d1220033dfc673b2df8c3ca4f94003393f6cc27493a246db94ee8ca71e4b5a8"} Jul 14 20:22:54.600563 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:54.600533 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sgvdx" Jul 14 20:22:54.601177 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:54.600533 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2r82p" Jul 14 20:22:54.601177 ip-10-0-141-61 kubenswrapper[2563]: E0714 20:22:54.600664 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-sgvdx" podUID="05045fa2-6bd3-4ee8-8f2a-e9056c8a316f" Jul 14 20:22:54.601177 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:54.600539 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rvhpg" Jul 14 20:22:54.601177 ip-10-0-141-61 kubenswrapper[2563]: E0714 20:22:54.600719 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2r82p" podUID="92f9c302-6d80-4679-9c71-1d37255dc852" Jul 14 20:22:54.601177 ip-10-0-141-61 kubenswrapper[2563]: E0714 20:22:54.600802 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rvhpg" podUID="92cbd473-9b2e-4faa-a232-810627648cc9" Jul 14 20:22:55.203472 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:55.203438 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/92cbd473-9b2e-4faa-a232-810627648cc9-metrics-certs\") pod \"network-metrics-daemon-rvhpg\" (UID: \"92cbd473-9b2e-4faa-a232-810627648cc9\") " pod="openshift-multus/network-metrics-daemon-rvhpg" Jul 14 20:22:55.203627 ip-10-0-141-61 kubenswrapper[2563]: E0714 20:22:55.203559 2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 14 20:22:55.203627 ip-10-0-141-61 kubenswrapper[2563]: E0714 20:22:55.203612 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/92cbd473-9b2e-4faa-a232-810627648cc9-metrics-certs podName:92cbd473-9b2e-4faa-a232-810627648cc9 nodeName:}" failed. No retries permitted until 2026-07-14 20:23:27.203597396 +0000 UTC m=+66.124760683 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/92cbd473-9b2e-4faa-a232-810627648cc9-metrics-certs") pod "network-metrics-daemon-rvhpg" (UID: "92cbd473-9b2e-4faa-a232-810627648cc9") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 14 20:22:55.304638 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:55.304599 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r8lsz\" (UniqueName: \"kubernetes.io/projected/92f9c302-6d80-4679-9c71-1d37255dc852-kube-api-access-r8lsz\") pod \"network-check-target-2r82p\" (UID: \"92f9c302-6d80-4679-9c71-1d37255dc852\") " pod="openshift-network-diagnostics/network-check-target-2r82p" Jul 14 20:22:55.304820 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:55.304695 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/05045fa2-6bd3-4ee8-8f2a-e9056c8a316f-original-pull-secret\") pod \"global-pull-secret-syncer-sgvdx\" (UID: \"05045fa2-6bd3-4ee8-8f2a-e9056c8a316f\") " pod="kube-system/global-pull-secret-syncer-sgvdx" Jul 14 20:22:55.304820 ip-10-0-141-61 kubenswrapper[2563]: E0714 20:22:55.304756 2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 14 20:22:55.304820 ip-10-0-141-61 kubenswrapper[2563]: E0714 20:22:55.304781 2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 14 20:22:55.304820 ip-10-0-141-61 kubenswrapper[2563]: E0714 20:22:55.304790 2563 projected.go:196] Error preparing data for projected volume kube-api-access-r8lsz for pod openshift-network-diagnostics/network-check-target-2r82p: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 14 20:22:55.304820 ip-10-0-141-61 kubenswrapper[2563]: E0714 20:22:55.304792 2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 14 20:22:55.305055 ip-10-0-141-61 kubenswrapper[2563]: E0714 20:22:55.304839 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/92f9c302-6d80-4679-9c71-1d37255dc852-kube-api-access-r8lsz podName:92f9c302-6d80-4679-9c71-1d37255dc852 nodeName:}" failed. No retries permitted until 2026-07-14 20:23:27.304822117 +0000 UTC m=+66.225985419 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-r8lsz" (UniqueName: "kubernetes.io/projected/92f9c302-6d80-4679-9c71-1d37255dc852-kube-api-access-r8lsz") pod "network-check-target-2r82p" (UID: "92f9c302-6d80-4679-9c71-1d37255dc852") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 14 20:22:55.305055 ip-10-0-141-61 kubenswrapper[2563]: E0714 20:22:55.304859 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/05045fa2-6bd3-4ee8-8f2a-e9056c8a316f-original-pull-secret podName:05045fa2-6bd3-4ee8-8f2a-e9056c8a316f nodeName:}" failed. No retries permitted until 2026-07-14 20:23:27.304850534 +0000 UTC m=+66.226013822 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/05045fa2-6bd3-4ee8-8f2a-e9056c8a316f-original-pull-secret") pod "global-pull-secret-syncer-sgvdx" (UID: "05045fa2-6bd3-4ee8-8f2a-e9056c8a316f") : object "kube-system"/"original-pull-secret" not registered Jul 14 20:22:56.388781 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:56.388757 2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-61.ec2.internal" event="NodeReady" Jul 14 20:22:56.389333 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:56.388891 2563 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 14 20:22:56.435251 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:56.435217 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-knrl4"] Jul 14 20:22:56.437218 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:56.437192 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-v2v5k"] Jul 14 20:22:56.437347 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:56.437332 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-knrl4" Jul 14 20:22:56.439175 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:56.439003 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-v2v5k" Jul 14 20:22:56.439318 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:56.439298 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 14 20:22:56.439395 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:56.439348 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-qv2tt\"" Jul 14 20:22:56.439445 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:56.439407 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 14 20:22:56.440991 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:56.440970 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 14 20:22:56.441248 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:56.441230 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 14 20:22:56.441350 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:56.441328 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 14 20:22:56.441426 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:56.441401 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-plh4p\"" Jul 14 20:22:56.447341 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:56.447283 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-v2v5k"] Jul 14 20:22:56.449575 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:56.449552 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-knrl4"] Jul 14 20:22:56.600851 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:56.600750 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rvhpg" Jul 14 20:22:56.600851 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:56.600774 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sgvdx" Jul 14 20:22:56.601071 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:56.600761 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2r82p" Jul 14 20:22:56.603398 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:56.603246 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-8rw2b\"" Jul 14 20:22:56.603398 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:56.603265 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 14 20:22:56.603398 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:56.603278 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 14 20:22:56.603398 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:56.603312 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-5hghx\"" Jul 14 20:22:56.603398 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:56.603252 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 14 20:22:56.603749 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:56.603584 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 14 20:22:56.613633 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:56.613601 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/b4e3e206-bf7b-47bf-b037-9c12fca3799b-config-volume\") pod \"dns-default-knrl4\" (UID: \"b4e3e206-bf7b-47bf-b037-9c12fca3799b\") " pod="openshift-dns/dns-default-knrl4" Jul 14 20:22:56.613749 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:56.613647 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b4e3e206-bf7b-47bf-b037-9c12fca3799b-tmp-dir\") pod \"dns-default-knrl4\" (UID: \"b4e3e206-bf7b-47bf-b037-9c12fca3799b\") " pod="openshift-dns/dns-default-knrl4" Jul 14 20:22:56.613749 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:56.613707 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b4e3e206-bf7b-47bf-b037-9c12fca3799b-metrics-tls\") pod \"dns-default-knrl4\" (UID: \"b4e3e206-bf7b-47bf-b037-9c12fca3799b\") " pod="openshift-dns/dns-default-knrl4" Jul 14 20:22:56.613872 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:56.613782 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pvtwz\" (UniqueName: \"kubernetes.io/projected/b4e3e206-bf7b-47bf-b037-9c12fca3799b-kube-api-access-pvtwz\") pod \"dns-default-knrl4\" (UID: \"b4e3e206-bf7b-47bf-b037-9c12fca3799b\") " pod="openshift-dns/dns-default-knrl4" Jul 14 20:22:56.613872 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:56.613815 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/cf457900-0b78-45d7-9e9e-99d9fe24bd1c-cert\") pod \"ingress-canary-v2v5k\" (UID: \"cf457900-0b78-45d7-9e9e-99d9fe24bd1c\") " pod="openshift-ingress-canary/ingress-canary-v2v5k" Jul 14 20:22:56.613988 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:56.613875 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pvrrw\" (UniqueName: \"kubernetes.io/projected/cf457900-0b78-45d7-9e9e-99d9fe24bd1c-kube-api-access-pvrrw\") pod \"ingress-canary-v2v5k\" (UID: \"cf457900-0b78-45d7-9e9e-99d9fe24bd1c\") " pod="openshift-ingress-canary/ingress-canary-v2v5k" Jul 14 20:22:56.714582 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:56.714538 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pvrrw\" (UniqueName: \"kubernetes.io/projected/cf457900-0b78-45d7-9e9e-99d9fe24bd1c-kube-api-access-pvrrw\") pod \"ingress-canary-v2v5k\" (UID: \"cf457900-0b78-45d7-9e9e-99d9fe24bd1c\") " pod="openshift-ingress-canary/ingress-canary-v2v5k" Jul 14 20:22:56.714767 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:56.714606 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/b4e3e206-bf7b-47bf-b037-9c12fca3799b-config-volume\") pod \"dns-default-knrl4\" (UID: \"b4e3e206-bf7b-47bf-b037-9c12fca3799b\") " pod="openshift-dns/dns-default-knrl4" Jul 14 20:22:56.714767 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:56.714629 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b4e3e206-bf7b-47bf-b037-9c12fca3799b-tmp-dir\") pod \"dns-default-knrl4\" (UID: \"b4e3e206-bf7b-47bf-b037-9c12fca3799b\") " pod="openshift-dns/dns-default-knrl4" Jul 14 20:22:56.714767 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:56.714661 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b4e3e206-bf7b-47bf-b037-9c12fca3799b-metrics-tls\") pod \"dns-default-knrl4\" (UID: \"b4e3e206-bf7b-47bf-b037-9c12fca3799b\") " pod="openshift-dns/dns-default-knrl4" Jul 14 20:22:56.714767 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:56.714706 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pvtwz\" (UniqueName: \"kubernetes.io/projected/b4e3e206-bf7b-47bf-b037-9c12fca3799b-kube-api-access-pvtwz\") pod \"dns-default-knrl4\" (UID: \"b4e3e206-bf7b-47bf-b037-9c12fca3799b\") " pod="openshift-dns/dns-default-knrl4" Jul 14 20:22:56.714767 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:56.714732 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/cf457900-0b78-45d7-9e9e-99d9fe24bd1c-cert\") pod \"ingress-canary-v2v5k\" (UID: \"cf457900-0b78-45d7-9e9e-99d9fe24bd1c\") " pod="openshift-ingress-canary/ingress-canary-v2v5k" Jul 14 20:22:56.715025 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:56.715006 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b4e3e206-bf7b-47bf-b037-9c12fca3799b-tmp-dir\") pod \"dns-default-knrl4\" (UID: \"b4e3e206-bf7b-47bf-b037-9c12fca3799b\") " pod="openshift-dns/dns-default-knrl4" Jul 14 20:22:56.715116 ip-10-0-141-61 kubenswrapper[2563]: E0714 20:22:56.715093 2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 14 20:22:56.715279 ip-10-0-141-61 kubenswrapper[2563]: E0714 20:22:56.715168 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b4e3e206-bf7b-47bf-b037-9c12fca3799b-metrics-tls podName:b4e3e206-bf7b-47bf-b037-9c12fca3799b nodeName:}" failed. No retries permitted until 2026-07-14 20:22:57.215150948 +0000 UTC m=+36.136314256 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b4e3e206-bf7b-47bf-b037-9c12fca3799b-metrics-tls") pod "dns-default-knrl4" (UID: "b4e3e206-bf7b-47bf-b037-9c12fca3799b") : secret "dns-default-metrics-tls" not found Jul 14 20:22:56.715279 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:56.715235 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/b4e3e206-bf7b-47bf-b037-9c12fca3799b-config-volume\") pod \"dns-default-knrl4\" (UID: \"b4e3e206-bf7b-47bf-b037-9c12fca3799b\") " pod="openshift-dns/dns-default-knrl4" Jul 14 20:22:56.719596 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:56.719569 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/cf457900-0b78-45d7-9e9e-99d9fe24bd1c-cert\") pod \"ingress-canary-v2v5k\" (UID: \"cf457900-0b78-45d7-9e9e-99d9fe24bd1c\") " pod="openshift-ingress-canary/ingress-canary-v2v5k" Jul 14 20:22:56.723542 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:56.723515 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pvtwz\" (UniqueName: \"kubernetes.io/projected/b4e3e206-bf7b-47bf-b037-9c12fca3799b-kube-api-access-pvtwz\") pod \"dns-default-knrl4\" (UID: \"b4e3e206-bf7b-47bf-b037-9c12fca3799b\") " pod="openshift-dns/dns-default-knrl4" Jul 14 20:22:56.724012 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:56.723993 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pvrrw\" (UniqueName: \"kubernetes.io/projected/cf457900-0b78-45d7-9e9e-99d9fe24bd1c-kube-api-access-pvrrw\") pod \"ingress-canary-v2v5k\" (UID: \"cf457900-0b78-45d7-9e9e-99d9fe24bd1c\") " pod="openshift-ingress-canary/ingress-canary-v2v5k" Jul 14 20:22:56.758830 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:56.758794 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-v2v5k" Jul 14 20:22:57.219043 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:57.219001 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b4e3e206-bf7b-47bf-b037-9c12fca3799b-metrics-tls\") pod \"dns-default-knrl4\" (UID: \"b4e3e206-bf7b-47bf-b037-9c12fca3799b\") " pod="openshift-dns/dns-default-knrl4" Jul 14 20:22:57.219233 ip-10-0-141-61 kubenswrapper[2563]: E0714 20:22:57.219183 2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 14 20:22:57.219298 ip-10-0-141-61 kubenswrapper[2563]: E0714 20:22:57.219260 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b4e3e206-bf7b-47bf-b037-9c12fca3799b-metrics-tls podName:b4e3e206-bf7b-47bf-b037-9c12fca3799b nodeName:}" failed. No retries permitted until 2026-07-14 20:22:58.219240463 +0000 UTC m=+37.140403762 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b4e3e206-bf7b-47bf-b037-9c12fca3799b-metrics-tls") pod "dns-default-knrl4" (UID: "b4e3e206-bf7b-47bf-b037-9c12fca3799b") : secret "dns-default-metrics-tls" not found Jul 14 20:22:58.155744 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:58.155565 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-v2v5k"] Jul 14 20:22:58.159896 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:22:58.159868 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcf457900_0b78_45d7_9e9e_99d9fe24bd1c.slice/crio-c1ac8df1ebd07aa45e509e43e82e899ac8cb9c3b9f14b9fa4f8d8f8a1fc0f4a2 WatchSource:0}: Error finding container c1ac8df1ebd07aa45e509e43e82e899ac8cb9c3b9f14b9fa4f8d8f8a1fc0f4a2: Status 404 returned error can't find the container with id c1ac8df1ebd07aa45e509e43e82e899ac8cb9c3b9f14b9fa4f8d8f8a1fc0f4a2 Jul 14 20:22:58.226919 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:58.226884 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b4e3e206-bf7b-47bf-b037-9c12fca3799b-metrics-tls\") pod \"dns-default-knrl4\" (UID: \"b4e3e206-bf7b-47bf-b037-9c12fca3799b\") " pod="openshift-dns/dns-default-knrl4" Jul 14 20:22:58.227082 ip-10-0-141-61 kubenswrapper[2563]: E0714 20:22:58.227038 2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 14 20:22:58.227136 ip-10-0-141-61 kubenswrapper[2563]: E0714 20:22:58.227108 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b4e3e206-bf7b-47bf-b037-9c12fca3799b-metrics-tls podName:b4e3e206-bf7b-47bf-b037-9c12fca3799b nodeName:}" failed. No retries permitted until 2026-07-14 20:23:00.22709089 +0000 UTC m=+39.148254177 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b4e3e206-bf7b-47bf-b037-9c12fca3799b-metrics-tls") pod "dns-default-knrl4" (UID: "b4e3e206-bf7b-47bf-b037-9c12fca3799b") : secret "dns-default-metrics-tls" not found Jul 14 20:22:58.779963 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:58.779929 2563 generic.go:356] "Generic (PLEG): container finished" podID="035331bf-20d9-4df2-99f6-b0e0db2e05f0" containerID="f52823ad100b13b0da36e22ce8a474d96380736a6a2d52d44e3fc187ed4bf88e" exitCode=0 Jul 14 20:22:58.780178 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:58.780019 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-jzpsb" event={"ID":"035331bf-20d9-4df2-99f6-b0e0db2e05f0","Type":"ContainerDied","Data":"f52823ad100b13b0da36e22ce8a474d96380736a6a2d52d44e3fc187ed4bf88e"} Jul 14 20:22:58.781405 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:58.781381 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-v2v5k" event={"ID":"cf457900-0b78-45d7-9e9e-99d9fe24bd1c","Type":"ContainerStarted","Data":"c1ac8df1ebd07aa45e509e43e82e899ac8cb9c3b9f14b9fa4f8d8f8a1fc0f4a2"} Jul 14 20:22:59.787988 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:59.787954 2563 generic.go:356] "Generic (PLEG): container finished" podID="035331bf-20d9-4df2-99f6-b0e0db2e05f0" containerID="d7c78dd7ecaa161d5676563b444b9854eb2153f3a8fd3ba5e8c3cfeede1fbc9a" exitCode=0 Jul 14 20:22:59.787988 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:59.787994 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-jzpsb" event={"ID":"035331bf-20d9-4df2-99f6-b0e0db2e05f0","Type":"ContainerDied","Data":"d7c78dd7ecaa161d5676563b444b9854eb2153f3a8fd3ba5e8c3cfeede1fbc9a"} Jul 14 20:22:59.841541 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:59.841508 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:22:59.898389 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:22:59.898354 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:23:00.241697 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:23:00.241642 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b4e3e206-bf7b-47bf-b037-9c12fca3799b-metrics-tls\") pod \"dns-default-knrl4\" (UID: \"b4e3e206-bf7b-47bf-b037-9c12fca3799b\") " pod="openshift-dns/dns-default-knrl4" Jul 14 20:23:00.241894 ip-10-0-141-61 kubenswrapper[2563]: E0714 20:23:00.241816 2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 14 20:23:00.241974 ip-10-0-141-61 kubenswrapper[2563]: E0714 20:23:00.241902 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b4e3e206-bf7b-47bf-b037-9c12fca3799b-metrics-tls podName:b4e3e206-bf7b-47bf-b037-9c12fca3799b nodeName:}" failed. No retries permitted until 2026-07-14 20:23:04.241880155 +0000 UTC m=+43.163043463 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b4e3e206-bf7b-47bf-b037-9c12fca3799b-metrics-tls") pod "dns-default-knrl4" (UID: "b4e3e206-bf7b-47bf-b037-9c12fca3799b") : secret "dns-default-metrics-tls" not found Jul 14 20:23:00.322149 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:23:00.322102 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:23:00.376897 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:23:00.376857 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:23:00.794379 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:23:00.794147 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-jzpsb" event={"ID":"035331bf-20d9-4df2-99f6-b0e0db2e05f0","Type":"ContainerStarted","Data":"3226bf1deb29870ce3a263342a58e8a9efbe941c844167d48c683f0288fe5278"} Jul 14 20:23:00.795720 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:23:00.795692 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-v2v5k" event={"ID":"cf457900-0b78-45d7-9e9e-99d9fe24bd1c","Type":"ContainerStarted","Data":"17c0af1a7fc5f0b3cb9f3583a23e874038ac44d47fe418f8cdec6e6653c8bdec"} Jul 14 20:23:00.817984 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:23:00.817914 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-jzpsb" podStartSLOduration=10.007521359 podStartE2EDuration="39.817895731s" totalImagesPullingTime="29.810374372s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 20:22:21 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 20:23:00.817598893 +0000 UTC m=+39.738762203" watchObservedRunningTime="2026-07-14 20:23:00.817895731 +0000 UTC m=+39.739059041" Jul 14 20:23:00.831823 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:23:00.831770 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-v2v5k" podStartSLOduration=2.339957373 podStartE2EDuration="4.831754468s" totalImagesPullingTime="2.491797095s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 20:22:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 20:23:00.830932202 +0000 UTC m=+39.752095511" watchObservedRunningTime="2026-07-14 20:23:00.831754468 +0000 UTC m=+39.752917787" Jul 14 20:23:00.910695 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:23:00.909339 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:23:00.967541 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:23:00.967504 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:23:01.025326 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:23:01.025252 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:23:01.135044 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:23:01.135014 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:23:04.268977 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:23:04.268935 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b4e3e206-bf7b-47bf-b037-9c12fca3799b-metrics-tls\") pod \"dns-default-knrl4\" (UID: \"b4e3e206-bf7b-47bf-b037-9c12fca3799b\") " pod="openshift-dns/dns-default-knrl4" Jul 14 20:23:04.269476 ip-10-0-141-61 kubenswrapper[2563]: E0714 20:23:04.269087 2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 14 20:23:04.269476 ip-10-0-141-61 kubenswrapper[2563]: E0714 20:23:04.269170 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b4e3e206-bf7b-47bf-b037-9c12fca3799b-metrics-tls podName:b4e3e206-bf7b-47bf-b037-9c12fca3799b nodeName:}" failed. No retries permitted until 2026-07-14 20:23:12.269154449 +0000 UTC m=+51.190317743 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b4e3e206-bf7b-47bf-b037-9c12fca3799b-metrics-tls") pod "dns-default-knrl4" (UID: "b4e3e206-bf7b-47bf-b037-9c12fca3799b") : secret "dns-default-metrics-tls" not found Jul 14 20:23:12.331501 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:23:12.331461 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b4e3e206-bf7b-47bf-b037-9c12fca3799b-metrics-tls\") pod \"dns-default-knrl4\" (UID: \"b4e3e206-bf7b-47bf-b037-9c12fca3799b\") " pod="openshift-dns/dns-default-knrl4" Jul 14 20:23:12.331956 ip-10-0-141-61 kubenswrapper[2563]: E0714 20:23:12.331596 2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 14 20:23:12.331956 ip-10-0-141-61 kubenswrapper[2563]: E0714 20:23:12.331662 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b4e3e206-bf7b-47bf-b037-9c12fca3799b-metrics-tls podName:b4e3e206-bf7b-47bf-b037-9c12fca3799b nodeName:}" failed. No retries permitted until 2026-07-14 20:23:28.331645796 +0000 UTC m=+67.252809084 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b4e3e206-bf7b-47bf-b037-9c12fca3799b-metrics-tls") pod "dns-default-knrl4" (UID: "b4e3e206-bf7b-47bf-b037-9c12fca3799b") : secret "dns-default-metrics-tls" not found Jul 14 20:23:14.388878 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:23:14.388836 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:23:14.437293 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:23:14.437260 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:23:16.432540 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:23:16.432506 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-55c6fbd889-rvg84"] Jul 14 20:23:16.435137 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:23:16.435105 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-55c6fbd889-rvg84" Jul 14 20:23:16.438038 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:23:16.438005 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\"" Jul 14 20:23:16.438930 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:23:16.438083 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Jul 14 20:23:16.438930 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:23:16.438087 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Jul 14 20:23:16.439207 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:23:16.439064 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Jul 14 20:23:16.445455 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:23:16.445433 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-55c6fbd889-rvg84"] Jul 14 20:23:16.462447 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:23:16.462417 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/f704a4c0-8d5e-41bf-a173-753ac3b54ff2-klusterlet-config\") pod \"klusterlet-addon-workmgr-55c6fbd889-rvg84\" (UID: \"f704a4c0-8d5e-41bf-a173-753ac3b54ff2\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-55c6fbd889-rvg84" Jul 14 20:23:16.462578 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:23:16.462471 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f704a4c0-8d5e-41bf-a173-753ac3b54ff2-tmp\") pod \"klusterlet-addon-workmgr-55c6fbd889-rvg84\" (UID: \"f704a4c0-8d5e-41bf-a173-753ac3b54ff2\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-55c6fbd889-rvg84" Jul 14 20:23:16.462578 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:23:16.462537 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9x78d\" (UniqueName: \"kubernetes.io/projected/f704a4c0-8d5e-41bf-a173-753ac3b54ff2-kube-api-access-9x78d\") pod \"klusterlet-addon-workmgr-55c6fbd889-rvg84\" (UID: \"f704a4c0-8d5e-41bf-a173-753ac3b54ff2\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-55c6fbd889-rvg84" Jul 14 20:23:16.468700 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:23:16.468676 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-76dff96bd-59zm4"] Jul 14 20:23:16.470535 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:23:16.470520 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-76dff96bd-59zm4" Jul 14 20:23:16.472438 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:23:16.472411 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-92pkj\"" Jul 14 20:23:16.472557 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:23:16.472411 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\"" Jul 14 20:23:16.478917 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:23:16.478892 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-76dff96bd-59zm4"] Jul 14 20:23:16.563404 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:23:16.563366 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mt6q6\" (UniqueName: \"kubernetes.io/projected/1447f395-a017-44cf-9176-2944ead9c090-kube-api-access-mt6q6\") pod \"managed-serviceaccount-addon-agent-76dff96bd-59zm4\" (UID: \"1447f395-a017-44cf-9176-2944ead9c090\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-76dff96bd-59zm4" Jul 14 20:23:16.563583 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:23:16.563426 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f704a4c0-8d5e-41bf-a173-753ac3b54ff2-tmp\") pod \"klusterlet-addon-workmgr-55c6fbd889-rvg84\" (UID: \"f704a4c0-8d5e-41bf-a173-753ac3b54ff2\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-55c6fbd889-rvg84" Jul 14 20:23:16.563583 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:23:16.563449 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9x78d\" (UniqueName: \"kubernetes.io/projected/f704a4c0-8d5e-41bf-a173-753ac3b54ff2-kube-api-access-9x78d\") pod \"klusterlet-addon-workmgr-55c6fbd889-rvg84\" (UID: \"f704a4c0-8d5e-41bf-a173-753ac3b54ff2\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-55c6fbd889-rvg84" Jul 14 20:23:16.563583 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:23:16.563487 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/1447f395-a017-44cf-9176-2944ead9c090-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-76dff96bd-59zm4\" (UID: \"1447f395-a017-44cf-9176-2944ead9c090\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-76dff96bd-59zm4" Jul 14 20:23:16.563583 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:23:16.563523 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/f704a4c0-8d5e-41bf-a173-753ac3b54ff2-klusterlet-config\") pod \"klusterlet-addon-workmgr-55c6fbd889-rvg84\" (UID: \"f704a4c0-8d5e-41bf-a173-753ac3b54ff2\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-55c6fbd889-rvg84" Jul 14 20:23:16.563851 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:23:16.563827 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f704a4c0-8d5e-41bf-a173-753ac3b54ff2-tmp\") pod \"klusterlet-addon-workmgr-55c6fbd889-rvg84\" (UID: \"f704a4c0-8d5e-41bf-a173-753ac3b54ff2\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-55c6fbd889-rvg84" Jul 14 20:23:16.567850 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:23:16.567830 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/f704a4c0-8d5e-41bf-a173-753ac3b54ff2-klusterlet-config\") pod \"klusterlet-addon-workmgr-55c6fbd889-rvg84\" (UID: \"f704a4c0-8d5e-41bf-a173-753ac3b54ff2\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-55c6fbd889-rvg84" Jul 14 20:23:16.571272 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:23:16.571241 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9x78d\" (UniqueName: \"kubernetes.io/projected/f704a4c0-8d5e-41bf-a173-753ac3b54ff2-kube-api-access-9x78d\") pod \"klusterlet-addon-workmgr-55c6fbd889-rvg84\" (UID: \"f704a4c0-8d5e-41bf-a173-753ac3b54ff2\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-55c6fbd889-rvg84" Jul 14 20:23:16.664335 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:23:16.664294 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mt6q6\" (UniqueName: \"kubernetes.io/projected/1447f395-a017-44cf-9176-2944ead9c090-kube-api-access-mt6q6\") pod \"managed-serviceaccount-addon-agent-76dff96bd-59zm4\" (UID: \"1447f395-a017-44cf-9176-2944ead9c090\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-76dff96bd-59zm4" Jul 14 20:23:16.664513 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:23:16.664384 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/1447f395-a017-44cf-9176-2944ead9c090-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-76dff96bd-59zm4\" (UID: \"1447f395-a017-44cf-9176-2944ead9c090\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-76dff96bd-59zm4" Jul 14 20:23:16.667033 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:23:16.667002 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/1447f395-a017-44cf-9176-2944ead9c090-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-76dff96bd-59zm4\" (UID: \"1447f395-a017-44cf-9176-2944ead9c090\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-76dff96bd-59zm4" Jul 14 20:23:16.672235 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:23:16.672210 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mt6q6\" (UniqueName: \"kubernetes.io/projected/1447f395-a017-44cf-9176-2944ead9c090-kube-api-access-mt6q6\") pod \"managed-serviceaccount-addon-agent-76dff96bd-59zm4\" (UID: \"1447f395-a017-44cf-9176-2944ead9c090\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-76dff96bd-59zm4" Jul 14 20:23:16.747592 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:23:16.747507 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-55c6fbd889-rvg84" Jul 14 20:23:16.791043 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:23:16.791011 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-76dff96bd-59zm4" Jul 14 20:23:16.882845 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:23:16.882768 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-55c6fbd889-rvg84"] Jul 14 20:23:16.888052 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:23:16.888024 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf704a4c0_8d5e_41bf_a173_753ac3b54ff2.slice/crio-d3496904437813a0ccfc0866a0019bed173f3178128685b97097f3a674853da8 WatchSource:0}: Error finding container d3496904437813a0ccfc0866a0019bed173f3178128685b97097f3a674853da8: Status 404 returned error can't find the container with id d3496904437813a0ccfc0866a0019bed173f3178128685b97097f3a674853da8 Jul 14 20:23:16.923054 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:23:16.923018 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-76dff96bd-59zm4"] Jul 14 20:23:16.926848 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:23:16.926819 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1447f395_a017_44cf_9176_2944ead9c090.slice/crio-9785fa8c855ca6aad47a8d3bb8431aef8762ed79f31c76740ddd2df457d3d73d WatchSource:0}: Error finding container 9785fa8c855ca6aad47a8d3bb8431aef8762ed79f31c76740ddd2df457d3d73d: Status 404 returned error can't find the container with id 9785fa8c855ca6aad47a8d3bb8431aef8762ed79f31c76740ddd2df457d3d73d Jul 14 20:23:17.838202 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:23:17.838145 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-76dff96bd-59zm4" event={"ID":"1447f395-a017-44cf-9176-2944ead9c090","Type":"ContainerStarted","Data":"9785fa8c855ca6aad47a8d3bb8431aef8762ed79f31c76740ddd2df457d3d73d"} Jul 14 20:23:17.840164 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:23:17.840079 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-55c6fbd889-rvg84" event={"ID":"f704a4c0-8d5e-41bf-a173-753ac3b54ff2","Type":"ContainerStarted","Data":"d3496904437813a0ccfc0866a0019bed173f3178128685b97097f3a674853da8"} Jul 14 20:23:19.846785 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:23:19.846750 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-76dff96bd-59zm4" event={"ID":"1447f395-a017-44cf-9176-2944ead9c090","Type":"ContainerStarted","Data":"d5d084c34b592eccdfb9697fed842e90f11417e759d52858e8ecd4285d5e2956"} Jul 14 20:23:19.860929 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:23:19.860772 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-76dff96bd-59zm4" podStartSLOduration=1.181088629 podStartE2EDuration="3.860749351s" totalImagesPullingTime="2.679660722s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 20:23:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 20:23:19.860244201 +0000 UTC m=+58.781407511" watchObservedRunningTime="2026-07-14 20:23:19.860749351 +0000 UTC m=+58.781912661" Jul 14 20:23:21.353214 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:23:21.353177 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:23:21.405934 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:23:21.405901 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:23:22.775228 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:23:22.775201 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-hzbx8" Jul 14 20:23:22.855889 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:23:22.855850 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-55c6fbd889-rvg84" event={"ID":"f704a4c0-8d5e-41bf-a173-753ac3b54ff2","Type":"ContainerStarted","Data":"3f022b8700a9c3f4adfd972ece2fb409ad6640b2d25c5a7212a88e4fb036d00d"} Jul 14 20:23:22.856091 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:23:22.856068 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-55c6fbd889-rvg84" Jul 14 20:23:22.857804 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:23:22.857782 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-55c6fbd889-rvg84" Jul 14 20:23:22.871229 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:23:22.871179 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-55c6fbd889-rvg84" podStartSLOduration=1.675815242 podStartE2EDuration="6.871161555s" totalImagesPullingTime="5.195346313s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 20:23:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 20:23:22.870239299 +0000 UTC m=+61.791402610" watchObservedRunningTime="2026-07-14 20:23:22.871161555 +0000 UTC m=+61.792324863" Jul 14 20:23:22.887428 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:23:22.887373 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:23:22.986166 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:23:22.986120 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:23:27.247409 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:23:27.247368 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/92cbd473-9b2e-4faa-a232-810627648cc9-metrics-certs\") pod \"network-metrics-daemon-rvhpg\" (UID: \"92cbd473-9b2e-4faa-a232-810627648cc9\") " pod="openshift-multus/network-metrics-daemon-rvhpg" Jul 14 20:23:27.249297 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:23:27.249276 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 14 20:23:27.257814 ip-10-0-141-61 kubenswrapper[2563]: E0714 20:23:27.257790 2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jul 14 20:23:27.257880 ip-10-0-141-61 kubenswrapper[2563]: E0714 20:23:27.257860 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/92cbd473-9b2e-4faa-a232-810627648cc9-metrics-certs podName:92cbd473-9b2e-4faa-a232-810627648cc9 nodeName:}" failed. No retries permitted until 2026-07-14 20:24:31.257841781 +0000 UTC m=+130.179005069 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/92cbd473-9b2e-4faa-a232-810627648cc9-metrics-certs") pod "network-metrics-daemon-rvhpg" (UID: "92cbd473-9b2e-4faa-a232-810627648cc9") : secret "metrics-daemon-secret" not found Jul 14 20:23:27.347862 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:23:27.347820 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/05045fa2-6bd3-4ee8-8f2a-e9056c8a316f-original-pull-secret\") pod \"global-pull-secret-syncer-sgvdx\" (UID: \"05045fa2-6bd3-4ee8-8f2a-e9056c8a316f\") " pod="kube-system/global-pull-secret-syncer-sgvdx" Jul 14 20:23:27.347862 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:23:27.347870 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r8lsz\" (UniqueName: \"kubernetes.io/projected/92f9c302-6d80-4679-9c71-1d37255dc852-kube-api-access-r8lsz\") pod \"network-check-target-2r82p\" (UID: \"92f9c302-6d80-4679-9c71-1d37255dc852\") " pod="openshift-network-diagnostics/network-check-target-2r82p" Jul 14 20:23:27.350118 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:23:27.350100 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 14 20:23:27.350195 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:23:27.350108 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 14 20:23:27.360602 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:23:27.360570 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 14 20:23:27.361213 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:23:27.361197 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/05045fa2-6bd3-4ee8-8f2a-e9056c8a316f-original-pull-secret\") pod \"global-pull-secret-syncer-sgvdx\" (UID: \"05045fa2-6bd3-4ee8-8f2a-e9056c8a316f\") " pod="kube-system/global-pull-secret-syncer-sgvdx" Jul 14 20:23:27.371324 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:23:27.371302 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r8lsz\" (UniqueName: \"kubernetes.io/projected/92f9c302-6d80-4679-9c71-1d37255dc852-kube-api-access-r8lsz\") pod \"network-check-target-2r82p\" (UID: \"92f9c302-6d80-4679-9c71-1d37255dc852\") " pod="openshift-network-diagnostics/network-check-target-2r82p" Jul 14 20:23:27.525152 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:23:27.525056 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-8rw2b\"" Jul 14 20:23:27.528449 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:23:27.528433 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-sgvdx" Jul 14 20:23:27.533923 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:23:27.533909 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2r82p" Jul 14 20:23:27.667536 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:23:27.667497 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-sgvdx"] Jul 14 20:23:27.671368 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:23:27.671339 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod05045fa2_6bd3_4ee8_8f2a_e9056c8a316f.slice/crio-dea395959598492281ba9166b86ee46792954b1d7c7ad1c555c90eba17325e82 WatchSource:0}: Error finding container dea395959598492281ba9166b86ee46792954b1d7c7ad1c555c90eba17325e82: Status 404 returned error can't find the container with id dea395959598492281ba9166b86ee46792954b1d7c7ad1c555c90eba17325e82 Jul 14 20:23:27.680240 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:23:27.680216 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-2r82p"] Jul 14 20:23:27.683236 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:23:27.683214 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod92f9c302_6d80_4679_9c71_1d37255dc852.slice/crio-4ec2ab57faf11769c36029625f979c6f79ffa02edbe8f59a4b97e7253193adf2 WatchSource:0}: Error finding container 4ec2ab57faf11769c36029625f979c6f79ffa02edbe8f59a4b97e7253193adf2: Status 404 returned error can't find the container with id 4ec2ab57faf11769c36029625f979c6f79ffa02edbe8f59a4b97e7253193adf2 Jul 14 20:23:27.869166 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:23:27.869039 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-2r82p" event={"ID":"92f9c302-6d80-4679-9c71-1d37255dc852","Type":"ContainerStarted","Data":"4ec2ab57faf11769c36029625f979c6f79ffa02edbe8f59a4b97e7253193adf2"} Jul 14 20:23:27.870105 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:23:27.870081 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-sgvdx" event={"ID":"05045fa2-6bd3-4ee8-8f2a-e9056c8a316f","Type":"ContainerStarted","Data":"dea395959598492281ba9166b86ee46792954b1d7c7ad1c555c90eba17325e82"} Jul 14 20:23:28.355053 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:23:28.355010 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b4e3e206-bf7b-47bf-b037-9c12fca3799b-metrics-tls\") pod \"dns-default-knrl4\" (UID: \"b4e3e206-bf7b-47bf-b037-9c12fca3799b\") " pod="openshift-dns/dns-default-knrl4" Jul 14 20:23:28.355498 ip-10-0-141-61 kubenswrapper[2563]: E0714 20:23:28.355178 2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 14 20:23:28.355498 ip-10-0-141-61 kubenswrapper[2563]: E0714 20:23:28.355250 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b4e3e206-bf7b-47bf-b037-9c12fca3799b-metrics-tls podName:b4e3e206-bf7b-47bf-b037-9c12fca3799b nodeName:}" failed. No retries permitted until 2026-07-14 20:24:00.355231077 +0000 UTC m=+99.276394379 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b4e3e206-bf7b-47bf-b037-9c12fca3799b-metrics-tls") pod "dns-default-knrl4" (UID: "b4e3e206-bf7b-47bf-b037-9c12fca3799b") : secret "dns-default-metrics-tls" not found Jul 14 20:23:29.053893 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:23:29.053857 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:23:32.887932 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:23:32.887833 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-sgvdx" event={"ID":"05045fa2-6bd3-4ee8-8f2a-e9056c8a316f","Type":"ContainerStarted","Data":"07dba18505ad1ee75af613615798460a484b2d3e2f471ef01dcb0aadc8d0581d"} Jul 14 20:23:32.889293 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:23:32.889265 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-2r82p" event={"ID":"92f9c302-6d80-4679-9c71-1d37255dc852","Type":"ContainerStarted","Data":"f695aa2c29736b8aefeddccf58552c9a27ef111d536548c02d85fe206f895873"} Jul 14 20:23:32.889406 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:23:32.889389 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-2r82p" Jul 14 20:23:32.901686 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:23:32.901639 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-sgvdx" podStartSLOduration=67.103305584 podStartE2EDuration="1m11.901621882s" totalImagesPullingTime="4.798316298s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 20:22:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 20:23:32.901038003 +0000 UTC m=+71.822201321" watchObservedRunningTime="2026-07-14 20:23:32.901621882 +0000 UTC m=+71.822785192" Jul 14 20:23:32.918961 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:23:32.918907 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-2r82p" podStartSLOduration=67.136629024 podStartE2EDuration="1m11.918891997s" totalImagesPullingTime="4.782262973s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 20:22:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 20:23:32.918191535 +0000 UTC m=+71.839354846" watchObservedRunningTime="2026-07-14 20:23:32.918891997 +0000 UTC m=+71.840055296" Jul 14 20:23:51.601904 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:23:51.601791 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:23:51.658330 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:23:51.658295 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:24:00.378455 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:24:00.378400 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b4e3e206-bf7b-47bf-b037-9c12fca3799b-metrics-tls\") pod \"dns-default-knrl4\" (UID: \"b4e3e206-bf7b-47bf-b037-9c12fca3799b\") " pod="openshift-dns/dns-default-knrl4" Jul 14 20:24:00.378937 ip-10-0-141-61 kubenswrapper[2563]: E0714 20:24:00.378561 2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 14 20:24:00.378937 ip-10-0-141-61 kubenswrapper[2563]: E0714 20:24:00.378634 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b4e3e206-bf7b-47bf-b037-9c12fca3799b-metrics-tls podName:b4e3e206-bf7b-47bf-b037-9c12fca3799b nodeName:}" failed. No retries permitted until 2026-07-14 20:25:04.378618052 +0000 UTC m=+163.299781339 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b4e3e206-bf7b-47bf-b037-9c12fca3799b-metrics-tls") pod "dns-default-knrl4" (UID: "b4e3e206-bf7b-47bf-b037-9c12fca3799b") : secret "dns-default-metrics-tls" not found Jul 14 20:24:03.896533 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:24:03.896498 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-2r82p" Jul 14 20:24:31.307899 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:24:31.307849 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/92cbd473-9b2e-4faa-a232-810627648cc9-metrics-certs\") pod \"network-metrics-daemon-rvhpg\" (UID: \"92cbd473-9b2e-4faa-a232-810627648cc9\") " pod="openshift-multus/network-metrics-daemon-rvhpg" Jul 14 20:24:31.308415 ip-10-0-141-61 kubenswrapper[2563]: E0714 20:24:31.307969 2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jul 14 20:24:31.308415 ip-10-0-141-61 kubenswrapper[2563]: E0714 20:24:31.308025 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/92cbd473-9b2e-4faa-a232-810627648cc9-metrics-certs podName:92cbd473-9b2e-4faa-a232-810627648cc9 nodeName:}" failed. No retries permitted until 2026-07-14 20:26:33.308009696 +0000 UTC m=+252.229172983 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/92cbd473-9b2e-4faa-a232-810627648cc9-metrics-certs") pod "network-metrics-daemon-rvhpg" (UID: "92cbd473-9b2e-4faa-a232-810627648cc9") : secret "metrics-daemon-secret" not found Jul 14 20:24:59.452538 ip-10-0-141-61 kubenswrapper[2563]: E0714 20:24:59.452488 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-knrl4" podUID="b4e3e206-bf7b-47bf-b037-9c12fca3799b" Jul 14 20:24:59.615971 ip-10-0-141-61 kubenswrapper[2563]: E0714 20:24:59.615928 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-rvhpg" podUID="92cbd473-9b2e-4faa-a232-810627648cc9" Jul 14 20:25:00.146451 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:25:00.146417 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-knrl4" Jul 14 20:25:04.441874 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:25:04.441825 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b4e3e206-bf7b-47bf-b037-9c12fca3799b-metrics-tls\") pod \"dns-default-knrl4\" (UID: \"b4e3e206-bf7b-47bf-b037-9c12fca3799b\") " pod="openshift-dns/dns-default-knrl4" Jul 14 20:25:04.442310 ip-10-0-141-61 kubenswrapper[2563]: E0714 20:25:04.441956 2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 14 20:25:04.442310 ip-10-0-141-61 kubenswrapper[2563]: E0714 20:25:04.442019 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b4e3e206-bf7b-47bf-b037-9c12fca3799b-metrics-tls podName:b4e3e206-bf7b-47bf-b037-9c12fca3799b nodeName:}" failed. No retries permitted until 2026-07-14 20:27:06.442005086 +0000 UTC m=+285.363168373 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b4e3e206-bf7b-47bf-b037-9c12fca3799b-metrics-tls") pod "dns-default-knrl4" (UID: "b4e3e206-bf7b-47bf-b037-9c12fca3799b") : secret "dns-default-metrics-tls" not found Jul 14 20:25:11.601901 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:25:11.601858 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rvhpg" Jul 14 20:25:20.202644 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:25:20.202612 2563 generic.go:356] "Generic (PLEG): container finished" podID="1447f395-a017-44cf-9176-2944ead9c090" containerID="d5d084c34b592eccdfb9697fed842e90f11417e759d52858e8ecd4285d5e2956" exitCode=255 Jul 14 20:25:20.203079 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:25:20.202664 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-76dff96bd-59zm4" event={"ID":"1447f395-a017-44cf-9176-2944ead9c090","Type":"ContainerDied","Data":"d5d084c34b592eccdfb9697fed842e90f11417e759d52858e8ecd4285d5e2956"} Jul 14 20:25:20.208189 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:25:20.208169 2563 scope.go:117] "RemoveContainer" containerID="d5d084c34b592eccdfb9697fed842e90f11417e759d52858e8ecd4285d5e2956" Jul 14 20:25:21.207329 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:25:21.207295 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-76dff96bd-59zm4" event={"ID":"1447f395-a017-44cf-9176-2944ead9c090","Type":"ContainerStarted","Data":"5066e8a3a8e625031e626373303931f07a874b9ac595017817cd95eed4fbc6b8"} Jul 14 20:25:22.857434 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:25:22.857376 2563 prober.go:120] "Probe failed" probeType="Readiness" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-55c6fbd889-rvg84" podUID="f704a4c0-8d5e-41bf-a173-753ac3b54ff2" containerName="acm-agent" probeResult="failure" output="Get \"http://10.134.0.8:8000/readyz\": dial tcp 10.134.0.8:8000: connect: connection refused" Jul 14 20:25:23.214171 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:25:23.214119 2563 generic.go:356] "Generic (PLEG): container finished" podID="f704a4c0-8d5e-41bf-a173-753ac3b54ff2" containerID="3f022b8700a9c3f4adfd972ece2fb409ad6640b2d25c5a7212a88e4fb036d00d" exitCode=1 Jul 14 20:25:23.214331 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:25:23.214190 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-55c6fbd889-rvg84" event={"ID":"f704a4c0-8d5e-41bf-a173-753ac3b54ff2","Type":"ContainerDied","Data":"3f022b8700a9c3f4adfd972ece2fb409ad6640b2d25c5a7212a88e4fb036d00d"} Jul 14 20:25:23.214512 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:25:23.214499 2563 scope.go:117] "RemoveContainer" containerID="3f022b8700a9c3f4adfd972ece2fb409ad6640b2d25c5a7212a88e4fb036d00d" Jul 14 20:25:24.219331 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:25:24.219295 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-55c6fbd889-rvg84" event={"ID":"f704a4c0-8d5e-41bf-a173-753ac3b54ff2","Type":"ContainerStarted","Data":"3516850081a3c01738d37f22d484a7359bfd73c18a1fa00755655a3cf4f43c28"} Jul 14 20:25:24.219743 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:25:24.219575 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-55c6fbd889-rvg84" Jul 14 20:25:24.220194 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:25:24.220178 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-55c6fbd889-rvg84" Jul 14 20:25:24.608911 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:25:24.608832 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:25:45.902251 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:25:45.902206 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:26:12.787643 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:12.787608 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-7695fc6c8c-dk2qr"] Jul 14 20:26:12.789350 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:12.789331 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-dk2qr" Jul 14 20:26:12.791315 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:12.791290 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\"" Jul 14 20:26:12.791431 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:12.791401 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\"" Jul 14 20:26:12.791483 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:12.791464 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\"" Jul 14 20:26:12.791994 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:12.791977 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-f9mdw\"" Jul 14 20:26:12.792066 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:12.792053 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\"" Jul 14 20:26:12.795591 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:12.795568 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-68d9dd94cf-spp52"] Jul 14 20:26:12.797461 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:12.797445 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-68d9dd94cf-spp52" Jul 14 20:26:12.799890 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:12.799869 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 14 20:26:12.799890 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:12.799873 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-4rcm4\"" Jul 14 20:26:12.800845 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:12.800826 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\"" Jul 14 20:26:12.800937 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:12.800829 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\"" Jul 14 20:26:12.805553 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:12.805532 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-7695fc6c8c-dk2qr"] Jul 14 20:26:12.808306 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:12.808286 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 14 20:26:12.808939 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:12.808925 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\"" Jul 14 20:26:12.819089 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:12.819066 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-68d9dd94cf-spp52"] Jul 14 20:26:12.819294 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:12.819198 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cb3ba939-0e7c-4afc-a64b-e073e5cf656c-service-ca-bundle\") pod \"insights-operator-68d9dd94cf-spp52\" (UID: \"cb3ba939-0e7c-4afc-a64b-e073e5cf656c\") " pod="openshift-insights/insights-operator-68d9dd94cf-spp52" Jul 14 20:26:12.819392 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:12.819357 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/cb3ba939-0e7c-4afc-a64b-e073e5cf656c-snapshots\") pod \"insights-operator-68d9dd94cf-spp52\" (UID: \"cb3ba939-0e7c-4afc-a64b-e073e5cf656c\") " pod="openshift-insights/insights-operator-68d9dd94cf-spp52" Jul 14 20:26:12.819392 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:12.819374 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/cb3ba939-0e7c-4afc-a64b-e073e5cf656c-serving-cert\") pod \"insights-operator-68d9dd94cf-spp52\" (UID: \"cb3ba939-0e7c-4afc-a64b-e073e5cf656c\") " pod="openshift-insights/insights-operator-68d9dd94cf-spp52" Jul 14 20:26:12.819483 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:12.819414 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/a00c85f6-2ec7-445d-8882-8c54e7ac223d-config\") pod \"service-ca-operator-7695fc6c8c-dk2qr\" (UID: \"a00c85f6-2ec7-445d-8882-8c54e7ac223d\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-dk2qr" Jul 14 20:26:12.819483 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:12.819433 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/cb3ba939-0e7c-4afc-a64b-e073e5cf656c-tmp\") pod \"insights-operator-68d9dd94cf-spp52\" (UID: \"cb3ba939-0e7c-4afc-a64b-e073e5cf656c\") " pod="openshift-insights/insights-operator-68d9dd94cf-spp52" Jul 14 20:26:12.819483 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:12.819448 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cb3ba939-0e7c-4afc-a64b-e073e5cf656c-trusted-ca-bundle\") pod \"insights-operator-68d9dd94cf-spp52\" (UID: \"cb3ba939-0e7c-4afc-a64b-e073e5cf656c\") " pod="openshift-insights/insights-operator-68d9dd94cf-spp52" Jul 14 20:26:12.819613 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:12.819513 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/a00c85f6-2ec7-445d-8882-8c54e7ac223d-serving-cert\") pod \"service-ca-operator-7695fc6c8c-dk2qr\" (UID: \"a00c85f6-2ec7-445d-8882-8c54e7ac223d\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-dk2qr" Jul 14 20:26:12.819613 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:12.819541 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hxs86\" (UniqueName: \"kubernetes.io/projected/cb3ba939-0e7c-4afc-a64b-e073e5cf656c-kube-api-access-hxs86\") pod \"insights-operator-68d9dd94cf-spp52\" (UID: \"cb3ba939-0e7c-4afc-a64b-e073e5cf656c\") " pod="openshift-insights/insights-operator-68d9dd94cf-spp52" Jul 14 20:26:12.819613 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:12.819577 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mzw95\" (UniqueName: \"kubernetes.io/projected/a00c85f6-2ec7-445d-8882-8c54e7ac223d-kube-api-access-mzw95\") pod \"service-ca-operator-7695fc6c8c-dk2qr\" (UID: \"a00c85f6-2ec7-445d-8882-8c54e7ac223d\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-dk2qr" Jul 14 20:26:12.920547 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:12.920513 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/cb3ba939-0e7c-4afc-a64b-e073e5cf656c-snapshots\") pod \"insights-operator-68d9dd94cf-spp52\" (UID: \"cb3ba939-0e7c-4afc-a64b-e073e5cf656c\") " pod="openshift-insights/insights-operator-68d9dd94cf-spp52" Jul 14 20:26:12.920547 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:12.920551 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/cb3ba939-0e7c-4afc-a64b-e073e5cf656c-serving-cert\") pod \"insights-operator-68d9dd94cf-spp52\" (UID: \"cb3ba939-0e7c-4afc-a64b-e073e5cf656c\") " pod="openshift-insights/insights-operator-68d9dd94cf-spp52" Jul 14 20:26:12.920823 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:12.920587 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/a00c85f6-2ec7-445d-8882-8c54e7ac223d-config\") pod \"service-ca-operator-7695fc6c8c-dk2qr\" (UID: \"a00c85f6-2ec7-445d-8882-8c54e7ac223d\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-dk2qr" Jul 14 20:26:12.920823 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:12.920604 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/cb3ba939-0e7c-4afc-a64b-e073e5cf656c-tmp\") pod \"insights-operator-68d9dd94cf-spp52\" (UID: \"cb3ba939-0e7c-4afc-a64b-e073e5cf656c\") " pod="openshift-insights/insights-operator-68d9dd94cf-spp52" Jul 14 20:26:12.920823 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:12.920725 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cb3ba939-0e7c-4afc-a64b-e073e5cf656c-trusted-ca-bundle\") pod \"insights-operator-68d9dd94cf-spp52\" (UID: \"cb3ba939-0e7c-4afc-a64b-e073e5cf656c\") " pod="openshift-insights/insights-operator-68d9dd94cf-spp52" Jul 14 20:26:12.920823 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:12.920769 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/a00c85f6-2ec7-445d-8882-8c54e7ac223d-serving-cert\") pod \"service-ca-operator-7695fc6c8c-dk2qr\" (UID: \"a00c85f6-2ec7-445d-8882-8c54e7ac223d\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-dk2qr" Jul 14 20:26:12.920823 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:12.920792 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hxs86\" (UniqueName: \"kubernetes.io/projected/cb3ba939-0e7c-4afc-a64b-e073e5cf656c-kube-api-access-hxs86\") pod \"insights-operator-68d9dd94cf-spp52\" (UID: \"cb3ba939-0e7c-4afc-a64b-e073e5cf656c\") " pod="openshift-insights/insights-operator-68d9dd94cf-spp52" Jul 14 20:26:12.921045 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:12.920832 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mzw95\" (UniqueName: \"kubernetes.io/projected/a00c85f6-2ec7-445d-8882-8c54e7ac223d-kube-api-access-mzw95\") pod \"service-ca-operator-7695fc6c8c-dk2qr\" (UID: \"a00c85f6-2ec7-445d-8882-8c54e7ac223d\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-dk2qr" Jul 14 20:26:12.921045 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:12.920871 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cb3ba939-0e7c-4afc-a64b-e073e5cf656c-service-ca-bundle\") pod \"insights-operator-68d9dd94cf-spp52\" (UID: \"cb3ba939-0e7c-4afc-a64b-e073e5cf656c\") " pod="openshift-insights/insights-operator-68d9dd94cf-spp52" Jul 14 20:26:12.921221 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:12.921196 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/a00c85f6-2ec7-445d-8882-8c54e7ac223d-config\") pod \"service-ca-operator-7695fc6c8c-dk2qr\" (UID: \"a00c85f6-2ec7-445d-8882-8c54e7ac223d\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-dk2qr" Jul 14 20:26:12.921297 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:12.921278 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/cb3ba939-0e7c-4afc-a64b-e073e5cf656c-tmp\") pod \"insights-operator-68d9dd94cf-spp52\" (UID: \"cb3ba939-0e7c-4afc-a64b-e073e5cf656c\") " pod="openshift-insights/insights-operator-68d9dd94cf-spp52" Jul 14 20:26:12.921379 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:12.921360 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/cb3ba939-0e7c-4afc-a64b-e073e5cf656c-snapshots\") pod \"insights-operator-68d9dd94cf-spp52\" (UID: \"cb3ba939-0e7c-4afc-a64b-e073e5cf656c\") " pod="openshift-insights/insights-operator-68d9dd94cf-spp52" Jul 14 20:26:12.921771 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:12.921748 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cb3ba939-0e7c-4afc-a64b-e073e5cf656c-service-ca-bundle\") pod \"insights-operator-68d9dd94cf-spp52\" (UID: \"cb3ba939-0e7c-4afc-a64b-e073e5cf656c\") " pod="openshift-insights/insights-operator-68d9dd94cf-spp52" Jul 14 20:26:12.922164 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:12.922119 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cb3ba939-0e7c-4afc-a64b-e073e5cf656c-trusted-ca-bundle\") pod \"insights-operator-68d9dd94cf-spp52\" (UID: \"cb3ba939-0e7c-4afc-a64b-e073e5cf656c\") " pod="openshift-insights/insights-operator-68d9dd94cf-spp52" Jul 14 20:26:12.923356 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:12.923335 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/a00c85f6-2ec7-445d-8882-8c54e7ac223d-serving-cert\") pod \"service-ca-operator-7695fc6c8c-dk2qr\" (UID: \"a00c85f6-2ec7-445d-8882-8c54e7ac223d\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-dk2qr" Jul 14 20:26:12.923439 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:12.923416 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/cb3ba939-0e7c-4afc-a64b-e073e5cf656c-serving-cert\") pod \"insights-operator-68d9dd94cf-spp52\" (UID: \"cb3ba939-0e7c-4afc-a64b-e073e5cf656c\") " pod="openshift-insights/insights-operator-68d9dd94cf-spp52" Jul 14 20:26:12.928359 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:12.928339 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mzw95\" (UniqueName: \"kubernetes.io/projected/a00c85f6-2ec7-445d-8882-8c54e7ac223d-kube-api-access-mzw95\") pod \"service-ca-operator-7695fc6c8c-dk2qr\" (UID: \"a00c85f6-2ec7-445d-8882-8c54e7ac223d\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-dk2qr" Jul 14 20:26:12.931735 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:12.931716 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hxs86\" (UniqueName: \"kubernetes.io/projected/cb3ba939-0e7c-4afc-a64b-e073e5cf656c-kube-api-access-hxs86\") pod \"insights-operator-68d9dd94cf-spp52\" (UID: \"cb3ba939-0e7c-4afc-a64b-e073e5cf656c\") " pod="openshift-insights/insights-operator-68d9dd94cf-spp52" Jul 14 20:26:12.969302 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:12.969268 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:26:13.070313 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:13.070221 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:26:13.098626 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:13.098596 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-dk2qr" Jul 14 20:26:13.105611 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:13.105589 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-68d9dd94cf-spp52" Jul 14 20:26:13.235773 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:13.235743 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-7695fc6c8c-dk2qr"] Jul 14 20:26:13.238949 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:26:13.238924 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda00c85f6_2ec7_445d_8882_8c54e7ac223d.slice/crio-4981de9c6040bed06346b12c6261db32c1d10bfb122e456f81182af4adfc0b30 WatchSource:0}: Error finding container 4981de9c6040bed06346b12c6261db32c1d10bfb122e456f81182af4adfc0b30: Status 404 returned error can't find the container with id 4981de9c6040bed06346b12c6261db32c1d10bfb122e456f81182af4adfc0b30 Jul 14 20:26:13.255808 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:13.255758 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-68d9dd94cf-spp52"] Jul 14 20:26:13.258802 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:26:13.258775 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcb3ba939_0e7c_4afc_a64b_e073e5cf656c.slice/crio-54e0e3fda37237f42383a8d23af2d171d0c0cdbe1aa5c4c63cb485c36177b1e8 WatchSource:0}: Error finding container 54e0e3fda37237f42383a8d23af2d171d0c0cdbe1aa5c4c63cb485c36177b1e8: Status 404 returned error can't find the container with id 54e0e3fda37237f42383a8d23af2d171d0c0cdbe1aa5c4c63cb485c36177b1e8 Jul 14 20:26:13.365193 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:13.365076 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-68d9dd94cf-spp52" event={"ID":"cb3ba939-0e7c-4afc-a64b-e073e5cf656c","Type":"ContainerStarted","Data":"54e0e3fda37237f42383a8d23af2d171d0c0cdbe1aa5c4c63cb485c36177b1e8"} Jul 14 20:26:13.366218 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:13.366192 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-dk2qr" event={"ID":"a00c85f6-2ec7-445d-8882-8c54e7ac223d","Type":"ContainerStarted","Data":"4981de9c6040bed06346b12c6261db32c1d10bfb122e456f81182af4adfc0b30"} Jul 14 20:26:15.804042 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:15.803950 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:26:15.856665 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:15.856600 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:26:16.376442 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:16.376407 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-68d9dd94cf-spp52" event={"ID":"cb3ba939-0e7c-4afc-a64b-e073e5cf656c","Type":"ContainerStarted","Data":"96e471328344c4464837a0c22d7754d19a2c8dd2201fc3d7f84a447ad7a39483"} Jul 14 20:26:16.377894 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:16.377822 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-dk2qr" event={"ID":"a00c85f6-2ec7-445d-8882-8c54e7ac223d","Type":"ContainerStarted","Data":"3c3cc8b980e1d25da2f1e4efed3d6da24981ebc3cbcc83a2b3503f3ec69d96f4"} Jul 14 20:26:16.393547 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:16.393487 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-68d9dd94cf-spp52" podStartSLOduration=1.52509295 podStartE2EDuration="4.393470038s" totalImagesPullingTime="2.868377088s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 20:26:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 20:26:16.392285836 +0000 UTC m=+235.313449145" watchObservedRunningTime="2026-07-14 20:26:16.393470038 +0000 UTC m=+235.314633346" Jul 14 20:26:16.405381 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:16.405315 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-dk2qr" podStartSLOduration=2.150152569 podStartE2EDuration="4.405294993s" totalImagesPullingTime="2.255142424s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 20:26:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 20:26:16.405242979 +0000 UTC m=+235.326406291" watchObservedRunningTime="2026-07-14 20:26:16.405294993 +0000 UTC m=+235.326458304" Jul 14 20:26:17.353273 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:17.353239 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:26:17.360948 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:17.360921 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7f59b54f6c-9mbjv"] Jul 14 20:26:17.362887 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:17.362872 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-9mbjv" Jul 14 20:26:17.364973 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:17.364948 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-6t7nj\"" Jul 14 20:26:17.364973 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:17.364961 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\"" Jul 14 20:26:17.364973 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:17.364970 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\"" Jul 14 20:26:17.372106 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:17.372082 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7f59b54f6c-9mbjv"] Jul 14 20:26:17.446999 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:17.446958 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:26:17.458550 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:17.458512 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-78gvw\" (UniqueName: \"kubernetes.io/projected/43dccdcb-04f9-4844-8574-90a15ec10a6d-kube-api-access-78gvw\") pod \"migrator-7f59b54f6c-9mbjv\" (UID: \"43dccdcb-04f9-4844-8574-90a15ec10a6d\") " pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-9mbjv" Jul 14 20:26:17.547795 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:17.547749 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:26:17.559777 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:17.559739 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-78gvw\" (UniqueName: \"kubernetes.io/projected/43dccdcb-04f9-4844-8574-90a15ec10a6d-kube-api-access-78gvw\") pod \"migrator-7f59b54f6c-9mbjv\" (UID: \"43dccdcb-04f9-4844-8574-90a15ec10a6d\") " pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-9mbjv" Jul 14 20:26:17.567371 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:17.567342 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-78gvw\" (UniqueName: \"kubernetes.io/projected/43dccdcb-04f9-4844-8574-90a15ec10a6d-kube-api-access-78gvw\") pod \"migrator-7f59b54f6c-9mbjv\" (UID: \"43dccdcb-04f9-4844-8574-90a15ec10a6d\") " pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-9mbjv" Jul 14 20:26:17.674159 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:17.673925 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-9mbjv" Jul 14 20:26:17.818522 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:17.818485 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7f59b54f6c-9mbjv"] Jul 14 20:26:17.821568 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:26:17.821538 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod43dccdcb_04f9_4844_8574_90a15ec10a6d.slice/crio-8010e5d23b613159416a16e77284d19a991c452ca245113d244144dafea9d97d WatchSource:0}: Error finding container 8010e5d23b613159416a16e77284d19a991c452ca245113d244144dafea9d97d: Status 404 returned error can't find the container with id 8010e5d23b613159416a16e77284d19a991c452ca245113d244144dafea9d97d Jul 14 20:26:17.932442 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:17.932329 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:26:18.038202 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:18.038158 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:26:18.048590 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:18.048521 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-tj9d5"] Jul 14 20:26:18.052147 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:18.052102 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-tj9d5" Jul 14 20:26:18.054687 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:18.054455 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 14 20:26:18.054687 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:18.054491 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 14 20:26:18.054687 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:18.054455 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-lxqqx\"" Jul 14 20:26:18.070313 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:18.070290 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-tj9d5"] Jul 14 20:26:18.124880 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:18.124844 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:26:18.164816 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:18.164781 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/cd97f132-9f22-4dc9-986f-315e50547b82-crio-socket\") pod \"insights-runtime-extractor-tj9d5\" (UID: \"cd97f132-9f22-4dc9-986f-315e50547b82\") " pod="openshift-insights/insights-runtime-extractor-tj9d5" Jul 14 20:26:18.164816 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:18.164819 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/cd97f132-9f22-4dc9-986f-315e50547b82-data-volume\") pod \"insights-runtime-extractor-tj9d5\" (UID: \"cd97f132-9f22-4dc9-986f-315e50547b82\") " pod="openshift-insights/insights-runtime-extractor-tj9d5" Jul 14 20:26:18.165024 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:18.164881 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/cd97f132-9f22-4dc9-986f-315e50547b82-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-tj9d5\" (UID: \"cd97f132-9f22-4dc9-986f-315e50547b82\") " pod="openshift-insights/insights-runtime-extractor-tj9d5" Jul 14 20:26:18.165024 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:18.164920 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/cd97f132-9f22-4dc9-986f-315e50547b82-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-tj9d5\" (UID: \"cd97f132-9f22-4dc9-986f-315e50547b82\") " pod="openshift-insights/insights-runtime-extractor-tj9d5" Jul 14 20:26:18.165024 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:18.164939 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p2gkh\" (UniqueName: \"kubernetes.io/projected/cd97f132-9f22-4dc9-986f-315e50547b82-kube-api-access-p2gkh\") pod \"insights-runtime-extractor-tj9d5\" (UID: \"cd97f132-9f22-4dc9-986f-315e50547b82\") " pod="openshift-insights/insights-runtime-extractor-tj9d5" Jul 14 20:26:18.266371 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:18.266280 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/cd97f132-9f22-4dc9-986f-315e50547b82-crio-socket\") pod \"insights-runtime-extractor-tj9d5\" (UID: \"cd97f132-9f22-4dc9-986f-315e50547b82\") " pod="openshift-insights/insights-runtime-extractor-tj9d5" Jul 14 20:26:18.266371 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:18.266318 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/cd97f132-9f22-4dc9-986f-315e50547b82-data-volume\") pod \"insights-runtime-extractor-tj9d5\" (UID: \"cd97f132-9f22-4dc9-986f-315e50547b82\") " pod="openshift-insights/insights-runtime-extractor-tj9d5" Jul 14 20:26:18.266371 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:18.266339 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/cd97f132-9f22-4dc9-986f-315e50547b82-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-tj9d5\" (UID: \"cd97f132-9f22-4dc9-986f-315e50547b82\") " pod="openshift-insights/insights-runtime-extractor-tj9d5" Jul 14 20:26:18.266571 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:18.266374 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/cd97f132-9f22-4dc9-986f-315e50547b82-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-tj9d5\" (UID: \"cd97f132-9f22-4dc9-986f-315e50547b82\") " pod="openshift-insights/insights-runtime-extractor-tj9d5" Jul 14 20:26:18.266571 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:18.266403 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p2gkh\" (UniqueName: \"kubernetes.io/projected/cd97f132-9f22-4dc9-986f-315e50547b82-kube-api-access-p2gkh\") pod \"insights-runtime-extractor-tj9d5\" (UID: \"cd97f132-9f22-4dc9-986f-315e50547b82\") " pod="openshift-insights/insights-runtime-extractor-tj9d5" Jul 14 20:26:18.266571 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:18.266414 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/cd97f132-9f22-4dc9-986f-315e50547b82-crio-socket\") pod \"insights-runtime-extractor-tj9d5\" (UID: \"cd97f132-9f22-4dc9-986f-315e50547b82\") " pod="openshift-insights/insights-runtime-extractor-tj9d5" Jul 14 20:26:18.266571 ip-10-0-141-61 kubenswrapper[2563]: E0714 20:26:18.266444 2563 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 14 20:26:18.266571 ip-10-0-141-61 kubenswrapper[2563]: E0714 20:26:18.266469 2563 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 14 20:26:18.266571 ip-10-0-141-61 kubenswrapper[2563]: E0714 20:26:18.266519 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cd97f132-9f22-4dc9-986f-315e50547b82-insights-runtime-extractor-tls podName:cd97f132-9f22-4dc9-986f-315e50547b82 nodeName:}" failed. No retries permitted until 2026-07-14 20:26:18.766498639 +0000 UTC m=+237.687661939 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/cd97f132-9f22-4dc9-986f-315e50547b82-insights-runtime-extractor-tls") pod "insights-runtime-extractor-tj9d5" (UID: "cd97f132-9f22-4dc9-986f-315e50547b82") : secret "insights-runtime-extractor-tls" not found Jul 14 20:26:18.266571 ip-10-0-141-61 kubenswrapper[2563]: E0714 20:26:18.266544 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/cd97f132-9f22-4dc9-986f-315e50547b82-kube-rbac-proxy-cm podName:cd97f132-9f22-4dc9-986f-315e50547b82 nodeName:}" failed. No retries permitted until 2026-07-14 20:26:18.766535406 +0000 UTC m=+237.687698736 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/cd97f132-9f22-4dc9-986f-315e50547b82-kube-rbac-proxy-cm") pod "insights-runtime-extractor-tj9d5" (UID: "cd97f132-9f22-4dc9-986f-315e50547b82") : configmap "kube-rbac-proxy" not found Jul 14 20:26:18.266789 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:18.266684 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/cd97f132-9f22-4dc9-986f-315e50547b82-data-volume\") pod \"insights-runtime-extractor-tj9d5\" (UID: \"cd97f132-9f22-4dc9-986f-315e50547b82\") " pod="openshift-insights/insights-runtime-extractor-tj9d5" Jul 14 20:26:18.274700 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:18.274674 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p2gkh\" (UniqueName: \"kubernetes.io/projected/cd97f132-9f22-4dc9-986f-315e50547b82-kube-api-access-p2gkh\") pod \"insights-runtime-extractor-tj9d5\" (UID: \"cd97f132-9f22-4dc9-986f-315e50547b82\") " pod="openshift-insights/insights-runtime-extractor-tj9d5" Jul 14 20:26:18.384917 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:18.384876 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-9mbjv" event={"ID":"43dccdcb-04f9-4844-8574-90a15ec10a6d","Type":"ContainerStarted","Data":"8010e5d23b613159416a16e77284d19a991c452ca245113d244144dafea9d97d"} Jul 14 20:26:18.655197 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:18.655145 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:26:18.753618 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:18.753573 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:26:18.770482 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:18.770445 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/cd97f132-9f22-4dc9-986f-315e50547b82-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-tj9d5\" (UID: \"cd97f132-9f22-4dc9-986f-315e50547b82\") " pod="openshift-insights/insights-runtime-extractor-tj9d5" Jul 14 20:26:18.770737 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:18.770508 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/cd97f132-9f22-4dc9-986f-315e50547b82-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-tj9d5\" (UID: \"cd97f132-9f22-4dc9-986f-315e50547b82\") " pod="openshift-insights/insights-runtime-extractor-tj9d5" Jul 14 20:26:18.770737 ip-10-0-141-61 kubenswrapper[2563]: E0714 20:26:18.770576 2563 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 14 20:26:18.770737 ip-10-0-141-61 kubenswrapper[2563]: E0714 20:26:18.770631 2563 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 14 20:26:18.770737 ip-10-0-141-61 kubenswrapper[2563]: E0714 20:26:18.770666 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cd97f132-9f22-4dc9-986f-315e50547b82-insights-runtime-extractor-tls podName:cd97f132-9f22-4dc9-986f-315e50547b82 nodeName:}" failed. No retries permitted until 2026-07-14 20:26:19.770641934 +0000 UTC m=+238.691805228 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/cd97f132-9f22-4dc9-986f-315e50547b82-insights-runtime-extractor-tls") pod "insights-runtime-extractor-tj9d5" (UID: "cd97f132-9f22-4dc9-986f-315e50547b82") : secret "insights-runtime-extractor-tls" not found Jul 14 20:26:18.770737 ip-10-0-141-61 kubenswrapper[2563]: E0714 20:26:18.770685 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/cd97f132-9f22-4dc9-986f-315e50547b82-kube-rbac-proxy-cm podName:cd97f132-9f22-4dc9-986f-315e50547b82 nodeName:}" failed. No retries permitted until 2026-07-14 20:26:19.770672633 +0000 UTC m=+238.691835920 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/cd97f132-9f22-4dc9-986f-315e50547b82-kube-rbac-proxy-cm") pod "insights-runtime-extractor-tj9d5" (UID: "cd97f132-9f22-4dc9-986f-315e50547b82") : configmap "kube-rbac-proxy" not found Jul 14 20:26:19.389406 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:19.389374 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-9mbjv" event={"ID":"43dccdcb-04f9-4844-8574-90a15ec10a6d","Type":"ContainerStarted","Data":"8d5dd26dbe3c8aa769cc0cbd524ae1afd05e0440a86f60df0d1f94c3b4c7c61a"} Jul 14 20:26:19.778756 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:19.778719 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/cd97f132-9f22-4dc9-986f-315e50547b82-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-tj9d5\" (UID: \"cd97f132-9f22-4dc9-986f-315e50547b82\") " pod="openshift-insights/insights-runtime-extractor-tj9d5" Jul 14 20:26:19.778962 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:19.778765 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/cd97f132-9f22-4dc9-986f-315e50547b82-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-tj9d5\" (UID: \"cd97f132-9f22-4dc9-986f-315e50547b82\") " pod="openshift-insights/insights-runtime-extractor-tj9d5" Jul 14 20:26:19.778962 ip-10-0-141-61 kubenswrapper[2563]: E0714 20:26:19.778840 2563 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 14 20:26:19.778962 ip-10-0-141-61 kubenswrapper[2563]: E0714 20:26:19.778892 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/cd97f132-9f22-4dc9-986f-315e50547b82-kube-rbac-proxy-cm podName:cd97f132-9f22-4dc9-986f-315e50547b82 nodeName:}" failed. No retries permitted until 2026-07-14 20:26:21.778878828 +0000 UTC m=+240.700042121 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/cd97f132-9f22-4dc9-986f-315e50547b82-kube-rbac-proxy-cm") pod "insights-runtime-extractor-tj9d5" (UID: "cd97f132-9f22-4dc9-986f-315e50547b82") : configmap "kube-rbac-proxy" not found Jul 14 20:26:19.778962 ip-10-0-141-61 kubenswrapper[2563]: E0714 20:26:19.778896 2563 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 14 20:26:19.778962 ip-10-0-141-61 kubenswrapper[2563]: E0714 20:26:19.778957 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cd97f132-9f22-4dc9-986f-315e50547b82-insights-runtime-extractor-tls podName:cd97f132-9f22-4dc9-986f-315e50547b82 nodeName:}" failed. No retries permitted until 2026-07-14 20:26:21.778940785 +0000 UTC m=+240.700104090 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/cd97f132-9f22-4dc9-986f-315e50547b82-insights-runtime-extractor-tls") pod "insights-runtime-extractor-tj9d5" (UID: "cd97f132-9f22-4dc9-986f-315e50547b82") : secret "insights-runtime-extractor-tls" not found Jul 14 20:26:20.240051 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:20.240017 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-79f68b9f4f-hzxn8"] Jul 14 20:26:20.242189 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:20.242172 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-79f68b9f4f-hzxn8" Jul 14 20:26:20.244224 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:20.244203 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\"" Jul 14 20:26:20.244343 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:20.244226 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\"" Jul 14 20:26:20.244693 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:20.244674 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-wnbvs\"" Jul 14 20:26:20.244796 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:20.244746 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\"" Jul 14 20:26:20.244796 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:20.244745 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\"" Jul 14 20:26:20.252193 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:20.252170 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-79f68b9f4f-hzxn8"] Jul 14 20:26:20.275369 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:20.275327 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:26:20.282400 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:20.282370 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/893660e4-d72a-44f7-a56e-f80d3f5863f2-signing-key\") pod \"service-ca-79f68b9f4f-hzxn8\" (UID: \"893660e4-d72a-44f7-a56e-f80d3f5863f2\") " pod="openshift-service-ca/service-ca-79f68b9f4f-hzxn8" Jul 14 20:26:20.282497 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:20.282414 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-27xjz\" (UniqueName: \"kubernetes.io/projected/893660e4-d72a-44f7-a56e-f80d3f5863f2-kube-api-access-27xjz\") pod \"service-ca-79f68b9f4f-hzxn8\" (UID: \"893660e4-d72a-44f7-a56e-f80d3f5863f2\") " pod="openshift-service-ca/service-ca-79f68b9f4f-hzxn8" Jul 14 20:26:20.282497 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:20.282484 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/893660e4-d72a-44f7-a56e-f80d3f5863f2-signing-cabundle\") pod \"service-ca-79f68b9f4f-hzxn8\" (UID: \"893660e4-d72a-44f7-a56e-f80d3f5863f2\") " pod="openshift-service-ca/service-ca-79f68b9f4f-hzxn8" Jul 14 20:26:20.332152 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:20.332079 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:26:20.382858 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:20.382816 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-27xjz\" (UniqueName: \"kubernetes.io/projected/893660e4-d72a-44f7-a56e-f80d3f5863f2-kube-api-access-27xjz\") pod \"service-ca-79f68b9f4f-hzxn8\" (UID: \"893660e4-d72a-44f7-a56e-f80d3f5863f2\") " pod="openshift-service-ca/service-ca-79f68b9f4f-hzxn8" Jul 14 20:26:20.382858 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:20.382863 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/893660e4-d72a-44f7-a56e-f80d3f5863f2-signing-cabundle\") pod \"service-ca-79f68b9f4f-hzxn8\" (UID: \"893660e4-d72a-44f7-a56e-f80d3f5863f2\") " pod="openshift-service-ca/service-ca-79f68b9f4f-hzxn8" Jul 14 20:26:20.383114 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:20.382937 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/893660e4-d72a-44f7-a56e-f80d3f5863f2-signing-key\") pod \"service-ca-79f68b9f4f-hzxn8\" (UID: \"893660e4-d72a-44f7-a56e-f80d3f5863f2\") " pod="openshift-service-ca/service-ca-79f68b9f4f-hzxn8" Jul 14 20:26:20.383585 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:20.383562 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/893660e4-d72a-44f7-a56e-f80d3f5863f2-signing-cabundle\") pod \"service-ca-79f68b9f4f-hzxn8\" (UID: \"893660e4-d72a-44f7-a56e-f80d3f5863f2\") " pod="openshift-service-ca/service-ca-79f68b9f4f-hzxn8" Jul 14 20:26:20.385470 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:20.385449 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/893660e4-d72a-44f7-a56e-f80d3f5863f2-signing-key\") pod \"service-ca-79f68b9f4f-hzxn8\" (UID: \"893660e4-d72a-44f7-a56e-f80d3f5863f2\") " pod="openshift-service-ca/service-ca-79f68b9f4f-hzxn8" Jul 14 20:26:20.391146 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:20.391105 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-27xjz\" (UniqueName: \"kubernetes.io/projected/893660e4-d72a-44f7-a56e-f80d3f5863f2-kube-api-access-27xjz\") pod \"service-ca-79f68b9f4f-hzxn8\" (UID: \"893660e4-d72a-44f7-a56e-f80d3f5863f2\") " pod="openshift-service-ca/service-ca-79f68b9f4f-hzxn8" Jul 14 20:26:20.394726 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:20.394693 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-9mbjv" event={"ID":"43dccdcb-04f9-4844-8574-90a15ec10a6d","Type":"ContainerStarted","Data":"1af625ce03161850aa4fc7b7979a16b18ed704ef9a6cf055de9e0b7b94b3d498"} Jul 14 20:26:20.409505 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:20.409447 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-9mbjv" podStartSLOduration=1.930122222 podStartE2EDuration="3.409425054s" totalImagesPullingTime="1.479302832s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 20:26:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 20:26:20.408849724 +0000 UTC m=+239.330013032" watchObservedRunningTime="2026-07-14 20:26:20.409425054 +0000 UTC m=+239.330588365" Jul 14 20:26:20.500034 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:20.499950 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:26:20.550956 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:20.550909 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-79f68b9f4f-hzxn8" Jul 14 20:26:20.683874 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:20.683842 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-79f68b9f4f-hzxn8"] Jul 14 20:26:20.686874 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:26:20.686847 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod893660e4_d72a_44f7_a56e_f80d3f5863f2.slice/crio-c0a72b273d4d889a5532a8954e9514dbae9bc0cb99132a5dc21ace8a6a966453 WatchSource:0}: Error finding container c0a72b273d4d889a5532a8954e9514dbae9bc0cb99132a5dc21ace8a6a966453: Status 404 returned error can't find the container with id c0a72b273d4d889a5532a8954e9514dbae9bc0cb99132a5dc21ace8a6a966453 Jul 14 20:26:20.851310 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:20.851275 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-dznvt_9d9cd7af-00ce-4f0a-8205-faf1e29a50e7/dns-node-resolver/0.log" Jul 14 20:26:21.389757 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:21.389671 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:26:21.403100 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:21.403060 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-79f68b9f4f-hzxn8" event={"ID":"893660e4-d72a-44f7-a56e-f80d3f5863f2","Type":"ContainerStarted","Data":"bc77f0747e365c6d70eefd0359a24229b3ae80350792c29424d80de27bd04aa1"} Jul 14 20:26:21.403765 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:21.403745 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-79f68b9f4f-hzxn8" event={"ID":"893660e4-d72a-44f7-a56e-f80d3f5863f2","Type":"ContainerStarted","Data":"c0a72b273d4d889a5532a8954e9514dbae9bc0cb99132a5dc21ace8a6a966453"} Jul 14 20:26:21.424077 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:21.424014 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-79f68b9f4f-hzxn8" podStartSLOduration=1.423988536 podStartE2EDuration="1.423988536s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 20:26:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 20:26:21.422491688 +0000 UTC m=+240.343654998" watchObservedRunningTime="2026-07-14 20:26:21.423988536 +0000 UTC m=+240.345151847" Jul 14 20:26:21.470768 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:21.470729 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:26:21.557260 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:21.557224 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:26:21.652395 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:21.652315 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-88985_afbb09bb-badd-4dc0-8082-581c48dda65b/node-ca/0.log" Jul 14 20:26:21.761384 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:21.761351 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:26:21.795567 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:21.795529 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/cd97f132-9f22-4dc9-986f-315e50547b82-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-tj9d5\" (UID: \"cd97f132-9f22-4dc9-986f-315e50547b82\") " pod="openshift-insights/insights-runtime-extractor-tj9d5" Jul 14 20:26:21.795567 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:21.795579 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/cd97f132-9f22-4dc9-986f-315e50547b82-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-tj9d5\" (UID: \"cd97f132-9f22-4dc9-986f-315e50547b82\") " pod="openshift-insights/insights-runtime-extractor-tj9d5" Jul 14 20:26:21.795822 ip-10-0-141-61 kubenswrapper[2563]: E0714 20:26:21.795679 2563 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 14 20:26:21.795822 ip-10-0-141-61 kubenswrapper[2563]: E0714 20:26:21.795740 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/cd97f132-9f22-4dc9-986f-315e50547b82-kube-rbac-proxy-cm podName:cd97f132-9f22-4dc9-986f-315e50547b82 nodeName:}" failed. No retries permitted until 2026-07-14 20:26:25.795720517 +0000 UTC m=+244.716883817 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/cd97f132-9f22-4dc9-986f-315e50547b82-kube-rbac-proxy-cm") pod "insights-runtime-extractor-tj9d5" (UID: "cd97f132-9f22-4dc9-986f-315e50547b82") : configmap "kube-rbac-proxy" not found Jul 14 20:26:21.796161 ip-10-0-141-61 kubenswrapper[2563]: E0714 20:26:21.796047 2563 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 14 20:26:21.796161 ip-10-0-141-61 kubenswrapper[2563]: E0714 20:26:21.796113 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cd97f132-9f22-4dc9-986f-315e50547b82-insights-runtime-extractor-tls podName:cd97f132-9f22-4dc9-986f-315e50547b82 nodeName:}" failed. No retries permitted until 2026-07-14 20:26:25.796095408 +0000 UTC m=+244.717258709 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/cd97f132-9f22-4dc9-986f-315e50547b82-insights-runtime-extractor-tls") pod "insights-runtime-extractor-tj9d5" (UID: "cd97f132-9f22-4dc9-986f-315e50547b82") : secret "insights-runtime-extractor-tls" not found Jul 14 20:26:21.863578 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:21.863537 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:26:21.964374 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:21.964340 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:26:22.146312 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:22.146259 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:26:22.851060 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:22.851015 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-v2v5k_cf457900-0b78-45d7-9e9e-99d9fe24bd1c/serve-healthcheck-canary/0.log" Jul 14 20:26:22.918326 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:22.918290 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:26:23.052470 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:23.052436 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-7f59b54f6c-9mbjv_43dccdcb-04f9-4844-8574-90a15ec10a6d/migrator/0.log" Jul 14 20:26:23.256283 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:23.256253 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-7f59b54f6c-9mbjv_43dccdcb-04f9-4844-8574-90a15ec10a6d/graceful-termination/0.log" Jul 14 20:26:23.325795 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:23.325758 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:26:23.747428 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:23.747390 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:26:23.935836 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:23.935798 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:26:24.329742 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:24.329704 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:26:24.899509 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:24.899468 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:26:24.967018 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:24.966972 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:26:25.154163 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:25.154050 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:26:25.324431 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:25.324394 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:26:25.532591 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:25.532559 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:26:25.725261 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:25.725215 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:26:25.829578 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:25.829492 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/cd97f132-9f22-4dc9-986f-315e50547b82-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-tj9d5\" (UID: \"cd97f132-9f22-4dc9-986f-315e50547b82\") " pod="openshift-insights/insights-runtime-extractor-tj9d5" Jul 14 20:26:25.829578 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:25.829533 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/cd97f132-9f22-4dc9-986f-315e50547b82-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-tj9d5\" (UID: \"cd97f132-9f22-4dc9-986f-315e50547b82\") " pod="openshift-insights/insights-runtime-extractor-tj9d5" Jul 14 20:26:25.829777 ip-10-0-141-61 kubenswrapper[2563]: E0714 20:26:25.829657 2563 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 14 20:26:25.829777 ip-10-0-141-61 kubenswrapper[2563]: E0714 20:26:25.829721 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cd97f132-9f22-4dc9-986f-315e50547b82-insights-runtime-extractor-tls podName:cd97f132-9f22-4dc9-986f-315e50547b82 nodeName:}" failed. No retries permitted until 2026-07-14 20:26:33.829704533 +0000 UTC m=+252.750867820 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/cd97f132-9f22-4dc9-986f-315e50547b82-insights-runtime-extractor-tls") pod "insights-runtime-extractor-tj9d5" (UID: "cd97f132-9f22-4dc9-986f-315e50547b82") : secret "insights-runtime-extractor-tls" not found Jul 14 20:26:25.829777 ip-10-0-141-61 kubenswrapper[2563]: E0714 20:26:25.829657 2563 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 14 20:26:25.829899 ip-10-0-141-61 kubenswrapper[2563]: E0714 20:26:25.829781 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/cd97f132-9f22-4dc9-986f-315e50547b82-kube-rbac-proxy-cm podName:cd97f132-9f22-4dc9-986f-315e50547b82 nodeName:}" failed. No retries permitted until 2026-07-14 20:26:33.829767898 +0000 UTC m=+252.750931202 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/cd97f132-9f22-4dc9-986f-315e50547b82-kube-rbac-proxy-cm") pod "insights-runtime-extractor-tj9d5" (UID: "cd97f132-9f22-4dc9-986f-315e50547b82") : configmap "kube-rbac-proxy" not found Jul 14 20:26:25.938011 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:25.937971 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:26:26.327248 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:26.327209 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:26:26.922038 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:26.922007 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:26:27.324245 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:27.324165 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:26:27.735173 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:27.735143 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:26:27.930177 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:27.930118 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:26:27.994461 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:27.994373 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:26:28.513973 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:28.513939 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:26:28.932572 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:28.932535 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:26:29.323550 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:29.323469 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:26:29.777264 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:29.777229 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:26:29.920148 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:29.920090 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:26:30.315112 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:30.315077 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:26:30.572804 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:30.572721 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:26:31.116297 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:31.116261 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:26:31.317621 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:31.317583 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:26:31.623864 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:31.623815 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:26:31.723298 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:31.723265 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:26:31.799717 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:31.799679 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:26:31.923183 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:31.923146 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:26:32.128517 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:32.128481 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:26:32.440075 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:32.440037 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:26:32.540743 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:32.540708 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:26:33.392194 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:33.392155 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/92cbd473-9b2e-4faa-a232-810627648cc9-metrics-certs\") pod \"network-metrics-daemon-rvhpg\" (UID: \"92cbd473-9b2e-4faa-a232-810627648cc9\") " pod="openshift-multus/network-metrics-daemon-rvhpg" Jul 14 20:26:33.394582 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:33.394554 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/92cbd473-9b2e-4faa-a232-810627648cc9-metrics-certs\") pod \"network-metrics-daemon-rvhpg\" (UID: \"92cbd473-9b2e-4faa-a232-810627648cc9\") " pod="openshift-multus/network-metrics-daemon-rvhpg" Jul 14 20:26:33.505473 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:33.505443 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-5hghx\"" Jul 14 20:26:33.514062 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:33.514033 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rvhpg" Jul 14 20:26:33.646505 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:33.646427 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-rvhpg"] Jul 14 20:26:33.649532 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:26:33.649508 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod92cbd473_9b2e_4faa_a232_810627648cc9.slice/crio-628d2a6db1bef93011b92c1eb0c45d50ab65d006ef12b52b10b50c3338f83b27 WatchSource:0}: Error finding container 628d2a6db1bef93011b92c1eb0c45d50ab65d006ef12b52b10b50c3338f83b27: Status 404 returned error can't find the container with id 628d2a6db1bef93011b92c1eb0c45d50ab65d006ef12b52b10b50c3338f83b27 Jul 14 20:26:33.896105 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:33.896070 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/cd97f132-9f22-4dc9-986f-315e50547b82-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-tj9d5\" (UID: \"cd97f132-9f22-4dc9-986f-315e50547b82\") " pod="openshift-insights/insights-runtime-extractor-tj9d5" Jul 14 20:26:33.896105 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:33.896108 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/cd97f132-9f22-4dc9-986f-315e50547b82-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-tj9d5\" (UID: \"cd97f132-9f22-4dc9-986f-315e50547b82\") " pod="openshift-insights/insights-runtime-extractor-tj9d5" Jul 14 20:26:33.896646 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:33.896596 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/cd97f132-9f22-4dc9-986f-315e50547b82-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-tj9d5\" (UID: \"cd97f132-9f22-4dc9-986f-315e50547b82\") " pod="openshift-insights/insights-runtime-extractor-tj9d5" Jul 14 20:26:33.898620 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:33.898595 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/cd97f132-9f22-4dc9-986f-315e50547b82-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-tj9d5\" (UID: \"cd97f132-9f22-4dc9-986f-315e50547b82\") " pod="openshift-insights/insights-runtime-extractor-tj9d5" Jul 14 20:26:33.964297 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:33.964262 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-lxqqx\"" Jul 14 20:26:33.972909 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:33.972891 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-tj9d5" Jul 14 20:26:34.115517 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:34.115485 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-tj9d5"] Jul 14 20:26:34.119720 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:26:34.119691 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcd97f132_9f22_4dc9_986f_315e50547b82.slice/crio-d8044b798b005621ee1258395b188d10c7a9366fa52b3eba792f5818dbf32525 WatchSource:0}: Error finding container d8044b798b005621ee1258395b188d10c7a9366fa52b3eba792f5818dbf32525: Status 404 returned error can't find the container with id d8044b798b005621ee1258395b188d10c7a9366fa52b3eba792f5818dbf32525 Jul 14 20:26:34.442790 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:34.442746 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-rvhpg" event={"ID":"92cbd473-9b2e-4faa-a232-810627648cc9","Type":"ContainerStarted","Data":"628d2a6db1bef93011b92c1eb0c45d50ab65d006ef12b52b10b50c3338f83b27"} Jul 14 20:26:34.443985 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:34.443951 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-tj9d5" event={"ID":"cd97f132-9f22-4dc9-986f-315e50547b82","Type":"ContainerStarted","Data":"d8044b798b005621ee1258395b188d10c7a9366fa52b3eba792f5818dbf32525"} Jul 14 20:26:35.449894 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:35.449857 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-rvhpg" event={"ID":"92cbd473-9b2e-4faa-a232-810627648cc9","Type":"ContainerStarted","Data":"350a54c768a2c50093e57f443aaddbf1531bb61ec48b8ae03cb1d99232d97853"} Jul 14 20:26:35.450372 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:35.449949 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-rvhpg" event={"ID":"92cbd473-9b2e-4faa-a232-810627648cc9","Type":"ContainerStarted","Data":"c9b8aa3803a2bae68b05788650ced01f3aff4acd9615c7499c31443a952e85e8"} Jul 14 20:26:35.466444 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:35.466384 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-rvhpg" podStartSLOduration=253.597595896 podStartE2EDuration="4m14.466363567s" totalImagesPullingTime="868.767671ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 20:22:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 20:26:35.465362429 +0000 UTC m=+254.386525739" watchObservedRunningTime="2026-07-14 20:26:35.466363567 +0000 UTC m=+254.387526877" Jul 14 20:26:35.975824 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:35.975781 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:26:36.076273 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:36.076237 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:26:36.176935 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:36.176885 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:26:36.277745 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:36.277657 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:26:36.640678 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:36.640580 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:26:36.699924 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:36.699890 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:26:37.074957 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:37.074922 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:26:37.296861 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:37.296832 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:26:37.458440 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:37.458398 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:26:37.590289 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:37.590256 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:26:37.691467 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:37.691423 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:26:37.749795 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:37.749705 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:26:37.900241 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:37.900149 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:26:38.012908 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:38.012880 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:26:38.461783 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:38.461751 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-tj9d5" event={"ID":"cd97f132-9f22-4dc9-986f-315e50547b82","Type":"ContainerStarted","Data":"ea2c5cd5d3cbb1e3938814f28b19738741077f3830d2a034e83be16cce23cc49"} Jul 14 20:26:38.574400 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:38.574369 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:26:38.631730 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:38.631680 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:26:39.466978 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:39.466943 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-tj9d5" event={"ID":"cd97f132-9f22-4dc9-986f-315e50547b82","Type":"ContainerStarted","Data":"b068d0096ec8465420e576cb999b7c41051a991bed56284a579fe065f03f1bdc"} Jul 14 20:26:41.475563 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:41.475523 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-tj9d5" event={"ID":"cd97f132-9f22-4dc9-986f-315e50547b82","Type":"ContainerStarted","Data":"6a6e60b297478725dd49ba74c524b90bdc33e8d34f5d97053d8f0ee5a448eb1d"} Jul 14 20:26:41.493002 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:41.492949 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-tj9d5" podStartSLOduration=16.482243004 podStartE2EDuration="23.492932544s" totalImagesPullingTime="7.01068954s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 20:26:18 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 20:26:41.491575017 +0000 UTC m=+260.412738325" watchObservedRunningTime="2026-07-14 20:26:41.492932544 +0000 UTC m=+260.414095854" Jul 14 20:26:44.691913 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:44.691881 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-x5h7q"] Jul 14 20:26:44.695012 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:44.694992 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-x5h7q" Jul 14 20:26:44.699010 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:44.698988 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\"" Jul 14 20:26:44.699393 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:44.699379 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-8v8mk\"" Jul 14 20:26:44.707715 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:44.707691 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-x5h7q"] Jul 14 20:26:44.784011 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:44.783980 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/f1c2ae6a-43b9-45fb-bb72-9a9a8197613d-tls-certificates\") pod \"prometheus-operator-admission-webhook-77d98bc786-x5h7q\" (UID: \"f1c2ae6a-43b9-45fb-bb72-9a9a8197613d\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-x5h7q" Jul 14 20:26:44.836921 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:44.836887 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:26:44.884403 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:44.884368 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/f1c2ae6a-43b9-45fb-bb72-9a9a8197613d-tls-certificates\") pod \"prometheus-operator-admission-webhook-77d98bc786-x5h7q\" (UID: \"f1c2ae6a-43b9-45fb-bb72-9a9a8197613d\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-x5h7q" Jul 14 20:26:44.887045 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:44.887014 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/f1c2ae6a-43b9-45fb-bb72-9a9a8197613d-tls-certificates\") pod \"prometheus-operator-admission-webhook-77d98bc786-x5h7q\" (UID: \"f1c2ae6a-43b9-45fb-bb72-9a9a8197613d\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-x5h7q" Jul 14 20:26:44.933644 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:44.933611 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:26:45.004227 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:45.004113 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-x5h7q" Jul 14 20:26:45.137369 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:45.137339 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-x5h7q"] Jul 14 20:26:45.140628 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:26:45.140603 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf1c2ae6a_43b9_45fb_bb72_9a9a8197613d.slice/crio-0d2c40f923863e510dc4d808fa284ef5b2d99823d0ac4894a484134694dbc928 WatchSource:0}: Error finding container 0d2c40f923863e510dc4d808fa284ef5b2d99823d0ac4894a484134694dbc928: Status 404 returned error can't find the container with id 0d2c40f923863e510dc4d808fa284ef5b2d99823d0ac4894a484134694dbc928 Jul 14 20:26:45.491252 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:45.491218 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-x5h7q" event={"ID":"f1c2ae6a-43b9-45fb-bb72-9a9a8197613d","Type":"ContainerStarted","Data":"0d2c40f923863e510dc4d808fa284ef5b2d99823d0ac4894a484134694dbc928"} Jul 14 20:26:47.499267 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:47.499230 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-x5h7q" event={"ID":"f1c2ae6a-43b9-45fb-bb72-9a9a8197613d","Type":"ContainerStarted","Data":"0fd91432e0c883ad57c6670e917e132acdfd14ee47464f70ba198fc9614ba2f9"} Jul 14 20:26:47.499667 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:47.499438 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-x5h7q" Jul 14 20:26:47.504188 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:47.504170 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-x5h7q" Jul 14 20:26:47.513546 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:47.513500 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-x5h7q" podStartSLOduration=2.121346132 podStartE2EDuration="3.513486288s" totalImagesPullingTime="1.392140156s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 20:26:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 20:26:47.512488558 +0000 UTC m=+266.433651871" watchObservedRunningTime="2026-07-14 20:26:47.513486288 +0000 UTC m=+266.434649597" Jul 14 20:26:47.793334 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:47.793251 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:26:47.856599 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:47.856565 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:26:47.970002 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:47.969959 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:26:48.046797 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:48.046707 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:26:48.292008 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:48.291972 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:26:50.800029 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:50.799999 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:26:52.089343 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:52.089304 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:26:52.954980 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:52.954950 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:26:53.190736 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:53.190688 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:26:53.233789 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:53.233708 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-84bd68b896-gkps8"] Jul 14 20:26:53.237245 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:53.237215 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-gkps8" Jul 14 20:26:53.239429 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:53.239406 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 14 20:26:53.239753 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:53.239731 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 14 20:26:53.240150 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:53.240114 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 14 20:26:53.240260 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:53.240178 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-flc4b\"" Jul 14 20:26:53.240260 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:53.240199 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\"" Jul 14 20:26:53.240260 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:53.240207 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\"" Jul 14 20:26:53.270298 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:53.270264 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-84bd68b896-gkps8"] Jul 14 20:26:53.271435 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:53.271408 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-szdqq"] Jul 14 20:26:53.275005 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:53.274984 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-szdqq" Jul 14 20:26:53.277804 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:53.277782 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 14 20:26:53.278526 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:53.278178 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 14 20:26:53.278741 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:53.278255 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-62ffc\"" Jul 14 20:26:53.278963 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:53.278305 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 14 20:26:53.302187 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:53.302161 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:26:53.343401 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:53.343358 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/5527cca4-4cdb-441c-9e12-0dfee7304f67-openshift-state-metrics-tls\") pod \"openshift-state-metrics-84bd68b896-gkps8\" (UID: \"5527cca4-4cdb-441c-9e12-0dfee7304f67\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-gkps8" Jul 14 20:26:53.343401 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:53.343406 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j2pzs\" (UniqueName: \"kubernetes.io/projected/5527cca4-4cdb-441c-9e12-0dfee7304f67-kube-api-access-j2pzs\") pod \"openshift-state-metrics-84bd68b896-gkps8\" (UID: \"5527cca4-4cdb-441c-9e12-0dfee7304f67\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-gkps8" Jul 14 20:26:53.343654 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:53.343434 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/5527cca4-4cdb-441c-9e12-0dfee7304f67-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-84bd68b896-gkps8\" (UID: \"5527cca4-4cdb-441c-9e12-0dfee7304f67\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-gkps8" Jul 14 20:26:53.343654 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:53.343525 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5527cca4-4cdb-441c-9e12-0dfee7304f67-metrics-client-ca\") pod \"openshift-state-metrics-84bd68b896-gkps8\" (UID: \"5527cca4-4cdb-441c-9e12-0dfee7304f67\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-gkps8" Jul 14 20:26:53.430648 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:53.430612 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:26:53.444743 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:53.444711 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/de1173b5-2d51-4f32-969c-40b0431bd7a0-node-exporter-tls\") pod \"node-exporter-szdqq\" (UID: \"de1173b5-2d51-4f32-969c-40b0431bd7a0\") " pod="openshift-monitoring/node-exporter-szdqq" Jul 14 20:26:53.444913 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:53.444759 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/5527cca4-4cdb-441c-9e12-0dfee7304f67-openshift-state-metrics-tls\") pod \"openshift-state-metrics-84bd68b896-gkps8\" (UID: \"5527cca4-4cdb-441c-9e12-0dfee7304f67\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-gkps8" Jul 14 20:26:53.444913 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:53.444786 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/de1173b5-2d51-4f32-969c-40b0431bd7a0-node-exporter-wtmp\") pod \"node-exporter-szdqq\" (UID: \"de1173b5-2d51-4f32-969c-40b0431bd7a0\") " pod="openshift-monitoring/node-exporter-szdqq" Jul 14 20:26:53.444913 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:53.444824 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j2pzs\" (UniqueName: \"kubernetes.io/projected/5527cca4-4cdb-441c-9e12-0dfee7304f67-kube-api-access-j2pzs\") pod \"openshift-state-metrics-84bd68b896-gkps8\" (UID: \"5527cca4-4cdb-441c-9e12-0dfee7304f67\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-gkps8" Jul 14 20:26:53.444913 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:53.444867 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/de1173b5-2d51-4f32-969c-40b0431bd7a0-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-szdqq\" (UID: \"de1173b5-2d51-4f32-969c-40b0431bd7a0\") " pod="openshift-monitoring/node-exporter-szdqq" Jul 14 20:26:53.445100 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:53.444921 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/de1173b5-2d51-4f32-969c-40b0431bd7a0-root\") pod \"node-exporter-szdqq\" (UID: \"de1173b5-2d51-4f32-969c-40b0431bd7a0\") " pod="openshift-monitoring/node-exporter-szdqq" Jul 14 20:26:53.445100 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:53.444947 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/de1173b5-2d51-4f32-969c-40b0431bd7a0-node-exporter-accelerators-collector-config\") pod \"node-exporter-szdqq\" (UID: \"de1173b5-2d51-4f32-969c-40b0431bd7a0\") " pod="openshift-monitoring/node-exporter-szdqq" Jul 14 20:26:53.445100 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:53.444992 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/5527cca4-4cdb-441c-9e12-0dfee7304f67-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-84bd68b896-gkps8\" (UID: \"5527cca4-4cdb-441c-9e12-0dfee7304f67\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-gkps8" Jul 14 20:26:53.445100 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:53.445017 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/de1173b5-2d51-4f32-969c-40b0431bd7a0-sys\") pod \"node-exporter-szdqq\" (UID: \"de1173b5-2d51-4f32-969c-40b0431bd7a0\") " pod="openshift-monitoring/node-exporter-szdqq" Jul 14 20:26:53.445100 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:53.445041 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5527cca4-4cdb-441c-9e12-0dfee7304f67-metrics-client-ca\") pod \"openshift-state-metrics-84bd68b896-gkps8\" (UID: \"5527cca4-4cdb-441c-9e12-0dfee7304f67\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-gkps8" Jul 14 20:26:53.445100 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:53.445080 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/de1173b5-2d51-4f32-969c-40b0431bd7a0-node-exporter-textfile\") pod \"node-exporter-szdqq\" (UID: \"de1173b5-2d51-4f32-969c-40b0431bd7a0\") " pod="openshift-monitoring/node-exporter-szdqq" Jul 14 20:26:53.445437 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:53.445144 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/de1173b5-2d51-4f32-969c-40b0431bd7a0-metrics-client-ca\") pod \"node-exporter-szdqq\" (UID: \"de1173b5-2d51-4f32-969c-40b0431bd7a0\") " pod="openshift-monitoring/node-exporter-szdqq" Jul 14 20:26:53.445437 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:53.445174 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cq2x6\" (UniqueName: \"kubernetes.io/projected/de1173b5-2d51-4f32-969c-40b0431bd7a0-kube-api-access-cq2x6\") pod \"node-exporter-szdqq\" (UID: \"de1173b5-2d51-4f32-969c-40b0431bd7a0\") " pod="openshift-monitoring/node-exporter-szdqq" Jul 14 20:26:53.445768 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:53.445747 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5527cca4-4cdb-441c-9e12-0dfee7304f67-metrics-client-ca\") pod \"openshift-state-metrics-84bd68b896-gkps8\" (UID: \"5527cca4-4cdb-441c-9e12-0dfee7304f67\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-gkps8" Jul 14 20:26:53.447445 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:53.447425 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/5527cca4-4cdb-441c-9e12-0dfee7304f67-openshift-state-metrics-tls\") pod \"openshift-state-metrics-84bd68b896-gkps8\" (UID: \"5527cca4-4cdb-441c-9e12-0dfee7304f67\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-gkps8" Jul 14 20:26:53.447542 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:53.447425 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/5527cca4-4cdb-441c-9e12-0dfee7304f67-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-84bd68b896-gkps8\" (UID: \"5527cca4-4cdb-441c-9e12-0dfee7304f67\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-gkps8" Jul 14 20:26:53.462928 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:53.462897 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j2pzs\" (UniqueName: \"kubernetes.io/projected/5527cca4-4cdb-441c-9e12-0dfee7304f67-kube-api-access-j2pzs\") pod \"openshift-state-metrics-84bd68b896-gkps8\" (UID: \"5527cca4-4cdb-441c-9e12-0dfee7304f67\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-gkps8" Jul 14 20:26:53.525240 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:53.525151 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:26:53.545916 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:53.545883 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/de1173b5-2d51-4f32-969c-40b0431bd7a0-metrics-client-ca\") pod \"node-exporter-szdqq\" (UID: \"de1173b5-2d51-4f32-969c-40b0431bd7a0\") " pod="openshift-monitoring/node-exporter-szdqq" Jul 14 20:26:53.545916 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:53.545918 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cq2x6\" (UniqueName: \"kubernetes.io/projected/de1173b5-2d51-4f32-969c-40b0431bd7a0-kube-api-access-cq2x6\") pod \"node-exporter-szdqq\" (UID: \"de1173b5-2d51-4f32-969c-40b0431bd7a0\") " pod="openshift-monitoring/node-exporter-szdqq" Jul 14 20:26:53.546157 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:53.545935 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/de1173b5-2d51-4f32-969c-40b0431bd7a0-node-exporter-tls\") pod \"node-exporter-szdqq\" (UID: \"de1173b5-2d51-4f32-969c-40b0431bd7a0\") " pod="openshift-monitoring/node-exporter-szdqq" Jul 14 20:26:53.546157 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:53.545959 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/de1173b5-2d51-4f32-969c-40b0431bd7a0-node-exporter-wtmp\") pod \"node-exporter-szdqq\" (UID: \"de1173b5-2d51-4f32-969c-40b0431bd7a0\") " pod="openshift-monitoring/node-exporter-szdqq" Jul 14 20:26:53.546157 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:53.546106 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/de1173b5-2d51-4f32-969c-40b0431bd7a0-node-exporter-wtmp\") pod \"node-exporter-szdqq\" (UID: \"de1173b5-2d51-4f32-969c-40b0431bd7a0\") " pod="openshift-monitoring/node-exporter-szdqq" Jul 14 20:26:53.546157 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:53.546106 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/de1173b5-2d51-4f32-969c-40b0431bd7a0-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-szdqq\" (UID: \"de1173b5-2d51-4f32-969c-40b0431bd7a0\") " pod="openshift-monitoring/node-exporter-szdqq" Jul 14 20:26:53.546366 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:53.546181 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/de1173b5-2d51-4f32-969c-40b0431bd7a0-root\") pod \"node-exporter-szdqq\" (UID: \"de1173b5-2d51-4f32-969c-40b0431bd7a0\") " pod="openshift-monitoring/node-exporter-szdqq" Jul 14 20:26:53.546366 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:53.546211 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/de1173b5-2d51-4f32-969c-40b0431bd7a0-node-exporter-accelerators-collector-config\") pod \"node-exporter-szdqq\" (UID: \"de1173b5-2d51-4f32-969c-40b0431bd7a0\") " pod="openshift-monitoring/node-exporter-szdqq" Jul 14 20:26:53.546366 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:53.546258 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/de1173b5-2d51-4f32-969c-40b0431bd7a0-sys\") pod \"node-exporter-szdqq\" (UID: \"de1173b5-2d51-4f32-969c-40b0431bd7a0\") " pod="openshift-monitoring/node-exporter-szdqq" Jul 14 20:26:53.546366 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:53.546297 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/de1173b5-2d51-4f32-969c-40b0431bd7a0-root\") pod \"node-exporter-szdqq\" (UID: \"de1173b5-2d51-4f32-969c-40b0431bd7a0\") " pod="openshift-monitoring/node-exporter-szdqq" Jul 14 20:26:53.546366 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:53.546305 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/de1173b5-2d51-4f32-969c-40b0431bd7a0-node-exporter-textfile\") pod \"node-exporter-szdqq\" (UID: \"de1173b5-2d51-4f32-969c-40b0431bd7a0\") " pod="openshift-monitoring/node-exporter-szdqq" Jul 14 20:26:53.546366 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:53.546351 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/de1173b5-2d51-4f32-969c-40b0431bd7a0-sys\") pod \"node-exporter-szdqq\" (UID: \"de1173b5-2d51-4f32-969c-40b0431bd7a0\") " pod="openshift-monitoring/node-exporter-szdqq" Jul 14 20:26:53.546632 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:53.546549 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/de1173b5-2d51-4f32-969c-40b0431bd7a0-node-exporter-textfile\") pod \"node-exporter-szdqq\" (UID: \"de1173b5-2d51-4f32-969c-40b0431bd7a0\") " pod="openshift-monitoring/node-exporter-szdqq" Jul 14 20:26:53.546776 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:53.546682 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/de1173b5-2d51-4f32-969c-40b0431bd7a0-metrics-client-ca\") pod \"node-exporter-szdqq\" (UID: \"de1173b5-2d51-4f32-969c-40b0431bd7a0\") " pod="openshift-monitoring/node-exporter-szdqq" Jul 14 20:26:53.546776 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:53.546751 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/de1173b5-2d51-4f32-969c-40b0431bd7a0-node-exporter-accelerators-collector-config\") pod \"node-exporter-szdqq\" (UID: \"de1173b5-2d51-4f32-969c-40b0431bd7a0\") " pod="openshift-monitoring/node-exporter-szdqq" Jul 14 20:26:53.548654 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:53.548630 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/de1173b5-2d51-4f32-969c-40b0431bd7a0-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-szdqq\" (UID: \"de1173b5-2d51-4f32-969c-40b0431bd7a0\") " pod="openshift-monitoring/node-exporter-szdqq" Jul 14 20:26:53.548654 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:53.548649 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/de1173b5-2d51-4f32-969c-40b0431bd7a0-node-exporter-tls\") pod \"node-exporter-szdqq\" (UID: \"de1173b5-2d51-4f32-969c-40b0431bd7a0\") " pod="openshift-monitoring/node-exporter-szdqq" Jul 14 20:26:53.548819 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:53.548722 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-gkps8" Jul 14 20:26:53.558738 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:53.558712 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cq2x6\" (UniqueName: \"kubernetes.io/projected/de1173b5-2d51-4f32-969c-40b0431bd7a0-kube-api-access-cq2x6\") pod \"node-exporter-szdqq\" (UID: \"de1173b5-2d51-4f32-969c-40b0431bd7a0\") " pod="openshift-monitoring/node-exporter-szdqq" Jul 14 20:26:53.593947 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:53.593913 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-szdqq" Jul 14 20:26:53.648841 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:53.648794 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:26:53.692200 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:53.692038 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-84bd68b896-gkps8"] Jul 14 20:26:53.694691 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:26:53.694664 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5527cca4_4cdb_441c_9e12_0dfee7304f67.slice/crio-21d12fa10159693b733992a9ffd7321389345ff4f220f33d556e2ed62761a853 WatchSource:0}: Error finding container 21d12fa10159693b733992a9ffd7321389345ff4f220f33d556e2ed62761a853: Status 404 returned error can't find the container with id 21d12fa10159693b733992a9ffd7321389345ff4f220f33d556e2ed62761a853 Jul 14 20:26:53.744917 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:53.744875 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:26:54.211572 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:54.211535 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:26:54.305648 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:54.305611 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:26:54.312410 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:54.312373 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 14 20:26:54.317840 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:54.317800 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 14 20:26:54.320051 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:54.320024 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jul 14 20:26:54.320209 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:54.320052 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jul 14 20:26:54.320362 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:54.320343 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jul 14 20:26:54.320660 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:54.320627 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jul 14 20:26:54.320768 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:54.320722 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jul 14 20:26:54.320858 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:54.320841 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jul 14 20:26:54.321040 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:54.321021 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jul 14 20:26:54.321330 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:54.321290 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jul 14 20:26:54.321581 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:54.321566 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-kslhj\"" Jul 14 20:26:54.322666 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:54.322644 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jul 14 20:26:54.333173 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:54.333114 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 14 20:26:54.427778 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:54.427729 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:26:54.455366 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:54.455334 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/3bd29b0f-00fb-4a66-97b1-c2727db1081a-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"3bd29b0f-00fb-4a66-97b1-c2727db1081a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 20:26:54.455489 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:54.455382 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/3bd29b0f-00fb-4a66-97b1-c2727db1081a-web-config\") pod \"alertmanager-main-0\" (UID: \"3bd29b0f-00fb-4a66-97b1-c2727db1081a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 20:26:54.455489 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:54.455448 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/3bd29b0f-00fb-4a66-97b1-c2727db1081a-tls-assets\") pod \"alertmanager-main-0\" (UID: \"3bd29b0f-00fb-4a66-97b1-c2727db1081a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 20:26:54.455489 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:54.455483 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/3bd29b0f-00fb-4a66-97b1-c2727db1081a-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"3bd29b0f-00fb-4a66-97b1-c2727db1081a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 20:26:54.455607 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:54.455508 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/3bd29b0f-00fb-4a66-97b1-c2727db1081a-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"3bd29b0f-00fb-4a66-97b1-c2727db1081a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 20:26:54.455607 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:54.455563 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/3bd29b0f-00fb-4a66-97b1-c2727db1081a-config-volume\") pod \"alertmanager-main-0\" (UID: \"3bd29b0f-00fb-4a66-97b1-c2727db1081a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 20:26:54.455684 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:54.455608 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3bd29b0f-00fb-4a66-97b1-c2727db1081a-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"3bd29b0f-00fb-4a66-97b1-c2727db1081a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 20:26:54.455684 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:54.455653 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/3bd29b0f-00fb-4a66-97b1-c2727db1081a-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"3bd29b0f-00fb-4a66-97b1-c2727db1081a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 20:26:54.455684 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:54.455669 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/3bd29b0f-00fb-4a66-97b1-c2727db1081a-config-out\") pod \"alertmanager-main-0\" (UID: \"3bd29b0f-00fb-4a66-97b1-c2727db1081a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 20:26:54.455788 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:54.455683 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/3bd29b0f-00fb-4a66-97b1-c2727db1081a-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"3bd29b0f-00fb-4a66-97b1-c2727db1081a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 20:26:54.455788 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:54.455736 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/3bd29b0f-00fb-4a66-97b1-c2727db1081a-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"3bd29b0f-00fb-4a66-97b1-c2727db1081a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 20:26:54.455855 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:54.455787 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3bd29b0f-00fb-4a66-97b1-c2727db1081a-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"3bd29b0f-00fb-4a66-97b1-c2727db1081a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 20:26:54.455855 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:54.455824 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5lhc5\" (UniqueName: \"kubernetes.io/projected/3bd29b0f-00fb-4a66-97b1-c2727db1081a-kube-api-access-5lhc5\") pod \"alertmanager-main-0\" (UID: \"3bd29b0f-00fb-4a66-97b1-c2727db1081a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 20:26:54.524351 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:54.524309 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-szdqq" event={"ID":"de1173b5-2d51-4f32-969c-40b0431bd7a0","Type":"ContainerStarted","Data":"80f68cd5ea6a09c4436b7da6d10ef02998dfbb77fb3478c124dc2f0c898ae063"} Jul 14 20:26:54.524543 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:54.524468 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-szdqq" event={"ID":"de1173b5-2d51-4f32-969c-40b0431bd7a0","Type":"ContainerStarted","Data":"4ecb0b0ec4db67e9119fad0941fab089ccd09143bca5459426d033c3795865cf"} Jul 14 20:26:54.526653 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:54.526619 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-gkps8" event={"ID":"5527cca4-4cdb-441c-9e12-0dfee7304f67","Type":"ContainerStarted","Data":"b8fa984577bf5dc469b42cd21446266f453dbb496a39ddd7e289f481c1389567"} Jul 14 20:26:54.526776 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:54.526668 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-gkps8" event={"ID":"5527cca4-4cdb-441c-9e12-0dfee7304f67","Type":"ContainerStarted","Data":"6eb2bd2a840df5935889342b85f6fcc91e28b46650fe8963adb4507f07797d3e"} Jul 14 20:26:54.526776 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:54.526687 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-gkps8" event={"ID":"5527cca4-4cdb-441c-9e12-0dfee7304f67","Type":"ContainerStarted","Data":"21d12fa10159693b733992a9ffd7321389345ff4f220f33d556e2ed62761a853"} Jul 14 20:26:54.556878 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:54.556842 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/3bd29b0f-00fb-4a66-97b1-c2727db1081a-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"3bd29b0f-00fb-4a66-97b1-c2727db1081a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 20:26:54.556878 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:54.556887 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/3bd29b0f-00fb-4a66-97b1-c2727db1081a-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"3bd29b0f-00fb-4a66-97b1-c2727db1081a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 20:26:54.557143 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:54.556906 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3bd29b0f-00fb-4a66-97b1-c2727db1081a-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"3bd29b0f-00fb-4a66-97b1-c2727db1081a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 20:26:54.557143 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:54.556927 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5lhc5\" (UniqueName: \"kubernetes.io/projected/3bd29b0f-00fb-4a66-97b1-c2727db1081a-kube-api-access-5lhc5\") pod \"alertmanager-main-0\" (UID: \"3bd29b0f-00fb-4a66-97b1-c2727db1081a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 20:26:54.557143 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:54.556971 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/3bd29b0f-00fb-4a66-97b1-c2727db1081a-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"3bd29b0f-00fb-4a66-97b1-c2727db1081a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 20:26:54.557143 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:54.556998 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/3bd29b0f-00fb-4a66-97b1-c2727db1081a-web-config\") pod \"alertmanager-main-0\" (UID: \"3bd29b0f-00fb-4a66-97b1-c2727db1081a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 20:26:54.557143 ip-10-0-141-61 kubenswrapper[2563]: E0714 20:26:54.557033 2563 secret.go:189] Couldn't get secret openshift-monitoring/alertmanager-main-tls: secret "alertmanager-main-tls" not found Jul 14 20:26:54.557143 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:54.557066 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/3bd29b0f-00fb-4a66-97b1-c2727db1081a-tls-assets\") pod \"alertmanager-main-0\" (UID: \"3bd29b0f-00fb-4a66-97b1-c2727db1081a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 20:26:54.557143 ip-10-0-141-61 kubenswrapper[2563]: E0714 20:26:54.557110 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3bd29b0f-00fb-4a66-97b1-c2727db1081a-secret-alertmanager-main-tls podName:3bd29b0f-00fb-4a66-97b1-c2727db1081a nodeName:}" failed. No retries permitted until 2026-07-14 20:26:55.057088402 +0000 UTC m=+273.978251692 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "secret-alertmanager-main-tls" (UniqueName: "kubernetes.io/secret/3bd29b0f-00fb-4a66-97b1-c2727db1081a-secret-alertmanager-main-tls") pod "alertmanager-main-0" (UID: "3bd29b0f-00fb-4a66-97b1-c2727db1081a") : secret "alertmanager-main-tls" not found Jul 14 20:26:54.557482 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:54.557155 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/3bd29b0f-00fb-4a66-97b1-c2727db1081a-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"3bd29b0f-00fb-4a66-97b1-c2727db1081a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 20:26:54.557482 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:54.557191 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/3bd29b0f-00fb-4a66-97b1-c2727db1081a-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"3bd29b0f-00fb-4a66-97b1-c2727db1081a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 20:26:54.557482 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:54.557222 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/3bd29b0f-00fb-4a66-97b1-c2727db1081a-config-volume\") pod \"alertmanager-main-0\" (UID: \"3bd29b0f-00fb-4a66-97b1-c2727db1081a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 20:26:54.557482 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:54.557255 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3bd29b0f-00fb-4a66-97b1-c2727db1081a-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"3bd29b0f-00fb-4a66-97b1-c2727db1081a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 20:26:54.557482 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:54.557282 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/3bd29b0f-00fb-4a66-97b1-c2727db1081a-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"3bd29b0f-00fb-4a66-97b1-c2727db1081a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 20:26:54.557482 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:54.557310 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/3bd29b0f-00fb-4a66-97b1-c2727db1081a-config-out\") pod \"alertmanager-main-0\" (UID: \"3bd29b0f-00fb-4a66-97b1-c2727db1081a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 20:26:54.557781 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:54.557597 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/3bd29b0f-00fb-4a66-97b1-c2727db1081a-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"3bd29b0f-00fb-4a66-97b1-c2727db1081a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 20:26:54.557932 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:54.557835 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3bd29b0f-00fb-4a66-97b1-c2727db1081a-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"3bd29b0f-00fb-4a66-97b1-c2727db1081a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 20:26:54.558543 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:54.558486 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3bd29b0f-00fb-4a66-97b1-c2727db1081a-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"3bd29b0f-00fb-4a66-97b1-c2727db1081a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 20:26:54.561193 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:54.561163 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/3bd29b0f-00fb-4a66-97b1-c2727db1081a-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"3bd29b0f-00fb-4a66-97b1-c2727db1081a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 20:26:54.561309 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:54.561288 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/3bd29b0f-00fb-4a66-97b1-c2727db1081a-web-config\") pod \"alertmanager-main-0\" (UID: \"3bd29b0f-00fb-4a66-97b1-c2727db1081a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 20:26:54.561381 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:54.561309 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/3bd29b0f-00fb-4a66-97b1-c2727db1081a-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"3bd29b0f-00fb-4a66-97b1-c2727db1081a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 20:26:54.561586 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:54.561536 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/3bd29b0f-00fb-4a66-97b1-c2727db1081a-config-out\") pod \"alertmanager-main-0\" (UID: \"3bd29b0f-00fb-4a66-97b1-c2727db1081a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 20:26:54.561695 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:54.561633 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/3bd29b0f-00fb-4a66-97b1-c2727db1081a-config-volume\") pod \"alertmanager-main-0\" (UID: \"3bd29b0f-00fb-4a66-97b1-c2727db1081a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 20:26:54.561846 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:54.561803 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/3bd29b0f-00fb-4a66-97b1-c2727db1081a-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"3bd29b0f-00fb-4a66-97b1-c2727db1081a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 20:26:54.561942 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:54.561923 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/3bd29b0f-00fb-4a66-97b1-c2727db1081a-tls-assets\") pod \"alertmanager-main-0\" (UID: \"3bd29b0f-00fb-4a66-97b1-c2727db1081a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 20:26:54.563871 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:54.563827 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/3bd29b0f-00fb-4a66-97b1-c2727db1081a-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"3bd29b0f-00fb-4a66-97b1-c2727db1081a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 20:26:54.567856 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:54.567832 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5lhc5\" (UniqueName: \"kubernetes.io/projected/3bd29b0f-00fb-4a66-97b1-c2727db1081a-kube-api-access-5lhc5\") pod \"alertmanager-main-0\" (UID: \"3bd29b0f-00fb-4a66-97b1-c2727db1081a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 20:26:54.919061 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:54.919024 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:26:55.062599 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:55.062549 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/3bd29b0f-00fb-4a66-97b1-c2727db1081a-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"3bd29b0f-00fb-4a66-97b1-c2727db1081a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 20:26:55.065643 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:55.065610 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/3bd29b0f-00fb-4a66-97b1-c2727db1081a-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"3bd29b0f-00fb-4a66-97b1-c2727db1081a\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 20:26:55.229837 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:55.229727 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 14 20:26:55.437768 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:55.437739 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 14 20:26:55.440430 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:26:55.440395 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3bd29b0f_00fb_4a66_97b1_c2727db1081a.slice/crio-d39d8be4eeaf5af406d68038d3375a731af6b28e59bc607dd6f3fe96fd14c18f WatchSource:0}: Error finding container d39d8be4eeaf5af406d68038d3375a731af6b28e59bc607dd6f3fe96fd14c18f: Status 404 returned error can't find the container with id d39d8be4eeaf5af406d68038d3375a731af6b28e59bc607dd6f3fe96fd14c18f Jul 14 20:26:55.531999 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:55.531911 2563 generic.go:356] "Generic (PLEG): container finished" podID="de1173b5-2d51-4f32-969c-40b0431bd7a0" containerID="80f68cd5ea6a09c4436b7da6d10ef02998dfbb77fb3478c124dc2f0c898ae063" exitCode=0 Jul 14 20:26:55.531999 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:55.531978 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-szdqq" event={"ID":"de1173b5-2d51-4f32-969c-40b0431bd7a0","Type":"ContainerDied","Data":"80f68cd5ea6a09c4436b7da6d10ef02998dfbb77fb3478c124dc2f0c898ae063"} Jul 14 20:26:55.533216 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:55.533192 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"3bd29b0f-00fb-4a66-97b1-c2727db1081a","Type":"ContainerStarted","Data":"d39d8be4eeaf5af406d68038d3375a731af6b28e59bc607dd6f3fe96fd14c18f"} Jul 14 20:26:55.535223 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:55.535196 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-gkps8" event={"ID":"5527cca4-4cdb-441c-9e12-0dfee7304f67","Type":"ContainerStarted","Data":"daafd4020d40b5b67e63d3e3a68d3ebc5eb0a08d4b10bf25c273c2024c6d1f3d"} Jul 14 20:26:55.569707 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:55.569657 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-gkps8" podStartSLOduration=1.045998134 podStartE2EDuration="2.5696417s" totalImagesPullingTime="1.523643566s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 20:26:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 20:26:55.568745944 +0000 UTC m=+274.489909255" watchObservedRunningTime="2026-07-14 20:26:55.5696417 +0000 UTC m=+274.490805008" Jul 14 20:26:56.129591 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:56.129553 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:26:56.217033 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:56.216999 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:26:56.344781 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:56.344744 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:26:56.410028 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:56.409951 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:26:56.541594 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:56.541560 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-szdqq" event={"ID":"de1173b5-2d51-4f32-969c-40b0431bd7a0","Type":"ContainerStarted","Data":"6a8de5492569965c318eb30f37bf880d9a9f3d35eeffcf777acdefc5ff65404d"} Jul 14 20:26:56.541753 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:56.541618 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-szdqq" event={"ID":"de1173b5-2d51-4f32-969c-40b0431bd7a0","Type":"ContainerStarted","Data":"3974c3fa9a72a945d0f48cbddde8505e91d3c7a3b26cee13a45eac36e9f40c43"} Jul 14 20:26:56.561748 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:56.561702 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-szdqq" podStartSLOduration=2.681439513 podStartE2EDuration="3.561685643s" totalImagesPullingTime="803.90212ms" totalInitContainerRuntime="76.34401ms" isStatefulPod=true podCreationTimestamp="2026-07-14 20:26:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 20:26:56.561054574 +0000 UTC m=+275.482217874" watchObservedRunningTime="2026-07-14 20:26:56.561685643 +0000 UTC m=+275.482848992" Jul 14 20:26:56.899538 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:56.899450 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:26:56.972331 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:56.972298 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:26:57.546518 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:57.546429 2563 generic.go:356] "Generic (PLEG): container finished" podID="3bd29b0f-00fb-4a66-97b1-c2727db1081a" containerID="a08f5ebe06e3cef45846ab4fa123374cc48127e25dc61b5836b6d0a5e0c4c121" exitCode=0 Jul 14 20:26:57.546518 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:57.546506 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"3bd29b0f-00fb-4a66-97b1-c2727db1081a","Type":"ContainerDied","Data":"a08f5ebe06e3cef45846ab4fa123374cc48127e25dc61b5836b6d0a5e0c4c121"} Jul 14 20:26:57.656001 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:57.655965 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-7cd4fc7897-d9pdf"] Jul 14 20:26:57.659307 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:57.659290 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-7cd4fc7897-d9pdf" Jul 14 20:26:57.661492 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:57.661463 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\"" Jul 14 20:26:57.661642 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:57.661546 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\"" Jul 14 20:26:57.661694 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:57.661671 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-zs442\"" Jul 14 20:26:57.661841 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:57.661813 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-1cqsgodmu5akk\"" Jul 14 20:26:57.661885 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:57.661841 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jul 14 20:26:57.661916 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:57.661850 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\"" Jul 14 20:26:57.670361 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:57.670338 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-7cd4fc7897-d9pdf"] Jul 14 20:26:57.787055 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:57.787017 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/896f8e23-c9f6-4fea-8eec-762519e4fe62-secret-metrics-server-tls\") pod \"metrics-server-7cd4fc7897-d9pdf\" (UID: \"896f8e23-c9f6-4fea-8eec-762519e4fe62\") " pod="openshift-monitoring/metrics-server-7cd4fc7897-d9pdf" Jul 14 20:26:57.787261 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:57.787069 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cgpdm\" (UniqueName: \"kubernetes.io/projected/896f8e23-c9f6-4fea-8eec-762519e4fe62-kube-api-access-cgpdm\") pod \"metrics-server-7cd4fc7897-d9pdf\" (UID: \"896f8e23-c9f6-4fea-8eec-762519e4fe62\") " pod="openshift-monitoring/metrics-server-7cd4fc7897-d9pdf" Jul 14 20:26:57.787261 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:57.787152 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/896f8e23-c9f6-4fea-8eec-762519e4fe62-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-7cd4fc7897-d9pdf\" (UID: \"896f8e23-c9f6-4fea-8eec-762519e4fe62\") " pod="openshift-monitoring/metrics-server-7cd4fc7897-d9pdf" Jul 14 20:26:57.787261 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:57.787187 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/896f8e23-c9f6-4fea-8eec-762519e4fe62-secret-metrics-server-client-certs\") pod \"metrics-server-7cd4fc7897-d9pdf\" (UID: \"896f8e23-c9f6-4fea-8eec-762519e4fe62\") " pod="openshift-monitoring/metrics-server-7cd4fc7897-d9pdf" Jul 14 20:26:57.787261 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:57.787226 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/896f8e23-c9f6-4fea-8eec-762519e4fe62-audit-log\") pod \"metrics-server-7cd4fc7897-d9pdf\" (UID: \"896f8e23-c9f6-4fea-8eec-762519e4fe62\") " pod="openshift-monitoring/metrics-server-7cd4fc7897-d9pdf" Jul 14 20:26:57.787400 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:57.787276 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/896f8e23-c9f6-4fea-8eec-762519e4fe62-client-ca-bundle\") pod \"metrics-server-7cd4fc7897-d9pdf\" (UID: \"896f8e23-c9f6-4fea-8eec-762519e4fe62\") " pod="openshift-monitoring/metrics-server-7cd4fc7897-d9pdf" Jul 14 20:26:57.787400 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:57.787320 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/896f8e23-c9f6-4fea-8eec-762519e4fe62-metrics-server-audit-profiles\") pod \"metrics-server-7cd4fc7897-d9pdf\" (UID: \"896f8e23-c9f6-4fea-8eec-762519e4fe62\") " pod="openshift-monitoring/metrics-server-7cd4fc7897-d9pdf" Jul 14 20:26:57.887962 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:57.887867 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/896f8e23-c9f6-4fea-8eec-762519e4fe62-secret-metrics-server-tls\") pod \"metrics-server-7cd4fc7897-d9pdf\" (UID: \"896f8e23-c9f6-4fea-8eec-762519e4fe62\") " pod="openshift-monitoring/metrics-server-7cd4fc7897-d9pdf" Jul 14 20:26:57.887962 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:57.887934 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cgpdm\" (UniqueName: \"kubernetes.io/projected/896f8e23-c9f6-4fea-8eec-762519e4fe62-kube-api-access-cgpdm\") pod \"metrics-server-7cd4fc7897-d9pdf\" (UID: \"896f8e23-c9f6-4fea-8eec-762519e4fe62\") " pod="openshift-monitoring/metrics-server-7cd4fc7897-d9pdf" Jul 14 20:26:57.888116 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:57.887976 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/896f8e23-c9f6-4fea-8eec-762519e4fe62-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-7cd4fc7897-d9pdf\" (UID: \"896f8e23-c9f6-4fea-8eec-762519e4fe62\") " pod="openshift-monitoring/metrics-server-7cd4fc7897-d9pdf" Jul 14 20:26:57.888116 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:57.888003 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/896f8e23-c9f6-4fea-8eec-762519e4fe62-secret-metrics-server-client-certs\") pod \"metrics-server-7cd4fc7897-d9pdf\" (UID: \"896f8e23-c9f6-4fea-8eec-762519e4fe62\") " pod="openshift-monitoring/metrics-server-7cd4fc7897-d9pdf" Jul 14 20:26:57.888116 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:57.888057 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/896f8e23-c9f6-4fea-8eec-762519e4fe62-audit-log\") pod \"metrics-server-7cd4fc7897-d9pdf\" (UID: \"896f8e23-c9f6-4fea-8eec-762519e4fe62\") " pod="openshift-monitoring/metrics-server-7cd4fc7897-d9pdf" Jul 14 20:26:57.888116 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:57.888084 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/896f8e23-c9f6-4fea-8eec-762519e4fe62-client-ca-bundle\") pod \"metrics-server-7cd4fc7897-d9pdf\" (UID: \"896f8e23-c9f6-4fea-8eec-762519e4fe62\") " pod="openshift-monitoring/metrics-server-7cd4fc7897-d9pdf" Jul 14 20:26:57.888116 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:57.888109 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/896f8e23-c9f6-4fea-8eec-762519e4fe62-metrics-server-audit-profiles\") pod \"metrics-server-7cd4fc7897-d9pdf\" (UID: \"896f8e23-c9f6-4fea-8eec-762519e4fe62\") " pod="openshift-monitoring/metrics-server-7cd4fc7897-d9pdf" Jul 14 20:26:57.888566 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:57.888541 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/896f8e23-c9f6-4fea-8eec-762519e4fe62-audit-log\") pod \"metrics-server-7cd4fc7897-d9pdf\" (UID: \"896f8e23-c9f6-4fea-8eec-762519e4fe62\") " pod="openshift-monitoring/metrics-server-7cd4fc7897-d9pdf" Jul 14 20:26:57.889048 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:57.889022 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/896f8e23-c9f6-4fea-8eec-762519e4fe62-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-7cd4fc7897-d9pdf\" (UID: \"896f8e23-c9f6-4fea-8eec-762519e4fe62\") " pod="openshift-monitoring/metrics-server-7cd4fc7897-d9pdf" Jul 14 20:26:57.889424 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:57.889392 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/896f8e23-c9f6-4fea-8eec-762519e4fe62-metrics-server-audit-profiles\") pod \"metrics-server-7cd4fc7897-d9pdf\" (UID: \"896f8e23-c9f6-4fea-8eec-762519e4fe62\") " pod="openshift-monitoring/metrics-server-7cd4fc7897-d9pdf" Jul 14 20:26:57.890791 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:57.890769 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/896f8e23-c9f6-4fea-8eec-762519e4fe62-secret-metrics-server-tls\") pod \"metrics-server-7cd4fc7897-d9pdf\" (UID: \"896f8e23-c9f6-4fea-8eec-762519e4fe62\") " pod="openshift-monitoring/metrics-server-7cd4fc7897-d9pdf" Jul 14 20:26:57.890878 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:57.890831 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/896f8e23-c9f6-4fea-8eec-762519e4fe62-client-ca-bundle\") pod \"metrics-server-7cd4fc7897-d9pdf\" (UID: \"896f8e23-c9f6-4fea-8eec-762519e4fe62\") " pod="openshift-monitoring/metrics-server-7cd4fc7897-d9pdf" Jul 14 20:26:57.890989 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:57.890974 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/896f8e23-c9f6-4fea-8eec-762519e4fe62-secret-metrics-server-client-certs\") pod \"metrics-server-7cd4fc7897-d9pdf\" (UID: \"896f8e23-c9f6-4fea-8eec-762519e4fe62\") " pod="openshift-monitoring/metrics-server-7cd4fc7897-d9pdf" Jul 14 20:26:57.895678 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:57.895656 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cgpdm\" (UniqueName: \"kubernetes.io/projected/896f8e23-c9f6-4fea-8eec-762519e4fe62-kube-api-access-cgpdm\") pod \"metrics-server-7cd4fc7897-d9pdf\" (UID: \"896f8e23-c9f6-4fea-8eec-762519e4fe62\") " pod="openshift-monitoring/metrics-server-7cd4fc7897-d9pdf" Jul 14 20:26:57.953013 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:57.952976 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:26:57.967976 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:57.967941 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-7cd4fc7897-d9pdf" Jul 14 20:26:58.102625 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:58.102300 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-7cd4fc7897-d9pdf"] Jul 14 20:26:58.104880 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:26:58.104853 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod896f8e23_c9f6_4fea_8eec_762519e4fe62.slice/crio-90080086728e76cf01230996375f705bb1c0f3dc397110e9f1985a81d912e9cd WatchSource:0}: Error finding container 90080086728e76cf01230996375f705bb1c0f3dc397110e9f1985a81d912e9cd: Status 404 returned error can't find the container with id 90080086728e76cf01230996375f705bb1c0f3dc397110e9f1985a81d912e9cd Jul 14 20:26:58.131968 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:58.131921 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:26:58.470966 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:58.470930 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-f598c6b45-bq5rt"] Jul 14 20:26:58.474713 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:58.474685 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-f598c6b45-bq5rt" Jul 14 20:26:58.478029 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:58.477844 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\"" Jul 14 20:26:58.478029 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:58.477858 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\"" Jul 14 20:26:58.478294 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:58.478141 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\"" Jul 14 20:26:58.478294 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:58.478210 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-5q4hz\"" Jul 14 20:26:58.478294 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:58.478272 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\"" Jul 14 20:26:58.478461 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:58.478390 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\"" Jul 14 20:26:58.485276 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:58.484584 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\"" Jul 14 20:26:58.487758 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:58.487734 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-f598c6b45-bq5rt"] Jul 14 20:26:58.551535 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:58.551502 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-7cd4fc7897-d9pdf" event={"ID":"896f8e23-c9f6-4fea-8eec-762519e4fe62","Type":"ContainerStarted","Data":"90080086728e76cf01230996375f705bb1c0f3dc397110e9f1985a81d912e9cd"} Jul 14 20:26:58.565421 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:58.565379 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:26:58.595415 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:58.595379 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e1657bdb-067b-430c-96b1-80d8a7c24f09-metrics-client-ca\") pod \"telemeter-client-f598c6b45-bq5rt\" (UID: \"e1657bdb-067b-430c-96b1-80d8a7c24f09\") " pod="openshift-monitoring/telemeter-client-f598c6b45-bq5rt" Jul 14 20:26:58.595557 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:58.595421 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e1657bdb-067b-430c-96b1-80d8a7c24f09-telemeter-trusted-ca-bundle\") pod \"telemeter-client-f598c6b45-bq5rt\" (UID: \"e1657bdb-067b-430c-96b1-80d8a7c24f09\") " pod="openshift-monitoring/telemeter-client-f598c6b45-bq5rt" Jul 14 20:26:58.595557 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:58.595450 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g6j2r\" (UniqueName: \"kubernetes.io/projected/e1657bdb-067b-430c-96b1-80d8a7c24f09-kube-api-access-g6j2r\") pod \"telemeter-client-f598c6b45-bq5rt\" (UID: \"e1657bdb-067b-430c-96b1-80d8a7c24f09\") " pod="openshift-monitoring/telemeter-client-f598c6b45-bq5rt" Jul 14 20:26:58.595557 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:58.595546 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e1657bdb-067b-430c-96b1-80d8a7c24f09-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-f598c6b45-bq5rt\" (UID: \"e1657bdb-067b-430c-96b1-80d8a7c24f09\") " pod="openshift-monitoring/telemeter-client-f598c6b45-bq5rt" Jul 14 20:26:58.595672 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:58.595588 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/e1657bdb-067b-430c-96b1-80d8a7c24f09-telemeter-client-tls\") pod \"telemeter-client-f598c6b45-bq5rt\" (UID: \"e1657bdb-067b-430c-96b1-80d8a7c24f09\") " pod="openshift-monitoring/telemeter-client-f598c6b45-bq5rt" Jul 14 20:26:58.595672 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:58.595634 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/e1657bdb-067b-430c-96b1-80d8a7c24f09-secret-telemeter-client\") pod \"telemeter-client-f598c6b45-bq5rt\" (UID: \"e1657bdb-067b-430c-96b1-80d8a7c24f09\") " pod="openshift-monitoring/telemeter-client-f598c6b45-bq5rt" Jul 14 20:26:58.595742 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:58.595673 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e1657bdb-067b-430c-96b1-80d8a7c24f09-serving-certs-ca-bundle\") pod \"telemeter-client-f598c6b45-bq5rt\" (UID: \"e1657bdb-067b-430c-96b1-80d8a7c24f09\") " pod="openshift-monitoring/telemeter-client-f598c6b45-bq5rt" Jul 14 20:26:58.595742 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:58.595722 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/e1657bdb-067b-430c-96b1-80d8a7c24f09-federate-client-tls\") pod \"telemeter-client-f598c6b45-bq5rt\" (UID: \"e1657bdb-067b-430c-96b1-80d8a7c24f09\") " pod="openshift-monitoring/telemeter-client-f598c6b45-bq5rt" Jul 14 20:26:58.682408 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:58.682369 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-6b66589b8c-lfhd2"] Jul 14 20:26:58.686055 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:58.686030 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6b66589b8c-lfhd2" Jul 14 20:26:58.688412 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:58.688384 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 14 20:26:58.688412 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:58.688399 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-2kpsj\"" Jul 14 20:26:58.688666 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:58.688439 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 14 20:26:58.688666 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:58.688389 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 14 20:26:58.688666 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:58.688404 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 14 20:26:58.688666 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:58.688443 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 14 20:26:58.688666 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:58.688384 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 14 20:26:58.688904 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:58.688833 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 14 20:26:58.694539 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:58.694507 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jul 14 20:26:58.695212 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:58.695178 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6b66589b8c-lfhd2"] Jul 14 20:26:58.696334 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:58.696308 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/e1657bdb-067b-430c-96b1-80d8a7c24f09-telemeter-client-tls\") pod \"telemeter-client-f598c6b45-bq5rt\" (UID: \"e1657bdb-067b-430c-96b1-80d8a7c24f09\") " pod="openshift-monitoring/telemeter-client-f598c6b45-bq5rt" Jul 14 20:26:58.696443 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:58.696357 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/e1657bdb-067b-430c-96b1-80d8a7c24f09-secret-telemeter-client\") pod \"telemeter-client-f598c6b45-bq5rt\" (UID: \"e1657bdb-067b-430c-96b1-80d8a7c24f09\") " pod="openshift-monitoring/telemeter-client-f598c6b45-bq5rt" Jul 14 20:26:58.696443 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:58.696394 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e1657bdb-067b-430c-96b1-80d8a7c24f09-serving-certs-ca-bundle\") pod \"telemeter-client-f598c6b45-bq5rt\" (UID: \"e1657bdb-067b-430c-96b1-80d8a7c24f09\") " pod="openshift-monitoring/telemeter-client-f598c6b45-bq5rt" Jul 14 20:26:58.696551 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:58.696444 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/e1657bdb-067b-430c-96b1-80d8a7c24f09-federate-client-tls\") pod \"telemeter-client-f598c6b45-bq5rt\" (UID: \"e1657bdb-067b-430c-96b1-80d8a7c24f09\") " pod="openshift-monitoring/telemeter-client-f598c6b45-bq5rt" Jul 14 20:26:58.696551 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:58.696480 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e1657bdb-067b-430c-96b1-80d8a7c24f09-metrics-client-ca\") pod \"telemeter-client-f598c6b45-bq5rt\" (UID: \"e1657bdb-067b-430c-96b1-80d8a7c24f09\") " pod="openshift-monitoring/telemeter-client-f598c6b45-bq5rt" Jul 14 20:26:58.696551 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:58.696504 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e1657bdb-067b-430c-96b1-80d8a7c24f09-telemeter-trusted-ca-bundle\") pod \"telemeter-client-f598c6b45-bq5rt\" (UID: \"e1657bdb-067b-430c-96b1-80d8a7c24f09\") " pod="openshift-monitoring/telemeter-client-f598c6b45-bq5rt" Jul 14 20:26:58.696551 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:58.696532 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g6j2r\" (UniqueName: \"kubernetes.io/projected/e1657bdb-067b-430c-96b1-80d8a7c24f09-kube-api-access-g6j2r\") pod \"telemeter-client-f598c6b45-bq5rt\" (UID: \"e1657bdb-067b-430c-96b1-80d8a7c24f09\") " pod="openshift-monitoring/telemeter-client-f598c6b45-bq5rt" Jul 14 20:26:58.696751 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:58.696593 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e1657bdb-067b-430c-96b1-80d8a7c24f09-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-f598c6b45-bq5rt\" (UID: \"e1657bdb-067b-430c-96b1-80d8a7c24f09\") " pod="openshift-monitoring/telemeter-client-f598c6b45-bq5rt" Jul 14 20:26:58.698787 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:58.697915 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e1657bdb-067b-430c-96b1-80d8a7c24f09-serving-certs-ca-bundle\") pod \"telemeter-client-f598c6b45-bq5rt\" (UID: \"e1657bdb-067b-430c-96b1-80d8a7c24f09\") " pod="openshift-monitoring/telemeter-client-f598c6b45-bq5rt" Jul 14 20:26:58.698787 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:58.698555 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e1657bdb-067b-430c-96b1-80d8a7c24f09-metrics-client-ca\") pod \"telemeter-client-f598c6b45-bq5rt\" (UID: \"e1657bdb-067b-430c-96b1-80d8a7c24f09\") " pod="openshift-monitoring/telemeter-client-f598c6b45-bq5rt" Jul 14 20:26:58.699571 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:58.699547 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e1657bdb-067b-430c-96b1-80d8a7c24f09-telemeter-trusted-ca-bundle\") pod \"telemeter-client-f598c6b45-bq5rt\" (UID: \"e1657bdb-067b-430c-96b1-80d8a7c24f09\") " pod="openshift-monitoring/telemeter-client-f598c6b45-bq5rt" Jul 14 20:26:58.700428 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:58.700396 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e1657bdb-067b-430c-96b1-80d8a7c24f09-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-f598c6b45-bq5rt\" (UID: \"e1657bdb-067b-430c-96b1-80d8a7c24f09\") " pod="openshift-monitoring/telemeter-client-f598c6b45-bq5rt" Jul 14 20:26:58.701166 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:58.701095 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/e1657bdb-067b-430c-96b1-80d8a7c24f09-secret-telemeter-client\") pod \"telemeter-client-f598c6b45-bq5rt\" (UID: \"e1657bdb-067b-430c-96b1-80d8a7c24f09\") " pod="openshift-monitoring/telemeter-client-f598c6b45-bq5rt" Jul 14 20:26:58.702189 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:58.702148 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/e1657bdb-067b-430c-96b1-80d8a7c24f09-federate-client-tls\") pod \"telemeter-client-f598c6b45-bq5rt\" (UID: \"e1657bdb-067b-430c-96b1-80d8a7c24f09\") " pod="openshift-monitoring/telemeter-client-f598c6b45-bq5rt" Jul 14 20:26:58.702618 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:58.702597 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/e1657bdb-067b-430c-96b1-80d8a7c24f09-telemeter-client-tls\") pod \"telemeter-client-f598c6b45-bq5rt\" (UID: \"e1657bdb-067b-430c-96b1-80d8a7c24f09\") " pod="openshift-monitoring/telemeter-client-f598c6b45-bq5rt" Jul 14 20:26:58.707854 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:58.707826 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g6j2r\" (UniqueName: \"kubernetes.io/projected/e1657bdb-067b-430c-96b1-80d8a7c24f09-kube-api-access-g6j2r\") pod \"telemeter-client-f598c6b45-bq5rt\" (UID: \"e1657bdb-067b-430c-96b1-80d8a7c24f09\") " pod="openshift-monitoring/telemeter-client-f598c6b45-bq5rt" Jul 14 20:26:58.789137 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:58.789038 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-f598c6b45-bq5rt" Jul 14 20:26:58.797012 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:58.796985 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3c07a280-eebb-4ab3-9381-e92e5f354120-console-config\") pod \"console-6b66589b8c-lfhd2\" (UID: \"3c07a280-eebb-4ab3-9381-e92e5f354120\") " pod="openshift-console/console-6b66589b8c-lfhd2" Jul 14 20:26:58.797169 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:58.797054 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3c07a280-eebb-4ab3-9381-e92e5f354120-service-ca\") pod \"console-6b66589b8c-lfhd2\" (UID: \"3c07a280-eebb-4ab3-9381-e92e5f354120\") " pod="openshift-console/console-6b66589b8c-lfhd2" Jul 14 20:26:58.797169 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:58.797092 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3c07a280-eebb-4ab3-9381-e92e5f354120-console-serving-cert\") pod \"console-6b66589b8c-lfhd2\" (UID: \"3c07a280-eebb-4ab3-9381-e92e5f354120\") " pod="openshift-console/console-6b66589b8c-lfhd2" Jul 14 20:26:58.797169 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:58.797155 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wkz58\" (UniqueName: \"kubernetes.io/projected/3c07a280-eebb-4ab3-9381-e92e5f354120-kube-api-access-wkz58\") pod \"console-6b66589b8c-lfhd2\" (UID: \"3c07a280-eebb-4ab3-9381-e92e5f354120\") " pod="openshift-console/console-6b66589b8c-lfhd2" Jul 14 20:26:58.797335 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:58.797186 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3c07a280-eebb-4ab3-9381-e92e5f354120-console-oauth-config\") pod \"console-6b66589b8c-lfhd2\" (UID: \"3c07a280-eebb-4ab3-9381-e92e5f354120\") " pod="openshift-console/console-6b66589b8c-lfhd2" Jul 14 20:26:58.797335 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:58.797211 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3c07a280-eebb-4ab3-9381-e92e5f354120-oauth-serving-cert\") pod \"console-6b66589b8c-lfhd2\" (UID: \"3c07a280-eebb-4ab3-9381-e92e5f354120\") " pod="openshift-console/console-6b66589b8c-lfhd2" Jul 14 20:26:58.797335 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:58.797254 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3c07a280-eebb-4ab3-9381-e92e5f354120-trusted-ca-bundle\") pod \"console-6b66589b8c-lfhd2\" (UID: \"3c07a280-eebb-4ab3-9381-e92e5f354120\") " pod="openshift-console/console-6b66589b8c-lfhd2" Jul 14 20:26:58.898755 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:58.898724 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3c07a280-eebb-4ab3-9381-e92e5f354120-console-config\") pod \"console-6b66589b8c-lfhd2\" (UID: \"3c07a280-eebb-4ab3-9381-e92e5f354120\") " pod="openshift-console/console-6b66589b8c-lfhd2" Jul 14 20:26:58.898918 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:58.898792 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3c07a280-eebb-4ab3-9381-e92e5f354120-service-ca\") pod \"console-6b66589b8c-lfhd2\" (UID: \"3c07a280-eebb-4ab3-9381-e92e5f354120\") " pod="openshift-console/console-6b66589b8c-lfhd2" Jul 14 20:26:58.898918 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:58.898821 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3c07a280-eebb-4ab3-9381-e92e5f354120-console-serving-cert\") pod \"console-6b66589b8c-lfhd2\" (UID: \"3c07a280-eebb-4ab3-9381-e92e5f354120\") " pod="openshift-console/console-6b66589b8c-lfhd2" Jul 14 20:26:58.898918 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:58.898845 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wkz58\" (UniqueName: \"kubernetes.io/projected/3c07a280-eebb-4ab3-9381-e92e5f354120-kube-api-access-wkz58\") pod \"console-6b66589b8c-lfhd2\" (UID: \"3c07a280-eebb-4ab3-9381-e92e5f354120\") " pod="openshift-console/console-6b66589b8c-lfhd2" Jul 14 20:26:58.898918 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:58.898870 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3c07a280-eebb-4ab3-9381-e92e5f354120-console-oauth-config\") pod \"console-6b66589b8c-lfhd2\" (UID: \"3c07a280-eebb-4ab3-9381-e92e5f354120\") " pod="openshift-console/console-6b66589b8c-lfhd2" Jul 14 20:26:58.898918 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:58.898897 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3c07a280-eebb-4ab3-9381-e92e5f354120-oauth-serving-cert\") pod \"console-6b66589b8c-lfhd2\" (UID: \"3c07a280-eebb-4ab3-9381-e92e5f354120\") " pod="openshift-console/console-6b66589b8c-lfhd2" Jul 14 20:26:58.899182 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:58.898936 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3c07a280-eebb-4ab3-9381-e92e5f354120-trusted-ca-bundle\") pod \"console-6b66589b8c-lfhd2\" (UID: \"3c07a280-eebb-4ab3-9381-e92e5f354120\") " pod="openshift-console/console-6b66589b8c-lfhd2" Jul 14 20:26:58.899980 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:58.899955 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3c07a280-eebb-4ab3-9381-e92e5f354120-oauth-serving-cert\") pod \"console-6b66589b8c-lfhd2\" (UID: \"3c07a280-eebb-4ab3-9381-e92e5f354120\") " pod="openshift-console/console-6b66589b8c-lfhd2" Jul 14 20:26:58.900075 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:58.900059 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3c07a280-eebb-4ab3-9381-e92e5f354120-console-config\") pod \"console-6b66589b8c-lfhd2\" (UID: \"3c07a280-eebb-4ab3-9381-e92e5f354120\") " pod="openshift-console/console-6b66589b8c-lfhd2" Jul 14 20:26:58.900201 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:58.900176 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3c07a280-eebb-4ab3-9381-e92e5f354120-service-ca\") pod \"console-6b66589b8c-lfhd2\" (UID: \"3c07a280-eebb-4ab3-9381-e92e5f354120\") " pod="openshift-console/console-6b66589b8c-lfhd2" Jul 14 20:26:58.900359 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:58.900318 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3c07a280-eebb-4ab3-9381-e92e5f354120-trusted-ca-bundle\") pod \"console-6b66589b8c-lfhd2\" (UID: \"3c07a280-eebb-4ab3-9381-e92e5f354120\") " pod="openshift-console/console-6b66589b8c-lfhd2" Jul 14 20:26:58.902715 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:58.902671 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3c07a280-eebb-4ab3-9381-e92e5f354120-console-oauth-config\") pod \"console-6b66589b8c-lfhd2\" (UID: \"3c07a280-eebb-4ab3-9381-e92e5f354120\") " pod="openshift-console/console-6b66589b8c-lfhd2" Jul 14 20:26:58.903010 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:58.902990 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3c07a280-eebb-4ab3-9381-e92e5f354120-console-serving-cert\") pod \"console-6b66589b8c-lfhd2\" (UID: \"3c07a280-eebb-4ab3-9381-e92e5f354120\") " pod="openshift-console/console-6b66589b8c-lfhd2" Jul 14 20:26:58.908380 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:58.908357 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wkz58\" (UniqueName: \"kubernetes.io/projected/3c07a280-eebb-4ab3-9381-e92e5f354120-kube-api-access-wkz58\") pod \"console-6b66589b8c-lfhd2\" (UID: \"3c07a280-eebb-4ab3-9381-e92e5f354120\") " pod="openshift-console/console-6b66589b8c-lfhd2" Jul 14 20:26:58.957368 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:58.957342 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-f598c6b45-bq5rt"] Jul 14 20:26:59.000653 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:59.000560 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6b66589b8c-lfhd2" Jul 14 20:26:59.292943 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:26:59.292900 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode1657bdb_067b_430c_96b1_80d8a7c24f09.slice/crio-cb6bcd8fca922ab780e743ecef28215c786d52accd05ecd3754e63123493b7b5 WatchSource:0}: Error finding container cb6bcd8fca922ab780e743ecef28215c786d52accd05ecd3754e63123493b7b5: Status 404 returned error can't find the container with id cb6bcd8fca922ab780e743ecef28215c786d52accd05ecd3754e63123493b7b5 Jul 14 20:26:59.431935 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:59.431903 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 14 20:26:59.436231 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:59.436205 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 14 20:26:59.439467 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:59.438534 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Jul 14 20:26:59.439467 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:59.438799 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Jul 14 20:26:59.439467 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:59.438941 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Jul 14 20:26:59.439467 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:59.438944 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Jul 14 20:26:59.439467 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:59.439034 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Jul 14 20:26:59.439926 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:59.439902 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Jul 14 20:26:59.440269 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:59.440247 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Jul 14 20:26:59.440603 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:59.440252 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Jul 14 20:26:59.440773 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:59.440332 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Jul 14 20:26:59.441323 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:59.441305 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-2c5221ssc5ite\"" Jul 14 20:26:59.441589 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:59.441566 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Jul 14 20:26:59.441758 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:59.441741 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Jul 14 20:26:59.441963 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:59.441946 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Jul 14 20:26:59.442059 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:59.442046 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Jul 14 20:26:59.442547 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:59.442249 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-8xsdz\"" Jul 14 20:26:59.444751 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:59.444732 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Jul 14 20:26:59.456673 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:59.456645 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 14 20:26:59.557494 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:59.557405 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-f598c6b45-bq5rt" event={"ID":"e1657bdb-067b-430c-96b1-80d8a7c24f09","Type":"ContainerStarted","Data":"cb6bcd8fca922ab780e743ecef28215c786d52accd05ecd3754e63123493b7b5"} Jul 14 20:26:59.606793 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:59.606757 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/28bc4a8b-af34-47be-bad7-d9be1b2c75b9-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"28bc4a8b-af34-47be-bad7-d9be1b2c75b9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 20:26:59.606997 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:59.606806 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/28bc4a8b-af34-47be-bad7-d9be1b2c75b9-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"28bc4a8b-af34-47be-bad7-d9be1b2c75b9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 20:26:59.606997 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:59.606837 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/28bc4a8b-af34-47be-bad7-d9be1b2c75b9-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"28bc4a8b-af34-47be-bad7-d9be1b2c75b9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 20:26:59.606997 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:59.606915 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/28bc4a8b-af34-47be-bad7-d9be1b2c75b9-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"28bc4a8b-af34-47be-bad7-d9be1b2c75b9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 20:26:59.606997 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:59.606940 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/28bc4a8b-af34-47be-bad7-d9be1b2c75b9-config\") pod \"prometheus-k8s-0\" (UID: \"28bc4a8b-af34-47be-bad7-d9be1b2c75b9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 20:26:59.606997 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:59.606977 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/28bc4a8b-af34-47be-bad7-d9be1b2c75b9-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"28bc4a8b-af34-47be-bad7-d9be1b2c75b9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 20:26:59.607287 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:59.607005 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/28bc4a8b-af34-47be-bad7-d9be1b2c75b9-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"28bc4a8b-af34-47be-bad7-d9be1b2c75b9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 20:26:59.607287 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:59.607074 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/28bc4a8b-af34-47be-bad7-d9be1b2c75b9-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"28bc4a8b-af34-47be-bad7-d9be1b2c75b9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 20:26:59.607287 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:59.607202 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/28bc4a8b-af34-47be-bad7-d9be1b2c75b9-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"28bc4a8b-af34-47be-bad7-d9be1b2c75b9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 20:26:59.607287 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:59.607227 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/28bc4a8b-af34-47be-bad7-d9be1b2c75b9-web-config\") pod \"prometheus-k8s-0\" (UID: \"28bc4a8b-af34-47be-bad7-d9be1b2c75b9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 20:26:59.607287 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:59.607272 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/28bc4a8b-af34-47be-bad7-d9be1b2c75b9-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"28bc4a8b-af34-47be-bad7-d9be1b2c75b9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 20:26:59.607526 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:59.607298 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/28bc4a8b-af34-47be-bad7-d9be1b2c75b9-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"28bc4a8b-af34-47be-bad7-d9be1b2c75b9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 20:26:59.607526 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:59.607319 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/28bc4a8b-af34-47be-bad7-d9be1b2c75b9-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"28bc4a8b-af34-47be-bad7-d9be1b2c75b9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 20:26:59.607526 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:59.607343 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/28bc4a8b-af34-47be-bad7-d9be1b2c75b9-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"28bc4a8b-af34-47be-bad7-d9be1b2c75b9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 20:26:59.607526 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:59.607394 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/28bc4a8b-af34-47be-bad7-d9be1b2c75b9-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"28bc4a8b-af34-47be-bad7-d9be1b2c75b9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 20:26:59.607526 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:59.607422 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/28bc4a8b-af34-47be-bad7-d9be1b2c75b9-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"28bc4a8b-af34-47be-bad7-d9be1b2c75b9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 20:26:59.607526 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:59.607464 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/28bc4a8b-af34-47be-bad7-d9be1b2c75b9-config-out\") pod \"prometheus-k8s-0\" (UID: \"28bc4a8b-af34-47be-bad7-d9be1b2c75b9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 20:26:59.607526 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:59.607494 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/28bc4a8b-af34-47be-bad7-d9be1b2c75b9-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"28bc4a8b-af34-47be-bad7-d9be1b2c75b9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 20:26:59.607526 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:59.607521 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sx97g\" (UniqueName: \"kubernetes.io/projected/28bc4a8b-af34-47be-bad7-d9be1b2c75b9-kube-api-access-sx97g\") pod \"prometheus-k8s-0\" (UID: \"28bc4a8b-af34-47be-bad7-d9be1b2c75b9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 20:26:59.607901 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:59.607542 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/28bc4a8b-af34-47be-bad7-d9be1b2c75b9-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"28bc4a8b-af34-47be-bad7-d9be1b2c75b9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 20:26:59.708233 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:59.708190 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/28bc4a8b-af34-47be-bad7-d9be1b2c75b9-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"28bc4a8b-af34-47be-bad7-d9be1b2c75b9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 20:26:59.708233 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:59.708232 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/28bc4a8b-af34-47be-bad7-d9be1b2c75b9-web-config\") pod \"prometheus-k8s-0\" (UID: \"28bc4a8b-af34-47be-bad7-d9be1b2c75b9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 20:26:59.708427 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:59.708358 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/28bc4a8b-af34-47be-bad7-d9be1b2c75b9-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"28bc4a8b-af34-47be-bad7-d9be1b2c75b9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 20:26:59.708427 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:59.708391 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/28bc4a8b-af34-47be-bad7-d9be1b2c75b9-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"28bc4a8b-af34-47be-bad7-d9be1b2c75b9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 20:26:59.708427 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:59.708412 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/28bc4a8b-af34-47be-bad7-d9be1b2c75b9-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"28bc4a8b-af34-47be-bad7-d9be1b2c75b9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 20:26:59.708572 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:59.708438 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/28bc4a8b-af34-47be-bad7-d9be1b2c75b9-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"28bc4a8b-af34-47be-bad7-d9be1b2c75b9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 20:26:59.708572 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:59.708470 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/28bc4a8b-af34-47be-bad7-d9be1b2c75b9-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"28bc4a8b-af34-47be-bad7-d9be1b2c75b9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 20:26:59.708572 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:59.708512 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/28bc4a8b-af34-47be-bad7-d9be1b2c75b9-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"28bc4a8b-af34-47be-bad7-d9be1b2c75b9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 20:26:59.708572 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:59.708556 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/28bc4a8b-af34-47be-bad7-d9be1b2c75b9-config-out\") pod \"prometheus-k8s-0\" (UID: \"28bc4a8b-af34-47be-bad7-d9be1b2c75b9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 20:26:59.708760 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:59.708587 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/28bc4a8b-af34-47be-bad7-d9be1b2c75b9-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"28bc4a8b-af34-47be-bad7-d9be1b2c75b9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 20:26:59.708760 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:59.708617 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sx97g\" (UniqueName: \"kubernetes.io/projected/28bc4a8b-af34-47be-bad7-d9be1b2c75b9-kube-api-access-sx97g\") pod \"prometheus-k8s-0\" (UID: \"28bc4a8b-af34-47be-bad7-d9be1b2c75b9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 20:26:59.708760 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:59.708646 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/28bc4a8b-af34-47be-bad7-d9be1b2c75b9-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"28bc4a8b-af34-47be-bad7-d9be1b2c75b9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 20:26:59.708760 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:59.708702 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/28bc4a8b-af34-47be-bad7-d9be1b2c75b9-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"28bc4a8b-af34-47be-bad7-d9be1b2c75b9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 20:26:59.708760 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:59.708734 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/28bc4a8b-af34-47be-bad7-d9be1b2c75b9-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"28bc4a8b-af34-47be-bad7-d9be1b2c75b9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 20:26:59.709005 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:59.708763 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/28bc4a8b-af34-47be-bad7-d9be1b2c75b9-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"28bc4a8b-af34-47be-bad7-d9be1b2c75b9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 20:26:59.709005 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:59.708796 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/28bc4a8b-af34-47be-bad7-d9be1b2c75b9-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"28bc4a8b-af34-47be-bad7-d9be1b2c75b9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 20:26:59.709005 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:59.708825 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/28bc4a8b-af34-47be-bad7-d9be1b2c75b9-config\") pod \"prometheus-k8s-0\" (UID: \"28bc4a8b-af34-47be-bad7-d9be1b2c75b9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 20:26:59.709005 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:59.708867 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/28bc4a8b-af34-47be-bad7-d9be1b2c75b9-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"28bc4a8b-af34-47be-bad7-d9be1b2c75b9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 20:26:59.709005 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:59.708905 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/28bc4a8b-af34-47be-bad7-d9be1b2c75b9-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"28bc4a8b-af34-47be-bad7-d9be1b2c75b9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 20:26:59.709005 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:59.708908 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/28bc4a8b-af34-47be-bad7-d9be1b2c75b9-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"28bc4a8b-af34-47be-bad7-d9be1b2c75b9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 20:26:59.709005 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:59.708932 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/28bc4a8b-af34-47be-bad7-d9be1b2c75b9-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"28bc4a8b-af34-47be-bad7-d9be1b2c75b9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 20:26:59.709533 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:59.709368 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/28bc4a8b-af34-47be-bad7-d9be1b2c75b9-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"28bc4a8b-af34-47be-bad7-d9be1b2c75b9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 20:26:59.709638 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:59.709612 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/28bc4a8b-af34-47be-bad7-d9be1b2c75b9-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"28bc4a8b-af34-47be-bad7-d9be1b2c75b9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 20:26:59.710350 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:59.710019 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/28bc4a8b-af34-47be-bad7-d9be1b2c75b9-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"28bc4a8b-af34-47be-bad7-d9be1b2c75b9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 20:26:59.711549 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:59.711334 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/28bc4a8b-af34-47be-bad7-d9be1b2c75b9-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"28bc4a8b-af34-47be-bad7-d9be1b2c75b9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 20:26:59.711651 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:59.711604 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/28bc4a8b-af34-47be-bad7-d9be1b2c75b9-web-config\") pod \"prometheus-k8s-0\" (UID: \"28bc4a8b-af34-47be-bad7-d9be1b2c75b9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 20:26:59.711708 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:59.711663 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/28bc4a8b-af34-47be-bad7-d9be1b2c75b9-config-out\") pod \"prometheus-k8s-0\" (UID: \"28bc4a8b-af34-47be-bad7-d9be1b2c75b9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 20:26:59.711931 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:59.711852 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/28bc4a8b-af34-47be-bad7-d9be1b2c75b9-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"28bc4a8b-af34-47be-bad7-d9be1b2c75b9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 20:26:59.711931 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:59.711870 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/28bc4a8b-af34-47be-bad7-d9be1b2c75b9-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"28bc4a8b-af34-47be-bad7-d9be1b2c75b9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 20:26:59.712069 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:59.712050 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/28bc4a8b-af34-47be-bad7-d9be1b2c75b9-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"28bc4a8b-af34-47be-bad7-d9be1b2c75b9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 20:26:59.712834 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:59.712614 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/28bc4a8b-af34-47be-bad7-d9be1b2c75b9-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"28bc4a8b-af34-47be-bad7-d9be1b2c75b9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 20:26:59.712834 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:59.712750 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/28bc4a8b-af34-47be-bad7-d9be1b2c75b9-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"28bc4a8b-af34-47be-bad7-d9be1b2c75b9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 20:26:59.713094 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:59.712985 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/28bc4a8b-af34-47be-bad7-d9be1b2c75b9-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"28bc4a8b-af34-47be-bad7-d9be1b2c75b9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 20:26:59.713094 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:59.712999 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/28bc4a8b-af34-47be-bad7-d9be1b2c75b9-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"28bc4a8b-af34-47be-bad7-d9be1b2c75b9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 20:26:59.714531 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:59.714510 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/28bc4a8b-af34-47be-bad7-d9be1b2c75b9-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"28bc4a8b-af34-47be-bad7-d9be1b2c75b9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 20:26:59.714953 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:59.714930 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/28bc4a8b-af34-47be-bad7-d9be1b2c75b9-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"28bc4a8b-af34-47be-bad7-d9be1b2c75b9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 20:26:59.715110 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:59.715090 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/28bc4a8b-af34-47be-bad7-d9be1b2c75b9-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"28bc4a8b-af34-47be-bad7-d9be1b2c75b9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 20:26:59.715382 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:59.715365 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/28bc4a8b-af34-47be-bad7-d9be1b2c75b9-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"28bc4a8b-af34-47be-bad7-d9be1b2c75b9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 20:26:59.715597 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:59.715574 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/28bc4a8b-af34-47be-bad7-d9be1b2c75b9-config\") pod \"prometheus-k8s-0\" (UID: \"28bc4a8b-af34-47be-bad7-d9be1b2c75b9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 20:26:59.717858 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:59.717826 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sx97g\" (UniqueName: \"kubernetes.io/projected/28bc4a8b-af34-47be-bad7-d9be1b2c75b9-kube-api-access-sx97g\") pod \"prometheus-k8s-0\" (UID: \"28bc4a8b-af34-47be-bad7-d9be1b2c75b9\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 20:26:59.746371 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:59.746339 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:26:59.750974 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:59.750952 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 14 20:26:59.910747 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:59.910706 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6b66589b8c-lfhd2"] Jul 14 20:26:59.916921 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:26:59.916896 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3c07a280_eebb_4ab3_9381_e92e5f354120.slice/crio-a9fdf37ac585f3bbff14234053d99a2c4b6d86dc69586d8923719fa61e2c354a WatchSource:0}: Error finding container a9fdf37ac585f3bbff14234053d99a2c4b6d86dc69586d8923719fa61e2c354a: Status 404 returned error can't find the container with id a9fdf37ac585f3bbff14234053d99a2c4b6d86dc69586d8923719fa61e2c354a Jul 14 20:26:59.932568 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:26:59.932544 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 14 20:26:59.937840 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:26:59.937799 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod28bc4a8b_af34_47be_bad7_d9be1b2c75b9.slice/crio-e57cfd03f3f7f4985bb882c871dc4ba1e2f6e086a000cc8db13d5f9427ac896e WatchSource:0}: Error finding container e57cfd03f3f7f4985bb882c871dc4ba1e2f6e086a000cc8db13d5f9427ac896e: Status 404 returned error can't find the container with id e57cfd03f3f7f4985bb882c871dc4ba1e2f6e086a000cc8db13d5f9427ac896e Jul 14 20:27:00.120851 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:27:00.120818 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:27:00.221505 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:27:00.221468 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:27:00.563782 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:27:00.563740 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6b66589b8c-lfhd2" event={"ID":"3c07a280-eebb-4ab3-9381-e92e5f354120","Type":"ContainerStarted","Data":"a9fdf37ac585f3bbff14234053d99a2c4b6d86dc69586d8923719fa61e2c354a"} Jul 14 20:27:00.565487 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:27:00.565459 2563 generic.go:356] "Generic (PLEG): container finished" podID="28bc4a8b-af34-47be-bad7-d9be1b2c75b9" containerID="77817d528b2fa089a23ac9fc25dd243c3ab43760686e7e35c1d243405780c831" exitCode=0 Jul 14 20:27:00.565647 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:27:00.565617 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"28bc4a8b-af34-47be-bad7-d9be1b2c75b9","Type":"ContainerDied","Data":"77817d528b2fa089a23ac9fc25dd243c3ab43760686e7e35c1d243405780c831"} Jul 14 20:27:00.565740 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:27:00.565687 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"28bc4a8b-af34-47be-bad7-d9be1b2c75b9","Type":"ContainerStarted","Data":"e57cfd03f3f7f4985bb882c871dc4ba1e2f6e086a000cc8db13d5f9427ac896e"} Jul 14 20:27:00.569550 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:27:00.569524 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"3bd29b0f-00fb-4a66-97b1-c2727db1081a","Type":"ContainerStarted","Data":"3c4a85c44a1b08c5bf049fc6aadc2d45861a137e3e03806bc44d2ad641ed0fb7"} Jul 14 20:27:00.569657 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:27:00.569585 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"3bd29b0f-00fb-4a66-97b1-c2727db1081a","Type":"ContainerStarted","Data":"4b8019dccad793e1ee2952ecd03269d63392e23289012cc28525d978f2d8ef8f"} Jul 14 20:27:00.569657 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:27:00.569620 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"3bd29b0f-00fb-4a66-97b1-c2727db1081a","Type":"ContainerStarted","Data":"11c0959594c39f4f4f97417dd5053740b7f4a6a8419a7f0df5cacce203f60277"} Jul 14 20:27:00.569657 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:27:00.569646 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"3bd29b0f-00fb-4a66-97b1-c2727db1081a","Type":"ContainerStarted","Data":"483313155a41d9be8f99aa2a6a7fef61ec30f2132b46e83e2eef66fd887ca6ce"} Jul 14 20:27:00.569801 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:27:00.569681 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"3bd29b0f-00fb-4a66-97b1-c2727db1081a","Type":"ContainerStarted","Data":"35a261aa2f7135e93e4fb2fcefd57469a2a31220f895da2810e59ab0fc4d286a"} Jul 14 20:27:00.571524 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:27:00.571455 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-7cd4fc7897-d9pdf" event={"ID":"896f8e23-c9f6-4fea-8eec-762519e4fe62","Type":"ContainerStarted","Data":"a198a95bd97ac29ae2f5f4258532ea71ead266afcbe15b7e20d5565b41a45b0b"} Jul 14 20:27:00.608252 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:27:00.608190 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-7cd4fc7897-d9pdf" podStartSLOduration=1.972325336 podStartE2EDuration="3.608167657s" totalImagesPullingTime="1.635842321s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 20:26:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 20:27:00.606320947 +0000 UTC m=+279.527484255" watchObservedRunningTime="2026-07-14 20:27:00.608167657 +0000 UTC m=+279.529330968" Jul 14 20:27:01.580532 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:27:01.580487 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"3bd29b0f-00fb-4a66-97b1-c2727db1081a","Type":"ContainerStarted","Data":"6e9731f4bbd5f4026d5ce21eba732197ff11c43392e10dd1dc9427af550d62f6"} Jul 14 20:27:01.614034 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:27:01.613973 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.472588852 podStartE2EDuration="7.613949235s" totalImagesPullingTime="5.124520858s" totalInitContainerRuntime="16.839525ms" isStatefulPod=false podCreationTimestamp="2026-07-14 20:26:54 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 20:27:01.61198262 +0000 UTC m=+280.533145928" watchObservedRunningTime="2026-07-14 20:27:01.613949235 +0000 UTC m=+280.535112545" Jul 14 20:27:03.147216 ip-10-0-141-61 kubenswrapper[2563]: E0714 20:27:03.147171 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-knrl4" podUID="b4e3e206-bf7b-47bf-b037-9c12fca3799b" Jul 14 20:27:03.527285 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:27:03.527253 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:27:03.600539 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:27:03.600507 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-knrl4" Jul 14 20:27:04.606245 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:27:04.606205 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6b66589b8c-lfhd2" event={"ID":"3c07a280-eebb-4ab3-9381-e92e5f354120","Type":"ContainerStarted","Data":"520cda8a389b0552aa11c7fc0569add29525c8a8ee56657f0129a3e311abe0d0"} Jul 14 20:27:04.608370 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:27:04.608338 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-f598c6b45-bq5rt" event={"ID":"e1657bdb-067b-430c-96b1-80d8a7c24f09","Type":"ContainerStarted","Data":"61210767a303cbf44bcd24efe0b01b75927ddbc6c5465aa71df3675c6d1be9fb"} Jul 14 20:27:04.624458 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:27:04.624398 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-6b66589b8c-lfhd2" podStartSLOduration=2.982599575 podStartE2EDuration="6.624375556s" totalImagesPullingTime="3.641775981s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 20:26:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 20:27:04.622204673 +0000 UTC m=+283.543367980" watchObservedRunningTime="2026-07-14 20:27:04.624375556 +0000 UTC m=+283.545538864" Jul 14 20:27:05.614923 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:27:05.614892 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"28bc4a8b-af34-47be-bad7-d9be1b2c75b9","Type":"ContainerStarted","Data":"6ce246e2ae983de72d36f83644143a417f38b98438e379236f09b2fe8bf65cda"} Jul 14 20:27:05.615318 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:27:05.614976 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"28bc4a8b-af34-47be-bad7-d9be1b2c75b9","Type":"ContainerStarted","Data":"35634f462d91ad1ce335dab920d61b1955698fc0fb19733fe2ec61d875b3d863"} Jul 14 20:27:05.616812 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:27:05.616782 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-f598c6b45-bq5rt" event={"ID":"e1657bdb-067b-430c-96b1-80d8a7c24f09","Type":"ContainerStarted","Data":"1642127d4128ded3df941335d2778c808479f9a61064dea306ba52dbbdfac0fa"} Jul 14 20:27:05.616914 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:27:05.616842 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-f598c6b45-bq5rt" event={"ID":"e1657bdb-067b-430c-96b1-80d8a7c24f09","Type":"ContainerStarted","Data":"50e44a908812d9c967311c26f974b113b6587a062a2c854eea9ffd9f93894bad"} Jul 14 20:27:05.637982 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:27:05.637926 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-f598c6b45-bq5rt" podStartSLOduration=3.368435869 podStartE2EDuration="7.637909285s" totalImagesPullingTime="4.269473416s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 20:26:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 20:27:05.636304786 +0000 UTC m=+284.557468094" watchObservedRunningTime="2026-07-14 20:27:05.637909285 +0000 UTC m=+284.559072595" Jul 14 20:27:06.478054 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:27:06.478015 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b4e3e206-bf7b-47bf-b037-9c12fca3799b-metrics-tls\") pod \"dns-default-knrl4\" (UID: \"b4e3e206-bf7b-47bf-b037-9c12fca3799b\") " pod="openshift-dns/dns-default-knrl4" Jul 14 20:27:06.480829 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:27:06.480801 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b4e3e206-bf7b-47bf-b037-9c12fca3799b-metrics-tls\") pod \"dns-default-knrl4\" (UID: \"b4e3e206-bf7b-47bf-b037-9c12fca3799b\") " pod="openshift-dns/dns-default-knrl4" Jul 14 20:27:06.603696 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:27:06.603661 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-qv2tt\"" Jul 14 20:27:06.612497 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:27:06.612459 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-knrl4" Jul 14 20:27:07.078335 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:27:07.078303 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-knrl4"] Jul 14 20:27:07.083172 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:27:07.082622 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb4e3e206_bf7b_47bf_b037_9c12fca3799b.slice/crio-c53d7966c5b92ae24144945cf407b9ae791a56e4b3f46f28f5cf36e64ef2c531 WatchSource:0}: Error finding container c53d7966c5b92ae24144945cf407b9ae791a56e4b3f46f28f5cf36e64ef2c531: Status 404 returned error can't find the container with id c53d7966c5b92ae24144945cf407b9ae791a56e4b3f46f28f5cf36e64ef2c531 Jul 14 20:27:07.628897 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:27:07.628807 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-knrl4" event={"ID":"b4e3e206-bf7b-47bf-b037-9c12fca3799b","Type":"ContainerStarted","Data":"c53d7966c5b92ae24144945cf407b9ae791a56e4b3f46f28f5cf36e64ef2c531"} Jul 14 20:27:07.632763 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:27:07.632731 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"28bc4a8b-af34-47be-bad7-d9be1b2c75b9","Type":"ContainerStarted","Data":"59bf3096727c83d6b45956c7830bc808a86f0dd745e2db10201564ef21a1d893"} Jul 14 20:27:07.632940 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:27:07.632812 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"28bc4a8b-af34-47be-bad7-d9be1b2c75b9","Type":"ContainerStarted","Data":"56cc1c7f62d121733fe4626b1ef37b96de3fccad3cd22d368e4215ecfc06a69e"} Jul 14 20:27:07.632940 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:27:07.632851 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"28bc4a8b-af34-47be-bad7-d9be1b2c75b9","Type":"ContainerStarted","Data":"4b3681e1a826480c6600fe6870249377e43a786d06fdea66e63271da33d55978"} Jul 14 20:27:07.632940 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:27:07.632888 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"28bc4a8b-af34-47be-bad7-d9be1b2c75b9","Type":"ContainerStarted","Data":"42ce2ee737882fd5ac081570f9978e43245de9b635cab2251685b03cfe0797c7"} Jul 14 20:27:07.661077 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:27:07.661009 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.439781573 podStartE2EDuration="8.660985855s" totalImagesPullingTime="6.200553623s" totalInitContainerRuntime="20.650659ms" isStatefulPod=false podCreationTimestamp="2026-07-14 20:26:59 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 20:27:07.660254867 +0000 UTC m=+286.581418177" watchObservedRunningTime="2026-07-14 20:27:07.660985855 +0000 UTC m=+286.582149164" Jul 14 20:27:08.502817 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:27:08.502768 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:27:08.605940 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:27:08.605909 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:27:09.001739 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:27:09.001700 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6b66589b8c-lfhd2" Jul 14 20:27:09.001930 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:27:09.001791 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-6b66589b8c-lfhd2" Jul 14 20:27:09.003253 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:27:09.003229 2563 patch_prober.go:28] interesting pod/console-6b66589b8c-lfhd2 container/console namespace/openshift-console: Startup probe status=failure output="Get \"https://10.134.0.20:8443/health\": dial tcp 10.134.0.20:8443: connect: connection refused" start-of-body= Jul 14 20:27:09.003310 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:27:09.003290 2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-console/console-6b66589b8c-lfhd2" podUID="3c07a280-eebb-4ab3-9381-e92e5f354120" containerName="console" probeResult="failure" output="Get \"https://10.134.0.20:8443/health\": dial tcp 10.134.0.20:8443: connect: connection refused" Jul 14 20:27:09.642238 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:27:09.642192 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-knrl4" event={"ID":"b4e3e206-bf7b-47bf-b037-9c12fca3799b","Type":"ContainerStarted","Data":"a730fd79f6c5a255e33fdb2010ce10d333bec8783a3b882d91960a74220cc229"} Jul 14 20:27:09.642631 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:27:09.642255 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-knrl4" event={"ID":"b4e3e206-bf7b-47bf-b037-9c12fca3799b","Type":"ContainerStarted","Data":"17e4b6f2050e976b77f8f6a9005fa6eadb098ed0b48a83827dd801b79e7e24c2"} Jul 14 20:27:09.642631 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:27:09.642307 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-knrl4" Jul 14 20:27:09.660919 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:27:09.660868 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-knrl4" podStartSLOduration=252.102366428 podStartE2EDuration="4m13.660852389s" totalImagesPullingTime="1.558485961s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 20:22:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 20:27:09.659626929 +0000 UTC m=+288.580790235" watchObservedRunningTime="2026-07-14 20:27:09.660852389 +0000 UTC m=+288.582015698" Jul 14 20:27:09.752111 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:27:09.752077 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 14 20:27:17.968065 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:27:17.968028 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-7cd4fc7897-d9pdf" Jul 14 20:27:17.968581 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:27:17.968098 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-7cd4fc7897-d9pdf" Jul 14 20:27:18.148859 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:27:18.148829 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:27:19.005679 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:27:19.005650 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-6b66589b8c-lfhd2" Jul 14 20:27:19.009731 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:27:19.009704 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-6b66589b8c-lfhd2" Jul 14 20:27:19.479842 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:27:19.479804 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:27:19.648794 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:27:19.648764 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-knrl4" Jul 14 20:27:20.689487 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:27:20.689457 2563 generic.go:356] "Generic (PLEG): container finished" podID="1447f395-a017-44cf-9176-2944ead9c090" containerID="5066e8a3a8e625031e626373303931f07a874b9ac595017817cd95eed4fbc6b8" exitCode=255 Jul 14 20:27:20.689935 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:27:20.689525 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-76dff96bd-59zm4" event={"ID":"1447f395-a017-44cf-9176-2944ead9c090","Type":"ContainerDied","Data":"5066e8a3a8e625031e626373303931f07a874b9ac595017817cd95eed4fbc6b8"} Jul 14 20:27:20.689935 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:27:20.689583 2563 scope.go:117] "RemoveContainer" containerID="d5d084c34b592eccdfb9697fed842e90f11417e759d52858e8ecd4285d5e2956" Jul 14 20:27:20.695932 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:27:20.695910 2563 scope.go:117] "RemoveContainer" containerID="5066e8a3a8e625031e626373303931f07a874b9ac595017817cd95eed4fbc6b8" Jul 14 20:27:20.696215 ip-10-0-141-61 kubenswrapper[2563]: E0714 20:27:20.696180 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"addon-agent\" with CrashLoopBackOff: \"back-off 10s restarting failed container=addon-agent pod=managed-serviceaccount-addon-agent-76dff96bd-59zm4_open-cluster-management-agent-addon(1447f395-a017-44cf-9176-2944ead9c090)\"" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-76dff96bd-59zm4" podUID="1447f395-a017-44cf-9176-2944ead9c090" Jul 14 20:27:21.457679 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:27:21.457648 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hzbx8_9588ab98-33e8-4c89-807d-a805240f1b81/ovn-acl-logging/0.log" Jul 14 20:27:21.459458 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:27:21.459438 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hzbx8_9588ab98-33e8-4c89-807d-a805240f1b81/ovn-acl-logging/0.log" Jul 14 20:27:21.467686 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:27:21.467663 2563 kubelet.go:1648] "Image garbage collection succeeded" Jul 14 20:27:22.403316 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:27:22.403279 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:27:22.451772 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:27:22.451737 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:27:23.436538 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:27:23.436496 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:27:23.536761 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:27:23.536732 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:27:23.704898 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:27:23.704867 2563 generic.go:356] "Generic (PLEG): container finished" podID="f704a4c0-8d5e-41bf-a173-753ac3b54ff2" containerID="3516850081a3c01738d37f22d484a7359bfd73c18a1fa00755655a3cf4f43c28" exitCode=1 Jul 14 20:27:23.705071 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:27:23.704944 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-55c6fbd889-rvg84" event={"ID":"f704a4c0-8d5e-41bf-a173-753ac3b54ff2","Type":"ContainerDied","Data":"3516850081a3c01738d37f22d484a7359bfd73c18a1fa00755655a3cf4f43c28"} Jul 14 20:27:23.705071 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:27:23.705010 2563 scope.go:117] "RemoveContainer" containerID="3f022b8700a9c3f4adfd972ece2fb409ad6640b2d25c5a7212a88e4fb036d00d" Jul 14 20:27:23.705379 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:27:23.705361 2563 scope.go:117] "RemoveContainer" containerID="3516850081a3c01738d37f22d484a7359bfd73c18a1fa00755655a3cf4f43c28" Jul 14 20:27:23.705585 ip-10-0-141-61 kubenswrapper[2563]: E0714 20:27:23.705564 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"acm-agent\" with CrashLoopBackOff: \"back-off 10s restarting failed container=acm-agent pod=klusterlet-addon-workmgr-55c6fbd889-rvg84_open-cluster-management-agent-addon(f704a4c0-8d5e-41bf-a173-753ac3b54ff2)\"" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-55c6fbd889-rvg84" podUID="f704a4c0-8d5e-41bf-a173-753ac3b54ff2" Jul 14 20:27:24.219589 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:27:24.219550 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-55c6fbd889-rvg84" Jul 14 20:27:24.710409 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:27:24.710376 2563 scope.go:117] "RemoveContainer" containerID="3516850081a3c01738d37f22d484a7359bfd73c18a1fa00755655a3cf4f43c28" Jul 14 20:27:24.710809 ip-10-0-141-61 kubenswrapper[2563]: E0714 20:27:24.710543 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"acm-agent\" with CrashLoopBackOff: \"back-off 10s restarting failed container=acm-agent pod=klusterlet-addon-workmgr-55c6fbd889-rvg84_open-cluster-management-agent-addon(f704a4c0-8d5e-41bf-a173-753ac3b54ff2)\"" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-55c6fbd889-rvg84" podUID="f704a4c0-8d5e-41bf-a173-753ac3b54ff2" Jul 14 20:27:25.260999 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:27:25.260966 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:27:25.315878 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:27:25.315845 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:27:26.038033 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:27:26.037998 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:27:26.134430 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:27:26.134390 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:27:26.178564 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:27:26.178516 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6b66589b8c-lfhd2"] Jul 14 20:27:26.281258 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:27:26.281221 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:27:26.330587 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:27:26.330485 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:27:26.424406 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:27:26.424353 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:27:26.719570 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:27:26.719538 2563 generic.go:356] "Generic (PLEG): container finished" podID="a00c85f6-2ec7-445d-8882-8c54e7ac223d" containerID="3c3cc8b980e1d25da2f1e4efed3d6da24981ebc3cbcc83a2b3503f3ec69d96f4" exitCode=0 Jul 14 20:27:26.719735 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:27:26.719607 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-dk2qr" event={"ID":"a00c85f6-2ec7-445d-8882-8c54e7ac223d","Type":"ContainerDied","Data":"3c3cc8b980e1d25da2f1e4efed3d6da24981ebc3cbcc83a2b3503f3ec69d96f4"} Jul 14 20:27:26.719924 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:27:26.719911 2563 scope.go:117] "RemoveContainer" containerID="3c3cc8b980e1d25da2f1e4efed3d6da24981ebc3cbcc83a2b3503f3ec69d96f4" Jul 14 20:27:26.747958 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:27:26.747925 2563 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-55c6fbd889-rvg84" Jul 14 20:27:26.748457 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:27:26.748438 2563 scope.go:117] "RemoveContainer" containerID="3516850081a3c01738d37f22d484a7359bfd73c18a1fa00755655a3cf4f43c28" Jul 14 20:27:26.748670 ip-10-0-141-61 kubenswrapper[2563]: E0714 20:27:26.748644 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"acm-agent\" with CrashLoopBackOff: \"back-off 10s restarting failed container=acm-agent pod=klusterlet-addon-workmgr-55c6fbd889-rvg84_open-cluster-management-agent-addon(f704a4c0-8d5e-41bf-a173-753ac3b54ff2)\"" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-55c6fbd889-rvg84" podUID="f704a4c0-8d5e-41bf-a173-753ac3b54ff2" Jul 14 20:27:26.791226 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:27:26.791189 2563 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-76dff96bd-59zm4" Jul 14 20:27:26.797560 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:27:26.797539 2563 scope.go:117] "RemoveContainer" containerID="5066e8a3a8e625031e626373303931f07a874b9ac595017817cd95eed4fbc6b8" Jul 14 20:27:26.797775 ip-10-0-141-61 kubenswrapper[2563]: E0714 20:27:26.797753 2563 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"addon-agent\" with CrashLoopBackOff: \"back-off 10s restarting failed container=addon-agent pod=managed-serviceaccount-addon-agent-76dff96bd-59zm4_open-cluster-management-agent-addon(1447f395-a017-44cf-9176-2944ead9c090)\"" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-76dff96bd-59zm4" podUID="1447f395-a017-44cf-9176-2944ead9c090" Jul 14 20:27:27.725471 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:27:27.725436 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-dk2qr" event={"ID":"a00c85f6-2ec7-445d-8882-8c54e7ac223d","Type":"ContainerStarted","Data":"465a716438250ad913674b67e5fce81c9b68154c292f4194aac9a509446298fe"} Jul 14 20:27:34.635628 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:27:34.635590 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:27:34.698666 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:27:34.698630 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:27:35.046753 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:27:35.046721 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:27:35.123681 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:27:35.123646 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:27:35.227176 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:27:35.227112 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:27:35.327459 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:27:35.327370 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:27:35.431282 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:27:35.431244 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:27:35.511033 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:27:35.511002 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:27:37.973067 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:27:37.973041 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-7cd4fc7897-d9pdf" Jul 14 20:27:37.976998 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:27:37.976976 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-7cd4fc7897-d9pdf" Jul 14 20:27:38.464209 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:27:38.464165 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:27:38.553606 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:27:38.553569 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:27:39.600872 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:27:39.600843 2563 scope.go:117] "RemoveContainer" containerID="5066e8a3a8e625031e626373303931f07a874b9ac595017817cd95eed4fbc6b8" Jul 14 20:27:39.775034 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:27:39.775004 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-76dff96bd-59zm4" event={"ID":"1447f395-a017-44cf-9176-2944ead9c090","Type":"ContainerStarted","Data":"ce4b969259c64eb00ff47f078603ea76938f8d5e4b0999bcf5beb94d30bedaf6"} Jul 14 20:27:41.604189 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:27:41.604160 2563 scope.go:117] "RemoveContainer" containerID="3516850081a3c01738d37f22d484a7359bfd73c18a1fa00755655a3cf4f43c28" Jul 14 20:27:41.785418 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:27:41.785388 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-55c6fbd889-rvg84" event={"ID":"f704a4c0-8d5e-41bf-a173-753ac3b54ff2","Type":"ContainerStarted","Data":"1e3d4c6c795b3cfa6223f526b308dce4d1ee120f4b38708bfd18f7e8a6ec6507"} Jul 14 20:27:41.785706 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:27:41.785687 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-55c6fbd889-rvg84" Jul 14 20:27:41.787375 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:27:41.787352 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-55c6fbd889-rvg84" Jul 14 20:27:43.715015 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:27:43.714977 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:27:43.778968 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:27:43.778923 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:27:46.809728 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:27:46.809699 2563 generic.go:356] "Generic (PLEG): container finished" podID="cb3ba939-0e7c-4afc-a64b-e073e5cf656c" containerID="96e471328344c4464837a0c22d7754d19a2c8dd2201fc3d7f84a447ad7a39483" exitCode=0 Jul 14 20:27:46.810108 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:27:46.809765 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-68d9dd94cf-spp52" event={"ID":"cb3ba939-0e7c-4afc-a64b-e073e5cf656c","Type":"ContainerDied","Data":"96e471328344c4464837a0c22d7754d19a2c8dd2201fc3d7f84a447ad7a39483"} Jul 14 20:27:46.810189 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:27:46.810110 2563 scope.go:117] "RemoveContainer" containerID="96e471328344c4464837a0c22d7754d19a2c8dd2201fc3d7f84a447ad7a39483" Jul 14 20:27:47.405784 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:27:47.405754 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-v2v5k_cf457900-0b78-45d7-9e9e-99d9fe24bd1c/serve-healthcheck-canary/0.log" Jul 14 20:27:47.815406 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:27:47.815321 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-68d9dd94cf-spp52" event={"ID":"cb3ba939-0e7c-4afc-a64b-e073e5cf656c","Type":"ContainerStarted","Data":"7cff7ec2a5a4e66391c7f32f6be61f5019cf923933d9ac52da1bc5f93a263258"} Jul 14 20:27:51.201013 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:27:51.200965 2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-6b66589b8c-lfhd2" podUID="3c07a280-eebb-4ab3-9381-e92e5f354120" containerName="console" containerID="cri-o://520cda8a389b0552aa11c7fc0569add29525c8a8ee56657f0129a3e311abe0d0" gracePeriod=15 Jul 14 20:27:51.461757 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:27:51.461702 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6b66589b8c-lfhd2" Jul 14 20:27:51.491244 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:27:51.491215 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3c07a280-eebb-4ab3-9381-e92e5f354120-service-ca\") pod \"3c07a280-eebb-4ab3-9381-e92e5f354120\" (UID: \"3c07a280-eebb-4ab3-9381-e92e5f354120\") " Jul 14 20:27:51.491433 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:27:51.491260 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3c07a280-eebb-4ab3-9381-e92e5f354120-oauth-serving-cert\") pod \"3c07a280-eebb-4ab3-9381-e92e5f354120\" (UID: \"3c07a280-eebb-4ab3-9381-e92e5f354120\") " Jul 14 20:27:51.491433 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:27:51.491415 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wkz58\" (UniqueName: \"kubernetes.io/projected/3c07a280-eebb-4ab3-9381-e92e5f354120-kube-api-access-wkz58\") pod \"3c07a280-eebb-4ab3-9381-e92e5f354120\" (UID: \"3c07a280-eebb-4ab3-9381-e92e5f354120\") " Jul 14 20:27:51.491552 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:27:51.491456 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3c07a280-eebb-4ab3-9381-e92e5f354120-console-config\") pod \"3c07a280-eebb-4ab3-9381-e92e5f354120\" (UID: \"3c07a280-eebb-4ab3-9381-e92e5f354120\") " Jul 14 20:27:51.491552 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:27:51.491491 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3c07a280-eebb-4ab3-9381-e92e5f354120-trusted-ca-bundle\") pod \"3c07a280-eebb-4ab3-9381-e92e5f354120\" (UID: \"3c07a280-eebb-4ab3-9381-e92e5f354120\") " Jul 14 20:27:51.491651 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:27:51.491549 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3c07a280-eebb-4ab3-9381-e92e5f354120-console-oauth-config\") pod \"3c07a280-eebb-4ab3-9381-e92e5f354120\" (UID: \"3c07a280-eebb-4ab3-9381-e92e5f354120\") " Jul 14 20:27:51.491651 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:27:51.491597 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3c07a280-eebb-4ab3-9381-e92e5f354120-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "3c07a280-eebb-4ab3-9381-e92e5f354120" (UID: "3c07a280-eebb-4ab3-9381-e92e5f354120"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 20:27:51.491651 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:27:51.491614 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3c07a280-eebb-4ab3-9381-e92e5f354120-console-serving-cert\") pod \"3c07a280-eebb-4ab3-9381-e92e5f354120\" (UID: \"3c07a280-eebb-4ab3-9381-e92e5f354120\") " Jul 14 20:27:51.491781 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:27:51.491623 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3c07a280-eebb-4ab3-9381-e92e5f354120-service-ca" (OuterVolumeSpecName: "service-ca") pod "3c07a280-eebb-4ab3-9381-e92e5f354120" (UID: "3c07a280-eebb-4ab3-9381-e92e5f354120"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 20:27:51.491851 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:27:51.491815 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3c07a280-eebb-4ab3-9381-e92e5f354120-console-config" (OuterVolumeSpecName: "console-config") pod "3c07a280-eebb-4ab3-9381-e92e5f354120" (UID: "3c07a280-eebb-4ab3-9381-e92e5f354120"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 20:27:51.491904 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:27:51.491885 2563 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3c07a280-eebb-4ab3-9381-e92e5f354120-service-ca\") on node \"ip-10-0-141-61.ec2.internal\" DevicePath \"\"" Jul 14 20:27:51.491947 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:27:51.491904 2563 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3c07a280-eebb-4ab3-9381-e92e5f354120-oauth-serving-cert\") on node \"ip-10-0-141-61.ec2.internal\" DevicePath \"\"" Jul 14 20:27:51.491947 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:27:51.491910 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3c07a280-eebb-4ab3-9381-e92e5f354120-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "3c07a280-eebb-4ab3-9381-e92e5f354120" (UID: "3c07a280-eebb-4ab3-9381-e92e5f354120"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 20:27:51.493886 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:27:51.493854 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3c07a280-eebb-4ab3-9381-e92e5f354120-kube-api-access-wkz58" (OuterVolumeSpecName: "kube-api-access-wkz58") pod "3c07a280-eebb-4ab3-9381-e92e5f354120" (UID: "3c07a280-eebb-4ab3-9381-e92e5f354120"). InnerVolumeSpecName "kube-api-access-wkz58". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 20:27:51.493999 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:27:51.493914 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3c07a280-eebb-4ab3-9381-e92e5f354120-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "3c07a280-eebb-4ab3-9381-e92e5f354120" (UID: "3c07a280-eebb-4ab3-9381-e92e5f354120"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 14 20:27:51.494176 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:27:51.494155 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3c07a280-eebb-4ab3-9381-e92e5f354120-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "3c07a280-eebb-4ab3-9381-e92e5f354120" (UID: "3c07a280-eebb-4ab3-9381-e92e5f354120"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 14 20:27:51.592378 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:27:51.592347 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wkz58\" (UniqueName: \"kubernetes.io/projected/3c07a280-eebb-4ab3-9381-e92e5f354120-kube-api-access-wkz58\") on node \"ip-10-0-141-61.ec2.internal\" DevicePath \"\"" Jul 14 20:27:51.592378 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:27:51.592379 2563 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3c07a280-eebb-4ab3-9381-e92e5f354120-console-config\") on node \"ip-10-0-141-61.ec2.internal\" DevicePath \"\"" Jul 14 20:27:51.592588 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:27:51.592392 2563 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3c07a280-eebb-4ab3-9381-e92e5f354120-trusted-ca-bundle\") on node \"ip-10-0-141-61.ec2.internal\" DevicePath \"\"" Jul 14 20:27:51.592588 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:27:51.592400 2563 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3c07a280-eebb-4ab3-9381-e92e5f354120-console-oauth-config\") on node \"ip-10-0-141-61.ec2.internal\" DevicePath \"\"" Jul 14 20:27:51.592588 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:27:51.592409 2563 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3c07a280-eebb-4ab3-9381-e92e5f354120-console-serving-cert\") on node \"ip-10-0-141-61.ec2.internal\" DevicePath \"\"" Jul 14 20:27:51.833982 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:27:51.833893 2563 generic.go:356] "Generic (PLEG): container finished" podID="3c07a280-eebb-4ab3-9381-e92e5f354120" containerID="520cda8a389b0552aa11c7fc0569add29525c8a8ee56657f0129a3e311abe0d0" exitCode=0 Jul 14 20:27:51.833982 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:27:51.833937 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6b66589b8c-lfhd2" Jul 14 20:27:51.834255 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:27:51.833982 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6b66589b8c-lfhd2" event={"ID":"3c07a280-eebb-4ab3-9381-e92e5f354120","Type":"ContainerDied","Data":"520cda8a389b0552aa11c7fc0569add29525c8a8ee56657f0129a3e311abe0d0"} Jul 14 20:27:51.834255 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:27:51.834037 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6b66589b8c-lfhd2" event={"ID":"3c07a280-eebb-4ab3-9381-e92e5f354120","Type":"ContainerDied","Data":"a9fdf37ac585f3bbff14234053d99a2c4b6d86dc69586d8923719fa61e2c354a"} Jul 14 20:27:51.834255 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:27:51.834068 2563 scope.go:117] "RemoveContainer" containerID="520cda8a389b0552aa11c7fc0569add29525c8a8ee56657f0129a3e311abe0d0" Jul 14 20:27:51.842724 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:27:51.842704 2563 scope.go:117] "RemoveContainer" containerID="520cda8a389b0552aa11c7fc0569add29525c8a8ee56657f0129a3e311abe0d0" Jul 14 20:27:51.842989 ip-10-0-141-61 kubenswrapper[2563]: E0714 20:27:51.842968 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"520cda8a389b0552aa11c7fc0569add29525c8a8ee56657f0129a3e311abe0d0\": container with ID starting with 520cda8a389b0552aa11c7fc0569add29525c8a8ee56657f0129a3e311abe0d0 not found: ID does not exist" containerID="520cda8a389b0552aa11c7fc0569add29525c8a8ee56657f0129a3e311abe0d0" Jul 14 20:27:51.843030 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:27:51.842999 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"520cda8a389b0552aa11c7fc0569add29525c8a8ee56657f0129a3e311abe0d0"} err="failed to get container status \"520cda8a389b0552aa11c7fc0569add29525c8a8ee56657f0129a3e311abe0d0\": rpc error: code = NotFound desc = could not find container \"520cda8a389b0552aa11c7fc0569add29525c8a8ee56657f0129a3e311abe0d0\": container with ID starting with 520cda8a389b0552aa11c7fc0569add29525c8a8ee56657f0129a3e311abe0d0 not found: ID does not exist" Jul 14 20:27:51.848747 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:27:51.848718 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6b66589b8c-lfhd2"] Jul 14 20:27:51.852479 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:27:51.852458 2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-6b66589b8c-lfhd2"] Jul 14 20:27:52.406972 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:27:52.406932 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:27:52.503154 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:27:52.503102 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:27:53.435319 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:27:53.435284 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:27:53.491223 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:27:53.491181 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:27:53.605538 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:27:53.605509 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3c07a280-eebb-4ab3-9381-e92e5f354120" path="/var/lib/kubelet/pods/3c07a280-eebb-4ab3-9381-e92e5f354120/volumes" Jul 14 20:27:59.753212 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:27:59.751675 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Jul 14 20:27:59.782048 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:27:59.782021 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Jul 14 20:27:59.894100 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:27:59.894072 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 14 20:28:01.220446 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:01.220416 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:28:01.321026 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:01.320992 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:28:08.443222 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:08.443176 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:28:08.543371 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:08.543337 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:28:09.301877 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:09.301840 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:28:09.403179 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:09.403148 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:28:13.343169 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:13.343066 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:28:13.398642 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:13.398604 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:28:13.562435 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:13.562398 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 14 20:28:13.562702 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:13.562681 2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="3bd29b0f-00fb-4a66-97b1-c2727db1081a" containerName="alertmanager" containerID="cri-o://35a261aa2f7135e93e4fb2fcefd57469a2a31220f895da2810e59ab0fc4d286a" gracePeriod=120 Jul 14 20:28:13.562794 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:13.562765 2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="3bd29b0f-00fb-4a66-97b1-c2727db1081a" containerName="kube-rbac-proxy-metric" containerID="cri-o://3c4a85c44a1b08c5bf049fc6aadc2d45861a137e3e03806bc44d2ad641ed0fb7" gracePeriod=120 Jul 14 20:28:13.563036 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:13.562818 2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="3bd29b0f-00fb-4a66-97b1-c2727db1081a" containerName="prom-label-proxy" containerID="cri-o://6e9731f4bbd5f4026d5ce21eba732197ff11c43392e10dd1dc9427af550d62f6" gracePeriod=120 Jul 14 20:28:13.563036 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:13.562883 2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="3bd29b0f-00fb-4a66-97b1-c2727db1081a" containerName="kube-rbac-proxy" containerID="cri-o://4b8019dccad793e1ee2952ecd03269d63392e23289012cc28525d978f2d8ef8f" gracePeriod=120 Jul 14 20:28:13.563036 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:13.562863 2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="3bd29b0f-00fb-4a66-97b1-c2727db1081a" containerName="kube-rbac-proxy-web" containerID="cri-o://11c0959594c39f4f4f97417dd5053740b7f4a6a8419a7f0df5cacce203f60277" gracePeriod=120 Jul 14 20:28:13.563036 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:13.562844 2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="3bd29b0f-00fb-4a66-97b1-c2727db1081a" containerName="config-reloader" containerID="cri-o://483313155a41d9be8f99aa2a6a7fef61ec30f2132b46e83e2eef66fd887ca6ce" gracePeriod=120 Jul 14 20:28:13.939722 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:13.939686 2563 generic.go:356] "Generic (PLEG): container finished" podID="3bd29b0f-00fb-4a66-97b1-c2727db1081a" containerID="6e9731f4bbd5f4026d5ce21eba732197ff11c43392e10dd1dc9427af550d62f6" exitCode=0 Jul 14 20:28:13.939722 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:13.939709 2563 generic.go:356] "Generic (PLEG): container finished" podID="3bd29b0f-00fb-4a66-97b1-c2727db1081a" containerID="4b8019dccad793e1ee2952ecd03269d63392e23289012cc28525d978f2d8ef8f" exitCode=0 Jul 14 20:28:13.939722 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:13.939718 2563 generic.go:356] "Generic (PLEG): container finished" podID="3bd29b0f-00fb-4a66-97b1-c2727db1081a" containerID="483313155a41d9be8f99aa2a6a7fef61ec30f2132b46e83e2eef66fd887ca6ce" exitCode=0 Jul 14 20:28:13.939722 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:13.939724 2563 generic.go:356] "Generic (PLEG): container finished" podID="3bd29b0f-00fb-4a66-97b1-c2727db1081a" containerID="35a261aa2f7135e93e4fb2fcefd57469a2a31220f895da2810e59ab0fc4d286a" exitCode=0 Jul 14 20:28:13.939975 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:13.939754 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"3bd29b0f-00fb-4a66-97b1-c2727db1081a","Type":"ContainerDied","Data":"6e9731f4bbd5f4026d5ce21eba732197ff11c43392e10dd1dc9427af550d62f6"} Jul 14 20:28:13.939975 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:13.939818 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"3bd29b0f-00fb-4a66-97b1-c2727db1081a","Type":"ContainerDied","Data":"4b8019dccad793e1ee2952ecd03269d63392e23289012cc28525d978f2d8ef8f"} Jul 14 20:28:13.939975 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:13.939847 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"3bd29b0f-00fb-4a66-97b1-c2727db1081a","Type":"ContainerDied","Data":"483313155a41d9be8f99aa2a6a7fef61ec30f2132b46e83e2eef66fd887ca6ce"} Jul 14 20:28:13.939975 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:13.939871 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"3bd29b0f-00fb-4a66-97b1-c2727db1081a","Type":"ContainerDied","Data":"35a261aa2f7135e93e4fb2fcefd57469a2a31220f895da2810e59ab0fc4d286a"} Jul 14 20:28:14.815900 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:14.815871 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 14 20:28:14.908767 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:14.908741 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/3bd29b0f-00fb-4a66-97b1-c2727db1081a-config-out\") pod \"3bd29b0f-00fb-4a66-97b1-c2727db1081a\" (UID: \"3bd29b0f-00fb-4a66-97b1-c2727db1081a\") " Jul 14 20:28:14.908933 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:14.908783 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/3bd29b0f-00fb-4a66-97b1-c2727db1081a-config-volume\") pod \"3bd29b0f-00fb-4a66-97b1-c2727db1081a\" (UID: \"3bd29b0f-00fb-4a66-97b1-c2727db1081a\") " Jul 14 20:28:14.908933 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:14.908815 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/3bd29b0f-00fb-4a66-97b1-c2727db1081a-secret-alertmanager-main-tls\") pod \"3bd29b0f-00fb-4a66-97b1-c2727db1081a\" (UID: \"3bd29b0f-00fb-4a66-97b1-c2727db1081a\") " Jul 14 20:28:14.908933 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:14.908842 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/3bd29b0f-00fb-4a66-97b1-c2727db1081a-secret-alertmanager-kube-rbac-proxy\") pod \"3bd29b0f-00fb-4a66-97b1-c2727db1081a\" (UID: \"3bd29b0f-00fb-4a66-97b1-c2727db1081a\") " Jul 14 20:28:14.909118 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:14.908932 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/3bd29b0f-00fb-4a66-97b1-c2727db1081a-secret-alertmanager-kube-rbac-proxy-metric\") pod \"3bd29b0f-00fb-4a66-97b1-c2727db1081a\" (UID: \"3bd29b0f-00fb-4a66-97b1-c2727db1081a\") " Jul 14 20:28:14.909118 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:14.908984 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5lhc5\" (UniqueName: \"kubernetes.io/projected/3bd29b0f-00fb-4a66-97b1-c2727db1081a-kube-api-access-5lhc5\") pod \"3bd29b0f-00fb-4a66-97b1-c2727db1081a\" (UID: \"3bd29b0f-00fb-4a66-97b1-c2727db1081a\") " Jul 14 20:28:14.909118 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:14.909019 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/3bd29b0f-00fb-4a66-97b1-c2727db1081a-tls-assets\") pod \"3bd29b0f-00fb-4a66-97b1-c2727db1081a\" (UID: \"3bd29b0f-00fb-4a66-97b1-c2727db1081a\") " Jul 14 20:28:14.909118 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:14.909048 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3bd29b0f-00fb-4a66-97b1-c2727db1081a-alertmanager-trusted-ca-bundle\") pod \"3bd29b0f-00fb-4a66-97b1-c2727db1081a\" (UID: \"3bd29b0f-00fb-4a66-97b1-c2727db1081a\") " Jul 14 20:28:14.909118 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:14.909091 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3bd29b0f-00fb-4a66-97b1-c2727db1081a-metrics-client-ca\") pod \"3bd29b0f-00fb-4a66-97b1-c2727db1081a\" (UID: \"3bd29b0f-00fb-4a66-97b1-c2727db1081a\") " Jul 14 20:28:14.909388 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:14.909152 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/3bd29b0f-00fb-4a66-97b1-c2727db1081a-web-config\") pod \"3bd29b0f-00fb-4a66-97b1-c2727db1081a\" (UID: \"3bd29b0f-00fb-4a66-97b1-c2727db1081a\") " Jul 14 20:28:14.909388 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:14.909176 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/3bd29b0f-00fb-4a66-97b1-c2727db1081a-secret-alertmanager-kube-rbac-proxy-web\") pod \"3bd29b0f-00fb-4a66-97b1-c2727db1081a\" (UID: \"3bd29b0f-00fb-4a66-97b1-c2727db1081a\") " Jul 14 20:28:14.909388 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:14.909208 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/3bd29b0f-00fb-4a66-97b1-c2727db1081a-alertmanager-main-db\") pod \"3bd29b0f-00fb-4a66-97b1-c2727db1081a\" (UID: \"3bd29b0f-00fb-4a66-97b1-c2727db1081a\") " Jul 14 20:28:14.909388 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:14.909375 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/3bd29b0f-00fb-4a66-97b1-c2727db1081a-cluster-tls-config\") pod \"3bd29b0f-00fb-4a66-97b1-c2727db1081a\" (UID: \"3bd29b0f-00fb-4a66-97b1-c2727db1081a\") " Jul 14 20:28:14.910839 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:14.910797 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3bd29b0f-00fb-4a66-97b1-c2727db1081a-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "3bd29b0f-00fb-4a66-97b1-c2727db1081a" (UID: "3bd29b0f-00fb-4a66-97b1-c2727db1081a"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 20:28:14.911226 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:14.911196 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3bd29b0f-00fb-4a66-97b1-c2727db1081a-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "3bd29b0f-00fb-4a66-97b1-c2727db1081a" (UID: "3bd29b0f-00fb-4a66-97b1-c2727db1081a"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 20:28:14.911425 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:14.911402 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3bd29b0f-00fb-4a66-97b1-c2727db1081a-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "3bd29b0f-00fb-4a66-97b1-c2727db1081a" (UID: "3bd29b0f-00fb-4a66-97b1-c2727db1081a"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 20:28:14.912186 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:14.912115 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3bd29b0f-00fb-4a66-97b1-c2727db1081a-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "3bd29b0f-00fb-4a66-97b1-c2727db1081a" (UID: "3bd29b0f-00fb-4a66-97b1-c2727db1081a"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 14 20:28:14.912444 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:14.912363 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3bd29b0f-00fb-4a66-97b1-c2727db1081a-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "3bd29b0f-00fb-4a66-97b1-c2727db1081a" (UID: "3bd29b0f-00fb-4a66-97b1-c2727db1081a"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 14 20:28:14.912525 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:14.912465 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3bd29b0f-00fb-4a66-97b1-c2727db1081a-config-volume" (OuterVolumeSpecName: "config-volume") pod "3bd29b0f-00fb-4a66-97b1-c2727db1081a" (UID: "3bd29b0f-00fb-4a66-97b1-c2727db1081a"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 14 20:28:14.913068 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:14.913044 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3bd29b0f-00fb-4a66-97b1-c2727db1081a-kube-api-access-5lhc5" (OuterVolumeSpecName: "kube-api-access-5lhc5") pod "3bd29b0f-00fb-4a66-97b1-c2727db1081a" (UID: "3bd29b0f-00fb-4a66-97b1-c2727db1081a"). InnerVolumeSpecName "kube-api-access-5lhc5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 20:28:14.913330 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:14.913306 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3bd29b0f-00fb-4a66-97b1-c2727db1081a-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "3bd29b0f-00fb-4a66-97b1-c2727db1081a" (UID: "3bd29b0f-00fb-4a66-97b1-c2727db1081a"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 20:28:14.913405 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:14.913384 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3bd29b0f-00fb-4a66-97b1-c2727db1081a-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "3bd29b0f-00fb-4a66-97b1-c2727db1081a" (UID: "3bd29b0f-00fb-4a66-97b1-c2727db1081a"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 14 20:28:14.913539 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:14.913522 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3bd29b0f-00fb-4a66-97b1-c2727db1081a-config-out" (OuterVolumeSpecName: "config-out") pod "3bd29b0f-00fb-4a66-97b1-c2727db1081a" (UID: "3bd29b0f-00fb-4a66-97b1-c2727db1081a"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 20:28:14.913746 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:14.913730 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3bd29b0f-00fb-4a66-97b1-c2727db1081a-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "3bd29b0f-00fb-4a66-97b1-c2727db1081a" (UID: "3bd29b0f-00fb-4a66-97b1-c2727db1081a"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 14 20:28:14.918278 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:14.918247 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3bd29b0f-00fb-4a66-97b1-c2727db1081a-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "3bd29b0f-00fb-4a66-97b1-c2727db1081a" (UID: "3bd29b0f-00fb-4a66-97b1-c2727db1081a"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 14 20:28:14.924309 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:14.924286 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3bd29b0f-00fb-4a66-97b1-c2727db1081a-web-config" (OuterVolumeSpecName: "web-config") pod "3bd29b0f-00fb-4a66-97b1-c2727db1081a" (UID: "3bd29b0f-00fb-4a66-97b1-c2727db1081a"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 14 20:28:14.947214 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:14.947187 2563 generic.go:356] "Generic (PLEG): container finished" podID="3bd29b0f-00fb-4a66-97b1-c2727db1081a" containerID="3c4a85c44a1b08c5bf049fc6aadc2d45861a137e3e03806bc44d2ad641ed0fb7" exitCode=0 Jul 14 20:28:14.947214 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:14.947211 2563 generic.go:356] "Generic (PLEG): container finished" podID="3bd29b0f-00fb-4a66-97b1-c2727db1081a" containerID="11c0959594c39f4f4f97417dd5053740b7f4a6a8419a7f0df5cacce203f60277" exitCode=0 Jul 14 20:28:14.947393 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:14.947247 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"3bd29b0f-00fb-4a66-97b1-c2727db1081a","Type":"ContainerDied","Data":"3c4a85c44a1b08c5bf049fc6aadc2d45861a137e3e03806bc44d2ad641ed0fb7"} Jul 14 20:28:14.947393 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:14.947259 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 14 20:28:14.947393 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:14.947305 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"3bd29b0f-00fb-4a66-97b1-c2727db1081a","Type":"ContainerDied","Data":"11c0959594c39f4f4f97417dd5053740b7f4a6a8419a7f0df5cacce203f60277"} Jul 14 20:28:14.947393 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:14.947331 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"3bd29b0f-00fb-4a66-97b1-c2727db1081a","Type":"ContainerDied","Data":"d39d8be4eeaf5af406d68038d3375a731af6b28e59bc607dd6f3fe96fd14c18f"} Jul 14 20:28:14.947393 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:14.947361 2563 scope.go:117] "RemoveContainer" containerID="6e9731f4bbd5f4026d5ce21eba732197ff11c43392e10dd1dc9427af550d62f6" Jul 14 20:28:14.955466 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:14.955399 2563 scope.go:117] "RemoveContainer" containerID="3c4a85c44a1b08c5bf049fc6aadc2d45861a137e3e03806bc44d2ad641ed0fb7" Jul 14 20:28:14.962904 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:14.962887 2563 scope.go:117] "RemoveContainer" containerID="4b8019dccad793e1ee2952ecd03269d63392e23289012cc28525d978f2d8ef8f" Jul 14 20:28:14.971481 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:14.971347 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 14 20:28:14.971556 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:14.971540 2563 scope.go:117] "RemoveContainer" containerID="11c0959594c39f4f4f97417dd5053740b7f4a6a8419a7f0df5cacce203f60277" Jul 14 20:28:14.973655 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:14.973635 2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 14 20:28:14.979473 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:14.979451 2563 scope.go:117] "RemoveContainer" containerID="483313155a41d9be8f99aa2a6a7fef61ec30f2132b46e83e2eef66fd887ca6ce" Jul 14 20:28:14.988300 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:14.988281 2563 scope.go:117] "RemoveContainer" containerID="35a261aa2f7135e93e4fb2fcefd57469a2a31220f895da2810e59ab0fc4d286a" Jul 14 20:28:14.996323 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:14.996304 2563 scope.go:117] "RemoveContainer" containerID="a08f5ebe06e3cef45846ab4fa123374cc48127e25dc61b5836b6d0a5e0c4c121" Jul 14 20:28:15.004078 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:15.004057 2563 scope.go:117] "RemoveContainer" containerID="6e9731f4bbd5f4026d5ce21eba732197ff11c43392e10dd1dc9427af550d62f6" Jul 14 20:28:15.004419 ip-10-0-141-61 kubenswrapper[2563]: E0714 20:28:15.004397 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6e9731f4bbd5f4026d5ce21eba732197ff11c43392e10dd1dc9427af550d62f6\": container with ID starting with 6e9731f4bbd5f4026d5ce21eba732197ff11c43392e10dd1dc9427af550d62f6 not found: ID does not exist" containerID="6e9731f4bbd5f4026d5ce21eba732197ff11c43392e10dd1dc9427af550d62f6" Jul 14 20:28:15.004486 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:15.004428 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6e9731f4bbd5f4026d5ce21eba732197ff11c43392e10dd1dc9427af550d62f6"} err="failed to get container status \"6e9731f4bbd5f4026d5ce21eba732197ff11c43392e10dd1dc9427af550d62f6\": rpc error: code = NotFound desc = could not find container \"6e9731f4bbd5f4026d5ce21eba732197ff11c43392e10dd1dc9427af550d62f6\": container with ID starting with 6e9731f4bbd5f4026d5ce21eba732197ff11c43392e10dd1dc9427af550d62f6 not found: ID does not exist" Jul 14 20:28:15.004486 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:15.004454 2563 scope.go:117] "RemoveContainer" containerID="3c4a85c44a1b08c5bf049fc6aadc2d45861a137e3e03806bc44d2ad641ed0fb7" Jul 14 20:28:15.004725 ip-10-0-141-61 kubenswrapper[2563]: E0714 20:28:15.004710 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3c4a85c44a1b08c5bf049fc6aadc2d45861a137e3e03806bc44d2ad641ed0fb7\": container with ID starting with 3c4a85c44a1b08c5bf049fc6aadc2d45861a137e3e03806bc44d2ad641ed0fb7 not found: ID does not exist" containerID="3c4a85c44a1b08c5bf049fc6aadc2d45861a137e3e03806bc44d2ad641ed0fb7" Jul 14 20:28:15.004766 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:15.004730 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3c4a85c44a1b08c5bf049fc6aadc2d45861a137e3e03806bc44d2ad641ed0fb7"} err="failed to get container status \"3c4a85c44a1b08c5bf049fc6aadc2d45861a137e3e03806bc44d2ad641ed0fb7\": rpc error: code = NotFound desc = could not find container \"3c4a85c44a1b08c5bf049fc6aadc2d45861a137e3e03806bc44d2ad641ed0fb7\": container with ID starting with 3c4a85c44a1b08c5bf049fc6aadc2d45861a137e3e03806bc44d2ad641ed0fb7 not found: ID does not exist" Jul 14 20:28:15.004766 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:15.004745 2563 scope.go:117] "RemoveContainer" containerID="4b8019dccad793e1ee2952ecd03269d63392e23289012cc28525d978f2d8ef8f" Jul 14 20:28:15.004989 ip-10-0-141-61 kubenswrapper[2563]: E0714 20:28:15.004970 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4b8019dccad793e1ee2952ecd03269d63392e23289012cc28525d978f2d8ef8f\": container with ID starting with 4b8019dccad793e1ee2952ecd03269d63392e23289012cc28525d978f2d8ef8f not found: ID does not exist" containerID="4b8019dccad793e1ee2952ecd03269d63392e23289012cc28525d978f2d8ef8f" Jul 14 20:28:15.005076 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:15.004993 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4b8019dccad793e1ee2952ecd03269d63392e23289012cc28525d978f2d8ef8f"} err="failed to get container status \"4b8019dccad793e1ee2952ecd03269d63392e23289012cc28525d978f2d8ef8f\": rpc error: code = NotFound desc = could not find container \"4b8019dccad793e1ee2952ecd03269d63392e23289012cc28525d978f2d8ef8f\": container with ID starting with 4b8019dccad793e1ee2952ecd03269d63392e23289012cc28525d978f2d8ef8f not found: ID does not exist" Jul 14 20:28:15.005076 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:15.005007 2563 scope.go:117] "RemoveContainer" containerID="11c0959594c39f4f4f97417dd5053740b7f4a6a8419a7f0df5cacce203f60277" Jul 14 20:28:15.005368 ip-10-0-141-61 kubenswrapper[2563]: E0714 20:28:15.005342 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"11c0959594c39f4f4f97417dd5053740b7f4a6a8419a7f0df5cacce203f60277\": container with ID starting with 11c0959594c39f4f4f97417dd5053740b7f4a6a8419a7f0df5cacce203f60277 not found: ID does not exist" containerID="11c0959594c39f4f4f97417dd5053740b7f4a6a8419a7f0df5cacce203f60277" Jul 14 20:28:15.005411 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:15.005375 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"11c0959594c39f4f4f97417dd5053740b7f4a6a8419a7f0df5cacce203f60277"} err="failed to get container status \"11c0959594c39f4f4f97417dd5053740b7f4a6a8419a7f0df5cacce203f60277\": rpc error: code = NotFound desc = could not find container \"11c0959594c39f4f4f97417dd5053740b7f4a6a8419a7f0df5cacce203f60277\": container with ID starting with 11c0959594c39f4f4f97417dd5053740b7f4a6a8419a7f0df5cacce203f60277 not found: ID does not exist" Jul 14 20:28:15.005411 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:15.005390 2563 scope.go:117] "RemoveContainer" containerID="483313155a41d9be8f99aa2a6a7fef61ec30f2132b46e83e2eef66fd887ca6ce" Jul 14 20:28:15.005661 ip-10-0-141-61 kubenswrapper[2563]: E0714 20:28:15.005642 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"483313155a41d9be8f99aa2a6a7fef61ec30f2132b46e83e2eef66fd887ca6ce\": container with ID starting with 483313155a41d9be8f99aa2a6a7fef61ec30f2132b46e83e2eef66fd887ca6ce not found: ID does not exist" containerID="483313155a41d9be8f99aa2a6a7fef61ec30f2132b46e83e2eef66fd887ca6ce" Jul 14 20:28:15.005718 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:15.005666 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"483313155a41d9be8f99aa2a6a7fef61ec30f2132b46e83e2eef66fd887ca6ce"} err="failed to get container status \"483313155a41d9be8f99aa2a6a7fef61ec30f2132b46e83e2eef66fd887ca6ce\": rpc error: code = NotFound desc = could not find container \"483313155a41d9be8f99aa2a6a7fef61ec30f2132b46e83e2eef66fd887ca6ce\": container with ID starting with 483313155a41d9be8f99aa2a6a7fef61ec30f2132b46e83e2eef66fd887ca6ce not found: ID does not exist" Jul 14 20:28:15.005718 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:15.005682 2563 scope.go:117] "RemoveContainer" containerID="35a261aa2f7135e93e4fb2fcefd57469a2a31220f895da2810e59ab0fc4d286a" Jul 14 20:28:15.005972 ip-10-0-141-61 kubenswrapper[2563]: E0714 20:28:15.005953 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"35a261aa2f7135e93e4fb2fcefd57469a2a31220f895da2810e59ab0fc4d286a\": container with ID starting with 35a261aa2f7135e93e4fb2fcefd57469a2a31220f895da2810e59ab0fc4d286a not found: ID does not exist" containerID="35a261aa2f7135e93e4fb2fcefd57469a2a31220f895da2810e59ab0fc4d286a" Jul 14 20:28:15.006209 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:15.005975 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"35a261aa2f7135e93e4fb2fcefd57469a2a31220f895da2810e59ab0fc4d286a"} err="failed to get container status \"35a261aa2f7135e93e4fb2fcefd57469a2a31220f895da2810e59ab0fc4d286a\": rpc error: code = NotFound desc = could not find container \"35a261aa2f7135e93e4fb2fcefd57469a2a31220f895da2810e59ab0fc4d286a\": container with ID starting with 35a261aa2f7135e93e4fb2fcefd57469a2a31220f895da2810e59ab0fc4d286a not found: ID does not exist" Jul 14 20:28:15.006209 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:15.005989 2563 scope.go:117] "RemoveContainer" containerID="a08f5ebe06e3cef45846ab4fa123374cc48127e25dc61b5836b6d0a5e0c4c121" Jul 14 20:28:15.006354 ip-10-0-141-61 kubenswrapper[2563]: E0714 20:28:15.006226 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a08f5ebe06e3cef45846ab4fa123374cc48127e25dc61b5836b6d0a5e0c4c121\": container with ID starting with a08f5ebe06e3cef45846ab4fa123374cc48127e25dc61b5836b6d0a5e0c4c121 not found: ID does not exist" containerID="a08f5ebe06e3cef45846ab4fa123374cc48127e25dc61b5836b6d0a5e0c4c121" Jul 14 20:28:15.006354 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:15.006247 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a08f5ebe06e3cef45846ab4fa123374cc48127e25dc61b5836b6d0a5e0c4c121"} err="failed to get container status \"a08f5ebe06e3cef45846ab4fa123374cc48127e25dc61b5836b6d0a5e0c4c121\": rpc error: code = NotFound desc = could not find container \"a08f5ebe06e3cef45846ab4fa123374cc48127e25dc61b5836b6d0a5e0c4c121\": container with ID starting with a08f5ebe06e3cef45846ab4fa123374cc48127e25dc61b5836b6d0a5e0c4c121 not found: ID does not exist" Jul 14 20:28:15.006354 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:15.006260 2563 scope.go:117] "RemoveContainer" containerID="6e9731f4bbd5f4026d5ce21eba732197ff11c43392e10dd1dc9427af550d62f6" Jul 14 20:28:15.006487 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:15.006469 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6e9731f4bbd5f4026d5ce21eba732197ff11c43392e10dd1dc9427af550d62f6"} err="failed to get container status \"6e9731f4bbd5f4026d5ce21eba732197ff11c43392e10dd1dc9427af550d62f6\": rpc error: code = NotFound desc = could not find container \"6e9731f4bbd5f4026d5ce21eba732197ff11c43392e10dd1dc9427af550d62f6\": container with ID starting with 6e9731f4bbd5f4026d5ce21eba732197ff11c43392e10dd1dc9427af550d62f6 not found: ID does not exist" Jul 14 20:28:15.006522 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:15.006487 2563 scope.go:117] "RemoveContainer" containerID="3c4a85c44a1b08c5bf049fc6aadc2d45861a137e3e03806bc44d2ad641ed0fb7" Jul 14 20:28:15.006707 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:15.006680 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3c4a85c44a1b08c5bf049fc6aadc2d45861a137e3e03806bc44d2ad641ed0fb7"} err="failed to get container status \"3c4a85c44a1b08c5bf049fc6aadc2d45861a137e3e03806bc44d2ad641ed0fb7\": rpc error: code = NotFound desc = could not find container \"3c4a85c44a1b08c5bf049fc6aadc2d45861a137e3e03806bc44d2ad641ed0fb7\": container with ID starting with 3c4a85c44a1b08c5bf049fc6aadc2d45861a137e3e03806bc44d2ad641ed0fb7 not found: ID does not exist" Jul 14 20:28:15.006791 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:15.006710 2563 scope.go:117] "RemoveContainer" containerID="4b8019dccad793e1ee2952ecd03269d63392e23289012cc28525d978f2d8ef8f" Jul 14 20:28:15.006948 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:15.006923 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4b8019dccad793e1ee2952ecd03269d63392e23289012cc28525d978f2d8ef8f"} err="failed to get container status \"4b8019dccad793e1ee2952ecd03269d63392e23289012cc28525d978f2d8ef8f\": rpc error: code = NotFound desc = could not find container \"4b8019dccad793e1ee2952ecd03269d63392e23289012cc28525d978f2d8ef8f\": container with ID starting with 4b8019dccad793e1ee2952ecd03269d63392e23289012cc28525d978f2d8ef8f not found: ID does not exist" Jul 14 20:28:15.007031 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:15.006950 2563 scope.go:117] "RemoveContainer" containerID="11c0959594c39f4f4f97417dd5053740b7f4a6a8419a7f0df5cacce203f60277" Jul 14 20:28:15.007239 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:15.007221 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"11c0959594c39f4f4f97417dd5053740b7f4a6a8419a7f0df5cacce203f60277"} err="failed to get container status \"11c0959594c39f4f4f97417dd5053740b7f4a6a8419a7f0df5cacce203f60277\": rpc error: code = NotFound desc = could not find container \"11c0959594c39f4f4f97417dd5053740b7f4a6a8419a7f0df5cacce203f60277\": container with ID starting with 11c0959594c39f4f4f97417dd5053740b7f4a6a8419a7f0df5cacce203f60277 not found: ID does not exist" Jul 14 20:28:15.007332 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:15.007241 2563 scope.go:117] "RemoveContainer" containerID="483313155a41d9be8f99aa2a6a7fef61ec30f2132b46e83e2eef66fd887ca6ce" Jul 14 20:28:15.007501 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:15.007474 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"483313155a41d9be8f99aa2a6a7fef61ec30f2132b46e83e2eef66fd887ca6ce"} err="failed to get container status \"483313155a41d9be8f99aa2a6a7fef61ec30f2132b46e83e2eef66fd887ca6ce\": rpc error: code = NotFound desc = could not find container \"483313155a41d9be8f99aa2a6a7fef61ec30f2132b46e83e2eef66fd887ca6ce\": container with ID starting with 483313155a41d9be8f99aa2a6a7fef61ec30f2132b46e83e2eef66fd887ca6ce not found: ID does not exist" Jul 14 20:28:15.007501 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:15.007493 2563 scope.go:117] "RemoveContainer" containerID="35a261aa2f7135e93e4fb2fcefd57469a2a31220f895da2810e59ab0fc4d286a" Jul 14 20:28:15.007744 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:15.007716 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"35a261aa2f7135e93e4fb2fcefd57469a2a31220f895da2810e59ab0fc4d286a"} err="failed to get container status \"35a261aa2f7135e93e4fb2fcefd57469a2a31220f895da2810e59ab0fc4d286a\": rpc error: code = NotFound desc = could not find container \"35a261aa2f7135e93e4fb2fcefd57469a2a31220f895da2810e59ab0fc4d286a\": container with ID starting with 35a261aa2f7135e93e4fb2fcefd57469a2a31220f895da2810e59ab0fc4d286a not found: ID does not exist" Jul 14 20:28:15.007811 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:15.007743 2563 scope.go:117] "RemoveContainer" containerID="a08f5ebe06e3cef45846ab4fa123374cc48127e25dc61b5836b6d0a5e0c4c121" Jul 14 20:28:15.008022 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:15.007998 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a08f5ebe06e3cef45846ab4fa123374cc48127e25dc61b5836b6d0a5e0c4c121"} err="failed to get container status \"a08f5ebe06e3cef45846ab4fa123374cc48127e25dc61b5836b6d0a5e0c4c121\": rpc error: code = NotFound desc = could not find container \"a08f5ebe06e3cef45846ab4fa123374cc48127e25dc61b5836b6d0a5e0c4c121\": container with ID starting with a08f5ebe06e3cef45846ab4fa123374cc48127e25dc61b5836b6d0a5e0c4c121 not found: ID does not exist" Jul 14 20:28:15.010519 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:15.010503 2563 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/3bd29b0f-00fb-4a66-97b1-c2727db1081a-config-out\") on node \"ip-10-0-141-61.ec2.internal\" DevicePath \"\"" Jul 14 20:28:15.010571 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:15.010522 2563 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/3bd29b0f-00fb-4a66-97b1-c2727db1081a-config-volume\") on node \"ip-10-0-141-61.ec2.internal\" DevicePath \"\"" Jul 14 20:28:15.010571 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:15.010532 2563 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/3bd29b0f-00fb-4a66-97b1-c2727db1081a-secret-alertmanager-main-tls\") on node \"ip-10-0-141-61.ec2.internal\" DevicePath \"\"" Jul 14 20:28:15.010571 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:15.010541 2563 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/3bd29b0f-00fb-4a66-97b1-c2727db1081a-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-141-61.ec2.internal\" DevicePath \"\"" Jul 14 20:28:15.010571 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:15.010551 2563 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/3bd29b0f-00fb-4a66-97b1-c2727db1081a-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-141-61.ec2.internal\" DevicePath \"\"" Jul 14 20:28:15.010571 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:15.010559 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5lhc5\" (UniqueName: \"kubernetes.io/projected/3bd29b0f-00fb-4a66-97b1-c2727db1081a-kube-api-access-5lhc5\") on node \"ip-10-0-141-61.ec2.internal\" DevicePath \"\"" Jul 14 20:28:15.010571 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:15.010567 2563 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/3bd29b0f-00fb-4a66-97b1-c2727db1081a-tls-assets\") on node \"ip-10-0-141-61.ec2.internal\" DevicePath \"\"" Jul 14 20:28:15.010739 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:15.010576 2563 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3bd29b0f-00fb-4a66-97b1-c2727db1081a-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-141-61.ec2.internal\" DevicePath \"\"" Jul 14 20:28:15.010739 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:15.010585 2563 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3bd29b0f-00fb-4a66-97b1-c2727db1081a-metrics-client-ca\") on node \"ip-10-0-141-61.ec2.internal\" DevicePath \"\"" Jul 14 20:28:15.010739 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:15.010594 2563 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/3bd29b0f-00fb-4a66-97b1-c2727db1081a-web-config\") on node \"ip-10-0-141-61.ec2.internal\" DevicePath \"\"" Jul 14 20:28:15.010739 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:15.010602 2563 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/3bd29b0f-00fb-4a66-97b1-c2727db1081a-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-141-61.ec2.internal\" DevicePath \"\"" Jul 14 20:28:15.010739 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:15.010610 2563 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/3bd29b0f-00fb-4a66-97b1-c2727db1081a-alertmanager-main-db\") on node \"ip-10-0-141-61.ec2.internal\" DevicePath \"\"" Jul 14 20:28:15.010739 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:15.010618 2563 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/3bd29b0f-00fb-4a66-97b1-c2727db1081a-cluster-tls-config\") on node \"ip-10-0-141-61.ec2.internal\" DevicePath \"\"" Jul 14 20:28:15.605388 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:15.605355 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3bd29b0f-00fb-4a66-97b1-c2727db1081a" path="/var/lib/kubelet/pods/3bd29b0f-00fb-4a66-97b1-c2727db1081a/volumes" Jul 14 20:28:17.790558 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:17.790504 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 14 20:28:17.791083 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:17.790839 2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="28bc4a8b-af34-47be-bad7-d9be1b2c75b9" containerName="prometheus" containerID="cri-o://35634f462d91ad1ce335dab920d61b1955698fc0fb19733fe2ec61d875b3d863" gracePeriod=600 Jul 14 20:28:17.791083 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:17.790906 2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="28bc4a8b-af34-47be-bad7-d9be1b2c75b9" containerName="kube-rbac-proxy-web" containerID="cri-o://4b3681e1a826480c6600fe6870249377e43a786d06fdea66e63271da33d55978" gracePeriod=600 Jul 14 20:28:17.791083 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:17.790968 2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="28bc4a8b-af34-47be-bad7-d9be1b2c75b9" containerName="thanos-sidecar" containerID="cri-o://42ce2ee737882fd5ac081570f9978e43245de9b635cab2251685b03cfe0797c7" gracePeriod=600 Jul 14 20:28:17.791083 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:17.791033 2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="28bc4a8b-af34-47be-bad7-d9be1b2c75b9" containerName="config-reloader" containerID="cri-o://6ce246e2ae983de72d36f83644143a417f38b98438e379236f09b2fe8bf65cda" gracePeriod=600 Jul 14 20:28:17.791083 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:17.790972 2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="28bc4a8b-af34-47be-bad7-d9be1b2c75b9" containerName="kube-rbac-proxy-thanos" containerID="cri-o://59bf3096727c83d6b45956c7830bc808a86f0dd745e2db10201564ef21a1d893" gracePeriod=600 Jul 14 20:28:17.791083 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:17.791065 2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="28bc4a8b-af34-47be-bad7-d9be1b2c75b9" containerName="kube-rbac-proxy" containerID="cri-o://56cc1c7f62d121733fe4626b1ef37b96de3fccad3cd22d368e4215ecfc06a69e" gracePeriod=600 Jul 14 20:28:17.963932 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:17.963898 2563 generic.go:356] "Generic (PLEG): container finished" podID="28bc4a8b-af34-47be-bad7-d9be1b2c75b9" containerID="59bf3096727c83d6b45956c7830bc808a86f0dd745e2db10201564ef21a1d893" exitCode=0 Jul 14 20:28:17.963932 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:17.963922 2563 generic.go:356] "Generic (PLEG): container finished" podID="28bc4a8b-af34-47be-bad7-d9be1b2c75b9" containerID="56cc1c7f62d121733fe4626b1ef37b96de3fccad3cd22d368e4215ecfc06a69e" exitCode=0 Jul 14 20:28:17.963932 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:17.963928 2563 generic.go:356] "Generic (PLEG): container finished" podID="28bc4a8b-af34-47be-bad7-d9be1b2c75b9" containerID="42ce2ee737882fd5ac081570f9978e43245de9b635cab2251685b03cfe0797c7" exitCode=0 Jul 14 20:28:17.963932 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:17.963934 2563 generic.go:356] "Generic (PLEG): container finished" podID="28bc4a8b-af34-47be-bad7-d9be1b2c75b9" containerID="6ce246e2ae983de72d36f83644143a417f38b98438e379236f09b2fe8bf65cda" exitCode=0 Jul 14 20:28:17.963932 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:17.963939 2563 generic.go:356] "Generic (PLEG): container finished" podID="28bc4a8b-af34-47be-bad7-d9be1b2c75b9" containerID="35634f462d91ad1ce335dab920d61b1955698fc0fb19733fe2ec61d875b3d863" exitCode=0 Jul 14 20:28:17.964273 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:17.963977 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"28bc4a8b-af34-47be-bad7-d9be1b2c75b9","Type":"ContainerDied","Data":"59bf3096727c83d6b45956c7830bc808a86f0dd745e2db10201564ef21a1d893"} Jul 14 20:28:17.964273 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:17.964050 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"28bc4a8b-af34-47be-bad7-d9be1b2c75b9","Type":"ContainerDied","Data":"56cc1c7f62d121733fe4626b1ef37b96de3fccad3cd22d368e4215ecfc06a69e"} Jul 14 20:28:17.964273 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:17.964078 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"28bc4a8b-af34-47be-bad7-d9be1b2c75b9","Type":"ContainerDied","Data":"42ce2ee737882fd5ac081570f9978e43245de9b635cab2251685b03cfe0797c7"} Jul 14 20:28:17.964273 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:17.964111 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"28bc4a8b-af34-47be-bad7-d9be1b2c75b9","Type":"ContainerDied","Data":"6ce246e2ae983de72d36f83644143a417f38b98438e379236f09b2fe8bf65cda"} Jul 14 20:28:17.964273 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:17.964200 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"28bc4a8b-af34-47be-bad7-d9be1b2c75b9","Type":"ContainerDied","Data":"35634f462d91ad1ce335dab920d61b1955698fc0fb19733fe2ec61d875b3d863"} Jul 14 20:28:18.971917 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:18.971889 2563 generic.go:356] "Generic (PLEG): container finished" podID="28bc4a8b-af34-47be-bad7-d9be1b2c75b9" containerID="4b3681e1a826480c6600fe6870249377e43a786d06fdea66e63271da33d55978" exitCode=0 Jul 14 20:28:18.972285 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:18.971924 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"28bc4a8b-af34-47be-bad7-d9be1b2c75b9","Type":"ContainerDied","Data":"4b3681e1a826480c6600fe6870249377e43a786d06fdea66e63271da33d55978"} Jul 14 20:28:19.061000 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:19.060972 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 14 20:28:19.152303 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:19.152215 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/28bc4a8b-af34-47be-bad7-d9be1b2c75b9-tls-assets\") pod \"28bc4a8b-af34-47be-bad7-d9be1b2c75b9\" (UID: \"28bc4a8b-af34-47be-bad7-d9be1b2c75b9\") " Jul 14 20:28:19.152303 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:19.152261 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/28bc4a8b-af34-47be-bad7-d9be1b2c75b9-configmap-metrics-client-ca\") pod \"28bc4a8b-af34-47be-bad7-d9be1b2c75b9\" (UID: \"28bc4a8b-af34-47be-bad7-d9be1b2c75b9\") " Jul 14 20:28:19.152303 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:19.152283 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/28bc4a8b-af34-47be-bad7-d9be1b2c75b9-prometheus-k8s-rulefiles-0\") pod \"28bc4a8b-af34-47be-bad7-d9be1b2c75b9\" (UID: \"28bc4a8b-af34-47be-bad7-d9be1b2c75b9\") " Jul 14 20:28:19.152303 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:19.152302 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/28bc4a8b-af34-47be-bad7-d9be1b2c75b9-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"28bc4a8b-af34-47be-bad7-d9be1b2c75b9\" (UID: \"28bc4a8b-af34-47be-bad7-d9be1b2c75b9\") " Jul 14 20:28:19.152604 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:19.152332 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/28bc4a8b-af34-47be-bad7-d9be1b2c75b9-web-config\") pod \"28bc4a8b-af34-47be-bad7-d9be1b2c75b9\" (UID: \"28bc4a8b-af34-47be-bad7-d9be1b2c75b9\") " Jul 14 20:28:19.152604 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:19.152359 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/28bc4a8b-af34-47be-bad7-d9be1b2c75b9-configmap-kubelet-serving-ca-bundle\") pod \"28bc4a8b-af34-47be-bad7-d9be1b2c75b9\" (UID: \"28bc4a8b-af34-47be-bad7-d9be1b2c75b9\") " Jul 14 20:28:19.152604 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:19.152391 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/28bc4a8b-af34-47be-bad7-d9be1b2c75b9-secret-grpc-tls\") pod \"28bc4a8b-af34-47be-bad7-d9be1b2c75b9\" (UID: \"28bc4a8b-af34-47be-bad7-d9be1b2c75b9\") " Jul 14 20:28:19.152604 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:19.152406 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sx97g\" (UniqueName: \"kubernetes.io/projected/28bc4a8b-af34-47be-bad7-d9be1b2c75b9-kube-api-access-sx97g\") pod \"28bc4a8b-af34-47be-bad7-d9be1b2c75b9\" (UID: \"28bc4a8b-af34-47be-bad7-d9be1b2c75b9\") " Jul 14 20:28:19.152604 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:19.152587 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/28bc4a8b-af34-47be-bad7-d9be1b2c75b9-secret-kube-rbac-proxy\") pod \"28bc4a8b-af34-47be-bad7-d9be1b2c75b9\" (UID: \"28bc4a8b-af34-47be-bad7-d9be1b2c75b9\") " Jul 14 20:28:19.152846 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:19.152612 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/28bc4a8b-af34-47be-bad7-d9be1b2c75b9-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"28bc4a8b-af34-47be-bad7-d9be1b2c75b9\" (UID: \"28bc4a8b-af34-47be-bad7-d9be1b2c75b9\") " Jul 14 20:28:19.152846 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:19.152670 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/28bc4a8b-af34-47be-bad7-d9be1b2c75b9-config\") pod \"28bc4a8b-af34-47be-bad7-d9be1b2c75b9\" (UID: \"28bc4a8b-af34-47be-bad7-d9be1b2c75b9\") " Jul 14 20:28:19.152846 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:19.152703 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/28bc4a8b-af34-47be-bad7-d9be1b2c75b9-prometheus-trusted-ca-bundle\") pod \"28bc4a8b-af34-47be-bad7-d9be1b2c75b9\" (UID: \"28bc4a8b-af34-47be-bad7-d9be1b2c75b9\") " Jul 14 20:28:19.152846 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:19.152744 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/28bc4a8b-af34-47be-bad7-d9be1b2c75b9-prometheus-k8s-db\") pod \"28bc4a8b-af34-47be-bad7-d9be1b2c75b9\" (UID: \"28bc4a8b-af34-47be-bad7-d9be1b2c75b9\") " Jul 14 20:28:19.152846 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:19.152771 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/28bc4a8b-af34-47be-bad7-d9be1b2c75b9-thanos-prometheus-http-client-file\") pod \"28bc4a8b-af34-47be-bad7-d9be1b2c75b9\" (UID: \"28bc4a8b-af34-47be-bad7-d9be1b2c75b9\") " Jul 14 20:28:19.152846 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:19.152796 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/28bc4a8b-af34-47be-bad7-d9be1b2c75b9-prometheus-k8s-rulefiles-1\") pod \"28bc4a8b-af34-47be-bad7-d9be1b2c75b9\" (UID: \"28bc4a8b-af34-47be-bad7-d9be1b2c75b9\") " Jul 14 20:28:19.152846 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:19.152791 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/28bc4a8b-af34-47be-bad7-d9be1b2c75b9-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "28bc4a8b-af34-47be-bad7-d9be1b2c75b9" (UID: "28bc4a8b-af34-47be-bad7-d9be1b2c75b9"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 20:28:19.152846 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:19.152835 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/28bc4a8b-af34-47be-bad7-d9be1b2c75b9-secret-metrics-client-certs\") pod \"28bc4a8b-af34-47be-bad7-d9be1b2c75b9\" (UID: \"28bc4a8b-af34-47be-bad7-d9be1b2c75b9\") " Jul 14 20:28:19.153490 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:19.152871 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/28bc4a8b-af34-47be-bad7-d9be1b2c75b9-secret-prometheus-k8s-tls\") pod \"28bc4a8b-af34-47be-bad7-d9be1b2c75b9\" (UID: \"28bc4a8b-af34-47be-bad7-d9be1b2c75b9\") " Jul 14 20:28:19.153490 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:19.152896 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/28bc4a8b-af34-47be-bad7-d9be1b2c75b9-config-out\") pod \"28bc4a8b-af34-47be-bad7-d9be1b2c75b9\" (UID: \"28bc4a8b-af34-47be-bad7-d9be1b2c75b9\") " Jul 14 20:28:19.153490 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:19.152925 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/28bc4a8b-af34-47be-bad7-d9be1b2c75b9-prometheus-k8s-rulefiles-2\") pod \"28bc4a8b-af34-47be-bad7-d9be1b2c75b9\" (UID: \"28bc4a8b-af34-47be-bad7-d9be1b2c75b9\") " Jul 14 20:28:19.153490 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:19.152959 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/28bc4a8b-af34-47be-bad7-d9be1b2c75b9-configmap-serving-certs-ca-bundle\") pod \"28bc4a8b-af34-47be-bad7-d9be1b2c75b9\" (UID: \"28bc4a8b-af34-47be-bad7-d9be1b2c75b9\") " Jul 14 20:28:19.153490 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:19.153065 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/28bc4a8b-af34-47be-bad7-d9be1b2c75b9-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "28bc4a8b-af34-47be-bad7-d9be1b2c75b9" (UID: "28bc4a8b-af34-47be-bad7-d9be1b2c75b9"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 20:28:19.153490 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:19.153425 2563 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/28bc4a8b-af34-47be-bad7-d9be1b2c75b9-configmap-metrics-client-ca\") on node \"ip-10-0-141-61.ec2.internal\" DevicePath \"\"" Jul 14 20:28:19.153490 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:19.153447 2563 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/28bc4a8b-af34-47be-bad7-d9be1b2c75b9-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-141-61.ec2.internal\" DevicePath \"\"" Jul 14 20:28:19.154018 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:19.153962 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/28bc4a8b-af34-47be-bad7-d9be1b2c75b9-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "28bc4a8b-af34-47be-bad7-d9be1b2c75b9" (UID: "28bc4a8b-af34-47be-bad7-d9be1b2c75b9"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 20:28:19.154500 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:19.154160 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/28bc4a8b-af34-47be-bad7-d9be1b2c75b9-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "28bc4a8b-af34-47be-bad7-d9be1b2c75b9" (UID: "28bc4a8b-af34-47be-bad7-d9be1b2c75b9"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 20:28:19.154602 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:19.154567 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/28bc4a8b-af34-47be-bad7-d9be1b2c75b9-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "28bc4a8b-af34-47be-bad7-d9be1b2c75b9" (UID: "28bc4a8b-af34-47be-bad7-d9be1b2c75b9"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 20:28:19.155111 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:19.154897 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/28bc4a8b-af34-47be-bad7-d9be1b2c75b9-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "28bc4a8b-af34-47be-bad7-d9be1b2c75b9" (UID: "28bc4a8b-af34-47be-bad7-d9be1b2c75b9"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 20:28:19.155111 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:19.155068 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/28bc4a8b-af34-47be-bad7-d9be1b2c75b9-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "28bc4a8b-af34-47be-bad7-d9be1b2c75b9" (UID: "28bc4a8b-af34-47be-bad7-d9be1b2c75b9"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 20:28:19.156792 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:19.155456 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/28bc4a8b-af34-47be-bad7-d9be1b2c75b9-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "28bc4a8b-af34-47be-bad7-d9be1b2c75b9" (UID: "28bc4a8b-af34-47be-bad7-d9be1b2c75b9"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 20:28:19.156792 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:19.155917 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/28bc4a8b-af34-47be-bad7-d9be1b2c75b9-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "28bc4a8b-af34-47be-bad7-d9be1b2c75b9" (UID: "28bc4a8b-af34-47be-bad7-d9be1b2c75b9"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 20:28:19.157383 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:19.157358 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/28bc4a8b-af34-47be-bad7-d9be1b2c75b9-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "28bc4a8b-af34-47be-bad7-d9be1b2c75b9" (UID: "28bc4a8b-af34-47be-bad7-d9be1b2c75b9"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 14 20:28:19.157478 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:19.157441 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/28bc4a8b-af34-47be-bad7-d9be1b2c75b9-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "28bc4a8b-af34-47be-bad7-d9be1b2c75b9" (UID: "28bc4a8b-af34-47be-bad7-d9be1b2c75b9"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 14 20:28:19.157589 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:19.157481 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/28bc4a8b-af34-47be-bad7-d9be1b2c75b9-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "28bc4a8b-af34-47be-bad7-d9be1b2c75b9" (UID: "28bc4a8b-af34-47be-bad7-d9be1b2c75b9"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 14 20:28:19.157589 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:19.157535 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/28bc4a8b-af34-47be-bad7-d9be1b2c75b9-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "28bc4a8b-af34-47be-bad7-d9be1b2c75b9" (UID: "28bc4a8b-af34-47be-bad7-d9be1b2c75b9"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 14 20:28:19.157697 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:19.157579 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/28bc4a8b-af34-47be-bad7-d9be1b2c75b9-kube-api-access-sx97g" (OuterVolumeSpecName: "kube-api-access-sx97g") pod "28bc4a8b-af34-47be-bad7-d9be1b2c75b9" (UID: "28bc4a8b-af34-47be-bad7-d9be1b2c75b9"). InnerVolumeSpecName "kube-api-access-sx97g". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 20:28:19.157852 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:19.157828 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/28bc4a8b-af34-47be-bad7-d9be1b2c75b9-config-out" (OuterVolumeSpecName: "config-out") pod "28bc4a8b-af34-47be-bad7-d9be1b2c75b9" (UID: "28bc4a8b-af34-47be-bad7-d9be1b2c75b9"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 20:28:19.157961 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:19.157881 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/28bc4a8b-af34-47be-bad7-d9be1b2c75b9-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "28bc4a8b-af34-47be-bad7-d9be1b2c75b9" (UID: "28bc4a8b-af34-47be-bad7-d9be1b2c75b9"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 14 20:28:19.158349 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:19.158325 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/28bc4a8b-af34-47be-bad7-d9be1b2c75b9-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "28bc4a8b-af34-47be-bad7-d9be1b2c75b9" (UID: "28bc4a8b-af34-47be-bad7-d9be1b2c75b9"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 14 20:28:19.158821 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:19.158803 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/28bc4a8b-af34-47be-bad7-d9be1b2c75b9-config" (OuterVolumeSpecName: "config") pod "28bc4a8b-af34-47be-bad7-d9be1b2c75b9" (UID: "28bc4a8b-af34-47be-bad7-d9be1b2c75b9"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 14 20:28:19.159171 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:19.159154 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/28bc4a8b-af34-47be-bad7-d9be1b2c75b9-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "28bc4a8b-af34-47be-bad7-d9be1b2c75b9" (UID: "28bc4a8b-af34-47be-bad7-d9be1b2c75b9"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 14 20:28:19.167405 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:19.167380 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/28bc4a8b-af34-47be-bad7-d9be1b2c75b9-web-config" (OuterVolumeSpecName: "web-config") pod "28bc4a8b-af34-47be-bad7-d9be1b2c75b9" (UID: "28bc4a8b-af34-47be-bad7-d9be1b2c75b9"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 14 20:28:19.254147 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:19.254101 2563 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/28bc4a8b-af34-47be-bad7-d9be1b2c75b9-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-141-61.ec2.internal\" DevicePath \"\"" Jul 14 20:28:19.254147 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:19.254156 2563 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/28bc4a8b-af34-47be-bad7-d9be1b2c75b9-web-config\") on node \"ip-10-0-141-61.ec2.internal\" DevicePath \"\"" Jul 14 20:28:19.254322 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:19.254166 2563 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/28bc4a8b-af34-47be-bad7-d9be1b2c75b9-secret-grpc-tls\") on node \"ip-10-0-141-61.ec2.internal\" DevicePath \"\"" Jul 14 20:28:19.254322 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:19.254175 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sx97g\" (UniqueName: \"kubernetes.io/projected/28bc4a8b-af34-47be-bad7-d9be1b2c75b9-kube-api-access-sx97g\") on node \"ip-10-0-141-61.ec2.internal\" DevicePath \"\"" Jul 14 20:28:19.254322 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:19.254184 2563 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/28bc4a8b-af34-47be-bad7-d9be1b2c75b9-secret-kube-rbac-proxy\") on node \"ip-10-0-141-61.ec2.internal\" DevicePath \"\"" Jul 14 20:28:19.254322 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:19.254192 2563 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/28bc4a8b-af34-47be-bad7-d9be1b2c75b9-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-141-61.ec2.internal\" DevicePath \"\"" Jul 14 20:28:19.254322 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:19.254200 2563 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/28bc4a8b-af34-47be-bad7-d9be1b2c75b9-config\") on node \"ip-10-0-141-61.ec2.internal\" DevicePath \"\"" Jul 14 20:28:19.254322 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:19.254208 2563 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/28bc4a8b-af34-47be-bad7-d9be1b2c75b9-prometheus-trusted-ca-bundle\") on node \"ip-10-0-141-61.ec2.internal\" DevicePath \"\"" Jul 14 20:28:19.254322 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:19.254217 2563 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/28bc4a8b-af34-47be-bad7-d9be1b2c75b9-prometheus-k8s-db\") on node \"ip-10-0-141-61.ec2.internal\" DevicePath \"\"" Jul 14 20:28:19.254322 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:19.254226 2563 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/28bc4a8b-af34-47be-bad7-d9be1b2c75b9-thanos-prometheus-http-client-file\") on node \"ip-10-0-141-61.ec2.internal\" DevicePath \"\"" Jul 14 20:28:19.254322 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:19.254234 2563 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/28bc4a8b-af34-47be-bad7-d9be1b2c75b9-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-141-61.ec2.internal\" DevicePath \"\"" Jul 14 20:28:19.254322 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:19.254244 2563 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/28bc4a8b-af34-47be-bad7-d9be1b2c75b9-secret-metrics-client-certs\") on node \"ip-10-0-141-61.ec2.internal\" DevicePath \"\"" Jul 14 20:28:19.254322 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:19.254252 2563 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/28bc4a8b-af34-47be-bad7-d9be1b2c75b9-secret-prometheus-k8s-tls\") on node \"ip-10-0-141-61.ec2.internal\" DevicePath \"\"" Jul 14 20:28:19.254322 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:19.254260 2563 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/28bc4a8b-af34-47be-bad7-d9be1b2c75b9-config-out\") on node \"ip-10-0-141-61.ec2.internal\" DevicePath \"\"" Jul 14 20:28:19.254322 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:19.254268 2563 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/28bc4a8b-af34-47be-bad7-d9be1b2c75b9-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-141-61.ec2.internal\" DevicePath \"\"" Jul 14 20:28:19.254322 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:19.254276 2563 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/28bc4a8b-af34-47be-bad7-d9be1b2c75b9-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-141-61.ec2.internal\" DevicePath \"\"" Jul 14 20:28:19.254322 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:19.254284 2563 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/28bc4a8b-af34-47be-bad7-d9be1b2c75b9-tls-assets\") on node \"ip-10-0-141-61.ec2.internal\" DevicePath \"\"" Jul 14 20:28:19.254322 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:19.254294 2563 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/28bc4a8b-af34-47be-bad7-d9be1b2c75b9-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-141-61.ec2.internal\" DevicePath \"\"" Jul 14 20:28:19.979553 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:19.979520 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 14 20:28:19.980015 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:19.979518 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"28bc4a8b-af34-47be-bad7-d9be1b2c75b9","Type":"ContainerDied","Data":"e57cfd03f3f7f4985bb882c871dc4ba1e2f6e086a000cc8db13d5f9427ac896e"} Jul 14 20:28:19.980015 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:19.979694 2563 scope.go:117] "RemoveContainer" containerID="59bf3096727c83d6b45956c7830bc808a86f0dd745e2db10201564ef21a1d893" Jul 14 20:28:19.990797 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:19.990780 2563 scope.go:117] "RemoveContainer" containerID="56cc1c7f62d121733fe4626b1ef37b96de3fccad3cd22d368e4215ecfc06a69e" Jul 14 20:28:19.998457 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:19.998436 2563 scope.go:117] "RemoveContainer" containerID="4b3681e1a826480c6600fe6870249377e43a786d06fdea66e63271da33d55978" Jul 14 20:28:19.999901 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:19.999879 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 14 20:28:20.003266 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:20.003239 2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 14 20:28:20.006730 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:20.006707 2563 scope.go:117] "RemoveContainer" containerID="42ce2ee737882fd5ac081570f9978e43245de9b635cab2251685b03cfe0797c7" Jul 14 20:28:20.014106 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:20.014087 2563 scope.go:117] "RemoveContainer" containerID="6ce246e2ae983de72d36f83644143a417f38b98438e379236f09b2fe8bf65cda" Jul 14 20:28:20.021675 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:20.021658 2563 scope.go:117] "RemoveContainer" containerID="35634f462d91ad1ce335dab920d61b1955698fc0fb19733fe2ec61d875b3d863" Jul 14 20:28:20.029618 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:20.029597 2563 scope.go:117] "RemoveContainer" containerID="77817d528b2fa089a23ac9fc25dd243c3ab43760686e7e35c1d243405780c831" Jul 14 20:28:20.030644 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:20.030593 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 14 20:28:20.030876 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:20.030864 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="28bc4a8b-af34-47be-bad7-d9be1b2c75b9" containerName="kube-rbac-proxy-web" Jul 14 20:28:20.030912 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:20.030878 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="28bc4a8b-af34-47be-bad7-d9be1b2c75b9" containerName="kube-rbac-proxy-web" Jul 14 20:28:20.030912 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:20.030889 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3bd29b0f-00fb-4a66-97b1-c2727db1081a" containerName="kube-rbac-proxy-metric" Jul 14 20:28:20.030912 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:20.030894 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="3bd29b0f-00fb-4a66-97b1-c2727db1081a" containerName="kube-rbac-proxy-metric" Jul 14 20:28:20.030912 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:20.030901 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="28bc4a8b-af34-47be-bad7-d9be1b2c75b9" containerName="prometheus" Jul 14 20:28:20.030912 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:20.030906 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="28bc4a8b-af34-47be-bad7-d9be1b2c75b9" containerName="prometheus" Jul 14 20:28:20.030912 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:20.030913 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3bd29b0f-00fb-4a66-97b1-c2727db1081a" containerName="config-reloader" Jul 14 20:28:20.031096 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:20.030918 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="3bd29b0f-00fb-4a66-97b1-c2727db1081a" containerName="config-reloader" Jul 14 20:28:20.031096 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:20.030928 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3bd29b0f-00fb-4a66-97b1-c2727db1081a" containerName="init-config-reloader" Jul 14 20:28:20.031096 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:20.030933 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="3bd29b0f-00fb-4a66-97b1-c2727db1081a" containerName="init-config-reloader" Jul 14 20:28:20.031096 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:20.030943 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="28bc4a8b-af34-47be-bad7-d9be1b2c75b9" containerName="kube-rbac-proxy-thanos" Jul 14 20:28:20.031096 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:20.030949 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="28bc4a8b-af34-47be-bad7-d9be1b2c75b9" containerName="kube-rbac-proxy-thanos" Jul 14 20:28:20.031096 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:20.030956 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3bd29b0f-00fb-4a66-97b1-c2727db1081a" containerName="kube-rbac-proxy-web" Jul 14 20:28:20.031096 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:20.030961 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="3bd29b0f-00fb-4a66-97b1-c2727db1081a" containerName="kube-rbac-proxy-web" Jul 14 20:28:20.031096 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:20.030966 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="28bc4a8b-af34-47be-bad7-d9be1b2c75b9" containerName="kube-rbac-proxy" Jul 14 20:28:20.031096 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:20.030971 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="28bc4a8b-af34-47be-bad7-d9be1b2c75b9" containerName="kube-rbac-proxy" Jul 14 20:28:20.031096 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:20.030977 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3bd29b0f-00fb-4a66-97b1-c2727db1081a" containerName="prom-label-proxy" Jul 14 20:28:20.031096 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:20.030982 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="3bd29b0f-00fb-4a66-97b1-c2727db1081a" containerName="prom-label-proxy" Jul 14 20:28:20.031096 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:20.030987 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3c07a280-eebb-4ab3-9381-e92e5f354120" containerName="console" Jul 14 20:28:20.031096 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:20.030991 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="3c07a280-eebb-4ab3-9381-e92e5f354120" containerName="console" Jul 14 20:28:20.031096 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:20.030997 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3bd29b0f-00fb-4a66-97b1-c2727db1081a" containerName="kube-rbac-proxy" Jul 14 20:28:20.031096 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:20.031002 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="3bd29b0f-00fb-4a66-97b1-c2727db1081a" containerName="kube-rbac-proxy" Jul 14 20:28:20.031096 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:20.031006 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="28bc4a8b-af34-47be-bad7-d9be1b2c75b9" containerName="config-reloader" Jul 14 20:28:20.031096 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:20.031012 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="28bc4a8b-af34-47be-bad7-d9be1b2c75b9" containerName="config-reloader" Jul 14 20:28:20.031096 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:20.031017 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3bd29b0f-00fb-4a66-97b1-c2727db1081a" containerName="alertmanager" Jul 14 20:28:20.031096 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:20.031022 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="3bd29b0f-00fb-4a66-97b1-c2727db1081a" containerName="alertmanager" Jul 14 20:28:20.031096 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:20.031030 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="28bc4a8b-af34-47be-bad7-d9be1b2c75b9" containerName="init-config-reloader" Jul 14 20:28:20.031096 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:20.031035 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="28bc4a8b-af34-47be-bad7-d9be1b2c75b9" containerName="init-config-reloader" Jul 14 20:28:20.031096 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:20.031041 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="28bc4a8b-af34-47be-bad7-d9be1b2c75b9" containerName="thanos-sidecar" Jul 14 20:28:20.031096 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:20.031046 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="28bc4a8b-af34-47be-bad7-d9be1b2c75b9" containerName="thanos-sidecar" Jul 14 20:28:20.031096 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:20.031089 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="3bd29b0f-00fb-4a66-97b1-c2727db1081a" containerName="alertmanager" Jul 14 20:28:20.031096 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:20.031096 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="3c07a280-eebb-4ab3-9381-e92e5f354120" containerName="console" Jul 14 20:28:20.031096 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:20.031104 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="3bd29b0f-00fb-4a66-97b1-c2727db1081a" containerName="kube-rbac-proxy-web" Jul 14 20:28:20.031943 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:20.031110 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="3bd29b0f-00fb-4a66-97b1-c2727db1081a" containerName="kube-rbac-proxy" Jul 14 20:28:20.031943 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:20.031116 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="28bc4a8b-af34-47be-bad7-d9be1b2c75b9" containerName="kube-rbac-proxy-web" Jul 14 20:28:20.031943 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:20.031139 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="3bd29b0f-00fb-4a66-97b1-c2727db1081a" containerName="kube-rbac-proxy-metric" Jul 14 20:28:20.031943 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:20.031146 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="28bc4a8b-af34-47be-bad7-d9be1b2c75b9" containerName="prometheus" Jul 14 20:28:20.031943 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:20.031150 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="28bc4a8b-af34-47be-bad7-d9be1b2c75b9" containerName="config-reloader" Jul 14 20:28:20.031943 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:20.031155 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="28bc4a8b-af34-47be-bad7-d9be1b2c75b9" containerName="kube-rbac-proxy" Jul 14 20:28:20.031943 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:20.031160 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="28bc4a8b-af34-47be-bad7-d9be1b2c75b9" containerName="kube-rbac-proxy-thanos" Jul 14 20:28:20.031943 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:20.031166 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="28bc4a8b-af34-47be-bad7-d9be1b2c75b9" containerName="thanos-sidecar" Jul 14 20:28:20.031943 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:20.031171 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="3bd29b0f-00fb-4a66-97b1-c2727db1081a" containerName="prom-label-proxy" Jul 14 20:28:20.031943 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:20.031176 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="3bd29b0f-00fb-4a66-97b1-c2727db1081a" containerName="config-reloader" Jul 14 20:28:20.035099 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:20.035081 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 14 20:28:20.037483 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:20.037394 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Jul 14 20:28:20.037821 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:20.037801 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-2c5221ssc5ite\"" Jul 14 20:28:20.037916 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:20.037897 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Jul 14 20:28:20.037984 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:20.037968 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Jul 14 20:28:20.038417 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:20.038381 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Jul 14 20:28:20.038417 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:20.038383 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Jul 14 20:28:20.038585 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:20.038419 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Jul 14 20:28:20.038752 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:20.038733 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Jul 14 20:28:20.038752 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:20.038743 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Jul 14 20:28:20.038902 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:20.038738 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Jul 14 20:28:20.038902 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:20.038762 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Jul 14 20:28:20.038902 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:20.038813 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Jul 14 20:28:20.038902 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:20.038743 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-8xsdz\"" Jul 14 20:28:20.039099 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:20.038955 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Jul 14 20:28:20.041774 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:20.041751 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Jul 14 20:28:20.046008 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:20.045984 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Jul 14 20:28:20.050677 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:20.050654 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 14 20:28:20.161214 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:20.161173 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wkvg8\" (UniqueName: \"kubernetes.io/projected/8c83ee98-d2c0-4eb9-bb01-279ea67f4ad8-kube-api-access-wkvg8\") pod \"prometheus-k8s-0\" (UID: \"8c83ee98-d2c0-4eb9-bb01-279ea67f4ad8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 20:28:20.161214 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:20.161211 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/8c83ee98-d2c0-4eb9-bb01-279ea67f4ad8-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"8c83ee98-d2c0-4eb9-bb01-279ea67f4ad8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 20:28:20.161445 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:20.161229 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/8c83ee98-d2c0-4eb9-bb01-279ea67f4ad8-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"8c83ee98-d2c0-4eb9-bb01-279ea67f4ad8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 20:28:20.161445 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:20.161297 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/8c83ee98-d2c0-4eb9-bb01-279ea67f4ad8-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"8c83ee98-d2c0-4eb9-bb01-279ea67f4ad8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 20:28:20.161445 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:20.161331 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/8c83ee98-d2c0-4eb9-bb01-279ea67f4ad8-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"8c83ee98-d2c0-4eb9-bb01-279ea67f4ad8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 20:28:20.161445 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:20.161370 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/8c83ee98-d2c0-4eb9-bb01-279ea67f4ad8-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"8c83ee98-d2c0-4eb9-bb01-279ea67f4ad8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 20:28:20.161445 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:20.161388 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/8c83ee98-d2c0-4eb9-bb01-279ea67f4ad8-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"8c83ee98-d2c0-4eb9-bb01-279ea67f4ad8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 20:28:20.161445 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:20.161409 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/8c83ee98-d2c0-4eb9-bb01-279ea67f4ad8-web-config\") pod \"prometheus-k8s-0\" (UID: \"8c83ee98-d2c0-4eb9-bb01-279ea67f4ad8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 20:28:20.161445 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:20.161429 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/8c83ee98-d2c0-4eb9-bb01-279ea67f4ad8-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"8c83ee98-d2c0-4eb9-bb01-279ea67f4ad8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 20:28:20.161679 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:20.161468 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/8c83ee98-d2c0-4eb9-bb01-279ea67f4ad8-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"8c83ee98-d2c0-4eb9-bb01-279ea67f4ad8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 20:28:20.161679 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:20.161556 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/8c83ee98-d2c0-4eb9-bb01-279ea67f4ad8-config-out\") pod \"prometheus-k8s-0\" (UID: \"8c83ee98-d2c0-4eb9-bb01-279ea67f4ad8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 20:28:20.161679 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:20.161577 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/8c83ee98-d2c0-4eb9-bb01-279ea67f4ad8-config\") pod \"prometheus-k8s-0\" (UID: \"8c83ee98-d2c0-4eb9-bb01-279ea67f4ad8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 20:28:20.161679 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:20.161591 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8c83ee98-d2c0-4eb9-bb01-279ea67f4ad8-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"8c83ee98-d2c0-4eb9-bb01-279ea67f4ad8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 20:28:20.161679 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:20.161612 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/8c83ee98-d2c0-4eb9-bb01-279ea67f4ad8-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"8c83ee98-d2c0-4eb9-bb01-279ea67f4ad8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 20:28:20.161679 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:20.161657 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/8c83ee98-d2c0-4eb9-bb01-279ea67f4ad8-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"8c83ee98-d2c0-4eb9-bb01-279ea67f4ad8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 20:28:20.161881 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:20.161689 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8c83ee98-d2c0-4eb9-bb01-279ea67f4ad8-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"8c83ee98-d2c0-4eb9-bb01-279ea67f4ad8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 20:28:20.161881 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:20.161708 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8c83ee98-d2c0-4eb9-bb01-279ea67f4ad8-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"8c83ee98-d2c0-4eb9-bb01-279ea67f4ad8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 20:28:20.161881 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:20.161726 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8c83ee98-d2c0-4eb9-bb01-279ea67f4ad8-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"8c83ee98-d2c0-4eb9-bb01-279ea67f4ad8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 20:28:20.161881 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:20.161779 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/8c83ee98-d2c0-4eb9-bb01-279ea67f4ad8-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"8c83ee98-d2c0-4eb9-bb01-279ea67f4ad8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 20:28:20.161881 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:20.161796 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/8c83ee98-d2c0-4eb9-bb01-279ea67f4ad8-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"8c83ee98-d2c0-4eb9-bb01-279ea67f4ad8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 20:28:20.263273 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:20.263190 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/8c83ee98-d2c0-4eb9-bb01-279ea67f4ad8-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"8c83ee98-d2c0-4eb9-bb01-279ea67f4ad8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 20:28:20.263273 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:20.263230 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/8c83ee98-d2c0-4eb9-bb01-279ea67f4ad8-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"8c83ee98-d2c0-4eb9-bb01-279ea67f4ad8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 20:28:20.263273 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:20.263249 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/8c83ee98-d2c0-4eb9-bb01-279ea67f4ad8-web-config\") pod \"prometheus-k8s-0\" (UID: \"8c83ee98-d2c0-4eb9-bb01-279ea67f4ad8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 20:28:20.263273 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:20.263273 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/8c83ee98-d2c0-4eb9-bb01-279ea67f4ad8-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"8c83ee98-d2c0-4eb9-bb01-279ea67f4ad8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 20:28:20.263568 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:20.263296 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/8c83ee98-d2c0-4eb9-bb01-279ea67f4ad8-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"8c83ee98-d2c0-4eb9-bb01-279ea67f4ad8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 20:28:20.263568 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:20.263330 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/8c83ee98-d2c0-4eb9-bb01-279ea67f4ad8-config-out\") pod \"prometheus-k8s-0\" (UID: \"8c83ee98-d2c0-4eb9-bb01-279ea67f4ad8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 20:28:20.263568 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:20.263355 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/8c83ee98-d2c0-4eb9-bb01-279ea67f4ad8-config\") pod \"prometheus-k8s-0\" (UID: \"8c83ee98-d2c0-4eb9-bb01-279ea67f4ad8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 20:28:20.263568 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:20.263376 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8c83ee98-d2c0-4eb9-bb01-279ea67f4ad8-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"8c83ee98-d2c0-4eb9-bb01-279ea67f4ad8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 20:28:20.263568 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:20.263403 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/8c83ee98-d2c0-4eb9-bb01-279ea67f4ad8-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"8c83ee98-d2c0-4eb9-bb01-279ea67f4ad8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 20:28:20.263568 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:20.263426 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/8c83ee98-d2c0-4eb9-bb01-279ea67f4ad8-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"8c83ee98-d2c0-4eb9-bb01-279ea67f4ad8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 20:28:20.263568 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:20.263453 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8c83ee98-d2c0-4eb9-bb01-279ea67f4ad8-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"8c83ee98-d2c0-4eb9-bb01-279ea67f4ad8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 20:28:20.263568 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:20.263479 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8c83ee98-d2c0-4eb9-bb01-279ea67f4ad8-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"8c83ee98-d2c0-4eb9-bb01-279ea67f4ad8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 20:28:20.263568 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:20.263500 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8c83ee98-d2c0-4eb9-bb01-279ea67f4ad8-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"8c83ee98-d2c0-4eb9-bb01-279ea67f4ad8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 20:28:20.263568 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:20.263528 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/8c83ee98-d2c0-4eb9-bb01-279ea67f4ad8-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"8c83ee98-d2c0-4eb9-bb01-279ea67f4ad8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 20:28:20.263568 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:20.263555 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/8c83ee98-d2c0-4eb9-bb01-279ea67f4ad8-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"8c83ee98-d2c0-4eb9-bb01-279ea67f4ad8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 20:28:20.264100 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:20.263584 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wkvg8\" (UniqueName: \"kubernetes.io/projected/8c83ee98-d2c0-4eb9-bb01-279ea67f4ad8-kube-api-access-wkvg8\") pod \"prometheus-k8s-0\" (UID: \"8c83ee98-d2c0-4eb9-bb01-279ea67f4ad8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 20:28:20.264100 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:20.263608 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/8c83ee98-d2c0-4eb9-bb01-279ea67f4ad8-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"8c83ee98-d2c0-4eb9-bb01-279ea67f4ad8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 20:28:20.264100 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:20.263637 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/8c83ee98-d2c0-4eb9-bb01-279ea67f4ad8-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"8c83ee98-d2c0-4eb9-bb01-279ea67f4ad8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 20:28:20.264100 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:20.263672 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/8c83ee98-d2c0-4eb9-bb01-279ea67f4ad8-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"8c83ee98-d2c0-4eb9-bb01-279ea67f4ad8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 20:28:20.264100 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:20.263699 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/8c83ee98-d2c0-4eb9-bb01-279ea67f4ad8-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"8c83ee98-d2c0-4eb9-bb01-279ea67f4ad8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 20:28:20.264982 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:20.264734 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8c83ee98-d2c0-4eb9-bb01-279ea67f4ad8-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"8c83ee98-d2c0-4eb9-bb01-279ea67f4ad8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 20:28:20.264982 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:20.264804 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/8c83ee98-d2c0-4eb9-bb01-279ea67f4ad8-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"8c83ee98-d2c0-4eb9-bb01-279ea67f4ad8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 20:28:20.266594 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:20.266558 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/8c83ee98-d2c0-4eb9-bb01-279ea67f4ad8-config\") pod \"prometheus-k8s-0\" (UID: \"8c83ee98-d2c0-4eb9-bb01-279ea67f4ad8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 20:28:20.266693 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:20.266593 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/8c83ee98-d2c0-4eb9-bb01-279ea67f4ad8-web-config\") pod \"prometheus-k8s-0\" (UID: \"8c83ee98-d2c0-4eb9-bb01-279ea67f4ad8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 20:28:20.266769 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:20.266706 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/8c83ee98-d2c0-4eb9-bb01-279ea67f4ad8-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"8c83ee98-d2c0-4eb9-bb01-279ea67f4ad8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 20:28:20.266809 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:20.266768 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/8c83ee98-d2c0-4eb9-bb01-279ea67f4ad8-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"8c83ee98-d2c0-4eb9-bb01-279ea67f4ad8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 20:28:20.266809 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:20.266791 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/8c83ee98-d2c0-4eb9-bb01-279ea67f4ad8-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"8c83ee98-d2c0-4eb9-bb01-279ea67f4ad8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 20:28:20.267230 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:20.267072 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/8c83ee98-d2c0-4eb9-bb01-279ea67f4ad8-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"8c83ee98-d2c0-4eb9-bb01-279ea67f4ad8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 20:28:20.267323 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:20.267278 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/8c83ee98-d2c0-4eb9-bb01-279ea67f4ad8-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"8c83ee98-d2c0-4eb9-bb01-279ea67f4ad8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 20:28:20.267690 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:20.267612 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/8c83ee98-d2c0-4eb9-bb01-279ea67f4ad8-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"8c83ee98-d2c0-4eb9-bb01-279ea67f4ad8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 20:28:20.267980 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:20.267928 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8c83ee98-d2c0-4eb9-bb01-279ea67f4ad8-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"8c83ee98-d2c0-4eb9-bb01-279ea67f4ad8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 20:28:20.268085 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:20.268025 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/8c83ee98-d2c0-4eb9-bb01-279ea67f4ad8-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"8c83ee98-d2c0-4eb9-bb01-279ea67f4ad8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 20:28:20.268295 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:20.268267 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8c83ee98-d2c0-4eb9-bb01-279ea67f4ad8-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"8c83ee98-d2c0-4eb9-bb01-279ea67f4ad8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 20:28:20.268464 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:20.268446 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8c83ee98-d2c0-4eb9-bb01-279ea67f4ad8-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"8c83ee98-d2c0-4eb9-bb01-279ea67f4ad8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 20:28:20.268724 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:20.268707 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/8c83ee98-d2c0-4eb9-bb01-279ea67f4ad8-config-out\") pod \"prometheus-k8s-0\" (UID: \"8c83ee98-d2c0-4eb9-bb01-279ea67f4ad8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 20:28:20.269146 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:20.269108 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/8c83ee98-d2c0-4eb9-bb01-279ea67f4ad8-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"8c83ee98-d2c0-4eb9-bb01-279ea67f4ad8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 20:28:20.269845 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:20.269819 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/8c83ee98-d2c0-4eb9-bb01-279ea67f4ad8-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"8c83ee98-d2c0-4eb9-bb01-279ea67f4ad8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 20:28:20.270096 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:20.270079 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/8c83ee98-d2c0-4eb9-bb01-279ea67f4ad8-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"8c83ee98-d2c0-4eb9-bb01-279ea67f4ad8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 20:28:20.271384 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:20.271362 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/8c83ee98-d2c0-4eb9-bb01-279ea67f4ad8-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"8c83ee98-d2c0-4eb9-bb01-279ea67f4ad8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 20:28:20.275143 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:20.275096 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wkvg8\" (UniqueName: \"kubernetes.io/projected/8c83ee98-d2c0-4eb9-bb01-279ea67f4ad8-kube-api-access-wkvg8\") pod \"prometheus-k8s-0\" (UID: \"8c83ee98-d2c0-4eb9-bb01-279ea67f4ad8\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 20:28:20.351304 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:20.351265 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 14 20:28:20.497783 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:20.497761 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 14 20:28:20.499950 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:28:20.499923 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8c83ee98_d2c0_4eb9_bb01_279ea67f4ad8.slice/crio-7bd8d129b92bfe53ba4c4ea383b3287afba8985a251587ead2747a0aafa5058e WatchSource:0}: Error finding container 7bd8d129b92bfe53ba4c4ea383b3287afba8985a251587ead2747a0aafa5058e: Status 404 returned error can't find the container with id 7bd8d129b92bfe53ba4c4ea383b3287afba8985a251587ead2747a0aafa5058e Jul 14 20:28:20.749926 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:20.749891 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:28:20.812861 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:20.812774 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:28:20.985778 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:20.985743 2563 generic.go:356] "Generic (PLEG): container finished" podID="8c83ee98-d2c0-4eb9-bb01-279ea67f4ad8" containerID="7d209351d6b32c317bc87feec2bf34d70b5b4f6b7591a92f31f25ce057046df3" exitCode=0 Jul 14 20:28:20.986193 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:20.985818 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8c83ee98-d2c0-4eb9-bb01-279ea67f4ad8","Type":"ContainerDied","Data":"7d209351d6b32c317bc87feec2bf34d70b5b4f6b7591a92f31f25ce057046df3"} Jul 14 20:28:20.986193 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:20.985896 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8c83ee98-d2c0-4eb9-bb01-279ea67f4ad8","Type":"ContainerStarted","Data":"7bd8d129b92bfe53ba4c4ea383b3287afba8985a251587ead2747a0aafa5058e"} Jul 14 20:28:21.612426 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:21.612348 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="28bc4a8b-af34-47be-bad7-d9be1b2c75b9" path="/var/lib/kubelet/pods/28bc4a8b-af34-47be-bad7-d9be1b2c75b9/volumes" Jul 14 20:28:21.651698 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:21.651659 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:28:21.993934 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:21.993880 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8c83ee98-d2c0-4eb9-bb01-279ea67f4ad8","Type":"ContainerStarted","Data":"85573e943e238cf6c60adbd80a7a490ef61a5083f0e1a4858d012f8f6378546f"} Jul 14 20:28:21.994449 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:21.993963 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8c83ee98-d2c0-4eb9-bb01-279ea67f4ad8","Type":"ContainerStarted","Data":"645eb5affbe3ee986563ed2c3c411d8c0b0efc877e5a74de12f80f6f5ddf14e4"} Jul 14 20:28:21.994449 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:21.994017 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8c83ee98-d2c0-4eb9-bb01-279ea67f4ad8","Type":"ContainerStarted","Data":"22960c04829f6ce7748c79ed87217167b17daff0fdc0e718ec266054282394e8"} Jul 14 20:28:21.994449 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:21.994060 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8c83ee98-d2c0-4eb9-bb01-279ea67f4ad8","Type":"ContainerStarted","Data":"f15e754a5f5ece1219e27e06572959f798f0bca0a16e0fbc6bf6cbc350ed3f01"} Jul 14 20:28:21.994449 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:21.994100 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8c83ee98-d2c0-4eb9-bb01-279ea67f4ad8","Type":"ContainerStarted","Data":"970ed4950a51a6b1a0f971484fecbd50556d95872f08e48cf9313d460d72542c"} Jul 14 20:28:21.994449 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:21.994169 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"8c83ee98-d2c0-4eb9-bb01-279ea67f4ad8","Type":"ContainerStarted","Data":"80cb213c5dc842896268ecb20e126fdaa911ff50fa61d97a7079fc9f2c56fb4c"} Jul 14 20:28:21.994710 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:21.994505 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:28:22.023275 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:22.023221 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.004332854 podStartE2EDuration="2.023193172s" totalImagesPullingTime="0s" totalInitContainerRuntime="18.860318ms" isStatefulPod=false podCreationTimestamp="2026-07-14 20:28:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 20:28:22.022673768 +0000 UTC m=+360.943837076" watchObservedRunningTime="2026-07-14 20:28:22.023193172 +0000 UTC m=+360.944356484" Jul 14 20:28:23.449370 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:23.449333 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:28:23.748186 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:23.748081 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:28:25.352424 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:25.352389 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 14 20:28:30.507629 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:30.507590 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:28:30.567143 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:30.567100 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:28:30.671051 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:30.671012 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:28:32.574312 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:32.574272 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:28:32.670676 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:28:32.670636 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:29:14.408257 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:29:14.408219 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:29:14.505835 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:29:14.505803 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:29:17.501323 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:29:17.501270 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:29:17.562337 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:29:17.562307 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:29:20.352461 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:29:20.352424 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Jul 14 20:29:20.369069 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:29:20.369042 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Jul 14 20:29:21.239232 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:29:21.239203 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 14 20:29:41.466298 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:29:41.466258 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:29:42.303704 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:29:42.303667 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:29:45.323315 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:29:45.323274 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:29:45.484078 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:29:45.484037 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:29:48.588471 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:29:48.588431 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:29:48.846232 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:29:48.846117 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:29:50.316568 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:29:50.316523 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:29:51.039141 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:29:51.039093 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:29:52.492732 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:29:52.492681 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:29:52.597279 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:29:52.597243 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:29:59.008535 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:29:59.008495 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:29:59.464655 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:29:59.464608 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:29:59.593728 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:29:59.593687 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:30:02.120485 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:30:02.120456 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:30:02.240696 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:30:02.240647 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:30:06.476613 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:30:06.476574 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:30:06.652931 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:30:06.652902 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:30:08.713237 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:30:08.713186 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:30:09.212281 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:30:09.212246 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-lpdrq"] Jul 14 20:30:09.215327 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:30:09.215312 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-lpdrq" Jul 14 20:30:09.219208 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:30:09.219188 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Jul 14 20:30:09.220598 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:30:09.220577 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Jul 14 20:30:09.221295 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:30:09.221273 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-5t4ls\"" Jul 14 20:30:09.231744 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:30:09.231719 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-lpdrq"] Jul 14 20:30:09.410769 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:30:09.410729 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jzq6m\" (UniqueName: \"kubernetes.io/projected/10ac8147-c207-4061-bbdd-9762d3b1943a-kube-api-access-jzq6m\") pod \"cert-manager-55b4d954d5-lpdrq\" (UID: \"10ac8147-c207-4061-bbdd-9762d3b1943a\") " pod="cert-manager/cert-manager-55b4d954d5-lpdrq" Jul 14 20:30:09.410945 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:30:09.410857 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/10ac8147-c207-4061-bbdd-9762d3b1943a-bound-sa-token\") pod \"cert-manager-55b4d954d5-lpdrq\" (UID: \"10ac8147-c207-4061-bbdd-9762d3b1943a\") " pod="cert-manager/cert-manager-55b4d954d5-lpdrq" Jul 14 20:30:09.512347 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:30:09.512253 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/10ac8147-c207-4061-bbdd-9762d3b1943a-bound-sa-token\") pod \"cert-manager-55b4d954d5-lpdrq\" (UID: \"10ac8147-c207-4061-bbdd-9762d3b1943a\") " pod="cert-manager/cert-manager-55b4d954d5-lpdrq" Jul 14 20:30:09.512347 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:30:09.512300 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jzq6m\" (UniqueName: \"kubernetes.io/projected/10ac8147-c207-4061-bbdd-9762d3b1943a-kube-api-access-jzq6m\") pod \"cert-manager-55b4d954d5-lpdrq\" (UID: \"10ac8147-c207-4061-bbdd-9762d3b1943a\") " pod="cert-manager/cert-manager-55b4d954d5-lpdrq" Jul 14 20:30:09.520763 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:30:09.520736 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/10ac8147-c207-4061-bbdd-9762d3b1943a-bound-sa-token\") pod \"cert-manager-55b4d954d5-lpdrq\" (UID: \"10ac8147-c207-4061-bbdd-9762d3b1943a\") " pod="cert-manager/cert-manager-55b4d954d5-lpdrq" Jul 14 20:30:09.520921 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:30:09.520904 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jzq6m\" (UniqueName: \"kubernetes.io/projected/10ac8147-c207-4061-bbdd-9762d3b1943a-kube-api-access-jzq6m\") pod \"cert-manager-55b4d954d5-lpdrq\" (UID: \"10ac8147-c207-4061-bbdd-9762d3b1943a\") " pod="cert-manager/cert-manager-55b4d954d5-lpdrq" Jul 14 20:30:09.524669 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:30:09.524652 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-lpdrq" Jul 14 20:30:09.664030 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:30:09.663998 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-lpdrq"] Jul 14 20:30:09.667560 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:30:09.667535 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod10ac8147_c207_4061_bbdd_9762d3b1943a.slice/crio-b9df0f3ad559fd5d4b2f5b347007c53455a235c9b7dfa5cd743159aad56553e9 WatchSource:0}: Error finding container b9df0f3ad559fd5d4b2f5b347007c53455a235c9b7dfa5cd743159aad56553e9: Status 404 returned error can't find the container with id b9df0f3ad559fd5d4b2f5b347007c53455a235c9b7dfa5cd743159aad56553e9 Jul 14 20:30:09.669523 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:30:09.669504 2563 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 14 20:30:10.419933 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:30:10.419859 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-lpdrq" event={"ID":"10ac8147-c207-4061-bbdd-9762d3b1943a","Type":"ContainerStarted","Data":"b9df0f3ad559fd5d4b2f5b347007c53455a235c9b7dfa5cd743159aad56553e9"} Jul 14 20:30:13.434482 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:30:13.434452 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-lpdrq" event={"ID":"10ac8147-c207-4061-bbdd-9762d3b1943a","Type":"ContainerStarted","Data":"094b7bc92236e22bd5f403f0465f22926227b64c1c49deaff9b9d89c02173ca7"} Jul 14 20:30:13.453622 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:30:13.453568 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-lpdrq" podStartSLOduration=1.599580529 podStartE2EDuration="4.45355257s" totalImagesPullingTime="2.853972041s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 20:30:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 20:30:13.452095463 +0000 UTC m=+472.373258776" watchObservedRunningTime="2026-07-14 20:30:13.45355257 +0000 UTC m=+472.374715877" Jul 14 20:30:16.623386 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:30:16.623343 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:30:16.712780 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:30:16.712742 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:30:16.732951 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:30:16.732922 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-78b47f4478-8sv5k"] Jul 14 20:30:16.736188 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:30:16.736170 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-78b47f4478-8sv5k" Jul 14 20:30:16.743500 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:30:16.743471 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Jul 14 20:30:16.743500 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:30:16.743507 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Jul 14 20:30:16.743699 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:30:16.743531 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-2cfhw\"" Jul 14 20:30:16.743699 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:30:16.743516 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 14 20:30:16.743699 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:30:16.743516 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Jul 14 20:30:16.743699 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:30:16.743507 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 14 20:30:16.756032 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:30:16.756001 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-78b47f4478-8sv5k"] Jul 14 20:30:16.776744 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:30:16.776715 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/997602e8-2fb7-46e0-9720-266e09a79f4b-manager-config\") pod \"lws-controller-manager-78b47f4478-8sv5k\" (UID: \"997602e8-2fb7-46e0-9720-266e09a79f4b\") " pod="openshift-lws-operator/lws-controller-manager-78b47f4478-8sv5k" Jul 14 20:30:16.776902 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:30:16.776771 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5lcwr\" (UniqueName: \"kubernetes.io/projected/997602e8-2fb7-46e0-9720-266e09a79f4b-kube-api-access-5lcwr\") pod \"lws-controller-manager-78b47f4478-8sv5k\" (UID: \"997602e8-2fb7-46e0-9720-266e09a79f4b\") " pod="openshift-lws-operator/lws-controller-manager-78b47f4478-8sv5k" Jul 14 20:30:16.776902 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:30:16.776846 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/997602e8-2fb7-46e0-9720-266e09a79f4b-metrics-cert\") pod \"lws-controller-manager-78b47f4478-8sv5k\" (UID: \"997602e8-2fb7-46e0-9720-266e09a79f4b\") " pod="openshift-lws-operator/lws-controller-manager-78b47f4478-8sv5k" Jul 14 20:30:16.776902 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:30:16.776876 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/997602e8-2fb7-46e0-9720-266e09a79f4b-cert\") pod \"lws-controller-manager-78b47f4478-8sv5k\" (UID: \"997602e8-2fb7-46e0-9720-266e09a79f4b\") " pod="openshift-lws-operator/lws-controller-manager-78b47f4478-8sv5k" Jul 14 20:30:16.877635 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:30:16.877541 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/997602e8-2fb7-46e0-9720-266e09a79f4b-manager-config\") pod \"lws-controller-manager-78b47f4478-8sv5k\" (UID: \"997602e8-2fb7-46e0-9720-266e09a79f4b\") " pod="openshift-lws-operator/lws-controller-manager-78b47f4478-8sv5k" Jul 14 20:30:16.877635 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:30:16.877624 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5lcwr\" (UniqueName: \"kubernetes.io/projected/997602e8-2fb7-46e0-9720-266e09a79f4b-kube-api-access-5lcwr\") pod \"lws-controller-manager-78b47f4478-8sv5k\" (UID: \"997602e8-2fb7-46e0-9720-266e09a79f4b\") " pod="openshift-lws-operator/lws-controller-manager-78b47f4478-8sv5k" Jul 14 20:30:16.877834 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:30:16.877670 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/997602e8-2fb7-46e0-9720-266e09a79f4b-metrics-cert\") pod \"lws-controller-manager-78b47f4478-8sv5k\" (UID: \"997602e8-2fb7-46e0-9720-266e09a79f4b\") " pod="openshift-lws-operator/lws-controller-manager-78b47f4478-8sv5k" Jul 14 20:30:16.877834 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:30:16.877687 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/997602e8-2fb7-46e0-9720-266e09a79f4b-cert\") pod \"lws-controller-manager-78b47f4478-8sv5k\" (UID: \"997602e8-2fb7-46e0-9720-266e09a79f4b\") " pod="openshift-lws-operator/lws-controller-manager-78b47f4478-8sv5k" Jul 14 20:30:16.878358 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:30:16.878332 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/997602e8-2fb7-46e0-9720-266e09a79f4b-manager-config\") pod \"lws-controller-manager-78b47f4478-8sv5k\" (UID: \"997602e8-2fb7-46e0-9720-266e09a79f4b\") " pod="openshift-lws-operator/lws-controller-manager-78b47f4478-8sv5k" Jul 14 20:30:16.880187 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:30:16.880154 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/997602e8-2fb7-46e0-9720-266e09a79f4b-cert\") pod \"lws-controller-manager-78b47f4478-8sv5k\" (UID: \"997602e8-2fb7-46e0-9720-266e09a79f4b\") " pod="openshift-lws-operator/lws-controller-manager-78b47f4478-8sv5k" Jul 14 20:30:16.880295 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:30:16.880268 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/997602e8-2fb7-46e0-9720-266e09a79f4b-metrics-cert\") pod \"lws-controller-manager-78b47f4478-8sv5k\" (UID: \"997602e8-2fb7-46e0-9720-266e09a79f4b\") " pod="openshift-lws-operator/lws-controller-manager-78b47f4478-8sv5k" Jul 14 20:30:16.888786 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:30:16.888760 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5lcwr\" (UniqueName: \"kubernetes.io/projected/997602e8-2fb7-46e0-9720-266e09a79f4b-kube-api-access-5lcwr\") pod \"lws-controller-manager-78b47f4478-8sv5k\" (UID: \"997602e8-2fb7-46e0-9720-266e09a79f4b\") " pod="openshift-lws-operator/lws-controller-manager-78b47f4478-8sv5k" Jul 14 20:30:17.045352 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:30:17.045309 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-78b47f4478-8sv5k" Jul 14 20:30:17.230079 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:30:17.230047 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-78b47f4478-8sv5k"] Jul 14 20:30:17.233568 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:30:17.233539 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod997602e8_2fb7_46e0_9720_266e09a79f4b.slice/crio-283903c24a16920954e0081621437554ee1662abaae4b51f4796f8b6e315a4f5 WatchSource:0}: Error finding container 283903c24a16920954e0081621437554ee1662abaae4b51f4796f8b6e315a4f5: Status 404 returned error can't find the container with id 283903c24a16920954e0081621437554ee1662abaae4b51f4796f8b6e315a4f5 Jul 14 20:30:17.457372 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:30:17.457338 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-78b47f4478-8sv5k" event={"ID":"997602e8-2fb7-46e0-9720-266e09a79f4b","Type":"ContainerStarted","Data":"283903c24a16920954e0081621437554ee1662abaae4b51f4796f8b6e315a4f5"} Jul 14 20:30:20.351660 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:30:20.351624 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:30:20.448070 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:30:20.448037 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:30:20.474303 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:30:20.474265 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-78b47f4478-8sv5k" event={"ID":"997602e8-2fb7-46e0-9720-266e09a79f4b","Type":"ContainerStarted","Data":"8a1059aa56f8b98840c7cb862d7af6c7c500a7e9c09f742eb668311ee1c831b8"} Jul 14 20:30:20.474486 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:30:20.474415 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-78b47f4478-8sv5k" Jul 14 20:30:20.496075 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:30:20.496024 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-78b47f4478-8sv5k" podStartSLOduration=1.937779073 podStartE2EDuration="4.496007752s" totalImagesPullingTime="2.558228679s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 20:30:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 20:30:20.493431875 +0000 UTC m=+479.414595199" watchObservedRunningTime="2026-07-14 20:30:20.496007752 +0000 UTC m=+479.417171061" Jul 14 20:30:21.085846 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:30:21.085805 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:30:21.522394 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:30:21.522362 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:30:21.658662 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:30:21.658626 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:30:21.691989 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:30:21.691955 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-79d49c7b87-bn8n6"] Jul 14 20:30:21.695460 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:30:21.695439 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-79d49c7b87-bn8n6" Jul 14 20:30:21.704644 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:30:21.704623 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 14 20:30:21.704753 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:30:21.704675 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 14 20:30:21.705275 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:30:21.705254 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 14 20:30:21.705539 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:30:21.705525 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 14 20:30:21.707168 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:30:21.707153 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-ptlqj\"" Jul 14 20:30:21.716544 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:30:21.716522 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-79d49c7b87-bn8n6"] Jul 14 20:30:21.723156 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:30:21.723106 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/af45e5e1-4dc3-4848-a094-f979fec50d9d-manifests\") pod \"opendatahub-operator-controller-manager-79d49c7b87-bn8n6\" (UID: \"af45e5e1-4dc3-4848-a094-f979fec50d9d\") " pod="opendatahub/opendatahub-operator-controller-manager-79d49c7b87-bn8n6" Jul 14 20:30:21.723274 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:30:21.723204 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/af45e5e1-4dc3-4848-a094-f979fec50d9d-webhook-cert\") pod \"opendatahub-operator-controller-manager-79d49c7b87-bn8n6\" (UID: \"af45e5e1-4dc3-4848-a094-f979fec50d9d\") " pod="opendatahub/opendatahub-operator-controller-manager-79d49c7b87-bn8n6" Jul 14 20:30:21.723274 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:30:21.723260 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/af45e5e1-4dc3-4848-a094-f979fec50d9d-tmp\") pod \"opendatahub-operator-controller-manager-79d49c7b87-bn8n6\" (UID: \"af45e5e1-4dc3-4848-a094-f979fec50d9d\") " pod="opendatahub/opendatahub-operator-controller-manager-79d49c7b87-bn8n6" Jul 14 20:30:21.723346 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:30:21.723318 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/af45e5e1-4dc3-4848-a094-f979fec50d9d-apiservice-cert\") pod \"opendatahub-operator-controller-manager-79d49c7b87-bn8n6\" (UID: \"af45e5e1-4dc3-4848-a094-f979fec50d9d\") " pod="opendatahub/opendatahub-operator-controller-manager-79d49c7b87-bn8n6" Jul 14 20:30:21.723390 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:30:21.723342 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kfgts\" (UniqueName: \"kubernetes.io/projected/af45e5e1-4dc3-4848-a094-f979fec50d9d-kube-api-access-kfgts\") pod \"opendatahub-operator-controller-manager-79d49c7b87-bn8n6\" (UID: \"af45e5e1-4dc3-4848-a094-f979fec50d9d\") " pod="opendatahub/opendatahub-operator-controller-manager-79d49c7b87-bn8n6" Jul 14 20:30:21.823797 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:30:21.823699 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/af45e5e1-4dc3-4848-a094-f979fec50d9d-apiservice-cert\") pod \"opendatahub-operator-controller-manager-79d49c7b87-bn8n6\" (UID: \"af45e5e1-4dc3-4848-a094-f979fec50d9d\") " pod="opendatahub/opendatahub-operator-controller-manager-79d49c7b87-bn8n6" Jul 14 20:30:21.823797 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:30:21.823734 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kfgts\" (UniqueName: \"kubernetes.io/projected/af45e5e1-4dc3-4848-a094-f979fec50d9d-kube-api-access-kfgts\") pod \"opendatahub-operator-controller-manager-79d49c7b87-bn8n6\" (UID: \"af45e5e1-4dc3-4848-a094-f979fec50d9d\") " pod="opendatahub/opendatahub-operator-controller-manager-79d49c7b87-bn8n6" Jul 14 20:30:21.823797 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:30:21.823769 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/af45e5e1-4dc3-4848-a094-f979fec50d9d-manifests\") pod \"opendatahub-operator-controller-manager-79d49c7b87-bn8n6\" (UID: \"af45e5e1-4dc3-4848-a094-f979fec50d9d\") " pod="opendatahub/opendatahub-operator-controller-manager-79d49c7b87-bn8n6" Jul 14 20:30:21.823797 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:30:21.823785 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/af45e5e1-4dc3-4848-a094-f979fec50d9d-webhook-cert\") pod \"opendatahub-operator-controller-manager-79d49c7b87-bn8n6\" (UID: \"af45e5e1-4dc3-4848-a094-f979fec50d9d\") " pod="opendatahub/opendatahub-operator-controller-manager-79d49c7b87-bn8n6" Jul 14 20:30:21.824177 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:30:21.823826 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/af45e5e1-4dc3-4848-a094-f979fec50d9d-tmp\") pod \"opendatahub-operator-controller-manager-79d49c7b87-bn8n6\" (UID: \"af45e5e1-4dc3-4848-a094-f979fec50d9d\") " pod="opendatahub/opendatahub-operator-controller-manager-79d49c7b87-bn8n6" Jul 14 20:30:21.824401 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:30:21.824371 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/af45e5e1-4dc3-4848-a094-f979fec50d9d-manifests\") pod \"opendatahub-operator-controller-manager-79d49c7b87-bn8n6\" (UID: \"af45e5e1-4dc3-4848-a094-f979fec50d9d\") " pod="opendatahub/opendatahub-operator-controller-manager-79d49c7b87-bn8n6" Jul 14 20:30:21.826395 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:30:21.826363 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/af45e5e1-4dc3-4848-a094-f979fec50d9d-tmp\") pod \"opendatahub-operator-controller-manager-79d49c7b87-bn8n6\" (UID: \"af45e5e1-4dc3-4848-a094-f979fec50d9d\") " pod="opendatahub/opendatahub-operator-controller-manager-79d49c7b87-bn8n6" Jul 14 20:30:21.826524 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:30:21.826500 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/af45e5e1-4dc3-4848-a094-f979fec50d9d-apiservice-cert\") pod \"opendatahub-operator-controller-manager-79d49c7b87-bn8n6\" (UID: \"af45e5e1-4dc3-4848-a094-f979fec50d9d\") " pod="opendatahub/opendatahub-operator-controller-manager-79d49c7b87-bn8n6" Jul 14 20:30:21.826570 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:30:21.826505 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/af45e5e1-4dc3-4848-a094-f979fec50d9d-webhook-cert\") pod \"opendatahub-operator-controller-manager-79d49c7b87-bn8n6\" (UID: \"af45e5e1-4dc3-4848-a094-f979fec50d9d\") " pod="opendatahub/opendatahub-operator-controller-manager-79d49c7b87-bn8n6" Jul 14 20:30:21.835096 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:30:21.835072 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kfgts\" (UniqueName: \"kubernetes.io/projected/af45e5e1-4dc3-4848-a094-f979fec50d9d-kube-api-access-kfgts\") pod \"opendatahub-operator-controller-manager-79d49c7b87-bn8n6\" (UID: \"af45e5e1-4dc3-4848-a094-f979fec50d9d\") " pod="opendatahub/opendatahub-operator-controller-manager-79d49c7b87-bn8n6" Jul 14 20:30:22.006397 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:30:22.006362 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-79d49c7b87-bn8n6" Jul 14 20:30:22.018698 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:30:22.018654 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:30:22.114519 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:30:22.114459 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:30:22.154187 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:30:22.153930 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-79d49c7b87-bn8n6"] Jul 14 20:30:22.158725 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:30:22.158699 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaf45e5e1_4dc3_4848_a094_f979fec50d9d.slice/crio-174bdfd670e86fd52a7f02f0fa6aeddd3e2b30f5e4456f8ad3f00e6da422fed2 WatchSource:0}: Error finding container 174bdfd670e86fd52a7f02f0fa6aeddd3e2b30f5e4456f8ad3f00e6da422fed2: Status 404 returned error can't find the container with id 174bdfd670e86fd52a7f02f0fa6aeddd3e2b30f5e4456f8ad3f00e6da422fed2 Jul 14 20:30:22.213307 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:30:22.213274 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:30:22.354051 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:30:22.353961 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:30:22.483434 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:30:22.483401 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-79d49c7b87-bn8n6" event={"ID":"af45e5e1-4dc3-4848-a094-f979fec50d9d","Type":"ContainerStarted","Data":"174bdfd670e86fd52a7f02f0fa6aeddd3e2b30f5e4456f8ad3f00e6da422fed2"} Jul 14 20:30:23.712908 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:30:23.712869 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:30:26.504026 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:30:26.503982 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-79d49c7b87-bn8n6" event={"ID":"af45e5e1-4dc3-4848-a094-f979fec50d9d","Type":"ContainerStarted","Data":"2fe61ac43a1684b870497f2e71d5cb7b0ce0c56ee787bf6e16ad105d91dffd1f"} Jul 14 20:30:27.514673 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:30:27.514636 2563 generic.go:356] "Generic (PLEG): container finished" podID="af45e5e1-4dc3-4848-a094-f979fec50d9d" containerID="2fe61ac43a1684b870497f2e71d5cb7b0ce0c56ee787bf6e16ad105d91dffd1f" exitCode=0 Jul 14 20:30:27.515079 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:30:27.514692 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-79d49c7b87-bn8n6" event={"ID":"af45e5e1-4dc3-4848-a094-f979fec50d9d","Type":"ContainerDied","Data":"2fe61ac43a1684b870497f2e71d5cb7b0ce0c56ee787bf6e16ad105d91dffd1f"} Jul 14 20:30:28.521730 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:30:28.521632 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-79d49c7b87-bn8n6" event={"ID":"af45e5e1-4dc3-4848-a094-f979fec50d9d","Type":"ContainerStarted","Data":"c95e2816ae8ab6d5edc530926a860d4aa7c1f9e26619eb73730cdc77dbb341ba"} Jul 14 20:30:28.522105 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:30:28.521783 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-79d49c7b87-bn8n6" Jul 14 20:30:28.546011 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:30:28.545955 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-79d49c7b87-bn8n6" podStartSLOduration=1.932728414 podStartE2EDuration="7.54589835s" totalImagesPullingTime="4.562526254s" totalInitContainerRuntime="1.050643682s" isStatefulPod=false podCreationTimestamp="2026-07-14 20:30:21 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 20:30:28.543889778 +0000 UTC m=+487.465053092" watchObservedRunningTime="2026-07-14 20:30:28.54589835 +0000 UTC m=+487.467061660" Jul 14 20:30:31.480073 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:30:31.480036 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-78b47f4478-8sv5k" Jul 14 20:30:39.527701 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:30:39.527670 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-79d49c7b87-bn8n6" Jul 14 20:30:41.833031 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:30:41.832995 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:30:42.030025 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:30:42.029985 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:30:45.101683 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:30:45.101645 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:30:45.295739 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:30:45.295698 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:30:45.468171 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:30:45.468107 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:30:45.575664 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:30:45.575634 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:30:46.052732 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:30:46.052695 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:30:46.156917 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:30:46.156878 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:30:46.433418 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:30:46.433379 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:30:46.629745 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:30:46.629690 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:30:46.710921 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:30:46.710808 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:30:47.707750 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:30:47.707709 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:30:47.979403 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:30:47.979314 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:30:52.506165 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:30:52.506113 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:30:52.607051 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:30:52.607014 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:30:53.013738 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:30:53.013709 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45cgj7"] Jul 14 20:30:53.016431 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:30:53.016413 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45cgj7" Jul 14 20:30:53.020000 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:30:53.019975 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 14 20:30:53.020169 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:30:53.020107 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 14 20:30:53.020169 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:30:53.020104 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 14 20:30:53.020539 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:30:53.020524 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-zss6m\"" Jul 14 20:30:53.041635 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:30:53.041598 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45cgj7"] Jul 14 20:30:53.100732 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:30:53.100690 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c4990631-d6e1-46cd-afdb-0692ca08c6ba-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf45cgj7\" (UID: \"c4990631-d6e1-46cd-afdb-0692ca08c6ba\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45cgj7" Jul 14 20:30:53.100732 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:30:53.100733 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c4990631-d6e1-46cd-afdb-0692ca08c6ba-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf45cgj7\" (UID: \"c4990631-d6e1-46cd-afdb-0692ca08c6ba\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45cgj7" Jul 14 20:30:53.100953 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:30:53.100759 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c4990631-d6e1-46cd-afdb-0692ca08c6ba-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf45cgj7\" (UID: \"c4990631-d6e1-46cd-afdb-0692ca08c6ba\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45cgj7" Jul 14 20:30:53.100953 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:30:53.100781 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c4990631-d6e1-46cd-afdb-0692ca08c6ba-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf45cgj7\" (UID: \"c4990631-d6e1-46cd-afdb-0692ca08c6ba\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45cgj7" Jul 14 20:30:53.100953 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:30:53.100810 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c4990631-d6e1-46cd-afdb-0692ca08c6ba-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf45cgj7\" (UID: \"c4990631-d6e1-46cd-afdb-0692ca08c6ba\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45cgj7" Jul 14 20:30:53.100953 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:30:53.100843 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dn77l\" (UniqueName: \"kubernetes.io/projected/c4990631-d6e1-46cd-afdb-0692ca08c6ba-kube-api-access-dn77l\") pod \"data-science-gateway-data-science-gateway-class-6576859cf45cgj7\" (UID: \"c4990631-d6e1-46cd-afdb-0692ca08c6ba\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45cgj7" Jul 14 20:30:53.100953 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:30:53.100881 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c4990631-d6e1-46cd-afdb-0692ca08c6ba-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf45cgj7\" (UID: \"c4990631-d6e1-46cd-afdb-0692ca08c6ba\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45cgj7" Jul 14 20:30:53.100953 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:30:53.100900 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c4990631-d6e1-46cd-afdb-0692ca08c6ba-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf45cgj7\" (UID: \"c4990631-d6e1-46cd-afdb-0692ca08c6ba\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45cgj7" Jul 14 20:30:53.100953 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:30:53.100944 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c4990631-d6e1-46cd-afdb-0692ca08c6ba-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf45cgj7\" (UID: \"c4990631-d6e1-46cd-afdb-0692ca08c6ba\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45cgj7" Jul 14 20:30:53.201865 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:30:53.201823 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c4990631-d6e1-46cd-afdb-0692ca08c6ba-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf45cgj7\" (UID: \"c4990631-d6e1-46cd-afdb-0692ca08c6ba\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45cgj7" Jul 14 20:30:53.201865 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:30:53.201867 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c4990631-d6e1-46cd-afdb-0692ca08c6ba-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf45cgj7\" (UID: \"c4990631-d6e1-46cd-afdb-0692ca08c6ba\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45cgj7" Jul 14 20:30:53.202175 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:30:53.201887 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c4990631-d6e1-46cd-afdb-0692ca08c6ba-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf45cgj7\" (UID: \"c4990631-d6e1-46cd-afdb-0692ca08c6ba\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45cgj7" Jul 14 20:30:53.202175 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:30:53.201914 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c4990631-d6e1-46cd-afdb-0692ca08c6ba-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf45cgj7\" (UID: \"c4990631-d6e1-46cd-afdb-0692ca08c6ba\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45cgj7" Jul 14 20:30:53.202175 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:30:53.201938 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c4990631-d6e1-46cd-afdb-0692ca08c6ba-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf45cgj7\" (UID: \"c4990631-d6e1-46cd-afdb-0692ca08c6ba\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45cgj7" Jul 14 20:30:53.202175 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:30:53.201972 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dn77l\" (UniqueName: \"kubernetes.io/projected/c4990631-d6e1-46cd-afdb-0692ca08c6ba-kube-api-access-dn77l\") pod \"data-science-gateway-data-science-gateway-class-6576859cf45cgj7\" (UID: \"c4990631-d6e1-46cd-afdb-0692ca08c6ba\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45cgj7" Jul 14 20:30:53.202175 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:30:53.202032 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c4990631-d6e1-46cd-afdb-0692ca08c6ba-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf45cgj7\" (UID: \"c4990631-d6e1-46cd-afdb-0692ca08c6ba\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45cgj7" Jul 14 20:30:53.202175 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:30:53.202056 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c4990631-d6e1-46cd-afdb-0692ca08c6ba-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf45cgj7\" (UID: \"c4990631-d6e1-46cd-afdb-0692ca08c6ba\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45cgj7" Jul 14 20:30:53.202175 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:30:53.202087 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c4990631-d6e1-46cd-afdb-0692ca08c6ba-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf45cgj7\" (UID: \"c4990631-d6e1-46cd-afdb-0692ca08c6ba\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45cgj7" Jul 14 20:30:53.202567 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:30:53.202379 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c4990631-d6e1-46cd-afdb-0692ca08c6ba-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf45cgj7\" (UID: \"c4990631-d6e1-46cd-afdb-0692ca08c6ba\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45cgj7" Jul 14 20:30:53.202567 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:30:53.202471 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c4990631-d6e1-46cd-afdb-0692ca08c6ba-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf45cgj7\" (UID: \"c4990631-d6e1-46cd-afdb-0692ca08c6ba\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45cgj7" Jul 14 20:30:53.202670 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:30:53.202643 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c4990631-d6e1-46cd-afdb-0692ca08c6ba-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf45cgj7\" (UID: \"c4990631-d6e1-46cd-afdb-0692ca08c6ba\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45cgj7" Jul 14 20:30:53.203005 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:30:53.202980 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c4990631-d6e1-46cd-afdb-0692ca08c6ba-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf45cgj7\" (UID: \"c4990631-d6e1-46cd-afdb-0692ca08c6ba\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45cgj7" Jul 14 20:30:53.203114 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:30:53.203051 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c4990631-d6e1-46cd-afdb-0692ca08c6ba-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf45cgj7\" (UID: \"c4990631-d6e1-46cd-afdb-0692ca08c6ba\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45cgj7" Jul 14 20:30:53.204567 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:30:53.204546 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c4990631-d6e1-46cd-afdb-0692ca08c6ba-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf45cgj7\" (UID: \"c4990631-d6e1-46cd-afdb-0692ca08c6ba\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45cgj7" Jul 14 20:30:53.204840 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:30:53.204819 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c4990631-d6e1-46cd-afdb-0692ca08c6ba-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf45cgj7\" (UID: \"c4990631-d6e1-46cd-afdb-0692ca08c6ba\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45cgj7" Jul 14 20:30:53.210753 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:30:53.210728 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c4990631-d6e1-46cd-afdb-0692ca08c6ba-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf45cgj7\" (UID: \"c4990631-d6e1-46cd-afdb-0692ca08c6ba\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45cgj7" Jul 14 20:30:53.210846 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:30:53.210801 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dn77l\" (UniqueName: \"kubernetes.io/projected/c4990631-d6e1-46cd-afdb-0692ca08c6ba-kube-api-access-dn77l\") pod \"data-science-gateway-data-science-gateway-class-6576859cf45cgj7\" (UID: \"c4990631-d6e1-46cd-afdb-0692ca08c6ba\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45cgj7" Jul 14 20:30:53.327402 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:30:53.327305 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45cgj7" Jul 14 20:30:53.392440 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:30:53.392400 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:30:53.494314 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:30:53.494288 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45cgj7"] Jul 14 20:30:53.496596 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:30:53.496569 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc4990631_d6e1_46cd_afdb_0692ca08c6ba.slice/crio-d0e83105214c5f7205fef9ce9b0066bdcc99c0f526fadc47509e34d03ab908ca WatchSource:0}: Error finding container d0e83105214c5f7205fef9ce9b0066bdcc99c0f526fadc47509e34d03ab908ca: Status 404 returned error can't find the container with id d0e83105214c5f7205fef9ce9b0066bdcc99c0f526fadc47509e34d03ab908ca Jul 14 20:30:53.547709 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:30:53.547673 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:30:53.632987 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:30:53.632894 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45cgj7" event={"ID":"c4990631-d6e1-46cd-afdb-0692ca08c6ba","Type":"ContainerStarted","Data":"d0e83105214c5f7205fef9ce9b0066bdcc99c0f526fadc47509e34d03ab908ca"} Jul 14 20:30:55.481165 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:30:55.481080 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-8jmfx"] Jul 14 20:30:55.483789 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:30:55.483763 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-8jmfx" Jul 14 20:30:55.488587 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:30:55.488533 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-v7vb9\"" Jul 14 20:30:55.488927 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:30:55.488902 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Jul 14 20:30:55.489042 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:30:55.488986 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Jul 14 20:30:55.498436 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:30:55.498415 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Jul 14 20:30:55.535591 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:30:55.535558 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-8jmfx"] Jul 14 20:30:55.625329 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:30:55.625295 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/42b909af-0bd9-4036-b180-2c5cd4394b30-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-8jmfx\" (UID: \"42b909af-0bd9-4036-b180-2c5cd4394b30\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-8jmfx" Jul 14 20:30:55.625527 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:30:55.625331 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/42b909af-0bd9-4036-b180-2c5cd4394b30-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-8jmfx\" (UID: \"42b909af-0bd9-4036-b180-2c5cd4394b30\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-8jmfx" Jul 14 20:30:55.625527 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:30:55.625374 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kknfk\" (UniqueName: \"kubernetes.io/projected/42b909af-0bd9-4036-b180-2c5cd4394b30-kube-api-access-kknfk\") pod \"istiod-openshift-gateway-94bb8fbfd-8jmfx\" (UID: \"42b909af-0bd9-4036-b180-2c5cd4394b30\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-8jmfx" Jul 14 20:30:55.625527 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:30:55.625429 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/42b909af-0bd9-4036-b180-2c5cd4394b30-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-8jmfx\" (UID: \"42b909af-0bd9-4036-b180-2c5cd4394b30\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-8jmfx" Jul 14 20:30:55.625527 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:30:55.625460 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/42b909af-0bd9-4036-b180-2c5cd4394b30-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-8jmfx\" (UID: \"42b909af-0bd9-4036-b180-2c5cd4394b30\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-8jmfx" Jul 14 20:30:55.625527 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:30:55.625497 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/42b909af-0bd9-4036-b180-2c5cd4394b30-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-8jmfx\" (UID: \"42b909af-0bd9-4036-b180-2c5cd4394b30\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-8jmfx" Jul 14 20:30:55.625527 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:30:55.625520 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/42b909af-0bd9-4036-b180-2c5cd4394b30-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-8jmfx\" (UID: \"42b909af-0bd9-4036-b180-2c5cd4394b30\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-8jmfx" Jul 14 20:30:55.726436 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:30:55.726398 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/42b909af-0bd9-4036-b180-2c5cd4394b30-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-8jmfx\" (UID: \"42b909af-0bd9-4036-b180-2c5cd4394b30\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-8jmfx" Jul 14 20:30:55.726631 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:30:55.726464 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/42b909af-0bd9-4036-b180-2c5cd4394b30-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-8jmfx\" (UID: \"42b909af-0bd9-4036-b180-2c5cd4394b30\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-8jmfx" Jul 14 20:30:55.726631 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:30:55.726580 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/42b909af-0bd9-4036-b180-2c5cd4394b30-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-8jmfx\" (UID: \"42b909af-0bd9-4036-b180-2c5cd4394b30\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-8jmfx" Jul 14 20:30:55.726763 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:30:55.726644 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/42b909af-0bd9-4036-b180-2c5cd4394b30-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-8jmfx\" (UID: \"42b909af-0bd9-4036-b180-2c5cd4394b30\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-8jmfx" Jul 14 20:30:55.726831 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:30:55.726724 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/42b909af-0bd9-4036-b180-2c5cd4394b30-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-8jmfx\" (UID: \"42b909af-0bd9-4036-b180-2c5cd4394b30\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-8jmfx" Jul 14 20:30:55.726831 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:30:55.726811 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/42b909af-0bd9-4036-b180-2c5cd4394b30-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-8jmfx\" (UID: \"42b909af-0bd9-4036-b180-2c5cd4394b30\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-8jmfx" Jul 14 20:30:55.726930 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:30:55.726844 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kknfk\" (UniqueName: \"kubernetes.io/projected/42b909af-0bd9-4036-b180-2c5cd4394b30-kube-api-access-kknfk\") pod \"istiod-openshift-gateway-94bb8fbfd-8jmfx\" (UID: \"42b909af-0bd9-4036-b180-2c5cd4394b30\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-8jmfx" Jul 14 20:30:55.728316 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:30:55.728288 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/42b909af-0bd9-4036-b180-2c5cd4394b30-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-8jmfx\" (UID: \"42b909af-0bd9-4036-b180-2c5cd4394b30\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-8jmfx" Jul 14 20:30:55.730986 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:30:55.730937 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/42b909af-0bd9-4036-b180-2c5cd4394b30-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-8jmfx\" (UID: \"42b909af-0bd9-4036-b180-2c5cd4394b30\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-8jmfx" Jul 14 20:30:55.731112 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:30:55.731072 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/42b909af-0bd9-4036-b180-2c5cd4394b30-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-8jmfx\" (UID: \"42b909af-0bd9-4036-b180-2c5cd4394b30\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-8jmfx" Jul 14 20:30:55.731477 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:30:55.731404 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/42b909af-0bd9-4036-b180-2c5cd4394b30-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-8jmfx\" (UID: \"42b909af-0bd9-4036-b180-2c5cd4394b30\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-8jmfx" Jul 14 20:30:55.731707 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:30:55.731685 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/42b909af-0bd9-4036-b180-2c5cd4394b30-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-8jmfx\" (UID: \"42b909af-0bd9-4036-b180-2c5cd4394b30\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-8jmfx" Jul 14 20:30:55.736261 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:30:55.736213 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kknfk\" (UniqueName: \"kubernetes.io/projected/42b909af-0bd9-4036-b180-2c5cd4394b30-kube-api-access-kknfk\") pod \"istiod-openshift-gateway-94bb8fbfd-8jmfx\" (UID: \"42b909af-0bd9-4036-b180-2c5cd4394b30\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-8jmfx" Jul 14 20:30:55.737672 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:30:55.737647 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/42b909af-0bd9-4036-b180-2c5cd4394b30-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-8jmfx\" (UID: \"42b909af-0bd9-4036-b180-2c5cd4394b30\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-8jmfx" Jul 14 20:30:55.795558 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:30:55.795525 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-8jmfx" Jul 14 20:30:55.966076 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:30:55.966049 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-8jmfx"] Jul 14 20:30:55.969056 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:30:55.969022 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod42b909af_0bd9_4036_b180_2c5cd4394b30.slice/crio-e7c7f86e93cc2c402c514c43d3c2f6fa5c30471603e65e5dbb75e33af780517b WatchSource:0}: Error finding container e7c7f86e93cc2c402c514c43d3c2f6fa5c30471603e65e5dbb75e33af780517b: Status 404 returned error can't find the container with id e7c7f86e93cc2c402c514c43d3c2f6fa5c30471603e65e5dbb75e33af780517b Jul 14 20:30:56.650939 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:30:56.650898 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-8jmfx" event={"ID":"42b909af-0bd9-4036-b180-2c5cd4394b30","Type":"ContainerStarted","Data":"e7c7f86e93cc2c402c514c43d3c2f6fa5c30471603e65e5dbb75e33af780517b"} Jul 14 20:30:57.579002 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:30:57.578959 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"} Jul 14 20:30:57.579114 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:30:57.579037 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"} Jul 14 20:30:57.579114 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:30:57.579073 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"} Jul 14 20:30:58.666190 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:30:58.666148 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45cgj7" event={"ID":"c4990631-d6e1-46cd-afdb-0692ca08c6ba","Type":"ContainerStarted","Data":"bf9df2293878faec530f313ab2df1046620a8e3d7663a35a69c36a70d2d042f5"} Jul 14 20:30:59.217360 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:30:59.217319 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:30:59.318310 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:30:59.318271 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:30:59.328257 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:30:59.328175 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45cgj7" Jul 14 20:30:59.333008 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:30:59.332982 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45cgj7" Jul 14 20:30:59.360356 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:30:59.360269 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45cgj7" podStartSLOduration=3.280206794 podStartE2EDuration="7.360249547s" totalImagesPullingTime="4.080042753s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 20:30:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 20:30:58.708726151 +0000 UTC m=+517.629889460" watchObservedRunningTime="2026-07-14 20:30:59.360249547 +0000 UTC m=+518.281412857" Jul 14 20:30:59.478528 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:30:59.478487 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"} Jul 14 20:30:59.478616 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:30:59.478552 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"} Jul 14 20:30:59.673030 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:30:59.672995 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-8jmfx" event={"ID":"42b909af-0bd9-4036-b180-2c5cd4394b30","Type":"ContainerStarted","Data":"aad08dfb9b1c696eff5f1f04b4be9252f85f3097dd96500bc94ac9081594d25e"} Jul 14 20:30:59.673468 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:30:59.673199 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-8jmfx" Jul 14 20:30:59.673468 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:30:59.673247 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45cgj7" Jul 14 20:30:59.674470 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:30:59.674454 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf45cgj7" Jul 14 20:30:59.695258 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:30:59.695200 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-8jmfx" podStartSLOduration=1.188703281 podStartE2EDuration="4.695179999s" totalImagesPullingTime="3.506476718s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 20:30:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 20:30:59.692921247 +0000 UTC m=+518.614084561" watchObservedRunningTime="2026-07-14 20:30:59.695179999 +0000 UTC m=+518.616343309" Jul 14 20:31:00.680274 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:31:00.680240 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-8jmfx" Jul 14 20:31:02.059980 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:31:02.059941 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:31:44.976494 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:31:44.976451 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:31:45.035509 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:31:45.035465 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:31:45.494925 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:31:45.494890 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:31:45.595364 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:31:45.595330 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:31:54.459331 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:31:54.459289 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:31:54.558737 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:31:54.558707 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:31:56.177427 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:31:56.177394 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:31:56.236001 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:31:56.235960 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:32:01.579777 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:32:01.579735 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:32:02.052181 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:32:02.052120 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:32:03.693393 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:32:03.693349 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:32:11.556364 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:32:11.556333 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:32:11.653493 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:32:11.653455 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:32:11.903188 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:32:11.903075 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:32:15.340248 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:32:15.340203 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:32:16.910483 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:32:16.910438 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:32:17.388269 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:32:17.388179 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:32:18.171440 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:32:18.171408 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:32:18.265472 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:32:18.265435 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:32:20.114866 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:32:20.114829 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:32:21.490537 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:32:21.490512 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hzbx8_9588ab98-33e8-4c89-807d-a805240f1b81/ovn-acl-logging/0.log" Jul 14 20:32:21.491001 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:32:21.490719 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hzbx8_9588ab98-33e8-4c89-807d-a805240f1b81/ovn-acl-logging/0.log" Jul 14 20:32:24.288852 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:32:24.288812 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:32:24.750197 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:32:24.750158 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:32:25.024810 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:32:25.024648 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:32:26.518179 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:32:26.518112 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:32:26.620098 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:32:26.620043 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:32:28.570036 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:32:28.569994 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:32:28.962085 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:32:28.962038 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:32:35.728731 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:32:35.728696 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:32:35.836771 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:32:35.836738 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:32:36.007440 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:32:36.007349 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:32:36.107000 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:32:36.106961 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:32:38.326703 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:32:38.326665 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:32:38.436589 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:32:38.436554 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:32:43.635824 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:32:43.635732 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:32:43.736035 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:32:43.736000 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:32:46.726273 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:32:46.726232 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:32:46.824694 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:32:46.824656 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:32:55.567437 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:32:55.567396 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:32:55.666047 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:32:55.666013 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:32:58.011337 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:32:58.011297 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:32:58.108458 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:32:58.108425 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:33:02.673655 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:33:02.673622 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:33:02.774505 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:33:02.774471 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:33:02.875146 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:33:02.875096 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:33:06.757070 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:33:06.757034 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:33:06.859142 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:33:06.859090 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:33:07.742326 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:33:07.742294 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-mzmqf"] Jul 14 20:33:07.744585 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:33:07.744567 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-mzmqf" Jul 14 20:33:07.746765 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:33:07.746745 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-vvmf4\"" Jul 14 20:33:07.757743 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:33:07.757717 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-mzmqf"] Jul 14 20:33:07.768163 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:33:07.768113 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:33:07.796546 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:33:07.796510 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/afb7dcce-c4ff-4a07-983f-8f52fa1409bd-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-mzmqf\" (UID: \"afb7dcce-c4ff-4a07-983f-8f52fa1409bd\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-mzmqf" Jul 14 20:33:07.796546 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:33:07.796552 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9g2g7\" (UniqueName: \"kubernetes.io/projected/afb7dcce-c4ff-4a07-983f-8f52fa1409bd-kube-api-access-9g2g7\") pod \"maas-default-gateway-openshift-default-56654dcf7-mzmqf\" (UID: \"afb7dcce-c4ff-4a07-983f-8f52fa1409bd\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-mzmqf" Jul 14 20:33:07.796750 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:33:07.796579 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/afb7dcce-c4ff-4a07-983f-8f52fa1409bd-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-mzmqf\" (UID: \"afb7dcce-c4ff-4a07-983f-8f52fa1409bd\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-mzmqf" Jul 14 20:33:07.796750 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:33:07.796643 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/afb7dcce-c4ff-4a07-983f-8f52fa1409bd-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-mzmqf\" (UID: \"afb7dcce-c4ff-4a07-983f-8f52fa1409bd\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-mzmqf" Jul 14 20:33:07.796750 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:33:07.796730 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/afb7dcce-c4ff-4a07-983f-8f52fa1409bd-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-mzmqf\" (UID: \"afb7dcce-c4ff-4a07-983f-8f52fa1409bd\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-mzmqf" Jul 14 20:33:07.796846 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:33:07.796777 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/afb7dcce-c4ff-4a07-983f-8f52fa1409bd-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-mzmqf\" (UID: \"afb7dcce-c4ff-4a07-983f-8f52fa1409bd\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-mzmqf" Jul 14 20:33:07.796846 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:33:07.796799 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/afb7dcce-c4ff-4a07-983f-8f52fa1409bd-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-mzmqf\" (UID: \"afb7dcce-c4ff-4a07-983f-8f52fa1409bd\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-mzmqf" Jul 14 20:33:07.796846 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:33:07.796828 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/afb7dcce-c4ff-4a07-983f-8f52fa1409bd-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-mzmqf\" (UID: \"afb7dcce-c4ff-4a07-983f-8f52fa1409bd\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-mzmqf" Jul 14 20:33:07.796937 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:33:07.796845 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/afb7dcce-c4ff-4a07-983f-8f52fa1409bd-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-mzmqf\" (UID: \"afb7dcce-c4ff-4a07-983f-8f52fa1409bd\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-mzmqf" Jul 14 20:33:07.897513 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:33:07.897474 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/afb7dcce-c4ff-4a07-983f-8f52fa1409bd-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-mzmqf\" (UID: \"afb7dcce-c4ff-4a07-983f-8f52fa1409bd\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-mzmqf" Jul 14 20:33:07.897711 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:33:07.897527 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9g2g7\" (UniqueName: \"kubernetes.io/projected/afb7dcce-c4ff-4a07-983f-8f52fa1409bd-kube-api-access-9g2g7\") pod \"maas-default-gateway-openshift-default-56654dcf7-mzmqf\" (UID: \"afb7dcce-c4ff-4a07-983f-8f52fa1409bd\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-mzmqf" Jul 14 20:33:07.897711 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:33:07.897566 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/afb7dcce-c4ff-4a07-983f-8f52fa1409bd-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-mzmqf\" (UID: \"afb7dcce-c4ff-4a07-983f-8f52fa1409bd\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-mzmqf" Jul 14 20:33:07.897711 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:33:07.897593 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/afb7dcce-c4ff-4a07-983f-8f52fa1409bd-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-mzmqf\" (UID: \"afb7dcce-c4ff-4a07-983f-8f52fa1409bd\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-mzmqf" Jul 14 20:33:07.897711 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:33:07.897662 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/afb7dcce-c4ff-4a07-983f-8f52fa1409bd-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-mzmqf\" (UID: \"afb7dcce-c4ff-4a07-983f-8f52fa1409bd\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-mzmqf" Jul 14 20:33:07.897991 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:33:07.897722 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/afb7dcce-c4ff-4a07-983f-8f52fa1409bd-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-mzmqf\" (UID: \"afb7dcce-c4ff-4a07-983f-8f52fa1409bd\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-mzmqf" Jul 14 20:33:07.897991 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:33:07.897745 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/afb7dcce-c4ff-4a07-983f-8f52fa1409bd-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-mzmqf\" (UID: \"afb7dcce-c4ff-4a07-983f-8f52fa1409bd\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-mzmqf" Jul 14 20:33:07.897991 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:33:07.897780 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/afb7dcce-c4ff-4a07-983f-8f52fa1409bd-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-mzmqf\" (UID: \"afb7dcce-c4ff-4a07-983f-8f52fa1409bd\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-mzmqf" Jul 14 20:33:07.897991 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:33:07.897811 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/afb7dcce-c4ff-4a07-983f-8f52fa1409bd-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-mzmqf\" (UID: \"afb7dcce-c4ff-4a07-983f-8f52fa1409bd\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-mzmqf" Jul 14 20:33:07.898225 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:33:07.898165 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/afb7dcce-c4ff-4a07-983f-8f52fa1409bd-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-mzmqf\" (UID: \"afb7dcce-c4ff-4a07-983f-8f52fa1409bd\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-mzmqf" Jul 14 20:33:07.898225 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:33:07.898209 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/afb7dcce-c4ff-4a07-983f-8f52fa1409bd-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-mzmqf\" (UID: \"afb7dcce-c4ff-4a07-983f-8f52fa1409bd\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-mzmqf" Jul 14 20:33:07.898439 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:33:07.898367 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/afb7dcce-c4ff-4a07-983f-8f52fa1409bd-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-mzmqf\" (UID: \"afb7dcce-c4ff-4a07-983f-8f52fa1409bd\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-mzmqf" Jul 14 20:33:07.898516 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:33:07.898474 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/afb7dcce-c4ff-4a07-983f-8f52fa1409bd-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-mzmqf\" (UID: \"afb7dcce-c4ff-4a07-983f-8f52fa1409bd\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-mzmqf" Jul 14 20:33:07.898790 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:33:07.898768 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/afb7dcce-c4ff-4a07-983f-8f52fa1409bd-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-mzmqf\" (UID: \"afb7dcce-c4ff-4a07-983f-8f52fa1409bd\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-mzmqf" Jul 14 20:33:07.900079 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:33:07.900051 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/afb7dcce-c4ff-4a07-983f-8f52fa1409bd-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-mzmqf\" (UID: \"afb7dcce-c4ff-4a07-983f-8f52fa1409bd\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-mzmqf" Jul 14 20:33:07.900292 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:33:07.900274 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/afb7dcce-c4ff-4a07-983f-8f52fa1409bd-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-mzmqf\" (UID: \"afb7dcce-c4ff-4a07-983f-8f52fa1409bd\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-mzmqf" Jul 14 20:33:07.906396 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:33:07.906369 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9g2g7\" (UniqueName: \"kubernetes.io/projected/afb7dcce-c4ff-4a07-983f-8f52fa1409bd-kube-api-access-9g2g7\") pod \"maas-default-gateway-openshift-default-56654dcf7-mzmqf\" (UID: \"afb7dcce-c4ff-4a07-983f-8f52fa1409bd\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-mzmqf" Jul 14 20:33:07.906704 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:33:07.906682 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/afb7dcce-c4ff-4a07-983f-8f52fa1409bd-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-mzmqf\" (UID: \"afb7dcce-c4ff-4a07-983f-8f52fa1409bd\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-mzmqf" Jul 14 20:33:08.046291 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:33:08.046207 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:33:08.055218 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:33:08.055191 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-mzmqf" Jul 14 20:33:08.198637 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:33:08.198603 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-mzmqf"] Jul 14 20:33:08.201981 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:33:08.201957 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podafb7dcce_c4ff_4a07_983f_8f52fa1409bd.slice/crio-dd9bf4a3f8b7cd583e20bc5d98fb6f7da75b0254d15a241d44b35dd9ba30500c WatchSource:0}: Error finding container dd9bf4a3f8b7cd583e20bc5d98fb6f7da75b0254d15a241d44b35dd9ba30500c: Status 404 returned error can't find the container with id dd9bf4a3f8b7cd583e20bc5d98fb6f7da75b0254d15a241d44b35dd9ba30500c Jul 14 20:33:08.204183 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:33:08.204151 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"} Jul 14 20:33:08.204294 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:33:08.204217 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"} Jul 14 20:33:08.204294 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:33:08.204252 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"} Jul 14 20:33:08.243560 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:33:08.243535 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-mzmqf" event={"ID":"afb7dcce-c4ff-4a07-983f-8f52fa1409bd","Type":"ContainerStarted","Data":"dd9bf4a3f8b7cd583e20bc5d98fb6f7da75b0254d15a241d44b35dd9ba30500c"} Jul 14 20:33:09.249203 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:33:09.249163 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-mzmqf" event={"ID":"afb7dcce-c4ff-4a07-983f-8f52fa1409bd","Type":"ContainerStarted","Data":"dfc3d8a4aabb4e7eb8a956044e4f20a8ebf997c52fc29203badc04f8aa6d6612"} Jul 14 20:33:09.268940 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:33:09.268882 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-mzmqf" podStartSLOduration=2.2688652510000002 podStartE2EDuration="2.268865251s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 20:33:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 20:33:09.267328222 +0000 UTC m=+648.188491530" watchObservedRunningTime="2026-07-14 20:33:09.268865251 +0000 UTC m=+648.190028584" Jul 14 20:33:10.055579 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:33:10.055536 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-mzmqf" Jul 14 20:33:10.060580 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:33:10.060551 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-mzmqf" Jul 14 20:33:10.253884 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:33:10.253858 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-mzmqf" Jul 14 20:33:10.254959 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:33:10.254943 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-mzmqf" Jul 14 20:33:10.289620 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:33:10.289588 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:33:10.406916 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:33:10.406878 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:33:10.508154 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:33:10.508092 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:33:11.132410 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:33:11.132362 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:33:11.223936 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:33:11.223904 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:33:16.745369 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:33:16.745336 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:33:16.845645 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:33:16.845607 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:33:17.375319 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:33:17.375280 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:33:17.487567 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:33:17.487529 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:33:19.040375 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:33:19.040335 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:33:21.159071 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:33:21.159032 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:33:21.218342 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:33:21.218303 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:33:25.160660 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:33:25.160627 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:33:25.616035 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:33:25.615944 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:33:31.581636 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:33:31.581601 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:33:31.685229 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:33:31.685191 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:33:32.064640 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:33:32.064583 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:33:43.870255 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:33:43.870219 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:33:44.930679 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:33:44.930645 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:33:50.581391 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:33:50.581357 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:33:50.689170 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:33:50.689138 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:33:50.784260 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:33:50.784225 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:33:50.788348 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:33:50.788321 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-5c7b6db686-g9zvp"] Jul 14 20:33:50.790525 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:33:50.790507 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5c7b6db686-g9zvp" Jul 14 20:33:50.792737 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:33:50.792717 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jul 14 20:33:50.792989 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:33:50.792973 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-2zmfq\"" Jul 14 20:33:50.793399 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:33:50.793382 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jul 14 20:33:50.804870 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:33:50.804843 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5c7b6db686-g9zvp"] Jul 14 20:33:50.886902 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:33:50.886813 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d10dc44c-c8b0-4f54-9c7e-fb99fd87d51d-webhook-cert\") pod \"maas-controller-5c7b6db686-g9zvp\" (UID: \"d10dc44c-c8b0-4f54-9c7e-fb99fd87d51d\") " pod="opendatahub/maas-controller-5c7b6db686-g9zvp" Jul 14 20:33:50.887061 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:33:50.886918 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-btpcb\" (UniqueName: \"kubernetes.io/projected/d10dc44c-c8b0-4f54-9c7e-fb99fd87d51d-kube-api-access-btpcb\") pod \"maas-controller-5c7b6db686-g9zvp\" (UID: \"d10dc44c-c8b0-4f54-9c7e-fb99fd87d51d\") " pod="opendatahub/maas-controller-5c7b6db686-g9zvp" Jul 14 20:33:50.987865 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:33:50.987826 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d10dc44c-c8b0-4f54-9c7e-fb99fd87d51d-webhook-cert\") pod \"maas-controller-5c7b6db686-g9zvp\" (UID: \"d10dc44c-c8b0-4f54-9c7e-fb99fd87d51d\") " pod="opendatahub/maas-controller-5c7b6db686-g9zvp" Jul 14 20:33:50.988064 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:33:50.987912 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-btpcb\" (UniqueName: \"kubernetes.io/projected/d10dc44c-c8b0-4f54-9c7e-fb99fd87d51d-kube-api-access-btpcb\") pod \"maas-controller-5c7b6db686-g9zvp\" (UID: \"d10dc44c-c8b0-4f54-9c7e-fb99fd87d51d\") " pod="opendatahub/maas-controller-5c7b6db686-g9zvp" Jul 14 20:33:50.988064 ip-10-0-141-61 kubenswrapper[2563]: E0714 20:33:50.987992 2563 secret.go:189] Couldn't get secret opendatahub/maas-controller-webhook-cert: secret "maas-controller-webhook-cert" not found Jul 14 20:33:50.988064 ip-10-0-141-61 kubenswrapper[2563]: E0714 20:33:50.988056 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d10dc44c-c8b0-4f54-9c7e-fb99fd87d51d-webhook-cert podName:d10dc44c-c8b0-4f54-9c7e-fb99fd87d51d nodeName:}" failed. No retries permitted until 2026-07-14 20:33:51.48803967 +0000 UTC m=+690.409202957 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "webhook-cert" (UniqueName: "kubernetes.io/secret/d10dc44c-c8b0-4f54-9c7e-fb99fd87d51d-webhook-cert") pod "maas-controller-5c7b6db686-g9zvp" (UID: "d10dc44c-c8b0-4f54-9c7e-fb99fd87d51d") : secret "maas-controller-webhook-cert" not found Jul 14 20:33:50.997159 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:33:50.997110 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-btpcb\" (UniqueName: \"kubernetes.io/projected/d10dc44c-c8b0-4f54-9c7e-fb99fd87d51d-kube-api-access-btpcb\") pod \"maas-controller-5c7b6db686-g9zvp\" (UID: \"d10dc44c-c8b0-4f54-9c7e-fb99fd87d51d\") " pod="opendatahub/maas-controller-5c7b6db686-g9zvp" Jul 14 20:33:51.150313 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:33:51.150210 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:33:51.492171 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:33:51.492090 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d10dc44c-c8b0-4f54-9c7e-fb99fd87d51d-webhook-cert\") pod \"maas-controller-5c7b6db686-g9zvp\" (UID: \"d10dc44c-c8b0-4f54-9c7e-fb99fd87d51d\") " pod="opendatahub/maas-controller-5c7b6db686-g9zvp" Jul 14 20:33:51.494760 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:33:51.494736 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d10dc44c-c8b0-4f54-9c7e-fb99fd87d51d-webhook-cert\") pod \"maas-controller-5c7b6db686-g9zvp\" (UID: \"d10dc44c-c8b0-4f54-9c7e-fb99fd87d51d\") " pod="opendatahub/maas-controller-5c7b6db686-g9zvp" Jul 14 20:33:51.701311 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:33:51.701253 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5c7b6db686-g9zvp" Jul 14 20:33:51.838365 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:33:51.838332 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5c7b6db686-g9zvp"] Jul 14 20:33:51.841604 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:33:51.841568 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd10dc44c_c8b0_4f54_9c7e_fb99fd87d51d.slice/crio-d9f93260e40fdac8ab5452767dae455a7c29cd3c7f8ecf1cc24d429becdcece4 WatchSource:0}: Error finding container d9f93260e40fdac8ab5452767dae455a7c29cd3c7f8ecf1cc24d429becdcece4: Status 404 returned error can't find the container with id d9f93260e40fdac8ab5452767dae455a7c29cd3c7f8ecf1cc24d429becdcece4 Jul 14 20:33:52.445163 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:33:52.445109 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5c7b6db686-g9zvp" event={"ID":"d10dc44c-c8b0-4f54-9c7e-fb99fd87d51d","Type":"ContainerStarted","Data":"d9f93260e40fdac8ab5452767dae455a7c29cd3c7f8ecf1cc24d429becdcece4"} Jul 14 20:33:55.462896 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:33:55.462862 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5c7b6db686-g9zvp" event={"ID":"d10dc44c-c8b0-4f54-9c7e-fb99fd87d51d","Type":"ContainerStarted","Data":"5a9da1253101c33bb9a9eb43257bb046a4ff79bdddd30a5fe52d3ce2e428d7a9"} Jul 14 20:33:55.463321 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:33:55.462986 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-5c7b6db686-g9zvp" Jul 14 20:33:55.502335 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:33:55.502298 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:33:55.566028 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:33:55.565998 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:34:06.469830 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:34:06.469800 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-5c7b6db686-g9zvp" Jul 14 20:34:06.486533 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:34:06.486473 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-5c7b6db686-g9zvp" podStartSLOduration=13.600920738 podStartE2EDuration="16.486455777s" totalImagesPullingTime="2.885535039s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 20:33:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 20:33:55.484718973 +0000 UTC m=+694.405882283" watchObservedRunningTime="2026-07-14 20:34:06.486455777 +0000 UTC m=+705.407619085" Jul 14 20:34:31.251010 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:34:31.250968 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:34:31.339222 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:34:31.339187 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:34:31.710856 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:34:31.710825 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:34:31.869202 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:34:31.869162 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-666cf7ccc9-nzxjw"] Jul 14 20:34:31.872714 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:34:31.872695 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-666cf7ccc9-nzxjw" Jul 14 20:34:31.875617 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:34:31.875588 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-l7ns7\"" Jul 14 20:34:31.876077 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:34:31.876057 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\"" Jul 14 20:34:31.876214 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:34:31.876085 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Jul 14 20:34:31.876837 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:34:31.876819 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 14 20:34:31.898581 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:34:31.898547 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-666cf7ccc9-nzxjw"] Jul 14 20:34:31.957726 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:34:31.957694 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1ea7f577-2c28-44be-ac99-5f5ed40f9c06-openshift-service-ca\") pod \"maas-api-666cf7ccc9-nzxjw\" (UID: \"1ea7f577-2c28-44be-ac99-5f5ed40f9c06\") " pod="odh-ai-gateway-infra/maas-api-666cf7ccc9-nzxjw" Jul 14 20:34:31.957902 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:34:31.957739 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tf6n8\" (UniqueName: \"kubernetes.io/projected/1ea7f577-2c28-44be-ac99-5f5ed40f9c06-kube-api-access-tf6n8\") pod \"maas-api-666cf7ccc9-nzxjw\" (UID: \"1ea7f577-2c28-44be-ac99-5f5ed40f9c06\") " pod="odh-ai-gateway-infra/maas-api-666cf7ccc9-nzxjw" Jul 14 20:34:31.957902 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:34:31.957834 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1ea7f577-2c28-44be-ac99-5f5ed40f9c06-maas-api-tls\") pod \"maas-api-666cf7ccc9-nzxjw\" (UID: \"1ea7f577-2c28-44be-ac99-5f5ed40f9c06\") " pod="odh-ai-gateway-infra/maas-api-666cf7ccc9-nzxjw" Jul 14 20:34:32.058725 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:34:32.058643 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1ea7f577-2c28-44be-ac99-5f5ed40f9c06-openshift-service-ca\") pod \"maas-api-666cf7ccc9-nzxjw\" (UID: \"1ea7f577-2c28-44be-ac99-5f5ed40f9c06\") " pod="odh-ai-gateway-infra/maas-api-666cf7ccc9-nzxjw" Jul 14 20:34:32.058725 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:34:32.058689 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tf6n8\" (UniqueName: \"kubernetes.io/projected/1ea7f577-2c28-44be-ac99-5f5ed40f9c06-kube-api-access-tf6n8\") pod \"maas-api-666cf7ccc9-nzxjw\" (UID: \"1ea7f577-2c28-44be-ac99-5f5ed40f9c06\") " pod="odh-ai-gateway-infra/maas-api-666cf7ccc9-nzxjw" Jul 14 20:34:32.058916 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:34:32.058749 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1ea7f577-2c28-44be-ac99-5f5ed40f9c06-maas-api-tls\") pod \"maas-api-666cf7ccc9-nzxjw\" (UID: \"1ea7f577-2c28-44be-ac99-5f5ed40f9c06\") " pod="odh-ai-gateway-infra/maas-api-666cf7ccc9-nzxjw" Jul 14 20:34:32.058916 ip-10-0-141-61 kubenswrapper[2563]: E0714 20:34:32.058876 2563 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-serving-cert: secret "maas-api-serving-cert" not found Jul 14 20:34:32.059072 ip-10-0-141-61 kubenswrapper[2563]: E0714 20:34:32.058942 2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1ea7f577-2c28-44be-ac99-5f5ed40f9c06-maas-api-tls podName:1ea7f577-2c28-44be-ac99-5f5ed40f9c06 nodeName:}" failed. No retries permitted until 2026-07-14 20:34:32.558921462 +0000 UTC m=+731.480084755 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/1ea7f577-2c28-44be-ac99-5f5ed40f9c06-maas-api-tls") pod "maas-api-666cf7ccc9-nzxjw" (UID: "1ea7f577-2c28-44be-ac99-5f5ed40f9c06") : secret "maas-api-serving-cert" not found Jul 14 20:34:32.059496 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:34:32.059473 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1ea7f577-2c28-44be-ac99-5f5ed40f9c06-openshift-service-ca\") pod \"maas-api-666cf7ccc9-nzxjw\" (UID: \"1ea7f577-2c28-44be-ac99-5f5ed40f9c06\") " pod="odh-ai-gateway-infra/maas-api-666cf7ccc9-nzxjw" Jul 14 20:34:32.068549 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:34:32.068522 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tf6n8\" (UniqueName: \"kubernetes.io/projected/1ea7f577-2c28-44be-ac99-5f5ed40f9c06-kube-api-access-tf6n8\") pod \"maas-api-666cf7ccc9-nzxjw\" (UID: \"1ea7f577-2c28-44be-ac99-5f5ed40f9c06\") " pod="odh-ai-gateway-infra/maas-api-666cf7ccc9-nzxjw" Jul 14 20:34:32.324451 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:34:32.324362 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:34:32.422831 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:34:32.422798 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:34:32.535813 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:34:32.535781 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:34:32.564793 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:34:32.564757 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1ea7f577-2c28-44be-ac99-5f5ed40f9c06-maas-api-tls\") pod \"maas-api-666cf7ccc9-nzxjw\" (UID: \"1ea7f577-2c28-44be-ac99-5f5ed40f9c06\") " pod="odh-ai-gateway-infra/maas-api-666cf7ccc9-nzxjw" Jul 14 20:34:32.567505 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:34:32.567476 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1ea7f577-2c28-44be-ac99-5f5ed40f9c06-maas-api-tls\") pod \"maas-api-666cf7ccc9-nzxjw\" (UID: \"1ea7f577-2c28-44be-ac99-5f5ed40f9c06\") " pod="odh-ai-gateway-infra/maas-api-666cf7ccc9-nzxjw" Jul 14 20:34:32.783811 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:34:32.783778 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-666cf7ccc9-nzxjw" Jul 14 20:34:32.927196 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:34:32.927162 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-666cf7ccc9-nzxjw"] Jul 14 20:34:32.930281 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:34:32.930256 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1ea7f577_2c28_44be_ac99_5f5ed40f9c06.slice/crio-7ae0c8a3e0d91e0ef38fac200a90d9c47304a4501d57756c24dfdaa8960e74ff WatchSource:0}: Error finding container 7ae0c8a3e0d91e0ef38fac200a90d9c47304a4501d57756c24dfdaa8960e74ff: Status 404 returned error can't find the container with id 7ae0c8a3e0d91e0ef38fac200a90d9c47304a4501d57756c24dfdaa8960e74ff Jul 14 20:34:33.646857 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:34:33.646822 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-666cf7ccc9-nzxjw" event={"ID":"1ea7f577-2c28-44be-ac99-5f5ed40f9c06","Type":"ContainerStarted","Data":"7ae0c8a3e0d91e0ef38fac200a90d9c47304a4501d57756c24dfdaa8960e74ff"} Jul 14 20:34:35.658669 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:34:35.658637 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-666cf7ccc9-nzxjw" event={"ID":"1ea7f577-2c28-44be-ac99-5f5ed40f9c06","Type":"ContainerStarted","Data":"feb154ba93d58c7ec6c8ccb2c11c0f9c7bd99df764da549a2ef1751349fcd2d0"} Jul 14 20:34:35.659076 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:34:35.658701 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-666cf7ccc9-nzxjw" Jul 14 20:34:35.675055 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:34:35.675004 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-666cf7ccc9-nzxjw" podStartSLOduration=2.775458392 podStartE2EDuration="4.674983887s" totalImagesPullingTime="1.899525495s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 20:34:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 20:34:35.67447866 +0000 UTC m=+734.595641981" watchObservedRunningTime="2026-07-14 20:34:35.674983887 +0000 UTC m=+734.596147199" Jul 14 20:34:41.157789 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:34:41.156835 2563 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-mzmqf container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.28:15021/healthz/ready\": read tcp 10.134.0.2:56600->10.134.0.28:15021: read: connection reset by peer" start-of-body= Jul 14 20:34:41.157789 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:34:41.156902 2563 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-mzmqf" podUID="afb7dcce-c4ff-4a07-983f-8f52fa1409bd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.28:15021/healthz/ready\": read tcp 10.134.0.2:56600->10.134.0.28:15021: read: connection reset by peer" Jul 14 20:34:41.671090 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:34:41.671062 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-666cf7ccc9-nzxjw" Jul 14 20:34:41.688734 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:34:41.688702 2563 generic.go:356] "Generic (PLEG): container finished" podID="afb7dcce-c4ff-4a07-983f-8f52fa1409bd" containerID="dfc3d8a4aabb4e7eb8a956044e4f20a8ebf997c52fc29203badc04f8aa6d6612" exitCode=0 Jul 14 20:34:41.688921 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:34:41.688777 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-mzmqf" event={"ID":"afb7dcce-c4ff-4a07-983f-8f52fa1409bd","Type":"ContainerDied","Data":"dfc3d8a4aabb4e7eb8a956044e4f20a8ebf997c52fc29203badc04f8aa6d6612"} Jul 14 20:34:41.689191 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:34:41.689177 2563 scope.go:117] "RemoveContainer" containerID="dfc3d8a4aabb4e7eb8a956044e4f20a8ebf997c52fc29203badc04f8aa6d6612" Jul 14 20:34:41.690192 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:34:41.690162 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"} Jul 14 20:34:41.690276 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:34:41.690242 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"} Jul 14 20:34:41.690371 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:34:41.690289 2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"} Jul 14 20:34:42.702981 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:34:42.702946 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-mzmqf" event={"ID":"afb7dcce-c4ff-4a07-983f-8f52fa1409bd","Type":"ContainerStarted","Data":"3707e0f42fd668414a4fd3f7c2ee1393a2b4ddff71099d49b62a12e2b2a27ce0"} Jul 14 20:34:42.738189 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:34:42.738152 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:34:42.835920 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:34:42.835889 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:34:43.056192 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:34:43.056095 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-mzmqf" Jul 14 20:34:43.056395 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:34:43.056370 2563 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-mzmqf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.28:15021/healthz/ready\": dial tcp 10.134.0.28:15021: connect: connection refused" start-of-body= Jul 14 20:34:43.056475 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:34:43.056430 2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-mzmqf" podUID="afb7dcce-c4ff-4a07-983f-8f52fa1409bd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.28:15021/healthz/ready\": dial tcp 10.134.0.28:15021: connect: connection refused" Jul 14 20:34:44.055868 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:34:44.055836 2563 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-mzmqf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.28:15021/healthz/ready\": dial tcp 10.134.0.28:15021: connect: connection refused" start-of-body= Jul 14 20:34:44.056276 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:34:44.055894 2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-mzmqf" podUID="afb7dcce-c4ff-4a07-983f-8f52fa1409bd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.28:15021/healthz/ready\": dial tcp 10.134.0.28:15021: connect: connection refused" Jul 14 20:34:45.056429 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:34:45.056391 2563 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-mzmqf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.28:15021/healthz/ready\": dial tcp 10.134.0.28:15021: connect: connection refused" start-of-body= Jul 14 20:34:45.056810 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:34:45.056455 2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-mzmqf" podUID="afb7dcce-c4ff-4a07-983f-8f52fa1409bd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.28:15021/healthz/ready\": dial tcp 10.134.0.28:15021: connect: connection refused" Jul 14 20:34:46.056427 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:34:46.056387 2563 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-mzmqf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.28:15021/healthz/ready\": dial tcp 10.134.0.28:15021: connect: connection refused" start-of-body= Jul 14 20:34:46.056620 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:34:46.056442 2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-mzmqf" podUID="afb7dcce-c4ff-4a07-983f-8f52fa1409bd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.28:15021/healthz/ready\": dial tcp 10.134.0.28:15021: connect: connection refused" Jul 14 20:34:47.059394 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:34:47.059362 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-mzmqf" Jul 14 20:34:47.059823 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:34:47.059798 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-mzmqf" Jul 14 20:34:47.060480 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:34:47.060466 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-mzmqf" Jul 14 20:34:54.175931 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:34:54.175889 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:35:02.369394 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:35:02.369365 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-6pcsl"] Jul 14 20:35:02.371651 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:35:02.371635 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6pcsl" Jul 14 20:35:02.377062 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:35:02.377031 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 14 20:35:02.377241 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:35:02.377040 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 14 20:35:02.377669 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:35:02.377651 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-dmmcr\"" Jul 14 20:35:02.377779 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:35:02.377678 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\"" Jul 14 20:35:02.386979 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:35:02.386952 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-6pcsl"] Jul 14 20:35:02.409413 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:35:02.409373 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:35:02.464386 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:35:02.464348 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6fae9e79-1b96-4de7-b8a5-0fc58eb948d8-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-6pcsl\" (UID: \"6fae9e79-1b96-4de7-b8a5-0fc58eb948d8\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6pcsl" Jul 14 20:35:02.464386 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:35:02.464383 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6fae9e79-1b96-4de7-b8a5-0fc58eb948d8-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-6pcsl\" (UID: \"6fae9e79-1b96-4de7-b8a5-0fc58eb948d8\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6pcsl" Jul 14 20:35:02.464665 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:35:02.464463 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6fae9e79-1b96-4de7-b8a5-0fc58eb948d8-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-6pcsl\" (UID: \"6fae9e79-1b96-4de7-b8a5-0fc58eb948d8\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6pcsl" Jul 14 20:35:02.464665 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:35:02.464507 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6fae9e79-1b96-4de7-b8a5-0fc58eb948d8-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-6pcsl\" (UID: \"6fae9e79-1b96-4de7-b8a5-0fc58eb948d8\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6pcsl" Jul 14 20:35:02.464665 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:35:02.464531 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6fae9e79-1b96-4de7-b8a5-0fc58eb948d8-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-6pcsl\" (UID: \"6fae9e79-1b96-4de7-b8a5-0fc58eb948d8\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6pcsl" Jul 14 20:35:02.464665 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:35:02.464551 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5wlrs\" (UniqueName: \"kubernetes.io/projected/6fae9e79-1b96-4de7-b8a5-0fc58eb948d8-kube-api-access-5wlrs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-6pcsl\" (UID: \"6fae9e79-1b96-4de7-b8a5-0fc58eb948d8\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6pcsl" Jul 14 20:35:02.565695 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:35:02.565653 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6fae9e79-1b96-4de7-b8a5-0fc58eb948d8-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-6pcsl\" (UID: \"6fae9e79-1b96-4de7-b8a5-0fc58eb948d8\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6pcsl" Jul 14 20:35:02.565695 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:35:02.565696 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6fae9e79-1b96-4de7-b8a5-0fc58eb948d8-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-6pcsl\" (UID: \"6fae9e79-1b96-4de7-b8a5-0fc58eb948d8\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6pcsl" Jul 14 20:35:02.565974 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:35:02.565850 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6fae9e79-1b96-4de7-b8a5-0fc58eb948d8-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-6pcsl\" (UID: \"6fae9e79-1b96-4de7-b8a5-0fc58eb948d8\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6pcsl" Jul 14 20:35:02.565974 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:35:02.565897 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5wlrs\" (UniqueName: \"kubernetes.io/projected/6fae9e79-1b96-4de7-b8a5-0fc58eb948d8-kube-api-access-5wlrs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-6pcsl\" (UID: \"6fae9e79-1b96-4de7-b8a5-0fc58eb948d8\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6pcsl" Jul 14 20:35:02.566087 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:35:02.566061 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6fae9e79-1b96-4de7-b8a5-0fc58eb948d8-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-6pcsl\" (UID: \"6fae9e79-1b96-4de7-b8a5-0fc58eb948d8\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6pcsl" Jul 14 20:35:02.566163 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:35:02.566087 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6fae9e79-1b96-4de7-b8a5-0fc58eb948d8-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-6pcsl\" (UID: \"6fae9e79-1b96-4de7-b8a5-0fc58eb948d8\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6pcsl" Jul 14 20:35:02.566227 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:35:02.566177 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6fae9e79-1b96-4de7-b8a5-0fc58eb948d8-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-6pcsl\" (UID: \"6fae9e79-1b96-4de7-b8a5-0fc58eb948d8\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6pcsl" Jul 14 20:35:02.566227 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:35:02.566201 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6fae9e79-1b96-4de7-b8a5-0fc58eb948d8-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-6pcsl\" (UID: \"6fae9e79-1b96-4de7-b8a5-0fc58eb948d8\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6pcsl" Jul 14 20:35:02.566389 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:35:02.566370 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6fae9e79-1b96-4de7-b8a5-0fc58eb948d8-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-6pcsl\" (UID: \"6fae9e79-1b96-4de7-b8a5-0fc58eb948d8\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6pcsl" Jul 14 20:35:02.568279 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:35:02.568258 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6fae9e79-1b96-4de7-b8a5-0fc58eb948d8-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-6pcsl\" (UID: \"6fae9e79-1b96-4de7-b8a5-0fc58eb948d8\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6pcsl" Jul 14 20:35:02.568560 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:35:02.568542 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6fae9e79-1b96-4de7-b8a5-0fc58eb948d8-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-6pcsl\" (UID: \"6fae9e79-1b96-4de7-b8a5-0fc58eb948d8\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6pcsl" Jul 14 20:35:02.574886 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:35:02.574859 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5wlrs\" (UniqueName: \"kubernetes.io/projected/6fae9e79-1b96-4de7-b8a5-0fc58eb948d8-kube-api-access-5wlrs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-6pcsl\" (UID: \"6fae9e79-1b96-4de7-b8a5-0fc58eb948d8\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6pcsl" Jul 14 20:35:02.683597 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:35:02.683559 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6pcsl" Jul 14 20:35:02.829814 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:35:02.829779 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-6pcsl"] Jul 14 20:35:02.833001 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:35:02.832978 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6fae9e79_1b96_4de7_b8a5_0fc58eb948d8.slice/crio-cc49a923dafee56f2221f2af2ff747075eaff847ae94970080fd877dd793eba6 WatchSource:0}: Error finding container cc49a923dafee56f2221f2af2ff747075eaff847ae94970080fd877dd793eba6: Status 404 returned error can't find the container with id cc49a923dafee56f2221f2af2ff747075eaff847ae94970080fd877dd793eba6 Jul 14 20:35:03.812327 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:35:03.812284 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6pcsl" event={"ID":"6fae9e79-1b96-4de7-b8a5-0fc58eb948d8","Type":"ContainerStarted","Data":"cc49a923dafee56f2221f2af2ff747075eaff847ae94970080fd877dd793eba6"} Jul 14 20:35:04.819241 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:35:04.819207 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6pcsl" event={"ID":"6fae9e79-1b96-4de7-b8a5-0fc58eb948d8","Type":"ContainerStarted","Data":"d8a83fd007e0286352699df84df91d6f7cb877aca8ffe9453292f1f0bc19da91"} Jul 14 20:35:04.839782 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:35:04.839734 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6pcsl" podStartSLOduration=1.563312774 podStartE2EDuration="2.839715291s" totalImagesPullingTime="1.276402517s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 20:35:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 20:35:04.83826534 +0000 UTC m=+763.759428649" watchObservedRunningTime="2026-07-14 20:35:04.839715291 +0000 UTC m=+763.760878600" Jul 14 20:35:12.684189 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:35:12.684155 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6pcsl" Jul 14 20:35:12.684715 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:35:12.684219 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6pcsl" Jul 14 20:35:12.696616 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:35:12.696590 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6pcsl" Jul 14 20:35:12.867699 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:35:12.867670 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-6pcsl" Jul 14 20:35:20.305593 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:35:20.305547 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-pq9zm"] Jul 14 20:35:20.309034 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:35:20.309010 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-pq9zm" Jul 14 20:35:20.311720 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:35:20.311695 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\"" Jul 14 20:35:20.338081 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:35:20.338042 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-pq9zm"] Jul 14 20:35:20.381065 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:35:20.381028 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:35:20.433413 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:35:20.433371 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a0c1f17c-3b0f-4668-b59e-4e0fe8f1df9a-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-pq9zm\" (UID: \"a0c1f17c-3b0f-4668-b59e-4e0fe8f1df9a\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-pq9zm" Jul 14 20:35:20.433584 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:35:20.433423 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a0c1f17c-3b0f-4668-b59e-4e0fe8f1df9a-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-pq9zm\" (UID: \"a0c1f17c-3b0f-4668-b59e-4e0fe8f1df9a\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-pq9zm" Jul 14 20:35:20.433584 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:35:20.433446 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4lnqn\" (UniqueName: \"kubernetes.io/projected/a0c1f17c-3b0f-4668-b59e-4e0fe8f1df9a-kube-api-access-4lnqn\") pod \"e2e-trlp-test-simulated-kserve-598466548d-pq9zm\" (UID: \"a0c1f17c-3b0f-4668-b59e-4e0fe8f1df9a\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-pq9zm" Jul 14 20:35:20.433673 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:35:20.433606 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a0c1f17c-3b0f-4668-b59e-4e0fe8f1df9a-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-pq9zm\" (UID: \"a0c1f17c-3b0f-4668-b59e-4e0fe8f1df9a\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-pq9zm" Jul 14 20:35:20.433728 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:35:20.433712 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a0c1f17c-3b0f-4668-b59e-4e0fe8f1df9a-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-pq9zm\" (UID: \"a0c1f17c-3b0f-4668-b59e-4e0fe8f1df9a\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-pq9zm" Jul 14 20:35:20.433766 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:35:20.433747 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a0c1f17c-3b0f-4668-b59e-4e0fe8f1df9a-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-pq9zm\" (UID: \"a0c1f17c-3b0f-4668-b59e-4e0fe8f1df9a\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-pq9zm" Jul 14 20:35:20.534577 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:35:20.534533 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a0c1f17c-3b0f-4668-b59e-4e0fe8f1df9a-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-pq9zm\" (UID: \"a0c1f17c-3b0f-4668-b59e-4e0fe8f1df9a\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-pq9zm" Jul 14 20:35:20.534789 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:35:20.534614 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a0c1f17c-3b0f-4668-b59e-4e0fe8f1df9a-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-pq9zm\" (UID: \"a0c1f17c-3b0f-4668-b59e-4e0fe8f1df9a\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-pq9zm" Jul 14 20:35:20.534789 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:35:20.534645 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a0c1f17c-3b0f-4668-b59e-4e0fe8f1df9a-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-pq9zm\" (UID: \"a0c1f17c-3b0f-4668-b59e-4e0fe8f1df9a\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-pq9zm" Jul 14 20:35:20.534789 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:35:20.534688 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a0c1f17c-3b0f-4668-b59e-4e0fe8f1df9a-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-pq9zm\" (UID: \"a0c1f17c-3b0f-4668-b59e-4e0fe8f1df9a\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-pq9zm" Jul 14 20:35:20.534789 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:35:20.534730 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a0c1f17c-3b0f-4668-b59e-4e0fe8f1df9a-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-pq9zm\" (UID: \"a0c1f17c-3b0f-4668-b59e-4e0fe8f1df9a\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-pq9zm" Jul 14 20:35:20.534789 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:35:20.534757 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4lnqn\" (UniqueName: \"kubernetes.io/projected/a0c1f17c-3b0f-4668-b59e-4e0fe8f1df9a-kube-api-access-4lnqn\") pod \"e2e-trlp-test-simulated-kserve-598466548d-pq9zm\" (UID: \"a0c1f17c-3b0f-4668-b59e-4e0fe8f1df9a\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-pq9zm" Jul 14 20:35:20.535090 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:35:20.535029 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a0c1f17c-3b0f-4668-b59e-4e0fe8f1df9a-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-pq9zm\" (UID: \"a0c1f17c-3b0f-4668-b59e-4e0fe8f1df9a\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-pq9zm" Jul 14 20:35:20.535183 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:35:20.535077 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a0c1f17c-3b0f-4668-b59e-4e0fe8f1df9a-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-pq9zm\" (UID: \"a0c1f17c-3b0f-4668-b59e-4e0fe8f1df9a\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-pq9zm" Jul 14 20:35:20.535230 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:35:20.535201 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a0c1f17c-3b0f-4668-b59e-4e0fe8f1df9a-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-pq9zm\" (UID: \"a0c1f17c-3b0f-4668-b59e-4e0fe8f1df9a\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-pq9zm" Jul 14 20:35:20.537105 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:35:20.537081 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a0c1f17c-3b0f-4668-b59e-4e0fe8f1df9a-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-pq9zm\" (UID: \"a0c1f17c-3b0f-4668-b59e-4e0fe8f1df9a\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-pq9zm" Jul 14 20:35:20.537311 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:35:20.537291 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a0c1f17c-3b0f-4668-b59e-4e0fe8f1df9a-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-pq9zm\" (UID: \"a0c1f17c-3b0f-4668-b59e-4e0fe8f1df9a\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-pq9zm" Jul 14 20:35:20.543201 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:35:20.543175 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4lnqn\" (UniqueName: \"kubernetes.io/projected/a0c1f17c-3b0f-4668-b59e-4e0fe8f1df9a-kube-api-access-4lnqn\") pod \"e2e-trlp-test-simulated-kserve-598466548d-pq9zm\" (UID: \"a0c1f17c-3b0f-4668-b59e-4e0fe8f1df9a\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-pq9zm" Jul 14 20:35:20.619571 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:35:20.619460 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-pq9zm" Jul 14 20:35:20.765003 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:35:20.764944 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-pq9zm"] Jul 14 20:35:20.768430 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:35:20.768397 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda0c1f17c_3b0f_4668_b59e_4e0fe8f1df9a.slice/crio-c9d5e263f3f7d5c3cfd357f404be42ba4114f4adeaec6b5b05d8413e71c6b9f1 WatchSource:0}: Error finding container c9d5e263f3f7d5c3cfd357f404be42ba4114f4adeaec6b5b05d8413e71c6b9f1: Status 404 returned error can't find the container with id c9d5e263f3f7d5c3cfd357f404be42ba4114f4adeaec6b5b05d8413e71c6b9f1 Jul 14 20:35:20.769698 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:35:20.769673 2563 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 14 20:35:20.895047 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:35:20.894954 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-pq9zm" event={"ID":"a0c1f17c-3b0f-4668-b59e-4e0fe8f1df9a","Type":"ContainerStarted","Data":"c9d5e263f3f7d5c3cfd357f404be42ba4114f4adeaec6b5b05d8413e71c6b9f1"} Jul 14 20:35:21.901723 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:35:21.901685 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-pq9zm" event={"ID":"a0c1f17c-3b0f-4668-b59e-4e0fe8f1df9a","Type":"ContainerStarted","Data":"40ccbe6ea77cd9864da6740b2ac9a5e8bae32429b8819b2440f3a049c7e0d3e7"} Jul 14 20:35:21.921708 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:35:21.921653 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-pq9zm" podStartSLOduration=1.689112986 podStartE2EDuration="1.921634994s" totalImagesPullingTime="232.522008ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 20:35:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 20:35:21.919752064 +0000 UTC m=+780.840915374" watchObservedRunningTime="2026-07-14 20:35:21.921634994 +0000 UTC m=+780.842798303" Jul 14 20:35:24.081658 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:35:24.081622 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:35:30.620027 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:35:30.619965 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-pq9zm" Jul 14 20:35:30.620459 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:35:30.620108 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-pq9zm" Jul 14 20:35:30.632531 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:35:30.632503 2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-pq9zm" Jul 14 20:35:30.956973 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:35:30.956936 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-pq9zm" Jul 14 20:35:39.104613 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:35:39.104566 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:35:51.080558 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:35:51.080457 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:35:55.762710 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:35:55.762671 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:35:55.865029 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:35:55.864998 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:37:01.474647 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:37:01.474608 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:37:01.584753 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:37:01.584719 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:37:02.706930 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:37:02.706892 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:37:02.802787 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:37:02.802754 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:37:21.522780 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:37:21.522705 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hzbx8_9588ab98-33e8-4c89-807d-a805240f1b81/ovn-acl-logging/0.log" Jul 14 20:37:21.523211 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:37:21.522902 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hzbx8_9588ab98-33e8-4c89-807d-a805240f1b81/ovn-acl-logging/0.log" Jul 14 20:37:56.252760 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:37:56.252718 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:38:56.198986 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:38:56.198892 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:39:32.035848 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:39:32.035804 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:39:32.137101 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:39:32.137066 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:39:39.238775 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:39:39.238728 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:40:46.035505 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:40:46.035419 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:40:46.144646 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:40:46.144611 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-5c7b6db686-g9zvp"] Jul 14 20:40:46.144863 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:40:46.144840 2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-5c7b6db686-g9zvp" podUID="d10dc44c-c8b0-4f54-9c7e-fb99fd87d51d" containerName="manager" containerID="cri-o://5a9da1253101c33bb9a9eb43257bb046a4ff79bdddd30a5fe52d3ce2e428d7a9" gracePeriod=30 Jul 14 20:40:46.402641 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:40:46.402615 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5c7b6db686-g9zvp" Jul 14 20:40:46.455488 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:40:46.455454 2563 generic.go:356] "Generic (PLEG): container finished" podID="d10dc44c-c8b0-4f54-9c7e-fb99fd87d51d" containerID="5a9da1253101c33bb9a9eb43257bb046a4ff79bdddd30a5fe52d3ce2e428d7a9" exitCode=0 Jul 14 20:40:46.455703 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:40:46.455507 2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5c7b6db686-g9zvp" Jul 14 20:40:46.455703 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:40:46.455519 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5c7b6db686-g9zvp" event={"ID":"d10dc44c-c8b0-4f54-9c7e-fb99fd87d51d","Type":"ContainerDied","Data":"5a9da1253101c33bb9a9eb43257bb046a4ff79bdddd30a5fe52d3ce2e428d7a9"} Jul 14 20:40:46.455703 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:40:46.455573 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5c7b6db686-g9zvp" event={"ID":"d10dc44c-c8b0-4f54-9c7e-fb99fd87d51d","Type":"ContainerDied","Data":"d9f93260e40fdac8ab5452767dae455a7c29cd3c7f8ecf1cc24d429becdcece4"} Jul 14 20:40:46.455703 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:40:46.455598 2563 scope.go:117] "RemoveContainer" containerID="5a9da1253101c33bb9a9eb43257bb046a4ff79bdddd30a5fe52d3ce2e428d7a9" Jul 14 20:40:46.466258 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:40:46.466238 2563 scope.go:117] "RemoveContainer" containerID="5a9da1253101c33bb9a9eb43257bb046a4ff79bdddd30a5fe52d3ce2e428d7a9" Jul 14 20:40:46.466562 ip-10-0-141-61 kubenswrapper[2563]: E0714 20:40:46.466542 2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5a9da1253101c33bb9a9eb43257bb046a4ff79bdddd30a5fe52d3ce2e428d7a9\": container with ID starting with 5a9da1253101c33bb9a9eb43257bb046a4ff79bdddd30a5fe52d3ce2e428d7a9 not found: ID does not exist" containerID="5a9da1253101c33bb9a9eb43257bb046a4ff79bdddd30a5fe52d3ce2e428d7a9" Jul 14 20:40:46.466610 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:40:46.466573 2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5a9da1253101c33bb9a9eb43257bb046a4ff79bdddd30a5fe52d3ce2e428d7a9"} err="failed to get container status \"5a9da1253101c33bb9a9eb43257bb046a4ff79bdddd30a5fe52d3ce2e428d7a9\": rpc error: code = NotFound desc = could not find container \"5a9da1253101c33bb9a9eb43257bb046a4ff79bdddd30a5fe52d3ce2e428d7a9\": container with ID starting with 5a9da1253101c33bb9a9eb43257bb046a4ff79bdddd30a5fe52d3ce2e428d7a9 not found: ID does not exist" Jul 14 20:40:46.533067 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:40:46.533032 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d10dc44c-c8b0-4f54-9c7e-fb99fd87d51d-webhook-cert\") pod \"d10dc44c-c8b0-4f54-9c7e-fb99fd87d51d\" (UID: \"d10dc44c-c8b0-4f54-9c7e-fb99fd87d51d\") " Jul 14 20:40:46.533263 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:40:46.533093 2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-btpcb\" (UniqueName: \"kubernetes.io/projected/d10dc44c-c8b0-4f54-9c7e-fb99fd87d51d-kube-api-access-btpcb\") pod \"d10dc44c-c8b0-4f54-9c7e-fb99fd87d51d\" (UID: \"d10dc44c-c8b0-4f54-9c7e-fb99fd87d51d\") " Jul 14 20:40:46.535489 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:40:46.535458 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d10dc44c-c8b0-4f54-9c7e-fb99fd87d51d-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "d10dc44c-c8b0-4f54-9c7e-fb99fd87d51d" (UID: "d10dc44c-c8b0-4f54-9c7e-fb99fd87d51d"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 14 20:40:46.535604 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:40:46.535564 2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d10dc44c-c8b0-4f54-9c7e-fb99fd87d51d-kube-api-access-btpcb" (OuterVolumeSpecName: "kube-api-access-btpcb") pod "d10dc44c-c8b0-4f54-9c7e-fb99fd87d51d" (UID: "d10dc44c-c8b0-4f54-9c7e-fb99fd87d51d"). InnerVolumeSpecName "kube-api-access-btpcb". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 20:40:46.634406 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:40:46.634316 2563 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d10dc44c-c8b0-4f54-9c7e-fb99fd87d51d-webhook-cert\") on node \"ip-10-0-141-61.ec2.internal\" DevicePath \"\"" Jul 14 20:40:46.634406 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:40:46.634349 2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-btpcb\" (UniqueName: \"kubernetes.io/projected/d10dc44c-c8b0-4f54-9c7e-fb99fd87d51d-kube-api-access-btpcb\") on node \"ip-10-0-141-61.ec2.internal\" DevicePath \"\"" Jul 14 20:40:46.778853 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:40:46.778821 2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-5c7b6db686-g9zvp"] Jul 14 20:40:46.782803 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:40:46.782774 2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-5c7b6db686-g9zvp"] Jul 14 20:40:47.472821 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:40:47.472789 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:40:47.593646 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:40:47.593607 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-5c7b6db686-dq5lx"] Jul 14 20:40:47.594151 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:40:47.594099 2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d10dc44c-c8b0-4f54-9c7e-fb99fd87d51d" containerName="manager" Jul 14 20:40:47.594151 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:40:47.594138 2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="d10dc44c-c8b0-4f54-9c7e-fb99fd87d51d" containerName="manager" Jul 14 20:40:47.594311 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:40:47.594195 2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="d10dc44c-c8b0-4f54-9c7e-fb99fd87d51d" containerName="manager" Jul 14 20:40:47.596312 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:40:47.596295 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5c7b6db686-dq5lx" Jul 14 20:40:47.598324 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:40:47.598303 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-2zmfq\"" Jul 14 20:40:47.598451 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:40:47.598303 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jul 14 20:40:47.598829 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:40:47.598812 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jul 14 20:40:47.606991 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:40:47.606959 2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d10dc44c-c8b0-4f54-9c7e-fb99fd87d51d" path="/var/lib/kubelet/pods/d10dc44c-c8b0-4f54-9c7e-fb99fd87d51d/volumes" Jul 14 20:40:47.607366 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:40:47.607348 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5c7b6db686-dq5lx"] Jul 14 20:40:47.745353 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:40:47.745251 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vt89x\" (UniqueName: \"kubernetes.io/projected/5b83138a-1f77-455b-8be5-d183995c46f1-kube-api-access-vt89x\") pod \"maas-controller-5c7b6db686-dq5lx\" (UID: \"5b83138a-1f77-455b-8be5-d183995c46f1\") " pod="opendatahub/maas-controller-5c7b6db686-dq5lx" Jul 14 20:40:47.745353 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:40:47.745307 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5b83138a-1f77-455b-8be5-d183995c46f1-webhook-cert\") pod \"maas-controller-5c7b6db686-dq5lx\" (UID: \"5b83138a-1f77-455b-8be5-d183995c46f1\") " pod="opendatahub/maas-controller-5c7b6db686-dq5lx" Jul 14 20:40:47.846654 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:40:47.846615 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vt89x\" (UniqueName: \"kubernetes.io/projected/5b83138a-1f77-455b-8be5-d183995c46f1-kube-api-access-vt89x\") pod \"maas-controller-5c7b6db686-dq5lx\" (UID: \"5b83138a-1f77-455b-8be5-d183995c46f1\") " pod="opendatahub/maas-controller-5c7b6db686-dq5lx" Jul 14 20:40:47.846654 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:40:47.846668 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5b83138a-1f77-455b-8be5-d183995c46f1-webhook-cert\") pod \"maas-controller-5c7b6db686-dq5lx\" (UID: \"5b83138a-1f77-455b-8be5-d183995c46f1\") " pod="opendatahub/maas-controller-5c7b6db686-dq5lx" Jul 14 20:40:47.849275 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:40:47.849251 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5b83138a-1f77-455b-8be5-d183995c46f1-webhook-cert\") pod \"maas-controller-5c7b6db686-dq5lx\" (UID: \"5b83138a-1f77-455b-8be5-d183995c46f1\") " pod="opendatahub/maas-controller-5c7b6db686-dq5lx" Jul 14 20:40:47.856091 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:40:47.856041 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vt89x\" (UniqueName: \"kubernetes.io/projected/5b83138a-1f77-455b-8be5-d183995c46f1-kube-api-access-vt89x\") pod \"maas-controller-5c7b6db686-dq5lx\" (UID: \"5b83138a-1f77-455b-8be5-d183995c46f1\") " pod="opendatahub/maas-controller-5c7b6db686-dq5lx" Jul 14 20:40:47.907961 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:40:47.907918 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5c7b6db686-dq5lx" Jul 14 20:40:48.048388 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:40:48.048361 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5c7b6db686-dq5lx"] Jul 14 20:40:48.051339 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:40:48.051312 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5b83138a_1f77_455b_8be5_d183995c46f1.slice/crio-00406f58b709c688ab8cff2e1dade5958f773ce59781b647af58f7e3dfbb1bd3 WatchSource:0}: Error finding container 00406f58b709c688ab8cff2e1dade5958f773ce59781b647af58f7e3dfbb1bd3: Status 404 returned error can't find the container with id 00406f58b709c688ab8cff2e1dade5958f773ce59781b647af58f7e3dfbb1bd3 Jul 14 20:40:48.053086 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:40:48.053067 2563 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 14 20:40:48.468567 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:40:48.468534 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5c7b6db686-dq5lx" event={"ID":"5b83138a-1f77-455b-8be5-d183995c46f1","Type":"ContainerStarted","Data":"00406f58b709c688ab8cff2e1dade5958f773ce59781b647af58f7e3dfbb1bd3"} Jul 14 20:40:49.474685 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:40:49.474650 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5c7b6db686-dq5lx" event={"ID":"5b83138a-1f77-455b-8be5-d183995c46f1","Type":"ContainerStarted","Data":"7dff25c3dd031a06573f4739b1c4d46fc7b69b2ea3d6474973225816e9c5747e"} Jul 14 20:40:49.475104 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:40:49.474741 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-5c7b6db686-dq5lx" Jul 14 20:40:49.497905 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:40:49.497847 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-5c7b6db686-dq5lx" podStartSLOduration=1.913920351 podStartE2EDuration="2.497828807s" totalImagesPullingTime="583.908456ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 20:40:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 20:40:49.495155158 +0000 UTC m=+1108.416318466" watchObservedRunningTime="2026-07-14 20:40:49.497828807 +0000 UTC m=+1108.418992116" Jul 14 20:41:00.482568 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:41:00.482533 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-5c7b6db686-dq5lx" Jul 14 20:41:37.042650 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:41:37.042613 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:41:37.118165 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:41:37.118112 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:41:59.076310 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:41:59.076274 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:42:05.345264 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:42:05.345224 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-5c7b6db686-dq5lx_5b83138a-1f77-455b-8be5-d183995c46f1/manager/0.log" Jul 14 20:42:05.889797 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:42:05.889770 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-79d49c7b87-bn8n6_af45e5e1-4dc3-4848-a094-f979fec50d9d/copy-manifests/0.log" Jul 14 20:42:05.902998 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:42:05.902967 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-79d49c7b87-bn8n6_af45e5e1-4dc3-4848-a094-f979fec50d9d/manager/0.log" Jul 14 20:42:06.392654 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:42:06.392616 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-666cf7ccc9-nzxjw_1ea7f577-2c28-44be-ac99-5f5ed40f9c06/maas-api/0.log" Jul 14 20:42:09.162962 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:42:09.162928 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf45cgj7_c4990631-d6e1-46cd-afdb-0692ca08c6ba/istio-proxy/0.log" Jul 14 20:42:09.295064 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:42:09.295028 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-8jmfx_42b909af-0bd9-4036-b180-2c5cd4394b30/discovery/0.log" Jul 14 20:42:09.677865 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:42:09.677832 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-mzmqf_afb7dcce-c4ff-4a07-983f-8f52fa1409bd/istio-proxy/1.log" Jul 14 20:42:10.526003 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:42:10.525947 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-6pcsl_6fae9e79-1b96-4de7-b8a5-0fc58eb948d8/main/0.log" Jul 14 20:42:10.641744 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:42:10.641706 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-pq9zm_a0c1f17c-3b0f-4668-b59e-4e0fe8f1df9a/main/0.log" Jul 14 20:42:15.378282 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:42:15.378247 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:42:15.456767 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:42:15.456707 2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" limit=1024 Jul 14 20:42:18.261378 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:42:18.261339 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-sgvdx_05045fa2-6bd3-4ee8-8f2a-e9056c8a316f/global-pull-secret-syncer/0.log" Jul 14 20:42:18.351749 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:42:18.351715 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-dwzbf_3d2553b9-0627-4fd5-b973-76e17eabf098/konnectivity-agent/0.log" Jul 14 20:42:18.448545 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:42:18.448514 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-141-61.ec2.internal_9a7e15c41e1da145327771a9c8cb3e39/haproxy/0.log" Jul 14 20:42:21.554888 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:42:21.554855 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hzbx8_9588ab98-33e8-4c89-807d-a805240f1b81/ovn-acl-logging/0.log" Jul 14 20:42:21.556315 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:42:21.556294 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hzbx8_9588ab98-33e8-4c89-807d-a805240f1b81/ovn-acl-logging/0.log" Jul 14 20:42:25.696488 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:42:25.696454 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-7cd4fc7897-d9pdf_896f8e23-c9f6-4fea-8eec-762519e4fe62/metrics-server/0.log" Jul 14 20:42:25.971060 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:42:25.971033 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-szdqq_de1173b5-2d51-4f32-969c-40b0431bd7a0/node-exporter/0.log" Jul 14 20:42:25.993308 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:42:25.993254 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-szdqq_de1173b5-2d51-4f32-969c-40b0431bd7a0/kube-rbac-proxy/0.log" Jul 14 20:42:26.016170 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:42:26.016116 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-szdqq_de1173b5-2d51-4f32-969c-40b0431bd7a0/init-textfile/0.log" Jul 14 20:42:26.045839 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:42:26.045797 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-84bd68b896-gkps8_5527cca4-4cdb-441c-9e12-0dfee7304f67/kube-rbac-proxy-main/0.log" Jul 14 20:42:26.068997 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:42:26.068971 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-84bd68b896-gkps8_5527cca4-4cdb-441c-9e12-0dfee7304f67/kube-rbac-proxy-self/0.log" Jul 14 20:42:26.091865 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:42:26.091828 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-84bd68b896-gkps8_5527cca4-4cdb-441c-9e12-0dfee7304f67/openshift-state-metrics/0.log" Jul 14 20:42:26.129888 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:42:26.129846 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_8c83ee98-d2c0-4eb9-bb01-279ea67f4ad8/prometheus/0.log" Jul 14 20:42:26.152853 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:42:26.152819 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_8c83ee98-d2c0-4eb9-bb01-279ea67f4ad8/config-reloader/0.log" Jul 14 20:42:26.177603 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:42:26.177567 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_8c83ee98-d2c0-4eb9-bb01-279ea67f4ad8/thanos-sidecar/0.log" Jul 14 20:42:26.200463 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:42:26.200414 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_8c83ee98-d2c0-4eb9-bb01-279ea67f4ad8/kube-rbac-proxy-web/0.log" Jul 14 20:42:26.224149 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:42:26.224029 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_8c83ee98-d2c0-4eb9-bb01-279ea67f4ad8/kube-rbac-proxy/0.log" Jul 14 20:42:26.250756 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:42:26.250723 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_8c83ee98-d2c0-4eb9-bb01-279ea67f4ad8/kube-rbac-proxy-thanos/0.log" Jul 14 20:42:26.272966 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:42:26.272943 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_8c83ee98-d2c0-4eb9-bb01-279ea67f4ad8/init-config-reloader/0.log" Jul 14 20:42:26.349696 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:42:26.349665 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-77d98bc786-x5h7q_f1c2ae6a-43b9-45fb-bb72-9a9a8197613d/prometheus-operator-admission-webhook/0.log" Jul 14 20:42:26.391171 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:42:26.391137 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-f598c6b45-bq5rt_e1657bdb-067b-430c-96b1-80d8a7c24f09/telemeter-client/0.log" Jul 14 20:42:26.414599 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:42:26.414564 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-f598c6b45-bq5rt_e1657bdb-067b-430c-96b1-80d8a7c24f09/reload/0.log" Jul 14 20:42:26.436483 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:42:26.436455 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-f598c6b45-bq5rt_e1657bdb-067b-430c-96b1-80d8a7c24f09/kube-rbac-proxy/0.log" Jul 14 20:42:28.845575 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:42:28.845537 2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-k7rsc/perf-node-gather-daemonset-pt5d6"] Jul 14 20:42:28.849111 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:42:28.849083 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-k7rsc/perf-node-gather-daemonset-pt5d6" Jul 14 20:42:28.851074 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:42:28.851052 2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-k7rsc\"/\"default-dockercfg-cqb4f\"" Jul 14 20:42:28.851671 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:42:28.851653 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-k7rsc\"/\"kube-root-ca.crt\"" Jul 14 20:42:28.851736 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:42:28.851699 2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-k7rsc\"/\"openshift-service-ca.crt\"" Jul 14 20:42:28.857907 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:42:28.857884 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-k7rsc/perf-node-gather-daemonset-pt5d6"] Jul 14 20:42:28.897282 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:42:28.897230 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/67886ff4-347d-4990-aee1-6f442b78549f-sys\") pod \"perf-node-gather-daemonset-pt5d6\" (UID: \"67886ff4-347d-4990-aee1-6f442b78549f\") " pod="openshift-must-gather-k7rsc/perf-node-gather-daemonset-pt5d6" Jul 14 20:42:28.897483 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:42:28.897363 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/67886ff4-347d-4990-aee1-6f442b78549f-lib-modules\") pod \"perf-node-gather-daemonset-pt5d6\" (UID: \"67886ff4-347d-4990-aee1-6f442b78549f\") " pod="openshift-must-gather-k7rsc/perf-node-gather-daemonset-pt5d6" Jul 14 20:42:28.897483 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:42:28.897402 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k5q2h\" (UniqueName: \"kubernetes.io/projected/67886ff4-347d-4990-aee1-6f442b78549f-kube-api-access-k5q2h\") pod \"perf-node-gather-daemonset-pt5d6\" (UID: \"67886ff4-347d-4990-aee1-6f442b78549f\") " pod="openshift-must-gather-k7rsc/perf-node-gather-daemonset-pt5d6" Jul 14 20:42:28.897483 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:42:28.897425 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/67886ff4-347d-4990-aee1-6f442b78549f-proc\") pod \"perf-node-gather-daemonset-pt5d6\" (UID: \"67886ff4-347d-4990-aee1-6f442b78549f\") " pod="openshift-must-gather-k7rsc/perf-node-gather-daemonset-pt5d6" Jul 14 20:42:28.897483 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:42:28.897446 2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/67886ff4-347d-4990-aee1-6f442b78549f-podres\") pod \"perf-node-gather-daemonset-pt5d6\" (UID: \"67886ff4-347d-4990-aee1-6f442b78549f\") " pod="openshift-must-gather-k7rsc/perf-node-gather-daemonset-pt5d6" Jul 14 20:42:28.998636 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:42:28.998584 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/67886ff4-347d-4990-aee1-6f442b78549f-sys\") pod \"perf-node-gather-daemonset-pt5d6\" (UID: \"67886ff4-347d-4990-aee1-6f442b78549f\") " pod="openshift-must-gather-k7rsc/perf-node-gather-daemonset-pt5d6" Jul 14 20:42:28.998847 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:42:28.998712 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/67886ff4-347d-4990-aee1-6f442b78549f-lib-modules\") pod \"perf-node-gather-daemonset-pt5d6\" (UID: \"67886ff4-347d-4990-aee1-6f442b78549f\") " pod="openshift-must-gather-k7rsc/perf-node-gather-daemonset-pt5d6" Jul 14 20:42:28.998847 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:42:28.998730 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/67886ff4-347d-4990-aee1-6f442b78549f-sys\") pod \"perf-node-gather-daemonset-pt5d6\" (UID: \"67886ff4-347d-4990-aee1-6f442b78549f\") " pod="openshift-must-gather-k7rsc/perf-node-gather-daemonset-pt5d6" Jul 14 20:42:28.998847 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:42:28.998738 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k5q2h\" (UniqueName: \"kubernetes.io/projected/67886ff4-347d-4990-aee1-6f442b78549f-kube-api-access-k5q2h\") pod \"perf-node-gather-daemonset-pt5d6\" (UID: \"67886ff4-347d-4990-aee1-6f442b78549f\") " pod="openshift-must-gather-k7rsc/perf-node-gather-daemonset-pt5d6" Jul 14 20:42:28.998847 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:42:28.998772 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/67886ff4-347d-4990-aee1-6f442b78549f-proc\") pod \"perf-node-gather-daemonset-pt5d6\" (UID: \"67886ff4-347d-4990-aee1-6f442b78549f\") " pod="openshift-must-gather-k7rsc/perf-node-gather-daemonset-pt5d6" Jul 14 20:42:28.998847 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:42:28.998798 2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/67886ff4-347d-4990-aee1-6f442b78549f-podres\") pod \"perf-node-gather-daemonset-pt5d6\" (UID: \"67886ff4-347d-4990-aee1-6f442b78549f\") " pod="openshift-must-gather-k7rsc/perf-node-gather-daemonset-pt5d6" Jul 14 20:42:28.999098 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:42:28.998884 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/67886ff4-347d-4990-aee1-6f442b78549f-lib-modules\") pod \"perf-node-gather-daemonset-pt5d6\" (UID: \"67886ff4-347d-4990-aee1-6f442b78549f\") " pod="openshift-must-gather-k7rsc/perf-node-gather-daemonset-pt5d6" Jul 14 20:42:28.999098 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:42:28.998919 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/67886ff4-347d-4990-aee1-6f442b78549f-proc\") pod \"perf-node-gather-daemonset-pt5d6\" (UID: \"67886ff4-347d-4990-aee1-6f442b78549f\") " pod="openshift-must-gather-k7rsc/perf-node-gather-daemonset-pt5d6" Jul 14 20:42:28.999098 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:42:28.998953 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/67886ff4-347d-4990-aee1-6f442b78549f-podres\") pod \"perf-node-gather-daemonset-pt5d6\" (UID: \"67886ff4-347d-4990-aee1-6f442b78549f\") " pod="openshift-must-gather-k7rsc/perf-node-gather-daemonset-pt5d6" Jul 14 20:42:29.007116 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:42:29.007084 2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k5q2h\" (UniqueName: \"kubernetes.io/projected/67886ff4-347d-4990-aee1-6f442b78549f-kube-api-access-k5q2h\") pod \"perf-node-gather-daemonset-pt5d6\" (UID: \"67886ff4-347d-4990-aee1-6f442b78549f\") " pod="openshift-must-gather-k7rsc/perf-node-gather-daemonset-pt5d6" Jul 14 20:42:29.160642 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:42:29.160602 2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-k7rsc/perf-node-gather-daemonset-pt5d6" Jul 14 20:42:29.302555 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:42:29.302529 2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-k7rsc/perf-node-gather-daemonset-pt5d6"] Jul 14 20:42:29.304500 ip-10-0-141-61 kubenswrapper[2563]: W0714 20:42:29.304464 2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod67886ff4_347d_4990_aee1_6f442b78549f.slice/crio-4e729c4fb0f55e4ead65277d1f2e4c2645e56363782acdf1fd816cecae98466a WatchSource:0}: Error finding container 4e729c4fb0f55e4ead65277d1f2e4c2645e56363782acdf1fd816cecae98466a: Status 404 returned error can't find the container with id 4e729c4fb0f55e4ead65277d1f2e4c2645e56363782acdf1fd816cecae98466a Jul 14 20:42:29.956777 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:42:29.956740 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-k7rsc/perf-node-gather-daemonset-pt5d6" event={"ID":"67886ff4-347d-4990-aee1-6f442b78549f","Type":"ContainerStarted","Data":"48dfd407fb2ebf0bba315355385e881fb220c82c6114c329cf1a2c189d805f41"} Jul 14 20:42:29.957197 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:42:29.956821 2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-k7rsc/perf-node-gather-daemonset-pt5d6" event={"ID":"67886ff4-347d-4990-aee1-6f442b78549f","Type":"ContainerStarted","Data":"4e729c4fb0f55e4ead65277d1f2e4c2645e56363782acdf1fd816cecae98466a"} Jul 14 20:42:29.957197 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:42:29.956864 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-k7rsc/perf-node-gather-daemonset-pt5d6" Jul 14 20:42:29.973036 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:42:29.972983 2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-k7rsc/perf-node-gather-daemonset-pt5d6" podStartSLOduration=1.972961736 podStartE2EDuration="1.972961736s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 20:42:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 20:42:29.972509877 +0000 UTC m=+1208.893673210" watchObservedRunningTime="2026-07-14 20:42:29.972961736 +0000 UTC m=+1208.894125046" Jul 14 20:42:30.815729 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:42:30.815689 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-knrl4_b4e3e206-bf7b-47bf-b037-9c12fca3799b/dns/0.log" Jul 14 20:42:30.837944 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:42:30.837920 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-knrl4_b4e3e206-bf7b-47bf-b037-9c12fca3799b/kube-rbac-proxy/0.log" Jul 14 20:42:30.944881 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:42:30.944851 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-dznvt_9d9cd7af-00ce-4f0a-8205-faf1e29a50e7/dns-node-resolver/0.log" Jul 14 20:42:31.550534 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:42:31.550503 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-88985_afbb09bb-badd-4dc0-8082-581c48dda65b/node-ca/0.log" Jul 14 20:42:32.564960 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:42:32.564927 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf45cgj7_c4990631-d6e1-46cd-afdb-0692ca08c6ba/istio-proxy/0.log" Jul 14 20:42:32.631072 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:42:32.631040 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-8jmfx_42b909af-0bd9-4036-b180-2c5cd4394b30/discovery/0.log" Jul 14 20:42:32.708109 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:42:32.708078 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-mzmqf_afb7dcce-c4ff-4a07-983f-8f52fa1409bd/istio-proxy/0.log" Jul 14 20:42:32.725642 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:42:32.725613 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-mzmqf_afb7dcce-c4ff-4a07-983f-8f52fa1409bd/istio-proxy/1.log" Jul 14 20:42:33.511453 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:42:33.511423 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-v2v5k_cf457900-0b78-45d7-9e9e-99d9fe24bd1c/serve-healthcheck-canary/0.log" Jul 14 20:42:34.007400 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:42:34.007368 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-68d9dd94cf-spp52_cb3ba939-0e7c-4afc-a64b-e073e5cf656c/insights-operator/1.log" Jul 14 20:42:34.008021 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:42:34.008002 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-68d9dd94cf-spp52_cb3ba939-0e7c-4afc-a64b-e073e5cf656c/insights-operator/0.log" Jul 14 20:42:34.188608 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:42:34.188575 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-tj9d5_cd97f132-9f22-4dc9-986f-315e50547b82/kube-rbac-proxy/0.log" Jul 14 20:42:34.210087 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:42:34.210051 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-tj9d5_cd97f132-9f22-4dc9-986f-315e50547b82/exporter/0.log" Jul 14 20:42:34.232078 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:42:34.232054 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-tj9d5_cd97f132-9f22-4dc9-986f-315e50547b82/extractor/0.log" Jul 14 20:42:35.972578 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:42:35.972550 2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-k7rsc/perf-node-gather-daemonset-pt5d6" Jul 14 20:42:36.431745 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:42:36.431688 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-5c7b6db686-dq5lx_5b83138a-1f77-455b-8be5-d183995c46f1/manager/0.log" Jul 14 20:42:36.595623 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:42:36.595581 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-79d49c7b87-bn8n6_af45e5e1-4dc3-4848-a094-f979fec50d9d/manager/0.log" Jul 14 20:42:36.616528 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:42:36.616503 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-79d49c7b87-bn8n6_af45e5e1-4dc3-4848-a094-f979fec50d9d/copy-manifests/0.log" Jul 14 20:42:37.995850 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:42:37.995819 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-78b47f4478-8sv5k_997602e8-2fb7-46e0-9720-266e09a79f4b/manager/0.log" Jul 14 20:42:38.854161 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:42:38.854112 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf45cgj7_c4990631-d6e1-46cd-afdb-0692ca08c6ba/istio-proxy/0.log" Jul 14 20:42:38.915951 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:42:38.915915 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-8jmfx_42b909af-0bd9-4036-b180-2c5cd4394b30/discovery/0.log" Jul 14 20:42:38.964604 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:42:38.964563 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-mzmqf_afb7dcce-c4ff-4a07-983f-8f52fa1409bd/istio-proxy/0.log" Jul 14 20:42:38.980973 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:42:38.980944 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-mzmqf_afb7dcce-c4ff-4a07-983f-8f52fa1409bd/istio-proxy/1.log" Jul 14 20:42:40.116555 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:42:40.116518 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf45cgj7_c4990631-d6e1-46cd-afdb-0692ca08c6ba/istio-proxy/0.log" Jul 14 20:42:40.179728 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:42:40.179696 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-8jmfx_42b909af-0bd9-4036-b180-2c5cd4394b30/discovery/0.log" Jul 14 20:42:40.248262 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:42:40.248230 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-mzmqf_afb7dcce-c4ff-4a07-983f-8f52fa1409bd/istio-proxy/0.log" Jul 14 20:42:40.265910 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:42:40.265869 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-mzmqf_afb7dcce-c4ff-4a07-983f-8f52fa1409bd/istio-proxy/1.log" Jul 14 20:42:43.293010 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:42:43.292974 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-7f59b54f6c-9mbjv_43dccdcb-04f9-4844-8574-90a15ec10a6d/migrator/0.log" Jul 14 20:42:43.316358 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:42:43.316326 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-7f59b54f6c-9mbjv_43dccdcb-04f9-4844-8574-90a15ec10a6d/graceful-termination/0.log" Jul 14 20:42:44.803612 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:42:44.803584 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-jzpsb_035331bf-20d9-4df2-99f6-b0e0db2e05f0/kube-multus-additional-cni-plugins/0.log" Jul 14 20:42:44.824778 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:42:44.824748 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-jzpsb_035331bf-20d9-4df2-99f6-b0e0db2e05f0/egress-router-binary-copy/0.log" Jul 14 20:42:44.848299 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:42:44.848271 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-jzpsb_035331bf-20d9-4df2-99f6-b0e0db2e05f0/cni-plugins/0.log" Jul 14 20:42:44.874009 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:42:44.873983 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-jzpsb_035331bf-20d9-4df2-99f6-b0e0db2e05f0/bond-cni-plugin/0.log" Jul 14 20:42:44.903233 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:42:44.903188 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-jzpsb_035331bf-20d9-4df2-99f6-b0e0db2e05f0/routeoverride-cni/0.log" Jul 14 20:42:44.925953 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:42:44.925925 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-jzpsb_035331bf-20d9-4df2-99f6-b0e0db2e05f0/whereabouts-cni-bincopy/0.log" Jul 14 20:42:44.948340 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:42:44.948311 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-jzpsb_035331bf-20d9-4df2-99f6-b0e0db2e05f0/whereabouts-cni/0.log" Jul 14 20:42:45.304688 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:42:45.304659 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-bbzpp_5835902d-ae0d-4395-9843-0ef04ad24976/kube-multus/0.log" Jul 14 20:42:45.419908 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:42:45.419878 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-rvhpg_92cbd473-9b2e-4faa-a232-810627648cc9/network-metrics-daemon/0.log" Jul 14 20:42:45.442388 ip-10-0-141-61 kubenswrapper[2563]: I0714 20:42:45.442353 2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-rvhpg_92cbd473-9b2e-4faa-a232-810627648cc9/kube-rbac-proxy/0.log"