Jul 29 22:46:23.599118 ip-10-0-138-112 systemd[1]: Starting Kubernetes Kubelet... Jul 29 22:46:23.970621 ip-10-0-138-112 kubenswrapper[2562]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 29 22:46:23.970621 ip-10-0-138-112 kubenswrapper[2562]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 29 22:46:23.970621 ip-10-0-138-112 kubenswrapper[2562]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 29 22:46:23.970621 ip-10-0-138-112 kubenswrapper[2562]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 29 22:46:23.970621 ip-10-0-138-112 kubenswrapper[2562]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 29 22:46:23.972098 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:23.972008 2562 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 29 22:46:23.978433 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.978407 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 29 22:46:23.978433 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.978430 2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 29 22:46:23.978433 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.978434 2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 29 22:46:23.978433 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.978438 2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 29 22:46:23.978616 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.978441 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 29 22:46:23.978616 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.978446 2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 29 22:46:23.978616 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.978452 2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 29 22:46:23.978616 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.978456 2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 29 22:46:23.978616 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.978459 2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 29 22:46:23.978616 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.978463 2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 29 22:46:23.978616 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.978466 2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 29 22:46:23.978616 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.978468 2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 29 22:46:23.978616 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.978471 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 29 22:46:23.978616 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.978474 2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 29 22:46:23.978616 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.978477 2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 29 22:46:23.978616 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.978480 2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 29 22:46:23.978616 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.978483 2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 29 22:46:23.978616 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.978486 2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 29 22:46:23.978616 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.978489 2562 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 29 22:46:23.978616 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.978491 2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 29 22:46:23.978616 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.978494 2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 29 22:46:23.978616 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.978497 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 29 22:46:23.978616 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.978499 2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 29 22:46:23.979096 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.978502 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 29 22:46:23.979096 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.978504 2562 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 29 22:46:23.979096 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.978507 2562 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 29 22:46:23.979096 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.978509 2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 29 22:46:23.979096 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.978512 2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 29 22:46:23.979096 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.978515 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 29 22:46:23.979096 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.978517 2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 29 22:46:23.979096 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.978520 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 29 22:46:23.979096 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.978522 2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 29 22:46:23.979096 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.978526 2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 29 22:46:23.979096 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.978529 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 29 22:46:23.979096 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.978531 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 29 22:46:23.979096 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.978534 2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 29 22:46:23.979096 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.978537 2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 29 22:46:23.979096 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.978541 2562 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 29 22:46:23.979096 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.978544 2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 29 22:46:23.979096 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.978546 2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 29 22:46:23.979096 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.978549 2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 29 22:46:23.979096 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.978552 2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 29 22:46:23.979096 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.978554 2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 29 22:46:23.979571 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.978557 2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 29 22:46:23.979571 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.978560 2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 29 22:46:23.979571 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.978563 2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 29 22:46:23.979571 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.978567 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 29 22:46:23.979571 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.978570 2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 29 22:46:23.979571 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.978573 2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 29 22:46:23.979571 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.978576 2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 29 22:46:23.979571 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.978579 2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 29 22:46:23.979571 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.978582 2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 29 22:46:23.979571 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.978584 2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 29 22:46:23.979571 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.978587 2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 29 22:46:23.979571 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.978590 2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 29 22:46:23.979571 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.978593 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 29 22:46:23.979571 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.978595 2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 29 22:46:23.979571 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.978598 2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 29 22:46:23.979571 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.978601 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 29 22:46:23.979571 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.978604 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 29 22:46:23.979571 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.978606 2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 29 22:46:23.979571 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.978609 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 29 22:46:23.980057 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.978611 2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 29 22:46:23.980057 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.978614 2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 29 22:46:23.980057 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.978617 2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 29 22:46:23.980057 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.978619 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 29 22:46:23.980057 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.978622 2562 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 29 22:46:23.980057 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.978625 2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 29 22:46:23.980057 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.978628 2562 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 29 22:46:23.980057 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.978631 2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 29 22:46:23.980057 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.978634 2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 29 22:46:23.980057 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.978637 2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 29 22:46:23.980057 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.978639 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 29 22:46:23.980057 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.978642 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 29 22:46:23.980057 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.978644 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 29 22:46:23.980057 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.978647 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 29 22:46:23.980057 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.978650 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 29 22:46:23.980057 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.978652 2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 29 22:46:23.980057 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.978655 2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 29 22:46:23.980057 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.978657 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 29 22:46:23.980057 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.978661 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 29 22:46:23.980057 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.978663 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 29 22:46:23.980572 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.978666 2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 29 22:46:23.980572 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.978669 2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 29 22:46:23.980572 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.978672 2562 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 29 22:46:23.980572 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.978674 2562 feature_gate.go:328] unrecognized feature gate: Example2 Jul 29 22:46:23.980572 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.978677 2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 29 22:46:23.980572 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.978680 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 29 22:46:23.980572 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.978682 2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 29 22:46:23.980572 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.978685 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 29 22:46:23.980572 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.978687 2562 feature_gate.go:328] unrecognized feature gate: Example Jul 29 22:46:23.980572 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.978690 2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 29 22:46:23.980572 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.978694 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 29 22:46:23.980572 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.978697 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 29 22:46:23.980572 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.978699 2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 29 22:46:23.980572 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.978702 2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 29 22:46:23.980572 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.978704 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 29 22:46:23.980572 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.978707 2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 29 22:46:23.980572 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.978709 2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 29 22:46:23.980572 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.979448 2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 29 22:46:23.980572 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.979455 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 29 22:46:23.981042 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.979458 2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 29 22:46:23.981042 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.979461 2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 29 22:46:23.981042 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.979464 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 29 22:46:23.981042 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.979467 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 29 22:46:23.981042 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.979470 2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 29 22:46:23.981042 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.979472 2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 29 22:46:23.981042 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.979475 2562 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 29 22:46:23.981042 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.979480 2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 29 22:46:23.981042 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.979484 2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 29 22:46:23.981042 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.979487 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 29 22:46:23.981042 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.979490 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 29 22:46:23.981042 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.979493 2562 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 29 22:46:23.981042 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.979495 2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 29 22:46:23.981042 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.979498 2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 29 22:46:23.981042 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.979501 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 29 22:46:23.981042 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.979504 2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 29 22:46:23.981042 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.979507 2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 29 22:46:23.981042 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.979510 2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 29 22:46:23.981042 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.979513 2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 29 22:46:23.981042 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.979516 2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 29 22:46:23.981530 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.979519 2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 29 22:46:23.981530 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.979521 2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 29 22:46:23.981530 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.979524 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 29 22:46:23.981530 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.979526 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 29 22:46:23.981530 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.979529 2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 29 22:46:23.981530 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.979531 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 29 22:46:23.981530 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.979534 2562 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 29 22:46:23.981530 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.979536 2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 29 22:46:23.981530 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.979539 2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 29 22:46:23.981530 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.979541 2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 29 22:46:23.981530 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.979544 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 29 22:46:23.981530 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.979547 2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 29 22:46:23.981530 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.979550 2562 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 29 22:46:23.981530 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.979553 2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 29 22:46:23.981530 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.979556 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 29 22:46:23.981530 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.979558 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 29 22:46:23.981530 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.979561 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 29 22:46:23.981530 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.979563 2562 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 29 22:46:23.981530 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.979566 2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 29 22:46:23.981530 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.979568 2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 29 22:46:23.982043 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.979571 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 29 22:46:23.982043 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.979574 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 29 22:46:23.982043 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.979576 2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 29 22:46:23.982043 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.979581 2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 29 22:46:23.982043 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.979584 2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 29 22:46:23.982043 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.979587 2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 29 22:46:23.982043 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.979591 2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 29 22:46:23.982043 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.979593 2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 29 22:46:23.982043 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.979597 2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 29 22:46:23.982043 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.979600 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 29 22:46:23.982043 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.979602 2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 29 22:46:23.982043 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.979605 2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 29 22:46:23.982043 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.979608 2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 29 22:46:23.982043 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.979610 2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 29 22:46:23.982043 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.979613 2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 29 22:46:23.982043 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.979615 2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 29 22:46:23.982043 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.979618 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 29 22:46:23.982043 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.979621 2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 29 22:46:23.982043 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.979624 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 29 22:46:23.982505 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.979626 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 29 22:46:23.982505 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.979629 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 29 22:46:23.982505 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.979631 2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 29 22:46:23.982505 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.979633 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 29 22:46:23.982505 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.979636 2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 29 22:46:23.982505 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.979639 2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 29 22:46:23.982505 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.979641 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 29 22:46:23.982505 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.979644 2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 29 22:46:23.982505 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.979646 2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 29 22:46:23.982505 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.979649 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 29 22:46:23.982505 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.979656 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 29 22:46:23.982505 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.979658 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 29 22:46:23.982505 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.979661 2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 29 22:46:23.982505 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.979663 2562 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 29 22:46:23.982505 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.979666 2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 29 22:46:23.982505 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.979668 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 29 22:46:23.982505 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.979671 2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 29 22:46:23.982505 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.979674 2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 29 22:46:23.982505 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.979676 2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 29 22:46:23.982992 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.979678 2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 29 22:46:23.982992 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.979681 2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 29 22:46:23.982992 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.979684 2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 29 22:46:23.982992 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.979687 2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 29 22:46:23.982992 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.979692 2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 29 22:46:23.982992 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.979695 2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 29 22:46:23.982992 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.979699 2562 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 29 22:46:23.982992 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.979702 2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 29 22:46:23.982992 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.979704 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 29 22:46:23.982992 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.979707 2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 29 22:46:23.982992 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.979709 2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 29 22:46:23.982992 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.979712 2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 29 22:46:23.982992 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.979715 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 29 22:46:23.982992 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.979717 2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 29 22:46:23.982992 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.979720 2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 29 22:46:23.982992 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.979722 2562 feature_gate.go:328] unrecognized feature gate: Example Jul 29 22:46:23.982992 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.979725 2562 feature_gate.go:328] unrecognized feature gate: Example2 Jul 29 22:46:23.982992 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.979728 2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 29 22:46:23.982992 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.979731 2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 29 22:46:23.982992 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.980141 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 29 22:46:23.983471 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.980147 2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 29 22:46:23.983471 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.980149 2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 29 22:46:23.983471 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.980152 2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 29 22:46:23.983471 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.980156 2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 29 22:46:23.983471 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.980159 2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 29 22:46:23.983471 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.980161 2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 29 22:46:23.983471 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.980164 2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 29 22:46:23.983471 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.980166 2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 29 22:46:23.983471 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.980169 2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 29 22:46:23.983471 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.980171 2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 29 22:46:23.983471 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.980174 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 29 22:46:23.983471 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.980177 2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 29 22:46:23.983471 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.980180 2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 29 22:46:23.983471 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.980182 2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 29 22:46:23.983471 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.980185 2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 29 22:46:23.983471 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.980188 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 29 22:46:23.983471 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.980190 2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 29 22:46:23.983471 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.980193 2562 feature_gate.go:328] unrecognized feature gate: Example Jul 29 22:46:23.983471 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.980196 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 29 22:46:23.983471 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.980198 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 29 22:46:23.983976 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.980201 2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 29 22:46:23.983976 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.980203 2562 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 29 22:46:23.983976 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.980206 2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 29 22:46:23.983976 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.980208 2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 29 22:46:23.983976 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.980211 2562 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 29 22:46:23.983976 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.980214 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 29 22:46:23.983976 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.980216 2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 29 22:46:23.983976 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.980219 2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 29 22:46:23.983976 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.980221 2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 29 22:46:23.983976 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.980224 2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 29 22:46:23.983976 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.980227 2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 29 22:46:23.983976 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.980230 2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 29 22:46:23.983976 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.980233 2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 29 22:46:23.983976 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.980235 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 29 22:46:23.983976 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.980238 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 29 22:46:23.983976 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.980240 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 29 22:46:23.983976 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.980243 2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 29 22:46:23.983976 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.980246 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 29 22:46:23.983976 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.980248 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 29 22:46:23.983976 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.980251 2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 29 22:46:23.984530 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.980253 2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 29 22:46:23.984530 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.980256 2562 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 29 22:46:23.984530 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.980259 2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 29 22:46:23.984530 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.980261 2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 29 22:46:23.984530 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.980264 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 29 22:46:23.984530 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.980266 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 29 22:46:23.984530 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.980268 2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 29 22:46:23.984530 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.980271 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 29 22:46:23.984530 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.980274 2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 29 22:46:23.984530 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.980276 2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 29 22:46:23.984530 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.980279 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 29 22:46:23.984530 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.980281 2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 29 22:46:23.984530 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.980284 2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 29 22:46:23.984530 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.980286 2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 29 22:46:23.984530 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.980289 2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 29 22:46:23.984530 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.980291 2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 29 22:46:23.984530 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.980294 2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 29 22:46:23.984530 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.980296 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 29 22:46:23.984530 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.980299 2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 29 22:46:23.984530 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.980302 2562 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 29 22:46:23.985058 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.980304 2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 29 22:46:23.985058 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.980307 2562 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 29 22:46:23.985058 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.980310 2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 29 22:46:23.985058 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.980314 2562 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 29 22:46:23.985058 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.980317 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 29 22:46:23.985058 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.980319 2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 29 22:46:23.985058 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.980322 2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 29 22:46:23.985058 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.980325 2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 29 22:46:23.985058 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.980327 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 29 22:46:23.985058 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.980330 2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 29 22:46:23.985058 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.980334 2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 29 22:46:23.985058 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.980337 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 29 22:46:23.985058 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.980341 2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 29 22:46:23.985058 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.980345 2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 29 22:46:23.985058 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.980348 2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 29 22:46:23.985058 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.980351 2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 29 22:46:23.985058 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.980354 2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 29 22:46:23.985058 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.980357 2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 29 22:46:23.985058 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.980359 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 29 22:46:23.985516 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.980362 2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 29 22:46:23.985516 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.980364 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 29 22:46:23.985516 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.980367 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 29 22:46:23.985516 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.980370 2562 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 29 22:46:23.985516 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.980372 2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 29 22:46:23.985516 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.980375 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 29 22:46:23.985516 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.980380 2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 29 22:46:23.985516 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.980383 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 29 22:46:23.985516 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.980386 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 29 22:46:23.985516 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.980389 2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 29 22:46:23.985516 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.980392 2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 29 22:46:23.985516 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.980395 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 29 22:46:23.985516 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.980398 2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 29 22:46:23.985516 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.980401 2562 feature_gate.go:328] unrecognized feature gate: Example2 Jul 29 22:46:23.985516 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.980404 2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 29 22:46:23.985516 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.980407 2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 29 22:46:23.985516 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.980410 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 29 22:46:23.985516 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.980412 2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 29 22:46:23.985516 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.980415 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 29 22:46:23.985516 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:23.980489 2562 flags.go:64] FLAG: --address="0.0.0.0" Jul 29 22:46:23.986069 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:23.980496 2562 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 29 22:46:23.986069 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:23.980503 2562 flags.go:64] FLAG: --anonymous-auth="true" Jul 29 22:46:23.986069 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:23.980510 2562 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 29 22:46:23.986069 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:23.980516 2562 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 29 22:46:23.986069 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:23.980521 2562 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 29 22:46:23.986069 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:23.980528 2562 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 29 22:46:23.986069 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:23.980534 2562 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 29 22:46:23.986069 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:23.980538 2562 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 29 22:46:23.986069 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:23.980541 2562 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 29 22:46:23.986069 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:23.980545 2562 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 29 22:46:23.986069 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:23.980548 2562 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 29 22:46:23.986069 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:23.980551 2562 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 29 22:46:23.986069 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:23.980554 2562 flags.go:64] FLAG: --cgroup-root="" Jul 29 22:46:23.986069 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:23.980557 2562 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 29 22:46:23.986069 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:23.980560 2562 flags.go:64] FLAG: --client-ca-file="" Jul 29 22:46:23.986069 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:23.980565 2562 flags.go:64] FLAG: --cloud-provider="external" Jul 29 22:46:23.986069 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:23.980568 2562 flags.go:64] FLAG: --cluster-dns="[]" Jul 29 22:46:23.986069 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:23.980572 2562 flags.go:64] FLAG: --cluster-domain="" Jul 29 22:46:23.986069 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:23.980575 2562 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 29 22:46:23.986069 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:23.980578 2562 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 29 22:46:23.986069 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:23.980582 2562 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 29 22:46:23.986069 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:23.980587 2562 flags.go:64] FLAG: --container-log-max-files="5" Jul 29 22:46:23.986069 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:23.980591 2562 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 29 22:46:23.986069 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:23.980594 2562 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 29 22:46:23.986639 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:23.980597 2562 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 29 22:46:23.986639 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:23.980600 2562 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 29 22:46:23.986639 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:23.980603 2562 flags.go:64] FLAG: --contention-profiling="false" Jul 29 22:46:23.986639 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:23.980606 2562 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 29 22:46:23.986639 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:23.980609 2562 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 29 22:46:23.986639 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:23.980612 2562 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 29 22:46:23.986639 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:23.980615 2562 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 29 22:46:23.986639 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:23.980619 2562 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 29 22:46:23.986639 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:23.980622 2562 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 29 22:46:23.986639 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:23.980625 2562 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 29 22:46:23.986639 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:23.980627 2562 flags.go:64] FLAG: --enable-load-reader="false" Jul 29 22:46:23.986639 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:23.980630 2562 flags.go:64] FLAG: --enable-server="true" Jul 29 22:46:23.986639 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:23.980633 2562 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 29 22:46:23.986639 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:23.980658 2562 flags.go:64] FLAG: --event-burst="100" Jul 29 22:46:23.986639 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:23.980662 2562 flags.go:64] FLAG: --event-qps="50" Jul 29 22:46:23.986639 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:23.980665 2562 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 29 22:46:23.986639 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:23.980668 2562 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 29 22:46:23.986639 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:23.980671 2562 flags.go:64] FLAG: --eviction-hard="" Jul 29 22:46:23.986639 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:23.980675 2562 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 29 22:46:23.986639 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:23.980678 2562 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 29 22:46:23.986639 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:23.980681 2562 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 29 22:46:23.986639 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:23.980684 2562 flags.go:64] FLAG: --eviction-soft="" Jul 29 22:46:23.986639 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:23.980686 2562 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 29 22:46:23.986639 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:23.980689 2562 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 29 22:46:23.986639 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:23.980693 2562 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 29 22:46:23.987306 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:23.980696 2562 flags.go:64] FLAG: --experimental-mounter-path="" Jul 29 22:46:23.987306 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:23.980698 2562 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 29 22:46:23.987306 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:23.980701 2562 flags.go:64] FLAG: --fail-swap-on="true" Jul 29 22:46:23.987306 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:23.980704 2562 flags.go:64] FLAG: --feature-gates="" Jul 29 22:46:23.987306 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:23.980709 2562 flags.go:64] FLAG: --file-check-frequency="20s" Jul 29 22:46:23.987306 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:23.980712 2562 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 29 22:46:23.987306 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:23.980715 2562 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 29 22:46:23.987306 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:23.980718 2562 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 29 22:46:23.987306 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:23.980721 2562 flags.go:64] FLAG: --healthz-port="10248" Jul 29 22:46:23.987306 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:23.980725 2562 flags.go:64] FLAG: --help="false" Jul 29 22:46:23.987306 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:23.980727 2562 flags.go:64] FLAG: --hostname-override="ip-10-0-138-112.ec2.internal" Jul 29 22:46:23.987306 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:23.980731 2562 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 29 22:46:23.987306 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:23.980733 2562 flags.go:64] FLAG: --http-check-frequency="20s" Jul 29 22:46:23.987306 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:23.980736 2562 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 29 22:46:23.987306 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:23.980740 2562 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 29 22:46:23.987306 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:23.980743 2562 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 29 22:46:23.987306 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:23.980746 2562 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 29 22:46:23.987306 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:23.980749 2562 flags.go:64] FLAG: --image-service-endpoint="" Jul 29 22:46:23.987306 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:23.980752 2562 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 29 22:46:23.987306 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:23.980754 2562 flags.go:64] FLAG: --kube-api-burst="100" Jul 29 22:46:23.987306 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:23.980757 2562 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 29 22:46:23.987306 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:23.980760 2562 flags.go:64] FLAG: --kube-api-qps="50" Jul 29 22:46:23.987306 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:23.980763 2562 flags.go:64] FLAG: --kube-reserved="" Jul 29 22:46:23.987306 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:23.980766 2562 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 29 22:46:23.987926 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:23.980768 2562 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 29 22:46:23.987926 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:23.980771 2562 flags.go:64] FLAG: --kubelet-cgroups="" Jul 29 22:46:23.987926 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:23.980774 2562 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 29 22:46:23.987926 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:23.980777 2562 flags.go:64] FLAG: --lock-file="" Jul 29 22:46:23.987926 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:23.980780 2562 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 29 22:46:23.987926 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:23.980783 2562 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 29 22:46:23.987926 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:23.980786 2562 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 29 22:46:23.987926 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:23.980791 2562 flags.go:64] FLAG: --log-json-split-stream="false" Jul 29 22:46:23.987926 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:23.980794 2562 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 29 22:46:23.987926 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:23.980797 2562 flags.go:64] FLAG: --log-text-split-stream="false" Jul 29 22:46:23.987926 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:23.980800 2562 flags.go:64] FLAG: --logging-format="text" Jul 29 22:46:23.987926 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:23.980802 2562 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 29 22:46:23.987926 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:23.980807 2562 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 29 22:46:23.987926 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:23.980810 2562 flags.go:64] FLAG: --manifest-url="" Jul 29 22:46:23.987926 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:23.980813 2562 flags.go:64] FLAG: --manifest-url-header="" Jul 29 22:46:23.987926 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:23.980818 2562 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 29 22:46:23.987926 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:23.980821 2562 flags.go:64] FLAG: --max-open-files="1000000" Jul 29 22:46:23.987926 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:23.980837 2562 flags.go:64] FLAG: --max-pods="110" Jul 29 22:46:23.987926 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:23.980841 2562 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 29 22:46:23.987926 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:23.980844 2562 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 29 22:46:23.987926 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:23.980847 2562 flags.go:64] FLAG: --memory-manager-policy="None" Jul 29 22:46:23.987926 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:23.980849 2562 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 29 22:46:23.987926 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:23.980852 2562 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 29 22:46:23.987926 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:23.980855 2562 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 29 22:46:23.987926 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:23.980858 2562 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 29 22:46:23.988538 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:23.980866 2562 flags.go:64] FLAG: --node-status-max-images="50" Jul 29 22:46:23.988538 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:23.980869 2562 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 29 22:46:23.988538 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:23.980872 2562 flags.go:64] FLAG: --oom-score-adj="-999" Jul 29 22:46:23.988538 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:23.980875 2562 flags.go:64] FLAG: --pod-cidr="" Jul 29 22:46:23.988538 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:23.980878 2562 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7505336a406394b8c1c77d14d890a9f628aa4cfff172312179cb9af8c28c30bb" Jul 29 22:46:23.988538 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:23.980884 2562 flags.go:64] FLAG: --pod-manifest-path="" Jul 29 22:46:23.988538 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:23.980887 2562 flags.go:64] FLAG: --pod-max-pids="-1" Jul 29 22:46:23.988538 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:23.980890 2562 flags.go:64] FLAG: --pods-per-core="0" Jul 29 22:46:23.988538 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:23.980893 2562 flags.go:64] FLAG: --port="10250" Jul 29 22:46:23.988538 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:23.980896 2562 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 29 22:46:23.988538 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:23.980900 2562 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0c953426b5d0ae612" Jul 29 22:46:23.988538 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:23.980903 2562 flags.go:64] FLAG: --qos-reserved="" Jul 29 22:46:23.988538 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:23.980906 2562 flags.go:64] FLAG: --read-only-port="10255" Jul 29 22:46:23.988538 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:23.980909 2562 flags.go:64] FLAG: --register-node="true" Jul 29 22:46:23.988538 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:23.980912 2562 flags.go:64] FLAG: --register-with-taints="" Jul 29 22:46:23.988538 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:23.980916 2562 flags.go:64] FLAG: --registry-burst="10" Jul 29 22:46:23.988538 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:23.980919 2562 flags.go:64] FLAG: --registry-qps="5" Jul 29 22:46:23.988538 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:23.980922 2562 flags.go:64] FLAG: --reserved-cpus="" Jul 29 22:46:23.988538 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:23.980925 2562 flags.go:64] FLAG: --reserved-memory="" Jul 29 22:46:23.988538 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:23.980929 2562 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 29 22:46:23.988538 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:23.980932 2562 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 29 22:46:23.988538 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:23.980935 2562 flags.go:64] FLAG: --rotate-certificates="false" Jul 29 22:46:23.988538 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:23.980941 2562 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 29 22:46:23.988538 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:23.980944 2562 flags.go:64] FLAG: --runonce="false" Jul 29 22:46:23.988538 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:23.980947 2562 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 29 22:46:23.989175 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:23.980949 2562 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 29 22:46:23.989175 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:23.980953 2562 flags.go:64] FLAG: --seccomp-default="false" Jul 29 22:46:23.989175 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:23.980955 2562 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 29 22:46:23.989175 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:23.980958 2562 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 29 22:46:23.989175 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:23.980961 2562 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 29 22:46:23.989175 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:23.980964 2562 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 29 22:46:23.989175 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:23.980967 2562 flags.go:64] FLAG: --storage-driver-password="root" Jul 29 22:46:23.989175 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:23.980970 2562 flags.go:64] FLAG: --storage-driver-secure="false" Jul 29 22:46:23.989175 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:23.980973 2562 flags.go:64] FLAG: --storage-driver-table="stats" Jul 29 22:46:23.989175 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:23.980976 2562 flags.go:64] FLAG: --storage-driver-user="root" Jul 29 22:46:23.989175 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:23.980979 2562 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 29 22:46:23.989175 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:23.980981 2562 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 29 22:46:23.989175 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:23.980985 2562 flags.go:64] FLAG: --system-cgroups="" Jul 29 22:46:23.989175 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:23.980987 2562 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 29 22:46:23.989175 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:23.980992 2562 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 29 22:46:23.989175 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:23.980995 2562 flags.go:64] FLAG: --tls-cert-file="" Jul 29 22:46:23.989175 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:23.980998 2562 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 29 22:46:23.989175 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:23.981002 2562 flags.go:64] FLAG: --tls-min-version="" Jul 29 22:46:23.989175 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:23.981004 2562 flags.go:64] FLAG: --tls-private-key-file="" Jul 29 22:46:23.989175 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:23.981007 2562 flags.go:64] FLAG: --topology-manager-policy="none" Jul 29 22:46:23.989175 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:23.981010 2562 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 29 22:46:23.989175 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:23.981013 2562 flags.go:64] FLAG: --topology-manager-scope="container" Jul 29 22:46:23.989175 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:23.981016 2562 flags.go:64] FLAG: --v="2" Jul 29 22:46:23.989175 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:23.981020 2562 flags.go:64] FLAG: --version="false" Jul 29 22:46:23.989175 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:23.981025 2562 flags.go:64] FLAG: --vmodule="" Jul 29 22:46:23.989879 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:23.981031 2562 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 29 22:46:23.989879 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:23.981036 2562 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 29 22:46:23.989879 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.981154 2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 29 22:46:23.989879 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.981159 2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 29 22:46:23.989879 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.981163 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 29 22:46:23.989879 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.981166 2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 29 22:46:23.989879 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.981170 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 29 22:46:23.989879 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.981173 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 29 22:46:23.989879 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.981176 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 29 22:46:23.989879 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.981178 2562 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 29 22:46:23.989879 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.981181 2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 29 22:46:23.989879 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.981183 2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 29 22:46:23.989879 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.981186 2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 29 22:46:23.989879 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.981188 2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 29 22:46:23.989879 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.981191 2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 29 22:46:23.989879 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.981194 2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 29 22:46:23.989879 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.981196 2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 29 22:46:23.989879 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.981199 2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 29 22:46:23.989879 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.981202 2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 29 22:46:23.989879 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.981205 2562 feature_gate.go:328] unrecognized feature gate: Example2 Jul 29 22:46:23.990376 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.981208 2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 29 22:46:23.990376 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.981210 2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 29 22:46:23.990376 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.981213 2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 29 22:46:23.990376 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.981216 2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 29 22:46:23.990376 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.981218 2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 29 22:46:23.990376 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.981221 2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 29 22:46:23.990376 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.981223 2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 29 22:46:23.990376 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.981226 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 29 22:46:23.990376 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.981228 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 29 22:46:23.990376 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.981231 2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 29 22:46:23.990376 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.981233 2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 29 22:46:23.990376 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.981236 2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 29 22:46:23.990376 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.981239 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 29 22:46:23.990376 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.981243 2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 29 22:46:23.990376 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.981246 2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 29 22:46:23.990376 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.981248 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 29 22:46:23.990376 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.981252 2562 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 29 22:46:23.990376 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.981255 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 29 22:46:23.990376 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.981258 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 29 22:46:23.990376 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.981260 2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 29 22:46:23.990905 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.981263 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 29 22:46:23.990905 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.981265 2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 29 22:46:23.990905 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.981268 2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 29 22:46:23.990905 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.981271 2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 29 22:46:23.990905 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.981273 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 29 22:46:23.990905 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.981276 2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 29 22:46:23.990905 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.981279 2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 29 22:46:23.990905 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.981282 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 29 22:46:23.990905 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.981284 2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 29 22:46:23.990905 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.981287 2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 29 22:46:23.990905 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.981289 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 29 22:46:23.990905 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.981292 2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 29 22:46:23.990905 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.981294 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 29 22:46:23.990905 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.981297 2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 29 22:46:23.990905 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.981300 2562 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 29 22:46:23.990905 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.981302 2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 29 22:46:23.990905 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.981305 2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 29 22:46:23.990905 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.981307 2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 29 22:46:23.990905 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.981310 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 29 22:46:23.990905 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.981312 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 29 22:46:23.991419 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.981315 2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 29 22:46:23.991419 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.981318 2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 29 22:46:23.991419 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.981322 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 29 22:46:23.991419 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.981325 2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 29 22:46:23.991419 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.981328 2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 29 22:46:23.991419 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.981333 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 29 22:46:23.991419 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.981335 2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 29 22:46:23.991419 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.981338 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 29 22:46:23.991419 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.981342 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 29 22:46:23.991419 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.981344 2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 29 22:46:23.991419 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.981347 2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 29 22:46:23.991419 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.981349 2562 feature_gate.go:328] unrecognized feature gate: Example Jul 29 22:46:23.991419 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.981352 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 29 22:46:23.991419 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.981355 2562 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 29 22:46:23.991419 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.981358 2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 29 22:46:23.991419 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.981362 2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 29 22:46:23.991419 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.981366 2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 29 22:46:23.991419 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.981369 2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 29 22:46:23.991419 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.981371 2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 29 22:46:23.991911 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.981374 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 29 22:46:23.991911 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.981377 2562 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 29 22:46:23.991911 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.981379 2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 29 22:46:23.991911 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.981382 2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 29 22:46:23.991911 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.981385 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 29 22:46:23.991911 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.981388 2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 29 22:46:23.991911 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.981391 2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 29 22:46:23.991911 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.981393 2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 29 22:46:23.991911 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.981396 2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 29 22:46:23.991911 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.981398 2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 29 22:46:23.991911 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.981401 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 29 22:46:23.991911 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.981403 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 29 22:46:23.991911 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.981406 2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 29 22:46:23.991911 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.981410 2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 29 22:46:23.991911 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.981413 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 29 22:46:23.991911 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.981416 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 29 22:46:23.991911 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.981419 2562 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 29 22:46:23.991911 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.981422 2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 29 22:46:23.991911 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.981426 2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 29 22:46:23.991911 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.981429 2562 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 29 22:46:23.992394 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.981432 2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 29 22:46:23.992394 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.981436 2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 29 22:46:23.992394 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:23.981895 2562 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 29 22:46:23.992394 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:23.988423 2562 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Jul 29 22:46:23.992394 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:23.988440 2562 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 29 22:46:23.992394 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:23.988467 2562 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 29 22:46:23.992394 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:23.988473 2562 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 29 22:46:23.992394 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.988533 2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 29 22:46:23.992394 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.988539 2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 29 22:46:23.992394 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.988542 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 29 22:46:23.992394 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.988546 2562 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 29 22:46:23.992394 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.988549 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 29 22:46:23.992394 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.988552 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 29 22:46:23.992394 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.988555 2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 29 22:46:23.992394 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.988559 2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 29 22:46:23.992771 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.988562 2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 29 22:46:23.992771 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.988565 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 29 22:46:23.992771 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.988569 2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 29 22:46:23.992771 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.988573 2562 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 29 22:46:23.992771 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.988576 2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 29 22:46:23.992771 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.988579 2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 29 22:46:23.992771 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.988582 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 29 22:46:23.992771 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.988585 2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 29 22:46:23.992771 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.988588 2562 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 29 22:46:23.992771 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.988591 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 29 22:46:23.992771 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.988594 2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 29 22:46:23.992771 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.988597 2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 29 22:46:23.992771 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.988599 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 29 22:46:23.992771 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.988602 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 29 22:46:23.992771 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.988605 2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 29 22:46:23.992771 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.988608 2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 29 22:46:23.992771 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.988611 2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 29 22:46:23.992771 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.988614 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 29 22:46:23.992771 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.988617 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 29 22:46:23.993259 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.988619 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 29 22:46:23.993259 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.988622 2562 feature_gate.go:328] unrecognized feature gate: Example Jul 29 22:46:23.993259 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.988626 2562 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 29 22:46:23.993259 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.988629 2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 29 22:46:23.993259 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.988632 2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 29 22:46:23.993259 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.988635 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 29 22:46:23.993259 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.988638 2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 29 22:46:23.993259 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.988640 2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 29 22:46:23.993259 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.988643 2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 29 22:46:23.993259 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.988646 2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 29 22:46:23.993259 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.988649 2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 29 22:46:23.993259 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.988651 2562 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 29 22:46:23.993259 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.988654 2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 29 22:46:23.993259 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.988657 2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 29 22:46:23.993259 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.988659 2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 29 22:46:23.993259 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.988662 2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 29 22:46:23.993259 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.988665 2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 29 22:46:23.993259 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.988667 2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 29 22:46:23.993259 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.988670 2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 29 22:46:23.993259 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.988672 2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 29 22:46:23.993759 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.988675 2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 29 22:46:23.993759 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.988678 2562 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 29 22:46:23.993759 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.988680 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 29 22:46:23.993759 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.988682 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 29 22:46:23.993759 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.988685 2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 29 22:46:23.993759 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.988688 2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 29 22:46:23.993759 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.988690 2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 29 22:46:23.993759 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.988693 2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 29 22:46:23.993759 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.988696 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 29 22:46:23.993759 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.988700 2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 29 22:46:23.993759 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.988703 2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 29 22:46:23.993759 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.988706 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 29 22:46:23.993759 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.988708 2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 29 22:46:23.993759 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.988711 2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 29 22:46:23.993759 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.988714 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 29 22:46:23.993759 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.988717 2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 29 22:46:23.993759 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.988721 2562 feature_gate.go:328] unrecognized feature gate: Example2 Jul 29 22:46:23.993759 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.988723 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 29 22:46:23.993759 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.988726 2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 29 22:46:23.993759 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.988728 2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 29 22:46:23.994253 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.988731 2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 29 22:46:23.994253 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.988734 2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 29 22:46:23.994253 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.988736 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 29 22:46:23.994253 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.988738 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 29 22:46:23.994253 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.988741 2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 29 22:46:23.994253 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.988744 2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 29 22:46:23.994253 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.988746 2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 29 22:46:23.994253 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.988749 2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 29 22:46:23.994253 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.988752 2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 29 22:46:23.994253 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.988754 2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 29 22:46:23.994253 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.988757 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 29 22:46:23.994253 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.988759 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 29 22:46:23.994253 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.988762 2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 29 22:46:23.994253 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.988765 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 29 22:46:23.994253 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.988767 2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 29 22:46:23.994253 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.988770 2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 29 22:46:23.994253 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.988774 2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 29 22:46:23.994253 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.988777 2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 29 22:46:23.994253 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.988780 2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 29 22:46:23.994757 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.988783 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 29 22:46:23.994757 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.988786 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 29 22:46:23.994757 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.988789 2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 29 22:46:23.994757 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.988791 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 29 22:46:23.994757 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.988795 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 29 22:46:23.994757 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.988797 2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 29 22:46:23.994757 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.988800 2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 29 22:46:23.994757 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.988804 2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 29 22:46:23.994757 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.988807 2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 29 22:46:23.994757 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.988810 2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 29 22:46:23.994757 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.988813 2562 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 29 22:46:23.994757 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.988816 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 29 22:46:23.994757 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.988819 2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 29 22:46:23.994757 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:23.988839 2562 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 29 22:46:23.994757 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.988952 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 29 22:46:23.995148 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.988957 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 29 22:46:23.995148 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.988960 2562 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 29 22:46:23.995148 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.988963 2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 29 22:46:23.995148 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.988966 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 29 22:46:23.995148 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.988969 2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 29 22:46:23.995148 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.988972 2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 29 22:46:23.995148 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.988974 2562 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 29 22:46:23.995148 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.988977 2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 29 22:46:23.995148 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.988979 2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 29 22:46:23.995148 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.988982 2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 29 22:46:23.995148 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.988984 2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 29 22:46:23.995148 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.988987 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 29 22:46:23.995148 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.988989 2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 29 22:46:23.995148 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.988992 2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 29 22:46:23.995148 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.988994 2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 29 22:46:23.995148 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.988996 2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 29 22:46:23.995148 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.988999 2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 29 22:46:23.995148 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.989001 2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 29 22:46:23.995148 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.989004 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 29 22:46:23.995148 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.989007 2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 29 22:46:23.995642 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.989011 2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 29 22:46:23.995642 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.989014 2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 29 22:46:23.995642 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.989017 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 29 22:46:23.995642 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.989020 2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 29 22:46:23.995642 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.989022 2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 29 22:46:23.995642 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.989025 2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 29 22:46:23.995642 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.989027 2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 29 22:46:23.995642 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.989030 2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 29 22:46:23.995642 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.989033 2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 29 22:46:23.995642 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.989035 2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 29 22:46:23.995642 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.989038 2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 29 22:46:23.995642 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.989040 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 29 22:46:23.995642 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.989043 2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 29 22:46:23.995642 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.989045 2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 29 22:46:23.995642 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.989048 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 29 22:46:23.995642 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.989050 2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 29 22:46:23.995642 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.989053 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 29 22:46:23.995642 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.989055 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 29 22:46:23.995642 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.989058 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 29 22:46:23.996157 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.989060 2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 29 22:46:23.996157 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.989063 2562 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 29 22:46:23.996157 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.989066 2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 29 22:46:23.996157 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.989070 2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 29 22:46:23.996157 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.989073 2562 feature_gate.go:328] unrecognized feature gate: Example Jul 29 22:46:23.996157 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.989076 2562 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 29 22:46:23.996157 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.989078 2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 29 22:46:23.996157 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.989081 2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 29 22:46:23.996157 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.989083 2562 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 29 22:46:23.996157 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.989085 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 29 22:46:23.996157 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.989088 2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 29 22:46:23.996157 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.989091 2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 29 22:46:23.996157 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.989093 2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 29 22:46:23.996157 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.989096 2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 29 22:46:23.996157 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.989099 2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 29 22:46:23.996157 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.989101 2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 29 22:46:23.996157 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.989104 2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 29 22:46:23.996157 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.989107 2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 29 22:46:23.996157 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.989110 2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 29 22:46:23.996632 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.989113 2562 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 29 22:46:23.996632 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.989115 2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 29 22:46:23.996632 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.989118 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 29 22:46:23.996632 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.989120 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 29 22:46:23.996632 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.989123 2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 29 22:46:23.996632 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.989125 2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 29 22:46:23.996632 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.989128 2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 29 22:46:23.996632 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.989130 2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 29 22:46:23.996632 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.989133 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 29 22:46:23.996632 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.989135 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 29 22:46:23.996632 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.989138 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 29 22:46:23.996632 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.989140 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 29 22:46:23.996632 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.989143 2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 29 22:46:23.996632 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.989146 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 29 22:46:23.996632 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.989148 2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 29 22:46:23.996632 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.989151 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 29 22:46:23.996632 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.989153 2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 29 22:46:23.996632 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.989156 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 29 22:46:23.996632 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.989159 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 29 22:46:23.996632 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.989161 2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 29 22:46:23.997133 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.989164 2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 29 22:46:23.997133 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.989166 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 29 22:46:23.997133 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.989168 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 29 22:46:23.997133 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.989171 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 29 22:46:23.997133 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.989173 2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 29 22:46:23.997133 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.989176 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 29 22:46:23.997133 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.989179 2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 29 22:46:23.997133 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.989181 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 29 22:46:23.997133 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.989184 2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 29 22:46:23.997133 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.989187 2562 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 29 22:46:23.997133 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.989190 2562 feature_gate.go:328] unrecognized feature gate: Example2 Jul 29 22:46:23.997133 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.989192 2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 29 22:46:23.997133 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.989195 2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 29 22:46:23.997133 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.989198 2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 29 22:46:23.997133 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.989201 2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 29 22:46:23.997133 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.989203 2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 29 22:46:23.997133 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.989206 2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 29 22:46:23.997133 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.989208 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 29 22:46:23.997133 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.989212 2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 29 22:46:23.997592 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:23.989215 2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 29 22:46:23.997592 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:23.989221 2562 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 29 22:46:23.997592 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:23.989336 2562 server.go:962] "Client rotation is on, will bootstrap in background" Jul 29 22:46:23.997592 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:23.991288 2562 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 29 22:46:23.997592 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:23.992816 2562 server.go:1019] "Starting client certificate rotation" Jul 29 22:46:23.997592 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:23.992930 2562 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 29 22:46:23.997592 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:23.993720 2562 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 29 22:46:23.997592 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:23.995636 2562 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 29 22:46:24.001433 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:23.998131 2562 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 29 22:46:24.009629 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:24.009611 2562 log.go:25] "Validated CRI v1 runtime API" Jul 29 22:46:24.013775 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:24.013757 2562 log.go:25] "Validated CRI v1 image API" Jul 29 22:46:24.015303 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:24.015289 2562 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 29 22:46:24.017808 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:24.017789 2562 fs.go:135] Filesystem UUIDs: map[1efb0a85-2849-4dcb-85dd-02080a9a090b:/dev/nvme0n1p3 40fc0263-700e-4ec4-8946-13c73d2a8f81:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2] Jul 29 22:46:24.017872 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:24.017808 2562 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 29 22:46:24.023091 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:24.022931 2562 manager.go:216] Machine: {Timestamp:2026-07-29 22:46:24.02176456 +0000 UTC m=+0.314161500 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3099278 MemoryCapacity:33164496896 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec28efc9f3eff2637e8f29ddfefc2b49 SystemUUID:ec28efc9-f3ef-f263-7e8f-29ddfefc2b49 BootID:71fdc0d2-078e-4b42-97ba-4ccb6d3f0fa5 Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:82:cd:1d:df:59 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:82:cd:1d:df:59 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:16:0a:e7:76:0f:6c Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164496896 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 29 22:46:24.023091 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:24.023082 2562 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 29 22:46:24.023203 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:24.023168 2562 manager.go:232] Version: {KernelVersion:5.14.0-570.129.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260717-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 29 22:46:24.026253 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:24.026228 2562 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 29 22:46:24.026402 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:24.026256 2562 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-138-112.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 29 22:46:24.026708 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:24.026692 2562 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 29 22:46:24.027850 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:24.027837 2562 topology_manager.go:138] "Creating topology manager with none policy" Jul 29 22:46:24.027888 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:24.027854 2562 container_manager_linux.go:309] "Creating device plugin manager" Jul 29 22:46:24.027888 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:24.027862 2562 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 29 22:46:24.028517 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:24.028505 2562 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 29 22:46:24.029887 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:24.029877 2562 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 29 22:46:24.032767 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:24.032755 2562 state_mem.go:36] "Initialized new in-memory state store" Jul 29 22:46:24.032906 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:24.032896 2562 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 29 22:46:24.032940 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:24.032926 2562 kubelet.go:486] "Attempting to sync node with API server" Jul 29 22:46:24.032940 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:24.032935 2562 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 29 22:46:24.032996 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:24.032953 2562 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 29 22:46:24.032996 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:24.032964 2562 kubelet.go:398] "Adding apiserver pod source" Jul 29 22:46:24.032996 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:24.032972 2562 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 29 22:46:24.033343 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:24.033331 2562 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 29 22:46:24.033391 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:24.033349 2562 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 29 22:46:24.036755 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:24.036735 2562 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1" Jul 29 22:46:24.038256 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:24.038243 2562 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 29 22:46:24.038312 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:24.038272 2562 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 29 22:46:24.038846 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:24.038819 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 29 22:46:24.038884 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:24.038856 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 29 22:46:24.038884 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:24.038863 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 29 22:46:24.038884 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:24.038868 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 29 22:46:24.038884 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:24.038876 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 29 22:46:24.038884 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:24.038882 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 29 22:46:24.039026 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:24.039001 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 29 22:46:24.039026 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:24.039007 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 29 22:46:24.039026 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:24.039014 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 29 22:46:24.039026 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:24.039019 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 29 22:46:24.039026 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:24.039027 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 29 22:46:24.039162 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:24.039035 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 29 22:46:24.039872 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:24.039860 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 29 22:46:24.039911 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:24.039872 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 29 22:46:24.042512 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:24.042495 2562 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-138-112.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 29 22:46:24.043072 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:24.043059 2562 server.go:1268] "Started kubelet" Jul 29 22:46:24.043129 ip-10-0-138-112 kubenswrapper[2562]: E0729 22:46:24.043084 2562 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-138-112.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 29 22:46:24.043129 ip-10-0-138-112 kubenswrapper[2562]: E0729 22:46:24.043099 2562 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 29 22:46:24.043222 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:24.043126 2562 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 29 22:46:24.043222 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:24.043142 2562 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 29 22:46:24.043306 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:24.043243 2562 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 29 22:46:24.043979 ip-10-0-138-112 systemd[1]: Started Kubernetes Kubelet. Jul 29 22:46:24.048390 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:24.048357 2562 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 29 22:46:24.049568 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:24.049547 2562 server.go:310] "Adding debug handlers to kubelet server" Jul 29 22:46:24.050196 ip-10-0-138-112 kubenswrapper[2562]: E0729 22:46:24.050175 2562 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 29 22:46:24.050700 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:24.050681 2562 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-tbbts" Jul 29 22:46:24.051621 ip-10-0-138-112 kubenswrapper[2562]: E0729 22:46:24.050676 2562 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-138-112.ec2.internal.18c6e42cf53c2163 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-138-112.ec2.internal,UID:ip-10-0-138-112.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-138-112.ec2.internal,},FirstTimestamp:2026-07-29 22:46:24.043032931 +0000 UTC m=+0.335429870,LastTimestamp:2026-07-29 22:46:24.043032931 +0000 UTC m=+0.335429870,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-138-112.ec2.internal,}" Jul 29 22:46:24.054485 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:24.054468 2562 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 29 22:46:24.054567 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:24.054483 2562 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 29 22:46:24.054567 ip-10-0-138-112 kubenswrapper[2562]: E0729 22:46:24.054536 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-112.ec2.internal\" not found" Jul 29 22:46:24.055013 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:24.054982 2562 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 29 22:46:24.055188 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:24.055065 2562 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 29 22:46:24.055188 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:24.055089 2562 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 29 22:46:24.055317 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:24.055223 2562 reconstruct.go:97] "Volume reconstruction finished" Jul 29 22:46:24.055317 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:24.055233 2562 reconciler.go:29] "Reconciler: start to sync state" Jul 29 22:46:24.055456 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:24.055434 2562 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 29 22:46:24.055456 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:24.055456 2562 factory.go:55] Registering systemd factory Jul 29 22:46:24.055606 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:24.055465 2562 factory.go:223] Registration of the systemd container factory successfully Jul 29 22:46:24.055688 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:24.055674 2562 factory.go:153] Registering CRI-O factory Jul 29 22:46:24.055726 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:24.055694 2562 factory.go:223] Registration of the crio container factory successfully Jul 29 22:46:24.055726 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:24.055715 2562 factory.go:103] Registering Raw factory Jul 29 22:46:24.055794 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:24.055728 2562 manager.go:1196] Started watching for new ooms in manager Jul 29 22:46:24.056234 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:24.056222 2562 manager.go:349] Starting recovery of all containers Jul 29 22:46:24.059444 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:24.059423 2562 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-tbbts" Jul 29 22:46:24.064076 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:24.064056 2562 manager.go:354] Recovery completed Jul 29 22:46:24.066300 ip-10-0-138-112 kubenswrapper[2562]: E0729 22:46:24.066272 2562 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 29 22:46:24.066300 ip-10-0-138-112 kubenswrapper[2562]: E0729 22:46:24.066285 2562 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-138-112.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 29 22:46:24.069062 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:24.069050 2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 29 22:46:24.071678 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:24.071659 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-112.ec2.internal" event="NodeHasSufficientMemory" Jul 29 22:46:24.071737 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:24.071699 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-112.ec2.internal" event="NodeHasNoDiskPressure" Jul 29 22:46:24.071737 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:24.071714 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-112.ec2.internal" event="NodeHasSufficientPID" Jul 29 22:46:24.072305 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:24.072289 2562 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 29 22:46:24.072305 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:24.072302 2562 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 29 22:46:24.072431 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:24.072304 2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 29 22:46:24.072431 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:24.072321 2562 state_mem.go:36] "Initialized new in-memory state store" Jul 29 22:46:24.073149 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:24.073132 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-112.ec2.internal" event="NodeHasSufficientMemory" Jul 29 22:46:24.073204 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:24.073165 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-112.ec2.internal" event="NodeHasNoDiskPressure" Jul 29 22:46:24.073204 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:24.073175 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-112.ec2.internal" event="NodeHasSufficientPID" Jul 29 22:46:24.073503 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:24.073492 2562 policy_none.go:49] "None policy: Start" Jul 29 22:46:24.073554 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:24.073508 2562 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 29 22:46:24.073554 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:24.073519 2562 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 29 22:46:24.074603 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:24.074591 2562 policy_none.go:47] "Start" Jul 29 22:46:24.103579 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:24.103557 2562 manager.go:337] "Starting Device Plugin manager" Jul 29 22:46:24.103701 ip-10-0-138-112 kubenswrapper[2562]: E0729 22:46:24.103596 2562 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 29 22:46:24.103701 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:24.103609 2562 server.go:87] "Starting device plugin registration server" Jul 29 22:46:24.103904 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:24.103892 2562 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 29 22:46:24.103985 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:24.103905 2562 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 29 22:46:24.104156 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:24.104133 2562 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 29 22:46:24.104490 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:24.104220 2562 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 29 22:46:24.104490 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:24.104230 2562 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 29 22:46:24.106452 ip-10-0-138-112 kubenswrapper[2562]: E0729 22:46:24.104783 2562 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 29 22:46:24.106452 ip-10-0-138-112 kubenswrapper[2562]: E0729 22:46:24.104841 2562 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-138-112.ec2.internal\" not found" Jul 29 22:46:24.187758 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:24.187721 2562 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 29 22:46:24.188964 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:24.188947 2562 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 29 22:46:24.189027 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:24.188988 2562 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 29 22:46:24.189027 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:24.189015 2562 kubelet.go:2448] "Starting kubelet main sync loop" Jul 29 22:46:24.189114 ip-10-0-138-112 kubenswrapper[2562]: E0729 22:46:24.189051 2562 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 29 22:46:24.192016 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:24.191991 2562 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 29 22:46:24.205114 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:24.205084 2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 29 22:46:24.206222 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:24.206196 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-112.ec2.internal" event="NodeHasSufficientMemory" Jul 29 22:46:24.206325 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:24.206236 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-112.ec2.internal" event="NodeHasNoDiskPressure" Jul 29 22:46:24.206325 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:24.206250 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-112.ec2.internal" event="NodeHasSufficientPID" Jul 29 22:46:24.206325 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:24.206277 2562 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-138-112.ec2.internal" Jul 29 22:46:24.214766 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:24.214745 2562 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-138-112.ec2.internal" Jul 29 22:46:24.214857 ip-10-0-138-112 kubenswrapper[2562]: E0729 22:46:24.214774 2562 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-138-112.ec2.internal\": node \"ip-10-0-138-112.ec2.internal\" not found" Jul 29 22:46:24.229254 ip-10-0-138-112 kubenswrapper[2562]: E0729 22:46:24.229185 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-112.ec2.internal\" not found" Jul 29 22:46:24.290217 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:24.290043 2562 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-112.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-138-112.ec2.internal"] Jul 29 22:46:24.290348 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:24.290256 2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 29 22:46:24.291861 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:24.291845 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-112.ec2.internal" event="NodeHasSufficientMemory" Jul 29 22:46:24.291932 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:24.291876 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-112.ec2.internal" event="NodeHasNoDiskPressure" Jul 29 22:46:24.291932 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:24.291890 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-112.ec2.internal" event="NodeHasSufficientPID" Jul 29 22:46:24.293286 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:24.293274 2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 29 22:46:24.293466 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:24.293453 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-112.ec2.internal" Jul 29 22:46:24.293503 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:24.293492 2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 29 22:46:24.294931 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:24.294908 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-112.ec2.internal" event="NodeHasSufficientMemory" Jul 29 22:46:24.295007 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:24.294934 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-112.ec2.internal" event="NodeHasNoDiskPressure" Jul 29 22:46:24.295007 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:24.294944 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-112.ec2.internal" event="NodeHasSufficientPID" Jul 29 22:46:24.295007 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:24.294909 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-112.ec2.internal" event="NodeHasSufficientMemory" Jul 29 22:46:24.295094 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:24.295010 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-112.ec2.internal" event="NodeHasNoDiskPressure" Jul 29 22:46:24.295094 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:24.295025 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-112.ec2.internal" event="NodeHasSufficientPID" Jul 29 22:46:24.296242 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:24.296227 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-112.ec2.internal" Jul 29 22:46:24.296294 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:24.296253 2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 29 22:46:24.297822 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:24.297800 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-112.ec2.internal" event="NodeHasSufficientMemory" Jul 29 22:46:24.297931 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:24.297847 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-112.ec2.internal" event="NodeHasNoDiskPressure" Jul 29 22:46:24.297931 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:24.297860 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-112.ec2.internal" event="NodeHasSufficientPID" Jul 29 22:46:24.322473 ip-10-0-138-112 kubenswrapper[2562]: E0729 22:46:24.322451 2562 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-138-112.ec2.internal\" not found" node="ip-10-0-138-112.ec2.internal" Jul 29 22:46:24.326751 ip-10-0-138-112 kubenswrapper[2562]: E0729 22:46:24.326735 2562 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-138-112.ec2.internal\" not found" node="ip-10-0-138-112.ec2.internal" Jul 29 22:46:24.329779 ip-10-0-138-112 kubenswrapper[2562]: E0729 22:46:24.329766 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-112.ec2.internal\" not found" Jul 29 22:46:24.356921 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:24.356899 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b8710411dd4204d4793d631cc5fc044b-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-138-112.ec2.internal\" (UID: \"b8710411dd4204d4793d631cc5fc044b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-112.ec2.internal" Jul 29 22:46:24.357002 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:24.356926 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/5376269631e55085670dee0434ce97e1-config\") pod \"kube-apiserver-proxy-ip-10-0-138-112.ec2.internal\" (UID: \"5376269631e55085670dee0434ce97e1\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-138-112.ec2.internal" Jul 29 22:46:24.357002 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:24.356943 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/b8710411dd4204d4793d631cc5fc044b-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-138-112.ec2.internal\" (UID: \"b8710411dd4204d4793d631cc5fc044b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-112.ec2.internal" Jul 29 22:46:24.430363 ip-10-0-138-112 kubenswrapper[2562]: E0729 22:46:24.430332 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-112.ec2.internal\" not found" Jul 29 22:46:24.457382 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:24.457350 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/b8710411dd4204d4793d631cc5fc044b-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-138-112.ec2.internal\" (UID: \"b8710411dd4204d4793d631cc5fc044b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-112.ec2.internal" Jul 29 22:46:24.457484 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:24.457387 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b8710411dd4204d4793d631cc5fc044b-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-138-112.ec2.internal\" (UID: \"b8710411dd4204d4793d631cc5fc044b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-112.ec2.internal" Jul 29 22:46:24.457484 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:24.457403 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/5376269631e55085670dee0434ce97e1-config\") pod \"kube-apiserver-proxy-ip-10-0-138-112.ec2.internal\" (UID: \"5376269631e55085670dee0434ce97e1\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-138-112.ec2.internal" Jul 29 22:46:24.457484 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:24.457454 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b8710411dd4204d4793d631cc5fc044b-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-138-112.ec2.internal\" (UID: \"b8710411dd4204d4793d631cc5fc044b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-112.ec2.internal" Jul 29 22:46:24.457484 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:24.457466 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/b8710411dd4204d4793d631cc5fc044b-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-138-112.ec2.internal\" (UID: \"b8710411dd4204d4793d631cc5fc044b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-112.ec2.internal" Jul 29 22:46:24.457623 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:24.457521 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/5376269631e55085670dee0434ce97e1-config\") pod \"kube-apiserver-proxy-ip-10-0-138-112.ec2.internal\" (UID: \"5376269631e55085670dee0434ce97e1\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-138-112.ec2.internal" Jul 29 22:46:24.530924 ip-10-0-138-112 kubenswrapper[2562]: E0729 22:46:24.530816 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-112.ec2.internal\" not found" Jul 29 22:46:24.625503 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:24.625467 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-112.ec2.internal" Jul 29 22:46:24.630743 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:24.630724 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-112.ec2.internal" Jul 29 22:46:24.631794 ip-10-0-138-112 kubenswrapper[2562]: E0729 22:46:24.631778 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-112.ec2.internal\" not found" Jul 29 22:46:24.732415 ip-10-0-138-112 kubenswrapper[2562]: E0729 22:46:24.732379 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-112.ec2.internal\" not found" Jul 29 22:46:24.832942 ip-10-0-138-112 kubenswrapper[2562]: E0729 22:46:24.832864 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-112.ec2.internal\" not found" Jul 29 22:46:24.933487 ip-10-0-138-112 kubenswrapper[2562]: E0729 22:46:24.933459 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-112.ec2.internal\" not found" Jul 29 22:46:24.991973 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:24.991937 2562 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 29 22:46:24.992431 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:24.992094 2562 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 29 22:46:25.034404 ip-10-0-138-112 kubenswrapper[2562]: E0729 22:46:25.034365 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-112.ec2.internal\" not found" Jul 29 22:46:25.063929 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:25.063884 2562 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-28 22:41:25 +0000 UTC" deadline="2027-12-29 00:11:28.818364428 +0000 UTC" Jul 29 22:46:25.063929 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:25.063914 2562 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12409h25m3.754453705s" Jul 29 22:46:25.097588 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:25.097500 2562 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 29 22:46:26.515411 ip-10-0-138-112 kubenswrapper[2562]: E0729 22:46:26.515375 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-112.ec2.internal\" not found" Jul 29 22:46:26.545615 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:26.545581 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5376269631e55085670dee0434ce97e1.slice/crio-a970f7544b4748c19788609562d9d9524837b50a4228e2b2542baf932db1d0c7 WatchSource:0}: Error finding container a970f7544b4748c19788609562d9d9524837b50a4228e2b2542baf932db1d0c7: Status 404 returned error can't find the container with id a970f7544b4748c19788609562d9d9524837b50a4228e2b2542baf932db1d0c7 Jul 29 22:46:26.545981 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:26.545962 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb8710411dd4204d4793d631cc5fc044b.slice/crio-7ed2ebd236a11eb52114f5fb7fb1eb33322706e47bdf1e5947f51325cff56972 WatchSource:0}: Error finding container 7ed2ebd236a11eb52114f5fb7fb1eb33322706e47bdf1e5947f51325cff56972: Status 404 returned error can't find the container with id 7ed2ebd236a11eb52114f5fb7fb1eb33322706e47bdf1e5947f51325cff56972 Jul 29 22:46:26.550728 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:26.550711 2562 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 29 22:46:26.573152 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:26.573098 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-112.ec2.internal" event={"ID":"b8710411dd4204d4793d631cc5fc044b","Type":"ContainerStarted","Data":"7ed2ebd236a11eb52114f5fb7fb1eb33322706e47bdf1e5947f51325cff56972"} Jul 29 22:46:26.574075 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:26.574055 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-112.ec2.internal" event={"ID":"5376269631e55085670dee0434ce97e1","Type":"ContainerStarted","Data":"a970f7544b4748c19788609562d9d9524837b50a4228e2b2542baf932db1d0c7"} Jul 29 22:46:26.616258 ip-10-0-138-112 kubenswrapper[2562]: E0729 22:46:26.616225 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-112.ec2.internal\" not found" Jul 29 22:46:26.716843 ip-10-0-138-112 kubenswrapper[2562]: E0729 22:46:26.716807 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-112.ec2.internal\" not found" Jul 29 22:46:26.765446 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:26.765364 2562 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 29 22:46:26.836090 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:26.836053 2562 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-112.ec2.internal" Jul 29 22:46:26.847745 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:26.847714 2562 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 29 22:46:26.849604 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:26.849590 2562 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-112.ec2.internal" Jul 29 22:46:26.860983 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:26.860964 2562 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 29 22:46:26.967184 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:26.967158 2562 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 29 22:46:27.209656 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:27.209576 2562 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 29 22:46:27.414610 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:27.414573 2562 apiserver.go:52] "Watching apiserver" Jul 29 22:46:27.424584 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:27.423744 2562 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 29 22:46:27.424584 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:27.424178 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/konnectivity-agent-k9nmc","openshift-dns/node-resolver-k8fl9","openshift-image-registry/node-ca-vwtj2","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-112.ec2.internal","openshift-multus/multus-2mwmf","openshift-multus/multus-additional-cni-plugins-h6flc","openshift-ovn-kubernetes/ovnkube-node-9g28r","kube-system/kube-apiserver-proxy-ip-10-0-138-112.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kqh77","openshift-cluster-node-tuning-operator/tuned-84tng","openshift-multus/network-metrics-daemon-fxb52","openshift-network-diagnostics/network-check-target-z92kj","openshift-network-operator/iptables-alerter-frd9g","kube-system/global-pull-secret-syncer-nkzq5"] Jul 29 22:46:27.425854 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:27.425826 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-84tng" Jul 29 22:46:27.427336 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:27.427314 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-frd9g" Jul 29 22:46:27.428518 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:27.428495 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-vwtj2" Jul 29 22:46:27.428890 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:27.428872 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-pnqj8\"" Jul 29 22:46:27.429131 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:27.429110 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 29 22:46:27.429359 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:27.429340 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 29 22:46:27.429985 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:27.429962 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-txbb4\"" Jul 29 22:46:27.430176 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:27.430159 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 29 22:46:27.430467 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:27.430448 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 29 22:46:27.430651 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:27.430636 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 29 22:46:27.430841 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:27.430827 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nkzq5" Jul 29 22:46:27.430913 ip-10-0-138-112 kubenswrapper[2562]: E0729 22:46:27.430891 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nkzq5" podUID="916b3077-78fa-4054-921c-2e7124dc35f8" Jul 29 22:46:27.431018 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:27.431000 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 29 22:46:27.431904 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:27.431661 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-2kdtk\"" Jul 29 22:46:27.431904 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:27.431687 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 29 22:46:27.431904 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:27.431738 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 29 22:46:27.431904 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:27.431891 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-2mwmf" Jul 29 22:46:27.433052 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:27.433032 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-h6flc" Jul 29 22:46:27.434389 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:27.434371 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 29 22:46:27.434792 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:27.434578 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-vmtsk\"" Jul 29 22:46:27.434792 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:27.434635 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kqh77" Jul 29 22:46:27.434792 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:27.434775 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 29 22:46:27.435072 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:27.435034 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 29 22:46:27.435250 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:27.435174 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 29 22:46:27.435321 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:27.435300 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-7p96z\"" Jul 29 22:46:27.435390 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:27.435373 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 29 22:46:27.435505 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:27.435489 2562 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 29 22:46:27.435850 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:27.435607 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 29 22:46:27.438151 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:27.437265 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-9g28r" Jul 29 22:46:27.438151 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:27.437356 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-4tcc7\"" Jul 29 22:46:27.438151 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:27.437675 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 29 22:46:27.438151 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:27.437894 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 29 22:46:27.438151 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:27.438025 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 29 22:46:27.439322 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:27.439303 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-k8fl9" Jul 29 22:46:27.440444 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:27.439977 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 29 22:46:27.440444 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:27.440134 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 29 22:46:27.440444 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:27.440170 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 29 22:46:27.440444 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:27.440220 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-bdm6x\"" Jul 29 22:46:27.440859 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:27.440840 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 29 22:46:27.441045 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:27.441007 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-k9nmc" Jul 29 22:46:27.441045 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:27.440847 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 29 22:46:27.441299 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:27.441052 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 29 22:46:27.441952 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:27.441933 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 29 22:46:27.442046 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:27.441974 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-9hnns\"" Jul 29 22:46:27.442422 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:27.442244 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 29 22:46:27.443189 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:27.442811 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fxb52" Jul 29 22:46:27.443189 ip-10-0-138-112 kubenswrapper[2562]: E0729 22:46:27.442876 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fxb52" podUID="f0a6baed-8bea-4992-8852-0b84808a78d1" Jul 29 22:46:27.444361 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:27.444336 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z92kj" Jul 29 22:46:27.444488 ip-10-0-138-112 kubenswrapper[2562]: E0729 22:46:27.444446 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z92kj" podUID="2771bdbc-d054-464b-a01f-82344308451f" Jul 29 22:46:27.447206 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:27.444689 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 29 22:46:27.447206 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:27.444745 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 29 22:46:27.447206 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:27.445598 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-hjj64\"" Jul 29 22:46:27.447569 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:27.447528 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/6a64a07e-1c4a-4990-90b9-80623be37f60-hostroot\") pod \"multus-2mwmf\" (UID: \"6a64a07e-1c4a-4990-90b9-80623be37f60\") " pod="openshift-multus/multus-2mwmf" Jul 29 22:46:27.447673 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:27.447592 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6a64a07e-1c4a-4990-90b9-80623be37f60-etc-kubernetes\") pod \"multus-2mwmf\" (UID: \"6a64a07e-1c4a-4990-90b9-80623be37f60\") " pod="openshift-multus/multus-2mwmf" Jul 29 22:46:27.447673 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:27.447626 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/d40a4cdf-f141-46d3-8159-986f680b2dec-cni-binary-copy\") pod \"multus-additional-cni-plugins-h6flc\" (UID: \"d40a4cdf-f141-46d3-8159-986f680b2dec\") " pod="openshift-multus/multus-additional-cni-plugins-h6flc" Jul 29 22:46:27.448006 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:27.447658 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cd64b\" (UniqueName: \"kubernetes.io/projected/d40a4cdf-f141-46d3-8159-986f680b2dec-kube-api-access-cd64b\") pod \"multus-additional-cni-plugins-h6flc\" (UID: \"d40a4cdf-f141-46d3-8159-986f680b2dec\") " pod="openshift-multus/multus-additional-cni-plugins-h6flc" Jul 29 22:46:27.448142 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:27.448033 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/6c54a7c1-2837-47d6-88f4-dc4d264397d4-etc-selinux\") pod \"aws-ebs-csi-driver-node-kqh77\" (UID: \"6c54a7c1-2837-47d6-88f4-dc4d264397d4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kqh77" Jul 29 22:46:27.448142 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:27.448060 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6050ad9f-817a-48e4-b939-12efa11a6a73-var-lib-openvswitch\") pod \"ovnkube-node-9g28r\" (UID: \"6050ad9f-817a-48e4-b939-12efa11a6a73\") " pod="openshift-ovn-kubernetes/ovnkube-node-9g28r" Jul 29 22:46:27.448142 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:27.448084 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/6050ad9f-817a-48e4-b939-12efa11a6a73-ovnkube-config\") pod \"ovnkube-node-9g28r\" (UID: \"6050ad9f-817a-48e4-b939-12efa11a6a73\") " pod="openshift-ovn-kubernetes/ovnkube-node-9g28r" Jul 29 22:46:27.448142 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:27.448107 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/4c31fa1a-fda4-43d6-84ef-e43f0bafdcb2-run\") pod \"tuned-84tng\" (UID: \"4c31fa1a-fda4-43d6-84ef-e43f0bafdcb2\") " pod="openshift-cluster-node-tuning-operator/tuned-84tng" Jul 29 22:46:27.448360 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:27.448161 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/f6d8c777-cb9a-4505-8e30-ffaa01e53239-serviceca\") pod \"node-ca-vwtj2\" (UID: \"f6d8c777-cb9a-4505-8e30-ffaa01e53239\") " pod="openshift-image-registry/node-ca-vwtj2" Jul 29 22:46:27.448360 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:27.448183 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/6a64a07e-1c4a-4990-90b9-80623be37f60-cnibin\") pod \"multus-2mwmf\" (UID: \"6a64a07e-1c4a-4990-90b9-80623be37f60\") " pod="openshift-multus/multus-2mwmf" Jul 29 22:46:27.448360 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:27.448207 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/d40a4cdf-f141-46d3-8159-986f680b2dec-tuning-conf-dir\") pod \"multus-additional-cni-plugins-h6flc\" (UID: \"d40a4cdf-f141-46d3-8159-986f680b2dec\") " pod="openshift-multus/multus-additional-cni-plugins-h6flc" Jul 29 22:46:27.448360 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:27.448256 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/6050ad9f-817a-48e4-b939-12efa11a6a73-node-log\") pod \"ovnkube-node-9g28r\" (UID: \"6050ad9f-817a-48e4-b939-12efa11a6a73\") " pod="openshift-ovn-kubernetes/ovnkube-node-9g28r" Jul 29 22:46:27.448360 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:27.448278 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/6050ad9f-817a-48e4-b939-12efa11a6a73-log-socket\") pod \"ovnkube-node-9g28r\" (UID: \"6050ad9f-817a-48e4-b939-12efa11a6a73\") " pod="openshift-ovn-kubernetes/ovnkube-node-9g28r" Jul 29 22:46:27.448360 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:27.448299 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6a64a07e-1c4a-4990-90b9-80623be37f60-multus-cni-dir\") pod \"multus-2mwmf\" (UID: \"6a64a07e-1c4a-4990-90b9-80623be37f60\") " pod="openshift-multus/multus-2mwmf" Jul 29 22:46:27.448360 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:27.448322 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/6a64a07e-1c4a-4990-90b9-80623be37f60-os-release\") pod \"multus-2mwmf\" (UID: \"6a64a07e-1c4a-4990-90b9-80623be37f60\") " pod="openshift-multus/multus-2mwmf" Jul 29 22:46:27.448360 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:27.448345 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/6a64a07e-1c4a-4990-90b9-80623be37f60-host-var-lib-cni-bin\") pod \"multus-2mwmf\" (UID: \"6a64a07e-1c4a-4990-90b9-80623be37f60\") " pod="openshift-multus/multus-2mwmf" Jul 29 22:46:27.448777 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:27.448353 2562 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 29 22:46:27.448777 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:27.448415 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d40a4cdf-f141-46d3-8159-986f680b2dec-system-cni-dir\") pod \"multus-additional-cni-plugins-h6flc\" (UID: \"d40a4cdf-f141-46d3-8159-986f680b2dec\") " pod="openshift-multus/multus-additional-cni-plugins-h6flc" Jul 29 22:46:27.448777 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:27.448458 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4c31fa1a-fda4-43d6-84ef-e43f0bafdcb2-sys\") pod \"tuned-84tng\" (UID: \"4c31fa1a-fda4-43d6-84ef-e43f0bafdcb2\") " pod="openshift-cluster-node-tuning-operator/tuned-84tng" Jul 29 22:46:27.448777 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:27.448495 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/f6d8c777-cb9a-4505-8e30-ffaa01e53239-host\") pod \"node-ca-vwtj2\" (UID: \"f6d8c777-cb9a-4505-8e30-ffaa01e53239\") " pod="openshift-image-registry/node-ca-vwtj2" Jul 29 22:46:27.448777 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:27.448542 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/6a64a07e-1c4a-4990-90b9-80623be37f60-multus-socket-dir-parent\") pod \"multus-2mwmf\" (UID: \"6a64a07e-1c4a-4990-90b9-80623be37f60\") " pod="openshift-multus/multus-2mwmf" Jul 29 22:46:27.448777 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:27.448599 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/6c54a7c1-2837-47d6-88f4-dc4d264397d4-socket-dir\") pod \"aws-ebs-csi-driver-node-kqh77\" (UID: \"6c54a7c1-2837-47d6-88f4-dc4d264397d4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kqh77" Jul 29 22:46:27.448777 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:27.448636 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/916b3077-78fa-4054-921c-2e7124dc35f8-dbus\") pod \"global-pull-secret-syncer-nkzq5\" (UID: \"916b3077-78fa-4054-921c-2e7124dc35f8\") " pod="kube-system/global-pull-secret-syncer-nkzq5" Jul 29 22:46:27.448777 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:27.448659 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4c31fa1a-fda4-43d6-84ef-e43f0bafdcb2-tmp\") pod \"tuned-84tng\" (UID: \"4c31fa1a-fda4-43d6-84ef-e43f0bafdcb2\") " pod="openshift-cluster-node-tuning-operator/tuned-84tng" Jul 29 22:46:27.448777 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:27.448685 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pmz5r\" (UniqueName: \"kubernetes.io/projected/6a64a07e-1c4a-4990-90b9-80623be37f60-kube-api-access-pmz5r\") pod \"multus-2mwmf\" (UID: \"6a64a07e-1c4a-4990-90b9-80623be37f60\") " pod="openshift-multus/multus-2mwmf" Jul 29 22:46:27.448777 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:27.448723 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/6c54a7c1-2837-47d6-88f4-dc4d264397d4-kubelet-dir\") pod \"aws-ebs-csi-driver-node-kqh77\" (UID: \"6c54a7c1-2837-47d6-88f4-dc4d264397d4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kqh77" Jul 29 22:46:27.448777 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:27.448754 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/6c54a7c1-2837-47d6-88f4-dc4d264397d4-registration-dir\") pod \"aws-ebs-csi-driver-node-kqh77\" (UID: \"6c54a7c1-2837-47d6-88f4-dc4d264397d4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kqh77" Jul 29 22:46:27.448777 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:27.448781 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/6c54a7c1-2837-47d6-88f4-dc4d264397d4-device-dir\") pod \"aws-ebs-csi-driver-node-kqh77\" (UID: \"6c54a7c1-2837-47d6-88f4-dc4d264397d4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kqh77" Jul 29 22:46:27.449335 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:27.448805 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/6050ad9f-817a-48e4-b939-12efa11a6a73-host-kubelet\") pod \"ovnkube-node-9g28r\" (UID: \"6050ad9f-817a-48e4-b939-12efa11a6a73\") " pod="openshift-ovn-kubernetes/ovnkube-node-9g28r" Jul 29 22:46:27.449335 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:27.448828 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9kjfg\" (UniqueName: \"kubernetes.io/projected/f6d8c777-cb9a-4505-8e30-ffaa01e53239-kube-api-access-9kjfg\") pod \"node-ca-vwtj2\" (UID: \"f6d8c777-cb9a-4505-8e30-ffaa01e53239\") " pod="openshift-image-registry/node-ca-vwtj2" Jul 29 22:46:27.449335 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:27.448849 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/6a64a07e-1c4a-4990-90b9-80623be37f60-cni-binary-copy\") pod \"multus-2mwmf\" (UID: \"6a64a07e-1c4a-4990-90b9-80623be37f60\") " pod="openshift-multus/multus-2mwmf" Jul 29 22:46:27.449335 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:27.448872 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/5ce0a47c-e715-472b-a0d9-761a5a6777dd-host-slash\") pod \"iptables-alerter-frd9g\" (UID: \"5ce0a47c-e715-472b-a0d9-761a5a6777dd\") " pod="openshift-network-operator/iptables-alerter-frd9g" Jul 29 22:46:27.449335 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:27.448893 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/6050ad9f-817a-48e4-b939-12efa11a6a73-host-cni-netd\") pod \"ovnkube-node-9g28r\" (UID: \"6050ad9f-817a-48e4-b939-12efa11a6a73\") " pod="openshift-ovn-kubernetes/ovnkube-node-9g28r" Jul 29 22:46:27.449335 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:27.448930 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/6050ad9f-817a-48e4-b939-12efa11a6a73-env-overrides\") pod \"ovnkube-node-9g28r\" (UID: \"6050ad9f-817a-48e4-b939-12efa11a6a73\") " pod="openshift-ovn-kubernetes/ovnkube-node-9g28r" Jul 29 22:46:27.449335 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:27.448951 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/6a64a07e-1c4a-4990-90b9-80623be37f60-multus-daemon-config\") pod \"multus-2mwmf\" (UID: \"6a64a07e-1c4a-4990-90b9-80623be37f60\") " pod="openshift-multus/multus-2mwmf" Jul 29 22:46:27.449335 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:27.448972 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/d40a4cdf-f141-46d3-8159-986f680b2dec-cnibin\") pod \"multus-additional-cni-plugins-h6flc\" (UID: \"d40a4cdf-f141-46d3-8159-986f680b2dec\") " pod="openshift-multus/multus-additional-cni-plugins-h6flc" Jul 29 22:46:27.449335 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:27.448993 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/6050ad9f-817a-48e4-b939-12efa11a6a73-ovnkube-script-lib\") pod \"ovnkube-node-9g28r\" (UID: \"6050ad9f-817a-48e4-b939-12efa11a6a73\") " pod="openshift-ovn-kubernetes/ovnkube-node-9g28r" Jul 29 22:46:27.449335 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:27.449045 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/916b3077-78fa-4054-921c-2e7124dc35f8-kubelet-config\") pod \"global-pull-secret-syncer-nkzq5\" (UID: \"916b3077-78fa-4054-921c-2e7124dc35f8\") " pod="kube-system/global-pull-secret-syncer-nkzq5" Jul 29 22:46:27.449335 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:27.449082 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/4c31fa1a-fda4-43d6-84ef-e43f0bafdcb2-etc-sysctl-conf\") pod \"tuned-84tng\" (UID: \"4c31fa1a-fda4-43d6-84ef-e43f0bafdcb2\") " pod="openshift-cluster-node-tuning-operator/tuned-84tng" Jul 29 22:46:27.449335 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:27.449139 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/6050ad9f-817a-48e4-b939-12efa11a6a73-ovn-node-metrics-cert\") pod \"ovnkube-node-9g28r\" (UID: \"6050ad9f-817a-48e4-b939-12efa11a6a73\") " pod="openshift-ovn-kubernetes/ovnkube-node-9g28r" Jul 29 22:46:27.449335 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:27.449164 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h8vm9\" (UniqueName: \"kubernetes.io/projected/5ce0a47c-e715-472b-a0d9-761a5a6777dd-kube-api-access-h8vm9\") pod \"iptables-alerter-frd9g\" (UID: \"5ce0a47c-e715-472b-a0d9-761a5a6777dd\") " pod="openshift-network-operator/iptables-alerter-frd9g" Jul 29 22:46:27.449335 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:27.449187 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/6050ad9f-817a-48e4-b939-12efa11a6a73-systemd-units\") pod \"ovnkube-node-9g28r\" (UID: \"6050ad9f-817a-48e4-b939-12efa11a6a73\") " pod="openshift-ovn-kubernetes/ovnkube-node-9g28r" Jul 29 22:46:27.449335 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:27.449207 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/4c31fa1a-fda4-43d6-84ef-e43f0bafdcb2-etc-sysctl-d\") pod \"tuned-84tng\" (UID: \"4c31fa1a-fda4-43d6-84ef-e43f0bafdcb2\") " pod="openshift-cluster-node-tuning-operator/tuned-84tng" Jul 29 22:46:27.449335 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:27.449232 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/d40a4cdf-f141-46d3-8159-986f680b2dec-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-h6flc\" (UID: \"d40a4cdf-f141-46d3-8159-986f680b2dec\") " pod="openshift-multus/multus-additional-cni-plugins-h6flc" Jul 29 22:46:27.450034 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:27.449257 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/916b3077-78fa-4054-921c-2e7124dc35f8-original-pull-secret\") pod \"global-pull-secret-syncer-nkzq5\" (UID: \"916b3077-78fa-4054-921c-2e7124dc35f8\") " pod="kube-system/global-pull-secret-syncer-nkzq5" Jul 29 22:46:27.450034 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:27.449281 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/d40a4cdf-f141-46d3-8159-986f680b2dec-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-h6flc\" (UID: \"d40a4cdf-f141-46d3-8159-986f680b2dec\") " pod="openshift-multus/multus-additional-cni-plugins-h6flc" Jul 29 22:46:27.450034 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:27.449302 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4c31fa1a-fda4-43d6-84ef-e43f0bafdcb2-var-lib-kubelet\") pod \"tuned-84tng\" (UID: \"4c31fa1a-fda4-43d6-84ef-e43f0bafdcb2\") " pod="openshift-cluster-node-tuning-operator/tuned-84tng" Jul 29 22:46:27.450034 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:27.449324 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/4c31fa1a-fda4-43d6-84ef-e43f0bafdcb2-etc-tuned\") pod \"tuned-84tng\" (UID: \"4c31fa1a-fda4-43d6-84ef-e43f0bafdcb2\") " pod="openshift-cluster-node-tuning-operator/tuned-84tng" Jul 29 22:46:27.450034 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:27.449357 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/6c54a7c1-2837-47d6-88f4-dc4d264397d4-sys-fs\") pod \"aws-ebs-csi-driver-node-kqh77\" (UID: \"6c54a7c1-2837-47d6-88f4-dc4d264397d4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kqh77" Jul 29 22:46:27.450034 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:27.449378 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4rf4l\" (UniqueName: \"kubernetes.io/projected/6c54a7c1-2837-47d6-88f4-dc4d264397d4-kube-api-access-4rf4l\") pod \"aws-ebs-csi-driver-node-kqh77\" (UID: \"6c54a7c1-2837-47d6-88f4-dc4d264397d4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kqh77" Jul 29 22:46:27.450034 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:27.449413 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/6a64a07e-1c4a-4990-90b9-80623be37f60-host-run-netns\") pod \"multus-2mwmf\" (UID: \"6a64a07e-1c4a-4990-90b9-80623be37f60\") " pod="openshift-multus/multus-2mwmf" Jul 29 22:46:27.450034 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:27.449431 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/5ce0a47c-e715-472b-a0d9-761a5a6777dd-iptables-alerter-script\") pod \"iptables-alerter-frd9g\" (UID: \"5ce0a47c-e715-472b-a0d9-761a5a6777dd\") " pod="openshift-network-operator/iptables-alerter-frd9g" Jul 29 22:46:27.450034 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:27.449446 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/6050ad9f-817a-48e4-b939-12efa11a6a73-host-slash\") pod \"ovnkube-node-9g28r\" (UID: \"6050ad9f-817a-48e4-b939-12efa11a6a73\") " pod="openshift-ovn-kubernetes/ovnkube-node-9g28r" Jul 29 22:46:27.450034 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:27.449465 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6050ad9f-817a-48e4-b939-12efa11a6a73-host-run-ovn-kubernetes\") pod \"ovnkube-node-9g28r\" (UID: \"6050ad9f-817a-48e4-b939-12efa11a6a73\") " pod="openshift-ovn-kubernetes/ovnkube-node-9g28r" Jul 29 22:46:27.450034 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:27.449486 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/4c31fa1a-fda4-43d6-84ef-e43f0bafdcb2-lib-modules\") pod \"tuned-84tng\" (UID: \"4c31fa1a-fda4-43d6-84ef-e43f0bafdcb2\") " pod="openshift-cluster-node-tuning-operator/tuned-84tng" Jul 29 22:46:27.450034 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:27.449508 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/6a64a07e-1c4a-4990-90b9-80623be37f60-host-run-k8s-cni-cncf-io\") pod \"multus-2mwmf\" (UID: \"6a64a07e-1c4a-4990-90b9-80623be37f60\") " pod="openshift-multus/multus-2mwmf" Jul 29 22:46:27.450034 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:27.449549 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/6a64a07e-1c4a-4990-90b9-80623be37f60-host-run-multus-certs\") pod \"multus-2mwmf\" (UID: \"6a64a07e-1c4a-4990-90b9-80623be37f60\") " pod="openshift-multus/multus-2mwmf" Jul 29 22:46:27.450034 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:27.449594 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/6050ad9f-817a-48e4-b939-12efa11a6a73-host-run-netns\") pod \"ovnkube-node-9g28r\" (UID: \"6050ad9f-817a-48e4-b939-12efa11a6a73\") " pod="openshift-ovn-kubernetes/ovnkube-node-9g28r" Jul 29 22:46:27.450034 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:27.449618 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5262s\" (UniqueName: \"kubernetes.io/projected/6050ad9f-817a-48e4-b939-12efa11a6a73-kube-api-access-5262s\") pod \"ovnkube-node-9g28r\" (UID: \"6050ad9f-817a-48e4-b939-12efa11a6a73\") " pod="openshift-ovn-kubernetes/ovnkube-node-9g28r" Jul 29 22:46:27.450034 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:27.449639 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/4c31fa1a-fda4-43d6-84ef-e43f0bafdcb2-etc-modprobe-d\") pod \"tuned-84tng\" (UID: \"4c31fa1a-fda4-43d6-84ef-e43f0bafdcb2\") " pod="openshift-cluster-node-tuning-operator/tuned-84tng" Jul 29 22:46:27.450751 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:27.449660 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/4c31fa1a-fda4-43d6-84ef-e43f0bafdcb2-host\") pod \"tuned-84tng\" (UID: \"4c31fa1a-fda4-43d6-84ef-e43f0bafdcb2\") " pod="openshift-cluster-node-tuning-operator/tuned-84tng" Jul 29 22:46:27.450751 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:27.449682 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/d40a4cdf-f141-46d3-8159-986f680b2dec-os-release\") pod \"multus-additional-cni-plugins-h6flc\" (UID: \"d40a4cdf-f141-46d3-8159-986f680b2dec\") " pod="openshift-multus/multus-additional-cni-plugins-h6flc" Jul 29 22:46:27.450751 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:27.449704 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4c31fa1a-fda4-43d6-84ef-e43f0bafdcb2-etc-kubernetes\") pod \"tuned-84tng\" (UID: \"4c31fa1a-fda4-43d6-84ef-e43f0bafdcb2\") " pod="openshift-cluster-node-tuning-operator/tuned-84tng" Jul 29 22:46:27.450751 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:27.449725 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/6a64a07e-1c4a-4990-90b9-80623be37f60-multus-conf-dir\") pod \"multus-2mwmf\" (UID: \"6a64a07e-1c4a-4990-90b9-80623be37f60\") " pod="openshift-multus/multus-2mwmf" Jul 29 22:46:27.450751 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:27.449747 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/6a64a07e-1c4a-4990-90b9-80623be37f60-host-var-lib-cni-multus\") pod \"multus-2mwmf\" (UID: \"6a64a07e-1c4a-4990-90b9-80623be37f60\") " pod="openshift-multus/multus-2mwmf" Jul 29 22:46:27.450751 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:27.449768 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/6050ad9f-817a-48e4-b939-12efa11a6a73-run-systemd\") pod \"ovnkube-node-9g28r\" (UID: \"6050ad9f-817a-48e4-b939-12efa11a6a73\") " pod="openshift-ovn-kubernetes/ovnkube-node-9g28r" Jul 29 22:46:27.450751 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:27.449790 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6050ad9f-817a-48e4-b939-12efa11a6a73-etc-openvswitch\") pod \"ovnkube-node-9g28r\" (UID: \"6050ad9f-817a-48e4-b939-12efa11a6a73\") " pod="openshift-ovn-kubernetes/ovnkube-node-9g28r" Jul 29 22:46:27.450751 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:27.449821 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6050ad9f-817a-48e4-b939-12efa11a6a73-run-openvswitch\") pod \"ovnkube-node-9g28r\" (UID: \"6050ad9f-817a-48e4-b939-12efa11a6a73\") " pod="openshift-ovn-kubernetes/ovnkube-node-9g28r" Jul 29 22:46:27.450751 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:27.449865 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/6050ad9f-817a-48e4-b939-12efa11a6a73-host-cni-bin\") pod \"ovnkube-node-9g28r\" (UID: \"6050ad9f-817a-48e4-b939-12efa11a6a73\") " pod="openshift-ovn-kubernetes/ovnkube-node-9g28r" Jul 29 22:46:27.450751 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:27.449914 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/4c31fa1a-fda4-43d6-84ef-e43f0bafdcb2-etc-sysconfig\") pod \"tuned-84tng\" (UID: \"4c31fa1a-fda4-43d6-84ef-e43f0bafdcb2\") " pod="openshift-cluster-node-tuning-operator/tuned-84tng" Jul 29 22:46:27.450751 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:27.449937 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/4c31fa1a-fda4-43d6-84ef-e43f0bafdcb2-etc-systemd\") pod \"tuned-84tng\" (UID: \"4c31fa1a-fda4-43d6-84ef-e43f0bafdcb2\") " pod="openshift-cluster-node-tuning-operator/tuned-84tng" Jul 29 22:46:27.450751 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:27.449960 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vkrlb\" (UniqueName: \"kubernetes.io/projected/4c31fa1a-fda4-43d6-84ef-e43f0bafdcb2-kube-api-access-vkrlb\") pod \"tuned-84tng\" (UID: \"4c31fa1a-fda4-43d6-84ef-e43f0bafdcb2\") " pod="openshift-cluster-node-tuning-operator/tuned-84tng" Jul 29 22:46:27.450751 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:27.449976 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6a64a07e-1c4a-4990-90b9-80623be37f60-host-var-lib-kubelet\") pod \"multus-2mwmf\" (UID: \"6a64a07e-1c4a-4990-90b9-80623be37f60\") " pod="openshift-multus/multus-2mwmf" Jul 29 22:46:27.450751 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:27.449993 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/6050ad9f-817a-48e4-b939-12efa11a6a73-run-ovn\") pod \"ovnkube-node-9g28r\" (UID: \"6050ad9f-817a-48e4-b939-12efa11a6a73\") " pod="openshift-ovn-kubernetes/ovnkube-node-9g28r" Jul 29 22:46:27.450751 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:27.450009 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6050ad9f-817a-48e4-b939-12efa11a6a73-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-9g28r\" (UID: \"6050ad9f-817a-48e4-b939-12efa11a6a73\") " pod="openshift-ovn-kubernetes/ovnkube-node-9g28r" Jul 29 22:46:27.450751 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:27.450023 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6a64a07e-1c4a-4990-90b9-80623be37f60-system-cni-dir\") pod \"multus-2mwmf\" (UID: \"6a64a07e-1c4a-4990-90b9-80623be37f60\") " pod="openshift-multus/multus-2mwmf" Jul 29 22:46:27.471950 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:27.471921 2562 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-lnnjf" Jul 29 22:46:27.480408 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:27.480367 2562 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-lnnjf" Jul 29 22:46:27.536647 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:27.536618 2562 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 29 22:46:27.550895 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:27.550860 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/916b3077-78fa-4054-921c-2e7124dc35f8-dbus\") pod \"global-pull-secret-syncer-nkzq5\" (UID: \"916b3077-78fa-4054-921c-2e7124dc35f8\") " pod="kube-system/global-pull-secret-syncer-nkzq5" Jul 29 22:46:27.550895 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:27.550899 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4c31fa1a-fda4-43d6-84ef-e43f0bafdcb2-tmp\") pod \"tuned-84tng\" (UID: \"4c31fa1a-fda4-43d6-84ef-e43f0bafdcb2\") " pod="openshift-cluster-node-tuning-operator/tuned-84tng" Jul 29 22:46:27.551129 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:27.550926 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pmz5r\" (UniqueName: \"kubernetes.io/projected/6a64a07e-1c4a-4990-90b9-80623be37f60-kube-api-access-pmz5r\") pod \"multus-2mwmf\" (UID: \"6a64a07e-1c4a-4990-90b9-80623be37f60\") " pod="openshift-multus/multus-2mwmf" Jul 29 22:46:27.551129 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:27.550952 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/6c54a7c1-2837-47d6-88f4-dc4d264397d4-kubelet-dir\") pod \"aws-ebs-csi-driver-node-kqh77\" (UID: \"6c54a7c1-2837-47d6-88f4-dc4d264397d4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kqh77" Jul 29 22:46:27.551129 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:27.550977 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/6c54a7c1-2837-47d6-88f4-dc4d264397d4-registration-dir\") pod \"aws-ebs-csi-driver-node-kqh77\" (UID: \"6c54a7c1-2837-47d6-88f4-dc4d264397d4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kqh77" Jul 29 22:46:27.551129 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:27.551051 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/6c54a7c1-2837-47d6-88f4-dc4d264397d4-device-dir\") pod \"aws-ebs-csi-driver-node-kqh77\" (UID: \"6c54a7c1-2837-47d6-88f4-dc4d264397d4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kqh77" Jul 29 22:46:27.551129 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:27.551068 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/6c54a7c1-2837-47d6-88f4-dc4d264397d4-kubelet-dir\") pod \"aws-ebs-csi-driver-node-kqh77\" (UID: \"6c54a7c1-2837-47d6-88f4-dc4d264397d4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kqh77" Jul 29 22:46:27.551129 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:27.551094 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/6c54a7c1-2837-47d6-88f4-dc4d264397d4-device-dir\") pod \"aws-ebs-csi-driver-node-kqh77\" (UID: \"6c54a7c1-2837-47d6-88f4-dc4d264397d4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kqh77" Jul 29 22:46:27.551129 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:27.551056 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/6c54a7c1-2837-47d6-88f4-dc4d264397d4-registration-dir\") pod \"aws-ebs-csi-driver-node-kqh77\" (UID: \"6c54a7c1-2837-47d6-88f4-dc4d264397d4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kqh77" Jul 29 22:46:27.551129 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:27.551096 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/6050ad9f-817a-48e4-b939-12efa11a6a73-host-kubelet\") pod \"ovnkube-node-9g28r\" (UID: \"6050ad9f-817a-48e4-b939-12efa11a6a73\") " pod="openshift-ovn-kubernetes/ovnkube-node-9g28r" Jul 29 22:46:27.551129 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:27.551115 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/916b3077-78fa-4054-921c-2e7124dc35f8-dbus\") pod \"global-pull-secret-syncer-nkzq5\" (UID: \"916b3077-78fa-4054-921c-2e7124dc35f8\") " pod="kube-system/global-pull-secret-syncer-nkzq5" Jul 29 22:46:27.551576 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:27.551132 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/6050ad9f-817a-48e4-b939-12efa11a6a73-host-kubelet\") pod \"ovnkube-node-9g28r\" (UID: \"6050ad9f-817a-48e4-b939-12efa11a6a73\") " pod="openshift-ovn-kubernetes/ovnkube-node-9g28r" Jul 29 22:46:27.551576 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:27.551140 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9kjfg\" (UniqueName: \"kubernetes.io/projected/f6d8c777-cb9a-4505-8e30-ffaa01e53239-kube-api-access-9kjfg\") pod \"node-ca-vwtj2\" (UID: \"f6d8c777-cb9a-4505-8e30-ffaa01e53239\") " pod="openshift-image-registry/node-ca-vwtj2" Jul 29 22:46:27.551576 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:27.551175 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/6a64a07e-1c4a-4990-90b9-80623be37f60-cni-binary-copy\") pod \"multus-2mwmf\" (UID: \"6a64a07e-1c4a-4990-90b9-80623be37f60\") " pod="openshift-multus/multus-2mwmf" Jul 29 22:46:27.551576 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:27.551201 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/5ce0a47c-e715-472b-a0d9-761a5a6777dd-host-slash\") pod \"iptables-alerter-frd9g\" (UID: \"5ce0a47c-e715-472b-a0d9-761a5a6777dd\") " pod="openshift-network-operator/iptables-alerter-frd9g" Jul 29 22:46:27.551576 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:27.551225 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/6050ad9f-817a-48e4-b939-12efa11a6a73-host-cni-netd\") pod \"ovnkube-node-9g28r\" (UID: \"6050ad9f-817a-48e4-b939-12efa11a6a73\") " pod="openshift-ovn-kubernetes/ovnkube-node-9g28r" Jul 29 22:46:27.551576 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:27.551249 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/6050ad9f-817a-48e4-b939-12efa11a6a73-env-overrides\") pod \"ovnkube-node-9g28r\" (UID: \"6050ad9f-817a-48e4-b939-12efa11a6a73\") " pod="openshift-ovn-kubernetes/ovnkube-node-9g28r" Jul 29 22:46:27.551576 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:27.551272 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/5ce0a47c-e715-472b-a0d9-761a5a6777dd-host-slash\") pod \"iptables-alerter-frd9g\" (UID: \"5ce0a47c-e715-472b-a0d9-761a5a6777dd\") " pod="openshift-network-operator/iptables-alerter-frd9g" Jul 29 22:46:27.551576 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:27.551273 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/6a64a07e-1c4a-4990-90b9-80623be37f60-multus-daemon-config\") pod \"multus-2mwmf\" (UID: \"6a64a07e-1c4a-4990-90b9-80623be37f60\") " pod="openshift-multus/multus-2mwmf" Jul 29 22:46:27.551576 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:27.551313 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/6050ad9f-817a-48e4-b939-12efa11a6a73-host-cni-netd\") pod \"ovnkube-node-9g28r\" (UID: \"6050ad9f-817a-48e4-b939-12efa11a6a73\") " pod="openshift-ovn-kubernetes/ovnkube-node-9g28r" Jul 29 22:46:27.551576 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:27.551331 2562 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 29 22:46:27.551576 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:27.551391 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/d40a4cdf-f141-46d3-8159-986f680b2dec-cnibin\") pod \"multus-additional-cni-plugins-h6flc\" (UID: \"d40a4cdf-f141-46d3-8159-986f680b2dec\") " pod="openshift-multus/multus-additional-cni-plugins-h6flc" Jul 29 22:46:27.551576 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:27.551438 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/6050ad9f-817a-48e4-b939-12efa11a6a73-ovnkube-script-lib\") pod \"ovnkube-node-9g28r\" (UID: \"6050ad9f-817a-48e4-b939-12efa11a6a73\") " pod="openshift-ovn-kubernetes/ovnkube-node-9g28r" Jul 29 22:46:27.551576 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:27.551467 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/916b3077-78fa-4054-921c-2e7124dc35f8-kubelet-config\") pod \"global-pull-secret-syncer-nkzq5\" (UID: \"916b3077-78fa-4054-921c-2e7124dc35f8\") " pod="kube-system/global-pull-secret-syncer-nkzq5" Jul 29 22:46:27.551576 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:27.551493 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/4c31fa1a-fda4-43d6-84ef-e43f0bafdcb2-etc-sysctl-conf\") pod \"tuned-84tng\" (UID: \"4c31fa1a-fda4-43d6-84ef-e43f0bafdcb2\") " pod="openshift-cluster-node-tuning-operator/tuned-84tng" Jul 29 22:46:27.551576 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:27.551521 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/ece97cfa-80dc-4fa8-ad31-61ba5c38aee3-agent-certs\") pod \"konnectivity-agent-k9nmc\" (UID: \"ece97cfa-80dc-4fa8-ad31-61ba5c38aee3\") " pod="kube-system/konnectivity-agent-k9nmc" Jul 29 22:46:27.551576 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:27.551555 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/6050ad9f-817a-48e4-b939-12efa11a6a73-ovn-node-metrics-cert\") pod \"ovnkube-node-9g28r\" (UID: \"6050ad9f-817a-48e4-b939-12efa11a6a73\") " pod="openshift-ovn-kubernetes/ovnkube-node-9g28r" Jul 29 22:46:27.551576 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:27.551577 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/916b3077-78fa-4054-921c-2e7124dc35f8-kubelet-config\") pod \"global-pull-secret-syncer-nkzq5\" (UID: \"916b3077-78fa-4054-921c-2e7124dc35f8\") " pod="kube-system/global-pull-secret-syncer-nkzq5" Jul 29 22:46:27.551576 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:27.551581 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h8vm9\" (UniqueName: \"kubernetes.io/projected/5ce0a47c-e715-472b-a0d9-761a5a6777dd-kube-api-access-h8vm9\") pod \"iptables-alerter-frd9g\" (UID: \"5ce0a47c-e715-472b-a0d9-761a5a6777dd\") " pod="openshift-network-operator/iptables-alerter-frd9g" Jul 29 22:46:27.552454 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:27.551613 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/6050ad9f-817a-48e4-b939-12efa11a6a73-systemd-units\") pod \"ovnkube-node-9g28r\" (UID: \"6050ad9f-817a-48e4-b939-12efa11a6a73\") " pod="openshift-ovn-kubernetes/ovnkube-node-9g28r" Jul 29 22:46:27.552454 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:27.551640 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/4c31fa1a-fda4-43d6-84ef-e43f0bafdcb2-etc-sysctl-d\") pod \"tuned-84tng\" (UID: \"4c31fa1a-fda4-43d6-84ef-e43f0bafdcb2\") " pod="openshift-cluster-node-tuning-operator/tuned-84tng" Jul 29 22:46:27.552454 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:27.551666 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/d40a4cdf-f141-46d3-8159-986f680b2dec-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-h6flc\" (UID: \"d40a4cdf-f141-46d3-8159-986f680b2dec\") " pod="openshift-multus/multus-additional-cni-plugins-h6flc" Jul 29 22:46:27.552454 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:27.551693 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/51d8cffe-16b4-4bab-9193-953163e45417-hosts-file\") pod \"node-resolver-k8fl9\" (UID: \"51d8cffe-16b4-4bab-9193-953163e45417\") " pod="openshift-dns/node-resolver-k8fl9" Jul 29 22:46:27.552454 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:27.551717 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/51d8cffe-16b4-4bab-9193-953163e45417-tmp-dir\") pod \"node-resolver-k8fl9\" (UID: \"51d8cffe-16b4-4bab-9193-953163e45417\") " pod="openshift-dns/node-resolver-k8fl9" Jul 29 22:46:27.552454 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:27.551745 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/916b3077-78fa-4054-921c-2e7124dc35f8-original-pull-secret\") pod \"global-pull-secret-syncer-nkzq5\" (UID: \"916b3077-78fa-4054-921c-2e7124dc35f8\") " pod="kube-system/global-pull-secret-syncer-nkzq5" Jul 29 22:46:27.552454 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:27.551773 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/d40a4cdf-f141-46d3-8159-986f680b2dec-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-h6flc\" (UID: \"d40a4cdf-f141-46d3-8159-986f680b2dec\") " pod="openshift-multus/multus-additional-cni-plugins-h6flc" Jul 29 22:46:27.552454 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:27.551521 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/d40a4cdf-f141-46d3-8159-986f680b2dec-cnibin\") pod \"multus-additional-cni-plugins-h6flc\" (UID: \"d40a4cdf-f141-46d3-8159-986f680b2dec\") " pod="openshift-multus/multus-additional-cni-plugins-h6flc" Jul 29 22:46:27.552454 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:27.551796 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4c31fa1a-fda4-43d6-84ef-e43f0bafdcb2-var-lib-kubelet\") pod \"tuned-84tng\" (UID: \"4c31fa1a-fda4-43d6-84ef-e43f0bafdcb2\") " pod="openshift-cluster-node-tuning-operator/tuned-84tng" Jul 29 22:46:27.552454 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:27.551808 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/6050ad9f-817a-48e4-b939-12efa11a6a73-env-overrides\") pod \"ovnkube-node-9g28r\" (UID: \"6050ad9f-817a-48e4-b939-12efa11a6a73\") " pod="openshift-ovn-kubernetes/ovnkube-node-9g28r" Jul 29 22:46:27.552454 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:27.551818 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/4c31fa1a-fda4-43d6-84ef-e43f0bafdcb2-etc-tuned\") pod \"tuned-84tng\" (UID: \"4c31fa1a-fda4-43d6-84ef-e43f0bafdcb2\") " pod="openshift-cluster-node-tuning-operator/tuned-84tng" Jul 29 22:46:27.552454 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:27.551822 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/6a64a07e-1c4a-4990-90b9-80623be37f60-multus-daemon-config\") pod \"multus-2mwmf\" (UID: \"6a64a07e-1c4a-4990-90b9-80623be37f60\") " pod="openshift-multus/multus-2mwmf" Jul 29 22:46:27.552454 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:27.551860 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/6c54a7c1-2837-47d6-88f4-dc4d264397d4-sys-fs\") pod \"aws-ebs-csi-driver-node-kqh77\" (UID: \"6c54a7c1-2837-47d6-88f4-dc4d264397d4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kqh77" Jul 29 22:46:27.552454 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:27.551824 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/6a64a07e-1c4a-4990-90b9-80623be37f60-cni-binary-copy\") pod \"multus-2mwmf\" (UID: \"6a64a07e-1c4a-4990-90b9-80623be37f60\") " pod="openshift-multus/multus-2mwmf" Jul 29 22:46:27.552454 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:27.551884 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4rf4l\" (UniqueName: \"kubernetes.io/projected/6c54a7c1-2837-47d6-88f4-dc4d264397d4-kube-api-access-4rf4l\") pod \"aws-ebs-csi-driver-node-kqh77\" (UID: \"6c54a7c1-2837-47d6-88f4-dc4d264397d4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kqh77" Jul 29 22:46:27.552454 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:27.551898 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/4c31fa1a-fda4-43d6-84ef-e43f0bafdcb2-etc-sysctl-conf\") pod \"tuned-84tng\" (UID: \"4c31fa1a-fda4-43d6-84ef-e43f0bafdcb2\") " pod="openshift-cluster-node-tuning-operator/tuned-84tng" Jul 29 22:46:27.552454 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:27.551908 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/6a64a07e-1c4a-4990-90b9-80623be37f60-host-run-netns\") pod \"multus-2mwmf\" (UID: \"6a64a07e-1c4a-4990-90b9-80623be37f60\") " pod="openshift-multus/multus-2mwmf" Jul 29 22:46:27.553216 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:27.551941 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/5ce0a47c-e715-472b-a0d9-761a5a6777dd-iptables-alerter-script\") pod \"iptables-alerter-frd9g\" (UID: \"5ce0a47c-e715-472b-a0d9-761a5a6777dd\") " pod="openshift-network-operator/iptables-alerter-frd9g" Jul 29 22:46:27.553216 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:27.551945 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/6a64a07e-1c4a-4990-90b9-80623be37f60-host-run-netns\") pod \"multus-2mwmf\" (UID: \"6a64a07e-1c4a-4990-90b9-80623be37f60\") " pod="openshift-multus/multus-2mwmf" Jul 29 22:46:27.553216 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:27.551957 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/4c31fa1a-fda4-43d6-84ef-e43f0bafdcb2-etc-sysctl-d\") pod \"tuned-84tng\" (UID: \"4c31fa1a-fda4-43d6-84ef-e43f0bafdcb2\") " pod="openshift-cluster-node-tuning-operator/tuned-84tng" Jul 29 22:46:27.553216 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:27.551966 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/6050ad9f-817a-48e4-b939-12efa11a6a73-host-slash\") pod \"ovnkube-node-9g28r\" (UID: \"6050ad9f-817a-48e4-b939-12efa11a6a73\") " pod="openshift-ovn-kubernetes/ovnkube-node-9g28r" Jul 29 22:46:27.553216 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:27.552000 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/6050ad9f-817a-48e4-b939-12efa11a6a73-systemd-units\") pod \"ovnkube-node-9g28r\" (UID: \"6050ad9f-817a-48e4-b939-12efa11a6a73\") " pod="openshift-ovn-kubernetes/ovnkube-node-9g28r" Jul 29 22:46:27.553216 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:27.552012 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/6050ad9f-817a-48e4-b939-12efa11a6a73-host-slash\") pod \"ovnkube-node-9g28r\" (UID: \"6050ad9f-817a-48e4-b939-12efa11a6a73\") " pod="openshift-ovn-kubernetes/ovnkube-node-9g28r" Jul 29 22:46:27.553216 ip-10-0-138-112 kubenswrapper[2562]: E0729 22:46:27.552070 2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 29 22:46:27.553216 ip-10-0-138-112 kubenswrapper[2562]: E0729 22:46:27.552132 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/916b3077-78fa-4054-921c-2e7124dc35f8-original-pull-secret podName:916b3077-78fa-4054-921c-2e7124dc35f8 nodeName:}" failed. No retries permitted until 2026-07-29 22:46:28.052113544 +0000 UTC m=+2.963939222 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/916b3077-78fa-4054-921c-2e7124dc35f8-original-pull-secret") pod "global-pull-secret-syncer-nkzq5" (UID: "916b3077-78fa-4054-921c-2e7124dc35f8") : object "kube-system"/"original-pull-secret" not registered Jul 29 22:46:27.553216 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:27.552354 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/6050ad9f-817a-48e4-b939-12efa11a6a73-ovnkube-script-lib\") pod \"ovnkube-node-9g28r\" (UID: \"6050ad9f-817a-48e4-b939-12efa11a6a73\") " pod="openshift-ovn-kubernetes/ovnkube-node-9g28r" Jul 29 22:46:27.553216 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:27.552380 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/d40a4cdf-f141-46d3-8159-986f680b2dec-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-h6flc\" (UID: \"d40a4cdf-f141-46d3-8159-986f680b2dec\") " pod="openshift-multus/multus-additional-cni-plugins-h6flc" Jul 29 22:46:27.553216 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:27.552441 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/6c54a7c1-2837-47d6-88f4-dc4d264397d4-sys-fs\") pod \"aws-ebs-csi-driver-node-kqh77\" (UID: \"6c54a7c1-2837-47d6-88f4-dc4d264397d4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kqh77" Jul 29 22:46:27.553216 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:27.552457 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4c31fa1a-fda4-43d6-84ef-e43f0bafdcb2-var-lib-kubelet\") pod \"tuned-84tng\" (UID: \"4c31fa1a-fda4-43d6-84ef-e43f0bafdcb2\") " pod="openshift-cluster-node-tuning-operator/tuned-84tng" Jul 29 22:46:27.553216 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:27.552461 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/d40a4cdf-f141-46d3-8159-986f680b2dec-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-h6flc\" (UID: \"d40a4cdf-f141-46d3-8159-986f680b2dec\") " pod="openshift-multus/multus-additional-cni-plugins-h6flc" Jul 29 22:46:27.553216 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:27.552498 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6050ad9f-817a-48e4-b939-12efa11a6a73-host-run-ovn-kubernetes\") pod \"ovnkube-node-9g28r\" (UID: \"6050ad9f-817a-48e4-b939-12efa11a6a73\") " pod="openshift-ovn-kubernetes/ovnkube-node-9g28r" Jul 29 22:46:27.553216 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:27.552522 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/4c31fa1a-fda4-43d6-84ef-e43f0bafdcb2-lib-modules\") pod \"tuned-84tng\" (UID: \"4c31fa1a-fda4-43d6-84ef-e43f0bafdcb2\") " pod="openshift-cluster-node-tuning-operator/tuned-84tng" Jul 29 22:46:27.553216 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:27.552542 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/5ce0a47c-e715-472b-a0d9-761a5a6777dd-iptables-alerter-script\") pod \"iptables-alerter-frd9g\" (UID: \"5ce0a47c-e715-472b-a0d9-761a5a6777dd\") " pod="openshift-network-operator/iptables-alerter-frd9g" Jul 29 22:46:27.553216 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:27.552562 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6050ad9f-817a-48e4-b939-12efa11a6a73-host-run-ovn-kubernetes\") pod \"ovnkube-node-9g28r\" (UID: \"6050ad9f-817a-48e4-b939-12efa11a6a73\") " pod="openshift-ovn-kubernetes/ovnkube-node-9g28r" Jul 29 22:46:27.554052 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:27.552571 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/6a64a07e-1c4a-4990-90b9-80623be37f60-host-run-k8s-cni-cncf-io\") pod \"multus-2mwmf\" (UID: \"6a64a07e-1c4a-4990-90b9-80623be37f60\") " pod="openshift-multus/multus-2mwmf" Jul 29 22:46:27.554052 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:27.552597 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/6a64a07e-1c4a-4990-90b9-80623be37f60-host-run-multus-certs\") pod \"multus-2mwmf\" (UID: \"6a64a07e-1c4a-4990-90b9-80623be37f60\") " pod="openshift-multus/multus-2mwmf" Jul 29 22:46:27.554052 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:27.552641 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/6050ad9f-817a-48e4-b939-12efa11a6a73-host-run-netns\") pod \"ovnkube-node-9g28r\" (UID: \"6050ad9f-817a-48e4-b939-12efa11a6a73\") " pod="openshift-ovn-kubernetes/ovnkube-node-9g28r" Jul 29 22:46:27.554052 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:27.552655 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/4c31fa1a-fda4-43d6-84ef-e43f0bafdcb2-lib-modules\") pod \"tuned-84tng\" (UID: \"4c31fa1a-fda4-43d6-84ef-e43f0bafdcb2\") " pod="openshift-cluster-node-tuning-operator/tuned-84tng" Jul 29 22:46:27.554052 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:27.552671 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/6a64a07e-1c4a-4990-90b9-80623be37f60-host-run-k8s-cni-cncf-io\") pod \"multus-2mwmf\" (UID: \"6a64a07e-1c4a-4990-90b9-80623be37f60\") " pod="openshift-multus/multus-2mwmf" Jul 29 22:46:27.554052 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:27.552683 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5262s\" (UniqueName: \"kubernetes.io/projected/6050ad9f-817a-48e4-b939-12efa11a6a73-kube-api-access-5262s\") pod \"ovnkube-node-9g28r\" (UID: \"6050ad9f-817a-48e4-b939-12efa11a6a73\") " pod="openshift-ovn-kubernetes/ovnkube-node-9g28r" Jul 29 22:46:27.554052 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:27.552688 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/6a64a07e-1c4a-4990-90b9-80623be37f60-host-run-multus-certs\") pod \"multus-2mwmf\" (UID: \"6a64a07e-1c4a-4990-90b9-80623be37f60\") " pod="openshift-multus/multus-2mwmf" Jul 29 22:46:27.554052 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:27.552709 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/4c31fa1a-fda4-43d6-84ef-e43f0bafdcb2-etc-modprobe-d\") pod \"tuned-84tng\" (UID: \"4c31fa1a-fda4-43d6-84ef-e43f0bafdcb2\") " pod="openshift-cluster-node-tuning-operator/tuned-84tng" Jul 29 22:46:27.554052 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:27.552714 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/6050ad9f-817a-48e4-b939-12efa11a6a73-host-run-netns\") pod \"ovnkube-node-9g28r\" (UID: \"6050ad9f-817a-48e4-b939-12efa11a6a73\") " pod="openshift-ovn-kubernetes/ovnkube-node-9g28r" Jul 29 22:46:27.554052 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:27.552741 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/4c31fa1a-fda4-43d6-84ef-e43f0bafdcb2-host\") pod \"tuned-84tng\" (UID: \"4c31fa1a-fda4-43d6-84ef-e43f0bafdcb2\") " pod="openshift-cluster-node-tuning-operator/tuned-84tng" Jul 29 22:46:27.554052 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:27.552764 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/d40a4cdf-f141-46d3-8159-986f680b2dec-os-release\") pod \"multus-additional-cni-plugins-h6flc\" (UID: \"d40a4cdf-f141-46d3-8159-986f680b2dec\") " pod="openshift-multus/multus-additional-cni-plugins-h6flc" Jul 29 22:46:27.554052 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:27.552791 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5kv55\" (UniqueName: \"kubernetes.io/projected/2771bdbc-d054-464b-a01f-82344308451f-kube-api-access-5kv55\") pod \"network-check-target-z92kj\" (UID: \"2771bdbc-d054-464b-a01f-82344308451f\") " pod="openshift-network-diagnostics/network-check-target-z92kj" Jul 29 22:46:27.554052 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:27.552853 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4c31fa1a-fda4-43d6-84ef-e43f0bafdcb2-etc-kubernetes\") pod \"tuned-84tng\" (UID: \"4c31fa1a-fda4-43d6-84ef-e43f0bafdcb2\") " pod="openshift-cluster-node-tuning-operator/tuned-84tng" Jul 29 22:46:27.554052 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:27.552814 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/4c31fa1a-fda4-43d6-84ef-e43f0bafdcb2-etc-modprobe-d\") pod \"tuned-84tng\" (UID: \"4c31fa1a-fda4-43d6-84ef-e43f0bafdcb2\") " pod="openshift-cluster-node-tuning-operator/tuned-84tng" Jul 29 22:46:27.554052 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:27.552879 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/6a64a07e-1c4a-4990-90b9-80623be37f60-multus-conf-dir\") pod \"multus-2mwmf\" (UID: \"6a64a07e-1c4a-4990-90b9-80623be37f60\") " pod="openshift-multus/multus-2mwmf" Jul 29 22:46:27.554052 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:27.552865 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/d40a4cdf-f141-46d3-8159-986f680b2dec-os-release\") pod \"multus-additional-cni-plugins-h6flc\" (UID: \"d40a4cdf-f141-46d3-8159-986f680b2dec\") " pod="openshift-multus/multus-additional-cni-plugins-h6flc" Jul 29 22:46:27.554052 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:27.552853 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/4c31fa1a-fda4-43d6-84ef-e43f0bafdcb2-host\") pod \"tuned-84tng\" (UID: \"4c31fa1a-fda4-43d6-84ef-e43f0bafdcb2\") " pod="openshift-cluster-node-tuning-operator/tuned-84tng" Jul 29 22:46:27.554855 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:27.552903 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4c31fa1a-fda4-43d6-84ef-e43f0bafdcb2-etc-kubernetes\") pod \"tuned-84tng\" (UID: \"4c31fa1a-fda4-43d6-84ef-e43f0bafdcb2\") " pod="openshift-cluster-node-tuning-operator/tuned-84tng" Jul 29 22:46:27.554855 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:27.552917 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/6a64a07e-1c4a-4990-90b9-80623be37f60-multus-conf-dir\") pod \"multus-2mwmf\" (UID: \"6a64a07e-1c4a-4990-90b9-80623be37f60\") " pod="openshift-multus/multus-2mwmf" Jul 29 22:46:27.554855 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:27.552945 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/6a64a07e-1c4a-4990-90b9-80623be37f60-host-var-lib-cni-multus\") pod \"multus-2mwmf\" (UID: \"6a64a07e-1c4a-4990-90b9-80623be37f60\") " pod="openshift-multus/multus-2mwmf" Jul 29 22:46:27.554855 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:27.552971 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sqxph\" (UniqueName: \"kubernetes.io/projected/51d8cffe-16b4-4bab-9193-953163e45417-kube-api-access-sqxph\") pod \"node-resolver-k8fl9\" (UID: \"51d8cffe-16b4-4bab-9193-953163e45417\") " pod="openshift-dns/node-resolver-k8fl9" Jul 29 22:46:27.554855 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:27.552995 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8dhhj\" (UniqueName: \"kubernetes.io/projected/f0a6baed-8bea-4992-8852-0b84808a78d1-kube-api-access-8dhhj\") pod \"network-metrics-daemon-fxb52\" (UID: \"f0a6baed-8bea-4992-8852-0b84808a78d1\") " pod="openshift-multus/network-metrics-daemon-fxb52" Jul 29 22:46:27.554855 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:27.553007 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/6a64a07e-1c4a-4990-90b9-80623be37f60-host-var-lib-cni-multus\") pod \"multus-2mwmf\" (UID: \"6a64a07e-1c4a-4990-90b9-80623be37f60\") " pod="openshift-multus/multus-2mwmf" Jul 29 22:46:27.554855 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:27.553015 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/ece97cfa-80dc-4fa8-ad31-61ba5c38aee3-konnectivity-ca\") pod \"konnectivity-agent-k9nmc\" (UID: \"ece97cfa-80dc-4fa8-ad31-61ba5c38aee3\") " pod="kube-system/konnectivity-agent-k9nmc" Jul 29 22:46:27.554855 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:27.553039 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/6050ad9f-817a-48e4-b939-12efa11a6a73-run-systemd\") pod \"ovnkube-node-9g28r\" (UID: \"6050ad9f-817a-48e4-b939-12efa11a6a73\") " pod="openshift-ovn-kubernetes/ovnkube-node-9g28r" Jul 29 22:46:27.554855 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:27.553060 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6050ad9f-817a-48e4-b939-12efa11a6a73-etc-openvswitch\") pod \"ovnkube-node-9g28r\" (UID: \"6050ad9f-817a-48e4-b939-12efa11a6a73\") " pod="openshift-ovn-kubernetes/ovnkube-node-9g28r" Jul 29 22:46:27.554855 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:27.553066 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/6050ad9f-817a-48e4-b939-12efa11a6a73-run-systemd\") pod \"ovnkube-node-9g28r\" (UID: \"6050ad9f-817a-48e4-b939-12efa11a6a73\") " pod="openshift-ovn-kubernetes/ovnkube-node-9g28r" Jul 29 22:46:27.554855 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:27.553083 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6050ad9f-817a-48e4-b939-12efa11a6a73-run-openvswitch\") pod \"ovnkube-node-9g28r\" (UID: \"6050ad9f-817a-48e4-b939-12efa11a6a73\") " pod="openshift-ovn-kubernetes/ovnkube-node-9g28r" Jul 29 22:46:27.554855 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:27.553107 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/6050ad9f-817a-48e4-b939-12efa11a6a73-host-cni-bin\") pod \"ovnkube-node-9g28r\" (UID: \"6050ad9f-817a-48e4-b939-12efa11a6a73\") " pod="openshift-ovn-kubernetes/ovnkube-node-9g28r" Jul 29 22:46:27.554855 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:27.553131 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/4c31fa1a-fda4-43d6-84ef-e43f0bafdcb2-etc-sysconfig\") pod \"tuned-84tng\" (UID: \"4c31fa1a-fda4-43d6-84ef-e43f0bafdcb2\") " pod="openshift-cluster-node-tuning-operator/tuned-84tng" Jul 29 22:46:27.554855 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:27.553152 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/4c31fa1a-fda4-43d6-84ef-e43f0bafdcb2-etc-systemd\") pod \"tuned-84tng\" (UID: \"4c31fa1a-fda4-43d6-84ef-e43f0bafdcb2\") " pod="openshift-cluster-node-tuning-operator/tuned-84tng" Jul 29 22:46:27.554855 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:27.553154 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6050ad9f-817a-48e4-b939-12efa11a6a73-run-openvswitch\") pod \"ovnkube-node-9g28r\" (UID: \"6050ad9f-817a-48e4-b939-12efa11a6a73\") " pod="openshift-ovn-kubernetes/ovnkube-node-9g28r" Jul 29 22:46:27.554855 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:27.553161 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/6050ad9f-817a-48e4-b939-12efa11a6a73-host-cni-bin\") pod \"ovnkube-node-9g28r\" (UID: \"6050ad9f-817a-48e4-b939-12efa11a6a73\") " pod="openshift-ovn-kubernetes/ovnkube-node-9g28r" Jul 29 22:46:27.554855 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:27.553178 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vkrlb\" (UniqueName: \"kubernetes.io/projected/4c31fa1a-fda4-43d6-84ef-e43f0bafdcb2-kube-api-access-vkrlb\") pod \"tuned-84tng\" (UID: \"4c31fa1a-fda4-43d6-84ef-e43f0bafdcb2\") " pod="openshift-cluster-node-tuning-operator/tuned-84tng" Jul 29 22:46:27.555571 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:27.553220 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6a64a07e-1c4a-4990-90b9-80623be37f60-host-var-lib-kubelet\") pod \"multus-2mwmf\" (UID: \"6a64a07e-1c4a-4990-90b9-80623be37f60\") " pod="openshift-multus/multus-2mwmf" Jul 29 22:46:27.555571 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:27.553265 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/4c31fa1a-fda4-43d6-84ef-e43f0bafdcb2-etc-systemd\") pod \"tuned-84tng\" (UID: \"4c31fa1a-fda4-43d6-84ef-e43f0bafdcb2\") " pod="openshift-cluster-node-tuning-operator/tuned-84tng" Jul 29 22:46:27.555571 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:27.553248 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/6050ad9f-817a-48e4-b939-12efa11a6a73-run-ovn\") pod \"ovnkube-node-9g28r\" (UID: \"6050ad9f-817a-48e4-b939-12efa11a6a73\") " pod="openshift-ovn-kubernetes/ovnkube-node-9g28r" Jul 29 22:46:27.555571 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:27.553287 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/4c31fa1a-fda4-43d6-84ef-e43f0bafdcb2-etc-sysconfig\") pod \"tuned-84tng\" (UID: \"4c31fa1a-fda4-43d6-84ef-e43f0bafdcb2\") " pod="openshift-cluster-node-tuning-operator/tuned-84tng" Jul 29 22:46:27.555571 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:27.553101 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6050ad9f-817a-48e4-b939-12efa11a6a73-etc-openvswitch\") pod \"ovnkube-node-9g28r\" (UID: \"6050ad9f-817a-48e4-b939-12efa11a6a73\") " pod="openshift-ovn-kubernetes/ovnkube-node-9g28r" Jul 29 22:46:27.555571 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:27.553298 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6050ad9f-817a-48e4-b939-12efa11a6a73-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-9g28r\" (UID: \"6050ad9f-817a-48e4-b939-12efa11a6a73\") " pod="openshift-ovn-kubernetes/ovnkube-node-9g28r" Jul 29 22:46:27.555571 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:27.553348 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6a64a07e-1c4a-4990-90b9-80623be37f60-host-var-lib-kubelet\") pod \"multus-2mwmf\" (UID: \"6a64a07e-1c4a-4990-90b9-80623be37f60\") " pod="openshift-multus/multus-2mwmf" Jul 29 22:46:27.555571 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:27.553298 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/6050ad9f-817a-48e4-b939-12efa11a6a73-run-ovn\") pod \"ovnkube-node-9g28r\" (UID: \"6050ad9f-817a-48e4-b939-12efa11a6a73\") " pod="openshift-ovn-kubernetes/ovnkube-node-9g28r" Jul 29 22:46:27.555571 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:27.553325 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6050ad9f-817a-48e4-b939-12efa11a6a73-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-9g28r\" (UID: \"6050ad9f-817a-48e4-b939-12efa11a6a73\") " pod="openshift-ovn-kubernetes/ovnkube-node-9g28r" Jul 29 22:46:27.555571 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:27.553365 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6a64a07e-1c4a-4990-90b9-80623be37f60-system-cni-dir\") pod \"multus-2mwmf\" (UID: \"6a64a07e-1c4a-4990-90b9-80623be37f60\") " pod="openshift-multus/multus-2mwmf" Jul 29 22:46:27.555571 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:27.553429 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/6a64a07e-1c4a-4990-90b9-80623be37f60-hostroot\") pod \"multus-2mwmf\" (UID: \"6a64a07e-1c4a-4990-90b9-80623be37f60\") " pod="openshift-multus/multus-2mwmf" Jul 29 22:46:27.555571 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:27.553463 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6a64a07e-1c4a-4990-90b9-80623be37f60-etc-kubernetes\") pod \"multus-2mwmf\" (UID: \"6a64a07e-1c4a-4990-90b9-80623be37f60\") " pod="openshift-multus/multus-2mwmf" Jul 29 22:46:27.555571 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:27.553471 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6a64a07e-1c4a-4990-90b9-80623be37f60-system-cni-dir\") pod \"multus-2mwmf\" (UID: \"6a64a07e-1c4a-4990-90b9-80623be37f60\") " pod="openshift-multus/multus-2mwmf" Jul 29 22:46:27.555571 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:27.553489 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/d40a4cdf-f141-46d3-8159-986f680b2dec-cni-binary-copy\") pod \"multus-additional-cni-plugins-h6flc\" (UID: \"d40a4cdf-f141-46d3-8159-986f680b2dec\") " pod="openshift-multus/multus-additional-cni-plugins-h6flc" Jul 29 22:46:27.555571 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:27.553517 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cd64b\" (UniqueName: \"kubernetes.io/projected/d40a4cdf-f141-46d3-8159-986f680b2dec-kube-api-access-cd64b\") pod \"multus-additional-cni-plugins-h6flc\" (UID: \"d40a4cdf-f141-46d3-8159-986f680b2dec\") " pod="openshift-multus/multus-additional-cni-plugins-h6flc" Jul 29 22:46:27.555571 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:27.553520 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/6a64a07e-1c4a-4990-90b9-80623be37f60-hostroot\") pod \"multus-2mwmf\" (UID: \"6a64a07e-1c4a-4990-90b9-80623be37f60\") " pod="openshift-multus/multus-2mwmf" Jul 29 22:46:27.555571 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:27.553540 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/6c54a7c1-2837-47d6-88f4-dc4d264397d4-etc-selinux\") pod \"aws-ebs-csi-driver-node-kqh77\" (UID: \"6c54a7c1-2837-47d6-88f4-dc4d264397d4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kqh77" Jul 29 22:46:27.556175 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:27.553562 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6050ad9f-817a-48e4-b939-12efa11a6a73-var-lib-openvswitch\") pod \"ovnkube-node-9g28r\" (UID: \"6050ad9f-817a-48e4-b939-12efa11a6a73\") " pod="openshift-ovn-kubernetes/ovnkube-node-9g28r" Jul 29 22:46:27.556175 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:27.553738 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6a64a07e-1c4a-4990-90b9-80623be37f60-etc-kubernetes\") pod \"multus-2mwmf\" (UID: \"6a64a07e-1c4a-4990-90b9-80623be37f60\") " pod="openshift-multus/multus-2mwmf" Jul 29 22:46:27.556175 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:27.553805 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/6c54a7c1-2837-47d6-88f4-dc4d264397d4-etc-selinux\") pod \"aws-ebs-csi-driver-node-kqh77\" (UID: \"6c54a7c1-2837-47d6-88f4-dc4d264397d4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kqh77" Jul 29 22:46:27.556175 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:27.553831 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6050ad9f-817a-48e4-b939-12efa11a6a73-var-lib-openvswitch\") pod \"ovnkube-node-9g28r\" (UID: \"6050ad9f-817a-48e4-b939-12efa11a6a73\") " pod="openshift-ovn-kubernetes/ovnkube-node-9g28r" Jul 29 22:46:27.556175 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:27.553935 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/d40a4cdf-f141-46d3-8159-986f680b2dec-cni-binary-copy\") pod \"multus-additional-cni-plugins-h6flc\" (UID: \"d40a4cdf-f141-46d3-8159-986f680b2dec\") " pod="openshift-multus/multus-additional-cni-plugins-h6flc" Jul 29 22:46:27.556175 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:27.554017 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/6050ad9f-817a-48e4-b939-12efa11a6a73-ovnkube-config\") pod \"ovnkube-node-9g28r\" (UID: \"6050ad9f-817a-48e4-b939-12efa11a6a73\") " pod="openshift-ovn-kubernetes/ovnkube-node-9g28r" Jul 29 22:46:27.556175 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:27.554043 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/4c31fa1a-fda4-43d6-84ef-e43f0bafdcb2-run\") pod \"tuned-84tng\" (UID: \"4c31fa1a-fda4-43d6-84ef-e43f0bafdcb2\") " pod="openshift-cluster-node-tuning-operator/tuned-84tng" Jul 29 22:46:27.556175 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:27.554063 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/f6d8c777-cb9a-4505-8e30-ffaa01e53239-serviceca\") pod \"node-ca-vwtj2\" (UID: \"f6d8c777-cb9a-4505-8e30-ffaa01e53239\") " pod="openshift-image-registry/node-ca-vwtj2" Jul 29 22:46:27.556175 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:27.554085 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/6a64a07e-1c4a-4990-90b9-80623be37f60-cnibin\") pod \"multus-2mwmf\" (UID: \"6a64a07e-1c4a-4990-90b9-80623be37f60\") " pod="openshift-multus/multus-2mwmf" Jul 29 22:46:27.556175 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:27.554106 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/d40a4cdf-f141-46d3-8159-986f680b2dec-tuning-conf-dir\") pod \"multus-additional-cni-plugins-h6flc\" (UID: \"d40a4cdf-f141-46d3-8159-986f680b2dec\") " pod="openshift-multus/multus-additional-cni-plugins-h6flc" Jul 29 22:46:27.556175 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:27.554128 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/6050ad9f-817a-48e4-b939-12efa11a6a73-node-log\") pod \"ovnkube-node-9g28r\" (UID: \"6050ad9f-817a-48e4-b939-12efa11a6a73\") " pod="openshift-ovn-kubernetes/ovnkube-node-9g28r" Jul 29 22:46:27.556175 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:27.554150 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/6050ad9f-817a-48e4-b939-12efa11a6a73-log-socket\") pod \"ovnkube-node-9g28r\" (UID: \"6050ad9f-817a-48e4-b939-12efa11a6a73\") " pod="openshift-ovn-kubernetes/ovnkube-node-9g28r" Jul 29 22:46:27.556175 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:27.554173 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6a64a07e-1c4a-4990-90b9-80623be37f60-multus-cni-dir\") pod \"multus-2mwmf\" (UID: \"6a64a07e-1c4a-4990-90b9-80623be37f60\") " pod="openshift-multus/multus-2mwmf" Jul 29 22:46:27.556175 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:27.554194 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/6a64a07e-1c4a-4990-90b9-80623be37f60-os-release\") pod \"multus-2mwmf\" (UID: \"6a64a07e-1c4a-4990-90b9-80623be37f60\") " pod="openshift-multus/multus-2mwmf" Jul 29 22:46:27.556175 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:27.554214 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/6a64a07e-1c4a-4990-90b9-80623be37f60-host-var-lib-cni-bin\") pod \"multus-2mwmf\" (UID: \"6a64a07e-1c4a-4990-90b9-80623be37f60\") " pod="openshift-multus/multus-2mwmf" Jul 29 22:46:27.556175 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:27.554232 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/6050ad9f-817a-48e4-b939-12efa11a6a73-ovnkube-config\") pod \"ovnkube-node-9g28r\" (UID: \"6050ad9f-817a-48e4-b939-12efa11a6a73\") " pod="openshift-ovn-kubernetes/ovnkube-node-9g28r" Jul 29 22:46:27.556175 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:27.554240 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d40a4cdf-f141-46d3-8159-986f680b2dec-system-cni-dir\") pod \"multus-additional-cni-plugins-h6flc\" (UID: \"d40a4cdf-f141-46d3-8159-986f680b2dec\") " pod="openshift-multus/multus-additional-cni-plugins-h6flc" Jul 29 22:46:27.556175 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:27.554268 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f0a6baed-8bea-4992-8852-0b84808a78d1-metrics-certs\") pod \"network-metrics-daemon-fxb52\" (UID: \"f0a6baed-8bea-4992-8852-0b84808a78d1\") " pod="openshift-multus/network-metrics-daemon-fxb52" Jul 29 22:46:27.556821 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:27.554276 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/6050ad9f-817a-48e4-b939-12efa11a6a73-node-log\") pod \"ovnkube-node-9g28r\" (UID: \"6050ad9f-817a-48e4-b939-12efa11a6a73\") " pod="openshift-ovn-kubernetes/ovnkube-node-9g28r" Jul 29 22:46:27.556821 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:27.554294 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4c31fa1a-fda4-43d6-84ef-e43f0bafdcb2-sys\") pod \"tuned-84tng\" (UID: \"4c31fa1a-fda4-43d6-84ef-e43f0bafdcb2\") " pod="openshift-cluster-node-tuning-operator/tuned-84tng" Jul 29 22:46:27.556821 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:27.554316 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/4c31fa1a-fda4-43d6-84ef-e43f0bafdcb2-run\") pod \"tuned-84tng\" (UID: \"4c31fa1a-fda4-43d6-84ef-e43f0bafdcb2\") " pod="openshift-cluster-node-tuning-operator/tuned-84tng" Jul 29 22:46:27.556821 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:27.554318 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/f6d8c777-cb9a-4505-8e30-ffaa01e53239-host\") pod \"node-ca-vwtj2\" (UID: \"f6d8c777-cb9a-4505-8e30-ffaa01e53239\") " pod="openshift-image-registry/node-ca-vwtj2" Jul 29 22:46:27.556821 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:27.554350 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/6a64a07e-1c4a-4990-90b9-80623be37f60-multus-socket-dir-parent\") pod \"multus-2mwmf\" (UID: \"6a64a07e-1c4a-4990-90b9-80623be37f60\") " pod="openshift-multus/multus-2mwmf" Jul 29 22:46:27.556821 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:27.554361 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/f6d8c777-cb9a-4505-8e30-ffaa01e53239-host\") pod \"node-ca-vwtj2\" (UID: \"f6d8c777-cb9a-4505-8e30-ffaa01e53239\") " pod="openshift-image-registry/node-ca-vwtj2" Jul 29 22:46:27.556821 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:27.554492 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/6c54a7c1-2837-47d6-88f4-dc4d264397d4-socket-dir\") pod \"aws-ebs-csi-driver-node-kqh77\" (UID: \"6c54a7c1-2837-47d6-88f4-dc4d264397d4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kqh77" Jul 29 22:46:27.556821 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:27.554544 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/6050ad9f-817a-48e4-b939-12efa11a6a73-log-socket\") pod \"ovnkube-node-9g28r\" (UID: \"6050ad9f-817a-48e4-b939-12efa11a6a73\") " pod="openshift-ovn-kubernetes/ovnkube-node-9g28r" Jul 29 22:46:27.556821 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:27.554605 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6a64a07e-1c4a-4990-90b9-80623be37f60-multus-cni-dir\") pod \"multus-2mwmf\" (UID: \"6a64a07e-1c4a-4990-90b9-80623be37f60\") " pod="openshift-multus/multus-2mwmf" Jul 29 22:46:27.556821 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:27.554610 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/d40a4cdf-f141-46d3-8159-986f680b2dec-tuning-conf-dir\") pod \"multus-additional-cni-plugins-h6flc\" (UID: \"d40a4cdf-f141-46d3-8159-986f680b2dec\") " pod="openshift-multus/multus-additional-cni-plugins-h6flc" Jul 29 22:46:27.556821 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:27.554652 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d40a4cdf-f141-46d3-8159-986f680b2dec-system-cni-dir\") pod \"multus-additional-cni-plugins-h6flc\" (UID: \"d40a4cdf-f141-46d3-8159-986f680b2dec\") " pod="openshift-multus/multus-additional-cni-plugins-h6flc" Jul 29 22:46:27.556821 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:27.554656 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/6a64a07e-1c4a-4990-90b9-80623be37f60-host-var-lib-cni-bin\") pod \"multus-2mwmf\" (UID: \"6a64a07e-1c4a-4990-90b9-80623be37f60\") " pod="openshift-multus/multus-2mwmf" Jul 29 22:46:27.556821 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:27.554659 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/f6d8c777-cb9a-4505-8e30-ffaa01e53239-serviceca\") pod \"node-ca-vwtj2\" (UID: \"f6d8c777-cb9a-4505-8e30-ffaa01e53239\") " pod="openshift-image-registry/node-ca-vwtj2" Jul 29 22:46:27.556821 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:27.554695 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/6a64a07e-1c4a-4990-90b9-80623be37f60-multus-socket-dir-parent\") pod \"multus-2mwmf\" (UID: \"6a64a07e-1c4a-4990-90b9-80623be37f60\") " pod="openshift-multus/multus-2mwmf" Jul 29 22:46:27.556821 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:27.554366 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/6c54a7c1-2837-47d6-88f4-dc4d264397d4-socket-dir\") pod \"aws-ebs-csi-driver-node-kqh77\" (UID: \"6c54a7c1-2837-47d6-88f4-dc4d264397d4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kqh77" Jul 29 22:46:27.556821 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:27.554710 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4c31fa1a-fda4-43d6-84ef-e43f0bafdcb2-sys\") pod \"tuned-84tng\" (UID: \"4c31fa1a-fda4-43d6-84ef-e43f0bafdcb2\") " pod="openshift-cluster-node-tuning-operator/tuned-84tng" Jul 29 22:46:27.556821 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:27.554721 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/6a64a07e-1c4a-4990-90b9-80623be37f60-cnibin\") pod \"multus-2mwmf\" (UID: \"6a64a07e-1c4a-4990-90b9-80623be37f60\") " pod="openshift-multus/multus-2mwmf" Jul 29 22:46:27.556821 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:27.554755 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/6a64a07e-1c4a-4990-90b9-80623be37f60-os-release\") pod \"multus-2mwmf\" (UID: \"6a64a07e-1c4a-4990-90b9-80623be37f60\") " pod="openshift-multus/multus-2mwmf" Jul 29 22:46:27.557583 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:27.555286 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4c31fa1a-fda4-43d6-84ef-e43f0bafdcb2-tmp\") pod \"tuned-84tng\" (UID: \"4c31fa1a-fda4-43d6-84ef-e43f0bafdcb2\") " pod="openshift-cluster-node-tuning-operator/tuned-84tng" Jul 29 22:46:27.557583 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:27.555338 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/4c31fa1a-fda4-43d6-84ef-e43f0bafdcb2-etc-tuned\") pod \"tuned-84tng\" (UID: \"4c31fa1a-fda4-43d6-84ef-e43f0bafdcb2\") " pod="openshift-cluster-node-tuning-operator/tuned-84tng" Jul 29 22:46:27.557583 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:27.555695 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/6050ad9f-817a-48e4-b939-12efa11a6a73-ovn-node-metrics-cert\") pod \"ovnkube-node-9g28r\" (UID: \"6050ad9f-817a-48e4-b939-12efa11a6a73\") " pod="openshift-ovn-kubernetes/ovnkube-node-9g28r" Jul 29 22:46:27.559703 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:27.559682 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h8vm9\" (UniqueName: \"kubernetes.io/projected/5ce0a47c-e715-472b-a0d9-761a5a6777dd-kube-api-access-h8vm9\") pod \"iptables-alerter-frd9g\" (UID: \"5ce0a47c-e715-472b-a0d9-761a5a6777dd\") " pod="openshift-network-operator/iptables-alerter-frd9g" Jul 29 22:46:27.560789 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:27.560749 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9kjfg\" (UniqueName: \"kubernetes.io/projected/f6d8c777-cb9a-4505-8e30-ffaa01e53239-kube-api-access-9kjfg\") pod \"node-ca-vwtj2\" (UID: \"f6d8c777-cb9a-4505-8e30-ffaa01e53239\") " pod="openshift-image-registry/node-ca-vwtj2" Jul 29 22:46:27.561279 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:27.561261 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pmz5r\" (UniqueName: \"kubernetes.io/projected/6a64a07e-1c4a-4990-90b9-80623be37f60-kube-api-access-pmz5r\") pod \"multus-2mwmf\" (UID: \"6a64a07e-1c4a-4990-90b9-80623be37f60\") " pod="openshift-multus/multus-2mwmf" Jul 29 22:46:27.562106 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:27.562084 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4rf4l\" (UniqueName: \"kubernetes.io/projected/6c54a7c1-2837-47d6-88f4-dc4d264397d4-kube-api-access-4rf4l\") pod \"aws-ebs-csi-driver-node-kqh77\" (UID: \"6c54a7c1-2837-47d6-88f4-dc4d264397d4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kqh77" Jul 29 22:46:27.562388 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:27.562357 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cd64b\" (UniqueName: \"kubernetes.io/projected/d40a4cdf-f141-46d3-8159-986f680b2dec-kube-api-access-cd64b\") pod \"multus-additional-cni-plugins-h6flc\" (UID: \"d40a4cdf-f141-46d3-8159-986f680b2dec\") " pod="openshift-multus/multus-additional-cni-plugins-h6flc" Jul 29 22:46:27.562567 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:27.562542 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5262s\" (UniqueName: \"kubernetes.io/projected/6050ad9f-817a-48e4-b939-12efa11a6a73-kube-api-access-5262s\") pod \"ovnkube-node-9g28r\" (UID: \"6050ad9f-817a-48e4-b939-12efa11a6a73\") " pod="openshift-ovn-kubernetes/ovnkube-node-9g28r" Jul 29 22:46:27.562682 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:27.562573 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vkrlb\" (UniqueName: \"kubernetes.io/projected/4c31fa1a-fda4-43d6-84ef-e43f0bafdcb2-kube-api-access-vkrlb\") pod \"tuned-84tng\" (UID: \"4c31fa1a-fda4-43d6-84ef-e43f0bafdcb2\") " pod="openshift-cluster-node-tuning-operator/tuned-84tng" Jul 29 22:46:27.655493 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:27.655457 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/51d8cffe-16b4-4bab-9193-953163e45417-hosts-file\") pod \"node-resolver-k8fl9\" (UID: \"51d8cffe-16b4-4bab-9193-953163e45417\") " pod="openshift-dns/node-resolver-k8fl9" Jul 29 22:46:27.655493 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:27.655500 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/51d8cffe-16b4-4bab-9193-953163e45417-tmp-dir\") pod \"node-resolver-k8fl9\" (UID: \"51d8cffe-16b4-4bab-9193-953163e45417\") " pod="openshift-dns/node-resolver-k8fl9" Jul 29 22:46:27.655754 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:27.655555 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5kv55\" (UniqueName: \"kubernetes.io/projected/2771bdbc-d054-464b-a01f-82344308451f-kube-api-access-5kv55\") pod \"network-check-target-z92kj\" (UID: \"2771bdbc-d054-464b-a01f-82344308451f\") " pod="openshift-network-diagnostics/network-check-target-z92kj" Jul 29 22:46:27.655754 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:27.655582 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sqxph\" (UniqueName: \"kubernetes.io/projected/51d8cffe-16b4-4bab-9193-953163e45417-kube-api-access-sqxph\") pod \"node-resolver-k8fl9\" (UID: \"51d8cffe-16b4-4bab-9193-953163e45417\") " pod="openshift-dns/node-resolver-k8fl9" Jul 29 22:46:27.655754 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:27.655603 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8dhhj\" (UniqueName: \"kubernetes.io/projected/f0a6baed-8bea-4992-8852-0b84808a78d1-kube-api-access-8dhhj\") pod \"network-metrics-daemon-fxb52\" (UID: \"f0a6baed-8bea-4992-8852-0b84808a78d1\") " pod="openshift-multus/network-metrics-daemon-fxb52" Jul 29 22:46:27.655754 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:27.655623 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/ece97cfa-80dc-4fa8-ad31-61ba5c38aee3-konnectivity-ca\") pod \"konnectivity-agent-k9nmc\" (UID: \"ece97cfa-80dc-4fa8-ad31-61ba5c38aee3\") " pod="kube-system/konnectivity-agent-k9nmc" Jul 29 22:46:27.655754 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:27.655662 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f0a6baed-8bea-4992-8852-0b84808a78d1-metrics-certs\") pod \"network-metrics-daemon-fxb52\" (UID: \"f0a6baed-8bea-4992-8852-0b84808a78d1\") " pod="openshift-multus/network-metrics-daemon-fxb52" Jul 29 22:46:27.655754 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:27.655698 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/ece97cfa-80dc-4fa8-ad31-61ba5c38aee3-agent-certs\") pod \"konnectivity-agent-k9nmc\" (UID: \"ece97cfa-80dc-4fa8-ad31-61ba5c38aee3\") " pod="kube-system/konnectivity-agent-k9nmc" Jul 29 22:46:27.655754 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:27.655617 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/51d8cffe-16b4-4bab-9193-953163e45417-hosts-file\") pod \"node-resolver-k8fl9\" (UID: \"51d8cffe-16b4-4bab-9193-953163e45417\") " pod="openshift-dns/node-resolver-k8fl9" Jul 29 22:46:27.656081 ip-10-0-138-112 kubenswrapper[2562]: E0729 22:46:27.655877 2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 29 22:46:27.656081 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:27.655919 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/51d8cffe-16b4-4bab-9193-953163e45417-tmp-dir\") pod \"node-resolver-k8fl9\" (UID: \"51d8cffe-16b4-4bab-9193-953163e45417\") " pod="openshift-dns/node-resolver-k8fl9" Jul 29 22:46:27.656081 ip-10-0-138-112 kubenswrapper[2562]: E0729 22:46:27.655962 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f0a6baed-8bea-4992-8852-0b84808a78d1-metrics-certs podName:f0a6baed-8bea-4992-8852-0b84808a78d1 nodeName:}" failed. No retries permitted until 2026-07-29 22:46:28.155938117 +0000 UTC m=+3.067763783 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f0a6baed-8bea-4992-8852-0b84808a78d1-metrics-certs") pod "network-metrics-daemon-fxb52" (UID: "f0a6baed-8bea-4992-8852-0b84808a78d1") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 29 22:46:27.656318 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:27.656281 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/ece97cfa-80dc-4fa8-ad31-61ba5c38aee3-konnectivity-ca\") pod \"konnectivity-agent-k9nmc\" (UID: \"ece97cfa-80dc-4fa8-ad31-61ba5c38aee3\") " pod="kube-system/konnectivity-agent-k9nmc" Jul 29 22:46:27.658290 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:27.658261 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/ece97cfa-80dc-4fa8-ad31-61ba5c38aee3-agent-certs\") pod \"konnectivity-agent-k9nmc\" (UID: \"ece97cfa-80dc-4fa8-ad31-61ba5c38aee3\") " pod="kube-system/konnectivity-agent-k9nmc" Jul 29 22:46:27.665012 ip-10-0-138-112 kubenswrapper[2562]: E0729 22:46:27.664975 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 29 22:46:27.665012 ip-10-0-138-112 kubenswrapper[2562]: E0729 22:46:27.665003 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 29 22:46:27.665012 ip-10-0-138-112 kubenswrapper[2562]: E0729 22:46:27.665017 2562 projected.go:196] Error preparing data for projected volume kube-api-access-5kv55 for pod openshift-network-diagnostics/network-check-target-z92kj: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 29 22:46:27.665265 ip-10-0-138-112 kubenswrapper[2562]: E0729 22:46:27.665097 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2771bdbc-d054-464b-a01f-82344308451f-kube-api-access-5kv55 podName:2771bdbc-d054-464b-a01f-82344308451f nodeName:}" failed. No retries permitted until 2026-07-29 22:46:28.165076662 +0000 UTC m=+3.076902343 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-5kv55" (UniqueName: "kubernetes.io/projected/2771bdbc-d054-464b-a01f-82344308451f-kube-api-access-5kv55") pod "network-check-target-z92kj" (UID: "2771bdbc-d054-464b-a01f-82344308451f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 29 22:46:27.667528 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:27.667503 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8dhhj\" (UniqueName: \"kubernetes.io/projected/f0a6baed-8bea-4992-8852-0b84808a78d1-kube-api-access-8dhhj\") pod \"network-metrics-daemon-fxb52\" (UID: \"f0a6baed-8bea-4992-8852-0b84808a78d1\") " pod="openshift-multus/network-metrics-daemon-fxb52" Jul 29 22:46:27.667650 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:27.667609 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sqxph\" (UniqueName: \"kubernetes.io/projected/51d8cffe-16b4-4bab-9193-953163e45417-kube-api-access-sqxph\") pod \"node-resolver-k8fl9\" (UID: \"51d8cffe-16b4-4bab-9193-953163e45417\") " pod="openshift-dns/node-resolver-k8fl9" Jul 29 22:46:27.740566 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:27.740482 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-84tng" Jul 29 22:46:27.748439 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:27.748412 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-frd9g" Jul 29 22:46:27.758341 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:27.758316 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-vwtj2" Jul 29 22:46:27.764230 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:27.764207 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-2mwmf" Jul 29 22:46:27.772067 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:27.772043 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-h6flc" Jul 29 22:46:27.780581 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:27.780558 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kqh77" Jul 29 22:46:27.787238 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:27.787218 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-9g28r" Jul 29 22:46:27.794893 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:27.794872 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-k8fl9" Jul 29 22:46:27.801718 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:27.801698 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-k9nmc" Jul 29 22:46:28.058284 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:28.058200 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/916b3077-78fa-4054-921c-2e7124dc35f8-original-pull-secret\") pod \"global-pull-secret-syncer-nkzq5\" (UID: \"916b3077-78fa-4054-921c-2e7124dc35f8\") " pod="kube-system/global-pull-secret-syncer-nkzq5" Jul 29 22:46:28.058456 ip-10-0-138-112 kubenswrapper[2562]: E0729 22:46:28.058334 2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 29 22:46:28.058456 ip-10-0-138-112 kubenswrapper[2562]: E0729 22:46:28.058424 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/916b3077-78fa-4054-921c-2e7124dc35f8-original-pull-secret podName:916b3077-78fa-4054-921c-2e7124dc35f8 nodeName:}" failed. No retries permitted until 2026-07-29 22:46:29.058390091 +0000 UTC m=+3.970215762 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/916b3077-78fa-4054-921c-2e7124dc35f8-original-pull-secret") pod "global-pull-secret-syncer-nkzq5" (UID: "916b3077-78fa-4054-921c-2e7124dc35f8") : object "kube-system"/"original-pull-secret" not registered Jul 29 22:46:28.159491 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:28.159452 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f0a6baed-8bea-4992-8852-0b84808a78d1-metrics-certs\") pod \"network-metrics-daemon-fxb52\" (UID: \"f0a6baed-8bea-4992-8852-0b84808a78d1\") " pod="openshift-multus/network-metrics-daemon-fxb52" Jul 29 22:46:28.159670 ip-10-0-138-112 kubenswrapper[2562]: E0729 22:46:28.159607 2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 29 22:46:28.159716 ip-10-0-138-112 kubenswrapper[2562]: E0729 22:46:28.159680 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f0a6baed-8bea-4992-8852-0b84808a78d1-metrics-certs podName:f0a6baed-8bea-4992-8852-0b84808a78d1 nodeName:}" failed. No retries permitted until 2026-07-29 22:46:29.15966172 +0000 UTC m=+4.071487388 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f0a6baed-8bea-4992-8852-0b84808a78d1-metrics-certs") pod "network-metrics-daemon-fxb52" (UID: "f0a6baed-8bea-4992-8852-0b84808a78d1") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 29 22:46:28.260574 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:28.260532 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5kv55\" (UniqueName: \"kubernetes.io/projected/2771bdbc-d054-464b-a01f-82344308451f-kube-api-access-5kv55\") pod \"network-check-target-z92kj\" (UID: \"2771bdbc-d054-464b-a01f-82344308451f\") " pod="openshift-network-diagnostics/network-check-target-z92kj" Jul 29 22:46:28.260766 ip-10-0-138-112 kubenswrapper[2562]: E0729 22:46:28.260725 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 29 22:46:28.260766 ip-10-0-138-112 kubenswrapper[2562]: E0729 22:46:28.260747 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 29 22:46:28.260766 ip-10-0-138-112 kubenswrapper[2562]: E0729 22:46:28.260757 2562 projected.go:196] Error preparing data for projected volume kube-api-access-5kv55 for pod openshift-network-diagnostics/network-check-target-z92kj: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 29 22:46:28.260916 ip-10-0-138-112 kubenswrapper[2562]: E0729 22:46:28.260814 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2771bdbc-d054-464b-a01f-82344308451f-kube-api-access-5kv55 podName:2771bdbc-d054-464b-a01f-82344308451f nodeName:}" failed. No retries permitted until 2026-07-29 22:46:29.260794564 +0000 UTC m=+4.172620235 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-5kv55" (UniqueName: "kubernetes.io/projected/2771bdbc-d054-464b-a01f-82344308451f-kube-api-access-5kv55") pod "network-check-target-z92kj" (UID: "2771bdbc-d054-464b-a01f-82344308451f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 29 22:46:28.429455 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:28.429387 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6c54a7c1_2837_47d6_88f4_dc4d264397d4.slice/crio-ffa92d4f81e50c1d80b61d11c4ffe1bb5932f9d38e7165421ac72b191cb04bbd WatchSource:0}: Error finding container ffa92d4f81e50c1d80b61d11c4ffe1bb5932f9d38e7165421ac72b191cb04bbd: Status 404 returned error can't find the container with id ffa92d4f81e50c1d80b61d11c4ffe1bb5932f9d38e7165421ac72b191cb04bbd Jul 29 22:46:28.430912 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:28.430883 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4c31fa1a_fda4_43d6_84ef_e43f0bafdcb2.slice/crio-39f6452525c4fda689334ae9dbd877c49a7e960dee23d4644cebcfdd4225fc74 WatchSource:0}: Error finding container 39f6452525c4fda689334ae9dbd877c49a7e960dee23d4644cebcfdd4225fc74: Status 404 returned error can't find the container with id 39f6452525c4fda689334ae9dbd877c49a7e960dee23d4644cebcfdd4225fc74 Jul 29 22:46:28.431951 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:28.431833 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd40a4cdf_f141_46d3_8159_986f680b2dec.slice/crio-5881eddaddaf618fd3a8f3276ce09f63183a416ab6ddffb2697b303270f852c1 WatchSource:0}: Error finding container 5881eddaddaf618fd3a8f3276ce09f63183a416ab6ddffb2697b303270f852c1: Status 404 returned error can't find the container with id 5881eddaddaf618fd3a8f3276ce09f63183a416ab6ddffb2697b303270f852c1 Jul 29 22:46:28.434220 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:28.434192 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podece97cfa_80dc_4fa8_ad31_61ba5c38aee3.slice/crio-e3334919a1cd4e95c03e5c0a8002d94820ccb888b879db77574df24749da7af0 WatchSource:0}: Error finding container e3334919a1cd4e95c03e5c0a8002d94820ccb888b879db77574df24749da7af0: Status 404 returned error can't find the container with id e3334919a1cd4e95c03e5c0a8002d94820ccb888b879db77574df24749da7af0 Jul 29 22:46:28.435291 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:28.435268 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6a64a07e_1c4a_4990_90b9_80623be37f60.slice/crio-20fad7ba9c8f5b2ecda4f7cc154e311788842bc23c3450a3b9fb09579cd0eb17 WatchSource:0}: Error finding container 20fad7ba9c8f5b2ecda4f7cc154e311788842bc23c3450a3b9fb09579cd0eb17: Status 404 returned error can't find the container with id 20fad7ba9c8f5b2ecda4f7cc154e311788842bc23c3450a3b9fb09579cd0eb17 Jul 29 22:46:28.437092 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:28.437065 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod51d8cffe_16b4_4bab_9193_953163e45417.slice/crio-42b9364a6e0c40020fae77d931ea833ceb1da59e3719334d4179119b190ce44b WatchSource:0}: Error finding container 42b9364a6e0c40020fae77d931ea833ceb1da59e3719334d4179119b190ce44b: Status 404 returned error can't find the container with id 42b9364a6e0c40020fae77d931ea833ceb1da59e3719334d4179119b190ce44b Jul 29 22:46:28.438303 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:28.437990 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6050ad9f_817a_48e4_b939_12efa11a6a73.slice/crio-491cafe067d588a054e5689f0844ca26c248b02ade587d5ef09a87a4b176737f WatchSource:0}: Error finding container 491cafe067d588a054e5689f0844ca26c248b02ade587d5ef09a87a4b176737f: Status 404 returned error can't find the container with id 491cafe067d588a054e5689f0844ca26c248b02ade587d5ef09a87a4b176737f Jul 29 22:46:28.439202 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:28.438938 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5ce0a47c_e715_472b_a0d9_761a5a6777dd.slice/crio-d206c628699d833ac56b7d96e9e5cffd2a26cad1f69382c2f076b52cbbe8bc93 WatchSource:0}: Error finding container d206c628699d833ac56b7d96e9e5cffd2a26cad1f69382c2f076b52cbbe8bc93: Status 404 returned error can't find the container with id d206c628699d833ac56b7d96e9e5cffd2a26cad1f69382c2f076b52cbbe8bc93 Jul 29 22:46:28.439872 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:46:28.439847 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf6d8c777_cb9a_4505_8e30_ffaa01e53239.slice/crio-6128a201335e903c59941be8b06693cf81d7b2f243856d7e2ed9d61e0df6f2d1 WatchSource:0}: Error finding container 6128a201335e903c59941be8b06693cf81d7b2f243856d7e2ed9d61e0df6f2d1: Status 404 returned error can't find the container with id 6128a201335e903c59941be8b06693cf81d7b2f243856d7e2ed9d61e0df6f2d1 Jul 29 22:46:28.483352 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:28.483321 2562 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-28 22:41:27 +0000 UTC" deadline="2028-03-28 16:08:31.307618604 +0000 UTC" Jul 29 22:46:28.483352 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:28.483350 2562 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14585h22m2.824272298s" Jul 29 22:46:28.570530 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:28.570495 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fxb52" Jul 29 22:46:28.570683 ip-10-0-138-112 kubenswrapper[2562]: E0729 22:46:28.570613 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fxb52" podUID="f0a6baed-8bea-4992-8852-0b84808a78d1" Jul 29 22:46:28.579820 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:28.579781 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-vwtj2" event={"ID":"f6d8c777-cb9a-4505-8e30-ffaa01e53239","Type":"ContainerStarted","Data":"6128a201335e903c59941be8b06693cf81d7b2f243856d7e2ed9d61e0df6f2d1"} Jul 29 22:46:28.580803 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:28.580770 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-frd9g" event={"ID":"5ce0a47c-e715-472b-a0d9-761a5a6777dd","Type":"ContainerStarted","Data":"d206c628699d833ac56b7d96e9e5cffd2a26cad1f69382c2f076b52cbbe8bc93"} Jul 29 22:46:28.581829 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:28.581804 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-k8fl9" event={"ID":"51d8cffe-16b4-4bab-9193-953163e45417","Type":"ContainerStarted","Data":"42b9364a6e0c40020fae77d931ea833ceb1da59e3719334d4179119b190ce44b"} Jul 29 22:46:28.582799 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:28.582776 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-k9nmc" event={"ID":"ece97cfa-80dc-4fa8-ad31-61ba5c38aee3","Type":"ContainerStarted","Data":"e3334919a1cd4e95c03e5c0a8002d94820ccb888b879db77574df24749da7af0"} Jul 29 22:46:28.583821 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:28.583799 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-84tng" event={"ID":"4c31fa1a-fda4-43d6-84ef-e43f0bafdcb2","Type":"ContainerStarted","Data":"39f6452525c4fda689334ae9dbd877c49a7e960dee23d4644cebcfdd4225fc74"} Jul 29 22:46:28.585544 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:28.585516 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-112.ec2.internal" event={"ID":"5376269631e55085670dee0434ce97e1","Type":"ContainerStarted","Data":"c13269180f661a38038bdc919b67eefbf246b7104e75828f125946a162e3a369"} Jul 29 22:46:28.586785 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:28.586757 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9g28r" event={"ID":"6050ad9f-817a-48e4-b939-12efa11a6a73","Type":"ContainerStarted","Data":"491cafe067d588a054e5689f0844ca26c248b02ade587d5ef09a87a4b176737f"} Jul 29 22:46:28.587978 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:28.587705 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-2mwmf" event={"ID":"6a64a07e-1c4a-4990-90b9-80623be37f60","Type":"ContainerStarted","Data":"20fad7ba9c8f5b2ecda4f7cc154e311788842bc23c3450a3b9fb09579cd0eb17"} Jul 29 22:46:28.588957 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:28.588936 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-h6flc" event={"ID":"d40a4cdf-f141-46d3-8159-986f680b2dec","Type":"ContainerStarted","Data":"5881eddaddaf618fd3a8f3276ce09f63183a416ab6ddffb2697b303270f852c1"} Jul 29 22:46:28.590149 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:28.590126 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kqh77" event={"ID":"6c54a7c1-2837-47d6-88f4-dc4d264397d4","Type":"ContainerStarted","Data":"ffa92d4f81e50c1d80b61d11c4ffe1bb5932f9d38e7165421ac72b191cb04bbd"} Jul 29 22:46:28.600520 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:28.600475 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-112.ec2.internal" podStartSLOduration=2.600461508 podStartE2EDuration="2.600461508s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 22:46:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 22:46:28.600366632 +0000 UTC m=+3.512192322" watchObservedRunningTime="2026-07-29 22:46:28.600461508 +0000 UTC m=+3.512287195" Jul 29 22:46:29.067208 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:29.067173 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/916b3077-78fa-4054-921c-2e7124dc35f8-original-pull-secret\") pod \"global-pull-secret-syncer-nkzq5\" (UID: \"916b3077-78fa-4054-921c-2e7124dc35f8\") " pod="kube-system/global-pull-secret-syncer-nkzq5" Jul 29 22:46:29.067380 ip-10-0-138-112 kubenswrapper[2562]: E0729 22:46:29.067336 2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 29 22:46:29.067480 ip-10-0-138-112 kubenswrapper[2562]: E0729 22:46:29.067414 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/916b3077-78fa-4054-921c-2e7124dc35f8-original-pull-secret podName:916b3077-78fa-4054-921c-2e7124dc35f8 nodeName:}" failed. No retries permitted until 2026-07-29 22:46:31.067381297 +0000 UTC m=+5.979206977 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/916b3077-78fa-4054-921c-2e7124dc35f8-original-pull-secret") pod "global-pull-secret-syncer-nkzq5" (UID: "916b3077-78fa-4054-921c-2e7124dc35f8") : object "kube-system"/"original-pull-secret" not registered Jul 29 22:46:29.168086 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:29.168046 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f0a6baed-8bea-4992-8852-0b84808a78d1-metrics-certs\") pod \"network-metrics-daemon-fxb52\" (UID: \"f0a6baed-8bea-4992-8852-0b84808a78d1\") " pod="openshift-multus/network-metrics-daemon-fxb52" Jul 29 22:46:29.168267 ip-10-0-138-112 kubenswrapper[2562]: E0729 22:46:29.168237 2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 29 22:46:29.168324 ip-10-0-138-112 kubenswrapper[2562]: E0729 22:46:29.168300 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f0a6baed-8bea-4992-8852-0b84808a78d1-metrics-certs podName:f0a6baed-8bea-4992-8852-0b84808a78d1 nodeName:}" failed. No retries permitted until 2026-07-29 22:46:31.168281111 +0000 UTC m=+6.080106783 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f0a6baed-8bea-4992-8852-0b84808a78d1-metrics-certs") pod "network-metrics-daemon-fxb52" (UID: "f0a6baed-8bea-4992-8852-0b84808a78d1") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 29 22:46:29.268588 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:29.268541 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5kv55\" (UniqueName: \"kubernetes.io/projected/2771bdbc-d054-464b-a01f-82344308451f-kube-api-access-5kv55\") pod \"network-check-target-z92kj\" (UID: \"2771bdbc-d054-464b-a01f-82344308451f\") " pod="openshift-network-diagnostics/network-check-target-z92kj" Jul 29 22:46:29.268756 ip-10-0-138-112 kubenswrapper[2562]: E0729 22:46:29.268721 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 29 22:46:29.268756 ip-10-0-138-112 kubenswrapper[2562]: E0729 22:46:29.268741 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 29 22:46:29.268756 ip-10-0-138-112 kubenswrapper[2562]: E0729 22:46:29.268754 2562 projected.go:196] Error preparing data for projected volume kube-api-access-5kv55 for pod openshift-network-diagnostics/network-check-target-z92kj: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 29 22:46:29.268945 ip-10-0-138-112 kubenswrapper[2562]: E0729 22:46:29.268814 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2771bdbc-d054-464b-a01f-82344308451f-kube-api-access-5kv55 podName:2771bdbc-d054-464b-a01f-82344308451f nodeName:}" failed. No retries permitted until 2026-07-29 22:46:31.268795571 +0000 UTC m=+6.180621239 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-5kv55" (UniqueName: "kubernetes.io/projected/2771bdbc-d054-464b-a01f-82344308451f-kube-api-access-5kv55") pod "network-check-target-z92kj" (UID: "2771bdbc-d054-464b-a01f-82344308451f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 29 22:46:29.484518 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:29.484345 2562 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-28 22:41:27 +0000 UTC" deadline="2028-03-06 11:29:12.687252933 +0000 UTC" Jul 29 22:46:29.484518 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:29.484420 2562 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14052h42m43.202836693s" Jul 29 22:46:29.573572 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:29.572591 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z92kj" Jul 29 22:46:29.573572 ip-10-0-138-112 kubenswrapper[2562]: E0729 22:46:29.572724 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z92kj" podUID="2771bdbc-d054-464b-a01f-82344308451f" Jul 29 22:46:29.573572 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:29.572802 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nkzq5" Jul 29 22:46:29.573572 ip-10-0-138-112 kubenswrapper[2562]: E0729 22:46:29.572875 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nkzq5" podUID="916b3077-78fa-4054-921c-2e7124dc35f8" Jul 29 22:46:29.608258 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:29.608219 2562 generic.go:356] "Generic (PLEG): container finished" podID="b8710411dd4204d4793d631cc5fc044b" containerID="7e76a96ca752aaa2aefe039c07261b53dd7e9eaeadadb4b4e4fc7ea0cef25406" exitCode=0 Jul 29 22:46:29.609297 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:29.609265 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-112.ec2.internal" event={"ID":"b8710411dd4204d4793d631cc5fc044b","Type":"ContainerDied","Data":"7e76a96ca752aaa2aefe039c07261b53dd7e9eaeadadb4b4e4fc7ea0cef25406"} Jul 29 22:46:30.570776 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:30.570741 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fxb52" Jul 29 22:46:30.571226 ip-10-0-138-112 kubenswrapper[2562]: E0729 22:46:30.570889 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fxb52" podUID="f0a6baed-8bea-4992-8852-0b84808a78d1" Jul 29 22:46:30.618375 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:30.617653 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-112.ec2.internal" event={"ID":"b8710411dd4204d4793d631cc5fc044b","Type":"ContainerStarted","Data":"39723ecb45dc1841b37c3bae9d04834abe9b61608212b7edda33618707ce28d2"} Jul 29 22:46:30.636283 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:30.636224 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-112.ec2.internal" podStartSLOduration=4.636201626 podStartE2EDuration="4.636201626s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 22:46:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 22:46:30.635901457 +0000 UTC m=+5.547727148" watchObservedRunningTime="2026-07-29 22:46:30.636201626 +0000 UTC m=+5.548027316" Jul 29 22:46:31.086791 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:31.086700 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/916b3077-78fa-4054-921c-2e7124dc35f8-original-pull-secret\") pod \"global-pull-secret-syncer-nkzq5\" (UID: \"916b3077-78fa-4054-921c-2e7124dc35f8\") " pod="kube-system/global-pull-secret-syncer-nkzq5" Jul 29 22:46:31.087000 ip-10-0-138-112 kubenswrapper[2562]: E0729 22:46:31.086866 2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 29 22:46:31.087000 ip-10-0-138-112 kubenswrapper[2562]: E0729 22:46:31.086929 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/916b3077-78fa-4054-921c-2e7124dc35f8-original-pull-secret podName:916b3077-78fa-4054-921c-2e7124dc35f8 nodeName:}" failed. No retries permitted until 2026-07-29 22:46:35.086909866 +0000 UTC m=+9.998735536 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/916b3077-78fa-4054-921c-2e7124dc35f8-original-pull-secret") pod "global-pull-secret-syncer-nkzq5" (UID: "916b3077-78fa-4054-921c-2e7124dc35f8") : object "kube-system"/"original-pull-secret" not registered Jul 29 22:46:31.187384 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:31.187343 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f0a6baed-8bea-4992-8852-0b84808a78d1-metrics-certs\") pod \"network-metrics-daemon-fxb52\" (UID: \"f0a6baed-8bea-4992-8852-0b84808a78d1\") " pod="openshift-multus/network-metrics-daemon-fxb52" Jul 29 22:46:31.187929 ip-10-0-138-112 kubenswrapper[2562]: E0729 22:46:31.187902 2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 29 22:46:31.188027 ip-10-0-138-112 kubenswrapper[2562]: E0729 22:46:31.188011 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f0a6baed-8bea-4992-8852-0b84808a78d1-metrics-certs podName:f0a6baed-8bea-4992-8852-0b84808a78d1 nodeName:}" failed. No retries permitted until 2026-07-29 22:46:35.187988821 +0000 UTC m=+10.099814490 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f0a6baed-8bea-4992-8852-0b84808a78d1-metrics-certs") pod "network-metrics-daemon-fxb52" (UID: "f0a6baed-8bea-4992-8852-0b84808a78d1") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 29 22:46:31.288117 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:31.288074 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5kv55\" (UniqueName: \"kubernetes.io/projected/2771bdbc-d054-464b-a01f-82344308451f-kube-api-access-5kv55\") pod \"network-check-target-z92kj\" (UID: \"2771bdbc-d054-464b-a01f-82344308451f\") " pod="openshift-network-diagnostics/network-check-target-z92kj" Jul 29 22:46:31.288317 ip-10-0-138-112 kubenswrapper[2562]: E0729 22:46:31.288300 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 29 22:46:31.288376 ip-10-0-138-112 kubenswrapper[2562]: E0729 22:46:31.288326 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 29 22:46:31.288376 ip-10-0-138-112 kubenswrapper[2562]: E0729 22:46:31.288337 2562 projected.go:196] Error preparing data for projected volume kube-api-access-5kv55 for pod openshift-network-diagnostics/network-check-target-z92kj: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 29 22:46:31.288510 ip-10-0-138-112 kubenswrapper[2562]: E0729 22:46:31.288430 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2771bdbc-d054-464b-a01f-82344308451f-kube-api-access-5kv55 podName:2771bdbc-d054-464b-a01f-82344308451f nodeName:}" failed. No retries permitted until 2026-07-29 22:46:35.288410155 +0000 UTC m=+10.200235835 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-5kv55" (UniqueName: "kubernetes.io/projected/2771bdbc-d054-464b-a01f-82344308451f-kube-api-access-5kv55") pod "network-check-target-z92kj" (UID: "2771bdbc-d054-464b-a01f-82344308451f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 29 22:46:31.571132 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:31.571019 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z92kj" Jul 29 22:46:31.571637 ip-10-0-138-112 kubenswrapper[2562]: E0729 22:46:31.571144 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z92kj" podUID="2771bdbc-d054-464b-a01f-82344308451f" Jul 29 22:46:31.571637 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:31.571492 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nkzq5" Jul 29 22:46:31.571637 ip-10-0-138-112 kubenswrapper[2562]: E0729 22:46:31.571612 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nkzq5" podUID="916b3077-78fa-4054-921c-2e7124dc35f8" Jul 29 22:46:32.570264 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:32.570216 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fxb52" Jul 29 22:46:32.570555 ip-10-0-138-112 kubenswrapper[2562]: E0729 22:46:32.570372 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fxb52" podUID="f0a6baed-8bea-4992-8852-0b84808a78d1" Jul 29 22:46:33.571122 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:33.570506 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z92kj" Jul 29 22:46:33.571122 ip-10-0-138-112 kubenswrapper[2562]: E0729 22:46:33.570644 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z92kj" podUID="2771bdbc-d054-464b-a01f-82344308451f" Jul 29 22:46:33.571122 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:33.570985 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nkzq5" Jul 29 22:46:33.571122 ip-10-0-138-112 kubenswrapper[2562]: E0729 22:46:33.571081 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nkzq5" podUID="916b3077-78fa-4054-921c-2e7124dc35f8" Jul 29 22:46:34.570506 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:34.570272 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fxb52" Jul 29 22:46:34.570506 ip-10-0-138-112 kubenswrapper[2562]: E0729 22:46:34.570450 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fxb52" podUID="f0a6baed-8bea-4992-8852-0b84808a78d1" Jul 29 22:46:35.122998 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:35.122922 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/916b3077-78fa-4054-921c-2e7124dc35f8-original-pull-secret\") pod \"global-pull-secret-syncer-nkzq5\" (UID: \"916b3077-78fa-4054-921c-2e7124dc35f8\") " pod="kube-system/global-pull-secret-syncer-nkzq5" Jul 29 22:46:35.123508 ip-10-0-138-112 kubenswrapper[2562]: E0729 22:46:35.123069 2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 29 22:46:35.123508 ip-10-0-138-112 kubenswrapper[2562]: E0729 22:46:35.123139 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/916b3077-78fa-4054-921c-2e7124dc35f8-original-pull-secret podName:916b3077-78fa-4054-921c-2e7124dc35f8 nodeName:}" failed. No retries permitted until 2026-07-29 22:46:43.12311868 +0000 UTC m=+18.034944345 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/916b3077-78fa-4054-921c-2e7124dc35f8-original-pull-secret") pod "global-pull-secret-syncer-nkzq5" (UID: "916b3077-78fa-4054-921c-2e7124dc35f8") : object "kube-system"/"original-pull-secret" not registered Jul 29 22:46:35.224157 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:35.224117 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f0a6baed-8bea-4992-8852-0b84808a78d1-metrics-certs\") pod \"network-metrics-daemon-fxb52\" (UID: \"f0a6baed-8bea-4992-8852-0b84808a78d1\") " pod="openshift-multus/network-metrics-daemon-fxb52" Jul 29 22:46:35.224353 ip-10-0-138-112 kubenswrapper[2562]: E0729 22:46:35.224290 2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 29 22:46:35.224448 ip-10-0-138-112 kubenswrapper[2562]: E0729 22:46:35.224358 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f0a6baed-8bea-4992-8852-0b84808a78d1-metrics-certs podName:f0a6baed-8bea-4992-8852-0b84808a78d1 nodeName:}" failed. No retries permitted until 2026-07-29 22:46:43.224339172 +0000 UTC m=+18.136164841 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f0a6baed-8bea-4992-8852-0b84808a78d1-metrics-certs") pod "network-metrics-daemon-fxb52" (UID: "f0a6baed-8bea-4992-8852-0b84808a78d1") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 29 22:46:35.325524 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:35.325479 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5kv55\" (UniqueName: \"kubernetes.io/projected/2771bdbc-d054-464b-a01f-82344308451f-kube-api-access-5kv55\") pod \"network-check-target-z92kj\" (UID: \"2771bdbc-d054-464b-a01f-82344308451f\") " pod="openshift-network-diagnostics/network-check-target-z92kj" Jul 29 22:46:35.325774 ip-10-0-138-112 kubenswrapper[2562]: E0729 22:46:35.325673 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 29 22:46:35.325774 ip-10-0-138-112 kubenswrapper[2562]: E0729 22:46:35.325704 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 29 22:46:35.325774 ip-10-0-138-112 kubenswrapper[2562]: E0729 22:46:35.325718 2562 projected.go:196] Error preparing data for projected volume kube-api-access-5kv55 for pod openshift-network-diagnostics/network-check-target-z92kj: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 29 22:46:35.325962 ip-10-0-138-112 kubenswrapper[2562]: E0729 22:46:35.325788 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2771bdbc-d054-464b-a01f-82344308451f-kube-api-access-5kv55 podName:2771bdbc-d054-464b-a01f-82344308451f nodeName:}" failed. No retries permitted until 2026-07-29 22:46:43.325767134 +0000 UTC m=+18.237592802 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-5kv55" (UniqueName: "kubernetes.io/projected/2771bdbc-d054-464b-a01f-82344308451f-kube-api-access-5kv55") pod "network-check-target-z92kj" (UID: "2771bdbc-d054-464b-a01f-82344308451f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 29 22:46:35.570984 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:35.570881 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z92kj" Jul 29 22:46:35.570984 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:35.570923 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nkzq5" Jul 29 22:46:35.571201 ip-10-0-138-112 kubenswrapper[2562]: E0729 22:46:35.571001 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z92kj" podUID="2771bdbc-d054-464b-a01f-82344308451f" Jul 29 22:46:35.571201 ip-10-0-138-112 kubenswrapper[2562]: E0729 22:46:35.571151 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nkzq5" podUID="916b3077-78fa-4054-921c-2e7124dc35f8" Jul 29 22:46:36.570589 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:36.570054 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fxb52" Jul 29 22:46:36.570589 ip-10-0-138-112 kubenswrapper[2562]: E0729 22:46:36.570202 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fxb52" podUID="f0a6baed-8bea-4992-8852-0b84808a78d1" Jul 29 22:46:37.572535 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:37.571956 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z92kj" Jul 29 22:46:37.572535 ip-10-0-138-112 kubenswrapper[2562]: E0729 22:46:37.572094 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z92kj" podUID="2771bdbc-d054-464b-a01f-82344308451f" Jul 29 22:46:37.572535 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:37.572145 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nkzq5" Jul 29 22:46:37.572535 ip-10-0-138-112 kubenswrapper[2562]: E0729 22:46:37.572266 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nkzq5" podUID="916b3077-78fa-4054-921c-2e7124dc35f8" Jul 29 22:46:38.570765 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:38.570728 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fxb52" Jul 29 22:46:38.570955 ip-10-0-138-112 kubenswrapper[2562]: E0729 22:46:38.570871 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fxb52" podUID="f0a6baed-8bea-4992-8852-0b84808a78d1" Jul 29 22:46:39.570294 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:39.570202 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z92kj" Jul 29 22:46:39.570749 ip-10-0-138-112 kubenswrapper[2562]: E0729 22:46:39.570311 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z92kj" podUID="2771bdbc-d054-464b-a01f-82344308451f" Jul 29 22:46:39.570749 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:39.570376 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nkzq5" Jul 29 22:46:39.570749 ip-10-0-138-112 kubenswrapper[2562]: E0729 22:46:39.570497 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nkzq5" podUID="916b3077-78fa-4054-921c-2e7124dc35f8" Jul 29 22:46:40.570773 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:40.570734 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fxb52" Jul 29 22:46:40.571258 ip-10-0-138-112 kubenswrapper[2562]: E0729 22:46:40.570886 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fxb52" podUID="f0a6baed-8bea-4992-8852-0b84808a78d1" Jul 29 22:46:41.570024 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:41.569985 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nkzq5" Jul 29 22:46:41.570205 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:41.570040 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z92kj" Jul 29 22:46:41.570205 ip-10-0-138-112 kubenswrapper[2562]: E0729 22:46:41.570152 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nkzq5" podUID="916b3077-78fa-4054-921c-2e7124dc35f8" Jul 29 22:46:41.570307 ip-10-0-138-112 kubenswrapper[2562]: E0729 22:46:41.570276 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z92kj" podUID="2771bdbc-d054-464b-a01f-82344308451f" Jul 29 22:46:42.570025 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:42.569992 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fxb52" Jul 29 22:46:42.570558 ip-10-0-138-112 kubenswrapper[2562]: E0729 22:46:42.570115 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fxb52" podUID="f0a6baed-8bea-4992-8852-0b84808a78d1" Jul 29 22:46:43.181213 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:43.181167 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/916b3077-78fa-4054-921c-2e7124dc35f8-original-pull-secret\") pod \"global-pull-secret-syncer-nkzq5\" (UID: \"916b3077-78fa-4054-921c-2e7124dc35f8\") " pod="kube-system/global-pull-secret-syncer-nkzq5" Jul 29 22:46:43.181391 ip-10-0-138-112 kubenswrapper[2562]: E0729 22:46:43.181307 2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 29 22:46:43.181391 ip-10-0-138-112 kubenswrapper[2562]: E0729 22:46:43.181362 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/916b3077-78fa-4054-921c-2e7124dc35f8-original-pull-secret podName:916b3077-78fa-4054-921c-2e7124dc35f8 nodeName:}" failed. No retries permitted until 2026-07-29 22:46:59.18134956 +0000 UTC m=+34.093175225 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/916b3077-78fa-4054-921c-2e7124dc35f8-original-pull-secret") pod "global-pull-secret-syncer-nkzq5" (UID: "916b3077-78fa-4054-921c-2e7124dc35f8") : object "kube-system"/"original-pull-secret" not registered Jul 29 22:46:43.282150 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:43.282111 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f0a6baed-8bea-4992-8852-0b84808a78d1-metrics-certs\") pod \"network-metrics-daemon-fxb52\" (UID: \"f0a6baed-8bea-4992-8852-0b84808a78d1\") " pod="openshift-multus/network-metrics-daemon-fxb52" Jul 29 22:46:43.282348 ip-10-0-138-112 kubenswrapper[2562]: E0729 22:46:43.282305 2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 29 22:46:43.282439 ip-10-0-138-112 kubenswrapper[2562]: E0729 22:46:43.282390 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f0a6baed-8bea-4992-8852-0b84808a78d1-metrics-certs podName:f0a6baed-8bea-4992-8852-0b84808a78d1 nodeName:}" failed. No retries permitted until 2026-07-29 22:46:59.282366786 +0000 UTC m=+34.194192471 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f0a6baed-8bea-4992-8852-0b84808a78d1-metrics-certs") pod "network-metrics-daemon-fxb52" (UID: "f0a6baed-8bea-4992-8852-0b84808a78d1") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 29 22:46:43.383196 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:43.383161 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5kv55\" (UniqueName: \"kubernetes.io/projected/2771bdbc-d054-464b-a01f-82344308451f-kube-api-access-5kv55\") pod \"network-check-target-z92kj\" (UID: \"2771bdbc-d054-464b-a01f-82344308451f\") " pod="openshift-network-diagnostics/network-check-target-z92kj" Jul 29 22:46:43.383411 ip-10-0-138-112 kubenswrapper[2562]: E0729 22:46:43.383300 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 29 22:46:43.383411 ip-10-0-138-112 kubenswrapper[2562]: E0729 22:46:43.383323 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 29 22:46:43.383411 ip-10-0-138-112 kubenswrapper[2562]: E0729 22:46:43.383335 2562 projected.go:196] Error preparing data for projected volume kube-api-access-5kv55 for pod openshift-network-diagnostics/network-check-target-z92kj: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 29 22:46:43.383584 ip-10-0-138-112 kubenswrapper[2562]: E0729 22:46:43.383415 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2771bdbc-d054-464b-a01f-82344308451f-kube-api-access-5kv55 podName:2771bdbc-d054-464b-a01f-82344308451f nodeName:}" failed. No retries permitted until 2026-07-29 22:46:59.383381889 +0000 UTC m=+34.295207585 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-5kv55" (UniqueName: "kubernetes.io/projected/2771bdbc-d054-464b-a01f-82344308451f-kube-api-access-5kv55") pod "network-check-target-z92kj" (UID: "2771bdbc-d054-464b-a01f-82344308451f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 29 22:46:43.570173 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:43.570088 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z92kj" Jul 29 22:46:43.570625 ip-10-0-138-112 kubenswrapper[2562]: E0729 22:46:43.570226 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z92kj" podUID="2771bdbc-d054-464b-a01f-82344308451f" Jul 29 22:46:43.570625 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:43.570288 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nkzq5" Jul 29 22:46:43.570625 ip-10-0-138-112 kubenswrapper[2562]: E0729 22:46:43.570428 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nkzq5" podUID="916b3077-78fa-4054-921c-2e7124dc35f8" Jul 29 22:46:44.569860 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:44.569826 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fxb52" Jul 29 22:46:44.570037 ip-10-0-138-112 kubenswrapper[2562]: E0729 22:46:44.569950 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fxb52" podUID="f0a6baed-8bea-4992-8852-0b84808a78d1" Jul 29 22:46:45.571064 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:45.571022 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z92kj" Jul 29 22:46:45.571539 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:45.571104 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nkzq5" Jul 29 22:46:45.571539 ip-10-0-138-112 kubenswrapper[2562]: E0729 22:46:45.571219 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z92kj" podUID="2771bdbc-d054-464b-a01f-82344308451f" Jul 29 22:46:45.571539 ip-10-0-138-112 kubenswrapper[2562]: E0729 22:46:45.571296 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nkzq5" podUID="916b3077-78fa-4054-921c-2e7124dc35f8" Jul 29 22:46:46.570180 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:46.570143 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fxb52" Jul 29 22:46:46.570362 ip-10-0-138-112 kubenswrapper[2562]: E0729 22:46:46.570273 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fxb52" podUID="f0a6baed-8bea-4992-8852-0b84808a78d1" Jul 29 22:46:47.570462 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:47.570271 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nkzq5" Jul 29 22:46:47.571080 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:47.570278 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z92kj" Jul 29 22:46:47.571080 ip-10-0-138-112 kubenswrapper[2562]: E0729 22:46:47.570571 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nkzq5" podUID="916b3077-78fa-4054-921c-2e7124dc35f8" Jul 29 22:46:47.571080 ip-10-0-138-112 kubenswrapper[2562]: E0729 22:46:47.570611 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z92kj" podUID="2771bdbc-d054-464b-a01f-82344308451f" Jul 29 22:46:47.653943 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:47.653916 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9g28r" event={"ID":"6050ad9f-817a-48e4-b939-12efa11a6a73","Type":"ContainerStarted","Data":"d37052312f34a30fd01bfc19113721b87679287dd56eb5170fc43e55896c0c99"} Jul 29 22:46:47.654067 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:47.653983 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9g28r" event={"ID":"6050ad9f-817a-48e4-b939-12efa11a6a73","Type":"ContainerStarted","Data":"f07167d78b5cf183962366b7e9daba49da161d9666fb8b298ba9c69f67016401"} Jul 29 22:46:47.654067 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:47.654015 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9g28r" event={"ID":"6050ad9f-817a-48e4-b939-12efa11a6a73","Type":"ContainerStarted","Data":"fa4709cb4ba612b623e81bcfaee95ae9f002ceb977bfa0b8f27aaf1b2ee5472a"} Jul 29 22:46:47.654067 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:47.654042 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9g28r" event={"ID":"6050ad9f-817a-48e4-b939-12efa11a6a73","Type":"ContainerStarted","Data":"609e60eddf5a857e84fcbcbc99d0751f0f571530f9184b4b4876278c68ff62a1"} Jul 29 22:46:47.655129 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:47.655109 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-2mwmf" event={"ID":"6a64a07e-1c4a-4990-90b9-80623be37f60","Type":"ContainerStarted","Data":"1c84b2c516fc07aca70075aef25b168f82629d219329f068868b909973cec7d5"} Jul 29 22:46:47.656323 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:47.656301 2562 generic.go:356] "Generic (PLEG): container finished" podID="d40a4cdf-f141-46d3-8159-986f680b2dec" containerID="a7dda68deb31cda9d20c63cf85aa0d50ef38bc86acf460445c893e29af62d4f8" exitCode=0 Jul 29 22:46:47.656448 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:47.656358 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-h6flc" event={"ID":"d40a4cdf-f141-46d3-8159-986f680b2dec","Type":"ContainerDied","Data":"a7dda68deb31cda9d20c63cf85aa0d50ef38bc86acf460445c893e29af62d4f8"} Jul 29 22:46:47.657838 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:47.657819 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kqh77" event={"ID":"6c54a7c1-2837-47d6-88f4-dc4d264397d4","Type":"ContainerStarted","Data":"ca29b7dc171c6c1b7a4efc6596bee3f77dc7695bee4f1b93763dac563214eb4f"} Jul 29 22:46:47.659185 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:47.659161 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-vwtj2" event={"ID":"f6d8c777-cb9a-4505-8e30-ffaa01e53239","Type":"ContainerStarted","Data":"938ff4cefdb3ad144fa40f36baf451efbe3222af6ceed633314b75b192462ca6"} Jul 29 22:46:47.660519 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:47.660500 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-k8fl9" event={"ID":"51d8cffe-16b4-4bab-9193-953163e45417","Type":"ContainerStarted","Data":"db0fea211c43502d6786d7d125f02d67c55d19dc2874a1e9c220e6e3e808c880"} Jul 29 22:46:47.661717 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:47.661698 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-k9nmc" event={"ID":"ece97cfa-80dc-4fa8-ad31-61ba5c38aee3","Type":"ContainerStarted","Data":"f7b9ef7f617ea09dd3c0d4201f7e55ca6552d7e49c74a2900dc82c2bd133392d"} Jul 29 22:46:47.663054 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:47.663030 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-84tng" event={"ID":"4c31fa1a-fda4-43d6-84ef-e43f0bafdcb2","Type":"ContainerStarted","Data":"fe6916e093205357926028f9246d747f45102481ff440bc06d4fb586a6a07cf4"} Jul 29 22:46:47.701307 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:47.701260 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-84tng" podStartSLOduration=4.044075293 podStartE2EDuration="22.701245049s" totalImagesPullingTime="18.657169756s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 22:46:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 22:46:47.701080472 +0000 UTC m=+22.612906162" watchObservedRunningTime="2026-07-29 22:46:47.701245049 +0000 UTC m=+22.613070756" Jul 29 22:46:47.717490 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:47.717429 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-k8fl9" podStartSLOduration=4.011622964 podStartE2EDuration="22.717391117s" totalImagesPullingTime="18.705768153s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 22:46:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 22:46:47.717171918 +0000 UTC m=+22.628997607" watchObservedRunningTime="2026-07-29 22:46:47.717391117 +0000 UTC m=+22.629216805" Jul 29 22:46:47.746610 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:47.746558 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-k9nmc" podStartSLOduration=4.092118432 podStartE2EDuration="22.746541833s" totalImagesPullingTime="18.654423401s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 22:46:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 22:46:47.731805706 +0000 UTC m=+22.643631392" watchObservedRunningTime="2026-07-29 22:46:47.746541833 +0000 UTC m=+22.658367568" Jul 29 22:46:47.763518 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:47.763477 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-vwtj2" podStartSLOduration=4.114735566 podStartE2EDuration="22.763460562s" totalImagesPullingTime="18.648724996s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 22:46:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 22:46:47.746085687 +0000 UTC m=+22.657911373" watchObservedRunningTime="2026-07-29 22:46:47.763460562 +0000 UTC m=+22.675286295" Jul 29 22:46:47.763659 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:47.763613 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-2mwmf" podStartSLOduration=3.884243331 podStartE2EDuration="22.763604504s" totalImagesPullingTime="18.879361173s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 22:46:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 22:46:47.762886869 +0000 UTC m=+22.674712556" watchObservedRunningTime="2026-07-29 22:46:47.763604504 +0000 UTC m=+22.675430192" Jul 29 22:46:48.570337 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:48.570258 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fxb52" Jul 29 22:46:48.570557 ip-10-0-138-112 kubenswrapper[2562]: E0729 22:46:48.570385 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fxb52" podUID="f0a6baed-8bea-4992-8852-0b84808a78d1" Jul 29 22:46:48.669980 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:48.669941 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9g28r" event={"ID":"6050ad9f-817a-48e4-b939-12efa11a6a73","Type":"ContainerStarted","Data":"985ab5d7d7f03977b3213320b2651ddeec2ffcd26df6bc18af2cfc93143b752a"} Jul 29 22:46:48.670148 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:48.670046 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9g28r" event={"ID":"6050ad9f-817a-48e4-b939-12efa11a6a73","Type":"ContainerStarted","Data":"c8da20bd187b6e3b22d33b9720b097ba154731c03745619b817dadda671e8433"} Jul 29 22:46:48.718584 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:48.718554 2562 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 29 22:46:49.496049 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:49.495943 2562 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-29T22:46:48.718577276Z","UUID":"05dfb9bd-ebe6-4444-899b-b3c2d661c5fe","Handler":null,"Name":"","Endpoint":""} Jul 29 22:46:49.498744 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:49.498719 2562 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 29 22:46:49.498939 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:49.498770 2562 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 29 22:46:49.570227 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:49.570191 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nkzq5" Jul 29 22:46:49.570528 ip-10-0-138-112 kubenswrapper[2562]: E0729 22:46:49.570329 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nkzq5" podUID="916b3077-78fa-4054-921c-2e7124dc35f8" Jul 29 22:46:49.570693 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:49.570191 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z92kj" Jul 29 22:46:49.571151 ip-10-0-138-112 kubenswrapper[2562]: E0729 22:46:49.570773 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z92kj" podUID="2771bdbc-d054-464b-a01f-82344308451f" Jul 29 22:46:49.677387 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:49.677114 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-frd9g" event={"ID":"5ce0a47c-e715-472b-a0d9-761a5a6777dd","Type":"ContainerStarted","Data":"ed11095367b659732d1bbd30f375420ce843be25a2569167147254fcc56a1e7f"} Jul 29 22:46:49.679067 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:49.679041 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kqh77" event={"ID":"6c54a7c1-2837-47d6-88f4-dc4d264397d4","Type":"ContainerStarted","Data":"c40c1c7a18482c92be84cde73dd26e83bc97f28e84e34db13120a3cad28bfc75"} Jul 29 22:46:49.692350 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:49.692301 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-frd9g" podStartSLOduration=6.042566257 podStartE2EDuration="24.692286429s" totalImagesPullingTime="18.649720172s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 22:46:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 22:46:49.6917618 +0000 UTC m=+24.603587491" watchObservedRunningTime="2026-07-29 22:46:49.692286429 +0000 UTC m=+24.604112116" Jul 29 22:46:50.570792 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:50.570756 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fxb52" Jul 29 22:46:50.571282 ip-10-0-138-112 kubenswrapper[2562]: E0729 22:46:50.570904 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fxb52" podUID="f0a6baed-8bea-4992-8852-0b84808a78d1" Jul 29 22:46:50.685256 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:50.685215 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9g28r" event={"ID":"6050ad9f-817a-48e4-b939-12efa11a6a73","Type":"ContainerStarted","Data":"e5d1d2002861f6a33fb5d888f971d97f47bdeb8740ef252a59ccc531854cc216"} Jul 29 22:46:50.687555 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:50.687506 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kqh77" event={"ID":"6c54a7c1-2837-47d6-88f4-dc4d264397d4","Type":"ContainerStarted","Data":"bdd224b9f6ddb08043a8a833f1bc1d2a3b5a8533492d8670272d62dfa290c281"} Jul 29 22:46:50.709676 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:50.709616 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kqh77" podStartSLOduration=4.6793594039999995 podStartE2EDuration="25.709596084s" totalImagesPullingTime="21.03023668s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 22:46:25 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 22:46:50.707730094 +0000 UTC m=+25.619555794" watchObservedRunningTime="2026-07-29 22:46:50.709596084 +0000 UTC m=+25.621421771" Jul 29 22:46:51.120791 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:51.120752 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-k9nmc" Jul 29 22:46:51.121506 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:51.121475 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-k9nmc" Jul 29 22:46:51.570490 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:51.570246 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nkzq5" Jul 29 22:46:51.570649 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:51.570306 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z92kj" Jul 29 22:46:51.570649 ip-10-0-138-112 kubenswrapper[2562]: E0729 22:46:51.570524 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nkzq5" podUID="916b3077-78fa-4054-921c-2e7124dc35f8" Jul 29 22:46:51.570649 ip-10-0-138-112 kubenswrapper[2562]: E0729 22:46:51.570587 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z92kj" podUID="2771bdbc-d054-464b-a01f-82344308451f" Jul 29 22:46:51.690143 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:51.690114 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-k9nmc" Jul 29 22:46:51.690591 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:51.690442 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-k9nmc" Jul 29 22:46:52.570028 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:52.569950 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fxb52" Jul 29 22:46:52.570160 ip-10-0-138-112 kubenswrapper[2562]: E0729 22:46:52.570064 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fxb52" podUID="f0a6baed-8bea-4992-8852-0b84808a78d1" Jul 29 22:46:52.695704 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:52.695671 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9g28r" event={"ID":"6050ad9f-817a-48e4-b939-12efa11a6a73","Type":"ContainerStarted","Data":"b3406aaccbc247f5bf6736317e733bc7a0165e64180954c323906bc7357e14b1"} Jul 29 22:46:52.696694 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:52.695827 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-9g28r" Jul 29 22:46:52.696694 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:52.696044 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-9g28r" Jul 29 22:46:52.696694 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:52.696109 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-9g28r" Jul 29 22:46:52.697700 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:52.697672 2562 generic.go:356] "Generic (PLEG): container finished" podID="d40a4cdf-f141-46d3-8159-986f680b2dec" containerID="c4cd7acfb34e418c6d9ad370373dbff48952d839406e7c350df7f32c7431e8c0" exitCode=0 Jul 29 22:46:52.697827 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:52.697760 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-h6flc" event={"ID":"d40a4cdf-f141-46d3-8159-986f680b2dec","Type":"ContainerDied","Data":"c4cd7acfb34e418c6d9ad370373dbff48952d839406e7c350df7f32c7431e8c0"} Jul 29 22:46:52.713170 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:52.713146 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-9g28r" Jul 29 22:46:52.713301 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:52.713285 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-9g28r" Jul 29 22:46:52.732774 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:52.732720 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-9g28r" podStartSLOduration=8.988539691 podStartE2EDuration="27.732705438s" totalImagesPullingTime="18.744165747s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 22:46:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 22:46:52.732471317 +0000 UTC m=+27.644297003" watchObservedRunningTime="2026-07-29 22:46:52.732705438 +0000 UTC m=+27.644531126" Jul 29 22:46:53.576411 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:53.576189 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nkzq5" Jul 29 22:46:53.576634 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:53.576292 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z92kj" Jul 29 22:46:53.576634 ip-10-0-138-112 kubenswrapper[2562]: E0729 22:46:53.576514 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nkzq5" podUID="916b3077-78fa-4054-921c-2e7124dc35f8" Jul 29 22:46:53.576634 ip-10-0-138-112 kubenswrapper[2562]: E0729 22:46:53.576579 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z92kj" podUID="2771bdbc-d054-464b-a01f-82344308451f" Jul 29 22:46:54.124996 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:54.124959 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-nkzq5"] Jul 29 22:46:54.125567 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:54.125162 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nkzq5" Jul 29 22:46:54.125761 ip-10-0-138-112 kubenswrapper[2562]: E0729 22:46:54.125721 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nkzq5" podUID="916b3077-78fa-4054-921c-2e7124dc35f8" Jul 29 22:46:54.127011 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:54.126705 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-z92kj"] Jul 29 22:46:54.127011 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:54.126848 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z92kj" Jul 29 22:46:54.128137 ip-10-0-138-112 kubenswrapper[2562]: E0729 22:46:54.126970 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z92kj" podUID="2771bdbc-d054-464b-a01f-82344308451f" Jul 29 22:46:54.130300 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:54.130268 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-fxb52"] Jul 29 22:46:54.130427 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:54.130364 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fxb52" Jul 29 22:46:54.130553 ip-10-0-138-112 kubenswrapper[2562]: E0729 22:46:54.130532 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fxb52" podUID="f0a6baed-8bea-4992-8852-0b84808a78d1" Jul 29 22:46:54.704949 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:54.704867 2562 generic.go:356] "Generic (PLEG): container finished" podID="d40a4cdf-f141-46d3-8159-986f680b2dec" containerID="45f30cb0943e5d9491330ea2097ef8ab16552acdfbe2810db98a677a812ec6b6" exitCode=0 Jul 29 22:46:54.705093 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:54.704952 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-h6flc" event={"ID":"d40a4cdf-f141-46d3-8159-986f680b2dec","Type":"ContainerDied","Data":"45f30cb0943e5d9491330ea2097ef8ab16552acdfbe2810db98a677a812ec6b6"} Jul 29 22:46:55.573085 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:55.573053 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z92kj" Jul 29 22:46:55.573544 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:55.573064 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nkzq5" Jul 29 22:46:55.573544 ip-10-0-138-112 kubenswrapper[2562]: E0729 22:46:55.573151 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z92kj" podUID="2771bdbc-d054-464b-a01f-82344308451f" Jul 29 22:46:55.573544 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:55.573165 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fxb52" Jul 29 22:46:55.573544 ip-10-0-138-112 kubenswrapper[2562]: E0729 22:46:55.573229 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nkzq5" podUID="916b3077-78fa-4054-921c-2e7124dc35f8" Jul 29 22:46:55.573544 ip-10-0-138-112 kubenswrapper[2562]: E0729 22:46:55.573328 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fxb52" podUID="f0a6baed-8bea-4992-8852-0b84808a78d1" Jul 29 22:46:56.713169 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:56.712991 2562 generic.go:356] "Generic (PLEG): container finished" podID="d40a4cdf-f141-46d3-8159-986f680b2dec" containerID="8083bca454e7fc66b3710b3eda1c3e8101e07c6e94b48cdd465f0aef84049955" exitCode=0 Jul 29 22:46:56.713169 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:56.713051 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-h6flc" event={"ID":"d40a4cdf-f141-46d3-8159-986f680b2dec","Type":"ContainerDied","Data":"8083bca454e7fc66b3710b3eda1c3e8101e07c6e94b48cdd465f0aef84049955"} Jul 29 22:46:57.569991 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:57.569961 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nkzq5" Jul 29 22:46:57.570184 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:57.569968 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fxb52" Jul 29 22:46:57.570184 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:57.570095 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z92kj" Jul 29 22:46:57.570184 ip-10-0-138-112 kubenswrapper[2562]: E0729 22:46:57.570088 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nkzq5" podUID="916b3077-78fa-4054-921c-2e7124dc35f8" Jul 29 22:46:57.570337 ip-10-0-138-112 kubenswrapper[2562]: E0729 22:46:57.570198 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fxb52" podUID="f0a6baed-8bea-4992-8852-0b84808a78d1" Jul 29 22:46:57.570337 ip-10-0-138-112 kubenswrapper[2562]: E0729 22:46:57.570302 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z92kj" podUID="2771bdbc-d054-464b-a01f-82344308451f" Jul 29 22:46:59.201348 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:59.201306 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/916b3077-78fa-4054-921c-2e7124dc35f8-original-pull-secret\") pod \"global-pull-secret-syncer-nkzq5\" (UID: \"916b3077-78fa-4054-921c-2e7124dc35f8\") " pod="kube-system/global-pull-secret-syncer-nkzq5" Jul 29 22:46:59.201992 ip-10-0-138-112 kubenswrapper[2562]: E0729 22:46:59.201509 2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 29 22:46:59.201992 ip-10-0-138-112 kubenswrapper[2562]: E0729 22:46:59.201586 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/916b3077-78fa-4054-921c-2e7124dc35f8-original-pull-secret podName:916b3077-78fa-4054-921c-2e7124dc35f8 nodeName:}" failed. No retries permitted until 2026-07-29 22:47:31.201570693 +0000 UTC m=+66.113396362 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/916b3077-78fa-4054-921c-2e7124dc35f8-original-pull-secret") pod "global-pull-secret-syncer-nkzq5" (UID: "916b3077-78fa-4054-921c-2e7124dc35f8") : object "kube-system"/"original-pull-secret" not registered Jul 29 22:46:59.302543 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:59.302503 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f0a6baed-8bea-4992-8852-0b84808a78d1-metrics-certs\") pod \"network-metrics-daemon-fxb52\" (UID: \"f0a6baed-8bea-4992-8852-0b84808a78d1\") " pod="openshift-multus/network-metrics-daemon-fxb52" Jul 29 22:46:59.302739 ip-10-0-138-112 kubenswrapper[2562]: E0729 22:46:59.302664 2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 29 22:46:59.302739 ip-10-0-138-112 kubenswrapper[2562]: E0729 22:46:59.302732 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f0a6baed-8bea-4992-8852-0b84808a78d1-metrics-certs podName:f0a6baed-8bea-4992-8852-0b84808a78d1 nodeName:}" failed. No retries permitted until 2026-07-29 22:47:31.30271745 +0000 UTC m=+66.214543119 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f0a6baed-8bea-4992-8852-0b84808a78d1-metrics-certs") pod "network-metrics-daemon-fxb52" (UID: "f0a6baed-8bea-4992-8852-0b84808a78d1") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 29 22:46:59.403042 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:59.403005 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5kv55\" (UniqueName: \"kubernetes.io/projected/2771bdbc-d054-464b-a01f-82344308451f-kube-api-access-5kv55\") pod \"network-check-target-z92kj\" (UID: \"2771bdbc-d054-464b-a01f-82344308451f\") " pod="openshift-network-diagnostics/network-check-target-z92kj" Jul 29 22:46:59.403210 ip-10-0-138-112 kubenswrapper[2562]: E0729 22:46:59.403148 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 29 22:46:59.403210 ip-10-0-138-112 kubenswrapper[2562]: E0729 22:46:59.403169 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 29 22:46:59.403210 ip-10-0-138-112 kubenswrapper[2562]: E0729 22:46:59.403180 2562 projected.go:196] Error preparing data for projected volume kube-api-access-5kv55 for pod openshift-network-diagnostics/network-check-target-z92kj: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 29 22:46:59.403353 ip-10-0-138-112 kubenswrapper[2562]: E0729 22:46:59.403239 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2771bdbc-d054-464b-a01f-82344308451f-kube-api-access-5kv55 podName:2771bdbc-d054-464b-a01f-82344308451f nodeName:}" failed. No retries permitted until 2026-07-29 22:47:31.403224232 +0000 UTC m=+66.315049897 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-5kv55" (UniqueName: "kubernetes.io/projected/2771bdbc-d054-464b-a01f-82344308451f-kube-api-access-5kv55") pod "network-check-target-z92kj" (UID: "2771bdbc-d054-464b-a01f-82344308451f") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 29 22:46:59.570540 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:59.570426 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fxb52" Jul 29 22:46:59.570540 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:59.570469 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nkzq5" Jul 29 22:46:59.570540 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:59.570426 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z92kj" Jul 29 22:46:59.570807 ip-10-0-138-112 kubenswrapper[2562]: E0729 22:46:59.570573 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fxb52" podUID="f0a6baed-8bea-4992-8852-0b84808a78d1" Jul 29 22:46:59.570807 ip-10-0-138-112 kubenswrapper[2562]: E0729 22:46:59.570630 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z92kj" podUID="2771bdbc-d054-464b-a01f-82344308451f" Jul 29 22:46:59.570807 ip-10-0-138-112 kubenswrapper[2562]: E0729 22:46:59.570719 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nkzq5" podUID="916b3077-78fa-4054-921c-2e7124dc35f8" Jul 29 22:46:59.899043 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:59.898964 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-112.ec2.internal" event="NodeReady" Jul 29 22:46:59.899194 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:59.899107 2562 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 29 22:46:59.937068 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:59.937025 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-6cb779f875-pfsr5"] Jul 29 22:46:59.939790 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:59.939768 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6cb779f875-pfsr5" Jul 29 22:46:59.944470 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:59.944432 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 29 22:46:59.944470 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:59.944458 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 29 22:46:59.944805 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:59.944782 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-vqjws\"" Jul 29 22:46:59.944984 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:59.944968 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 29 22:46:59.951579 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:59.951556 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-ncs4k"] Jul 29 22:46:59.953049 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:59.953026 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 29 22:46:59.953548 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:59.953531 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-ncs4k" Jul 29 22:46:59.957743 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:59.957722 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 29 22:46:59.957840 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:59.957754 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-c5tdl"] Jul 29 22:46:59.957840 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:59.957791 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 29 22:46:59.958011 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:59.957969 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-tpckt\"" Jul 29 22:46:59.959729 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:59.959708 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6cb779f875-pfsr5"] Jul 29 22:46:59.959829 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:59.959817 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-c5tdl" Jul 29 22:46:59.962898 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:59.962846 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 29 22:46:59.963250 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:59.963229 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 29 22:46:59.963346 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:59.963241 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 29 22:46:59.963346 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:59.963270 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-f8hzf\"" Jul 29 22:46:59.976066 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:59.976037 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-c5tdl"] Jul 29 22:46:59.976747 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:46:59.976728 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-ncs4k"] Jul 29 22:47:00.108442 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:00.108381 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qnk7m\" (UniqueName: \"kubernetes.io/projected/f05d8f86-4adc-47ed-9a27-1e9a00f68a69-kube-api-access-qnk7m\") pod \"image-registry-6cb779f875-pfsr5\" (UID: \"f05d8f86-4adc-47ed-9a27-1e9a00f68a69\") " pod="openshift-image-registry/image-registry-6cb779f875-pfsr5" Jul 29 22:47:00.108629 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:00.108491 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/f05d8f86-4adc-47ed-9a27-1e9a00f68a69-registry-certificates\") pod \"image-registry-6cb779f875-pfsr5\" (UID: \"f05d8f86-4adc-47ed-9a27-1e9a00f68a69\") " pod="openshift-image-registry/image-registry-6cb779f875-pfsr5" Jul 29 22:47:00.108629 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:00.108538 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f05d8f86-4adc-47ed-9a27-1e9a00f68a69-registry-tls\") pod \"image-registry-6cb779f875-pfsr5\" (UID: \"f05d8f86-4adc-47ed-9a27-1e9a00f68a69\") " pod="openshift-image-registry/image-registry-6cb779f875-pfsr5" Jul 29 22:47:00.108629 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:00.108563 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/f05d8f86-4adc-47ed-9a27-1e9a00f68a69-ca-trust-extracted\") pod \"image-registry-6cb779f875-pfsr5\" (UID: \"f05d8f86-4adc-47ed-9a27-1e9a00f68a69\") " pod="openshift-image-registry/image-registry-6cb779f875-pfsr5" Jul 29 22:47:00.108629 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:00.108598 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/cb3010ad-c703-4901-979e-88a3e1b07ec3-config-volume\") pod \"dns-default-ncs4k\" (UID: \"cb3010ad-c703-4901-979e-88a3e1b07ec3\") " pod="openshift-dns/dns-default-ncs4k" Jul 29 22:47:00.108629 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:00.108625 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mgxxz\" (UniqueName: \"kubernetes.io/projected/cb3010ad-c703-4901-979e-88a3e1b07ec3-kube-api-access-mgxxz\") pod \"dns-default-ncs4k\" (UID: \"cb3010ad-c703-4901-979e-88a3e1b07ec3\") " pod="openshift-dns/dns-default-ncs4k" Jul 29 22:47:00.108890 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:00.108659 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9b18dc65-d191-4eef-b23e-14561cbe95f1-cert\") pod \"ingress-canary-c5tdl\" (UID: \"9b18dc65-d191-4eef-b23e-14561cbe95f1\") " pod="openshift-ingress-canary/ingress-canary-c5tdl" Jul 29 22:47:00.108890 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:00.108708 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/f05d8f86-4adc-47ed-9a27-1e9a00f68a69-image-registry-private-configuration\") pod \"image-registry-6cb779f875-pfsr5\" (UID: \"f05d8f86-4adc-47ed-9a27-1e9a00f68a69\") " pod="openshift-image-registry/image-registry-6cb779f875-pfsr5" Jul 29 22:47:00.108890 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:00.108738 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f05d8f86-4adc-47ed-9a27-1e9a00f68a69-trusted-ca\") pod \"image-registry-6cb779f875-pfsr5\" (UID: \"f05d8f86-4adc-47ed-9a27-1e9a00f68a69\") " pod="openshift-image-registry/image-registry-6cb779f875-pfsr5" Jul 29 22:47:00.108890 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:00.108760 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/f05d8f86-4adc-47ed-9a27-1e9a00f68a69-installation-pull-secrets\") pod \"image-registry-6cb779f875-pfsr5\" (UID: \"f05d8f86-4adc-47ed-9a27-1e9a00f68a69\") " pod="openshift-image-registry/image-registry-6cb779f875-pfsr5" Jul 29 22:47:00.108890 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:00.108784 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f05d8f86-4adc-47ed-9a27-1e9a00f68a69-bound-sa-token\") pod \"image-registry-6cb779f875-pfsr5\" (UID: \"f05d8f86-4adc-47ed-9a27-1e9a00f68a69\") " pod="openshift-image-registry/image-registry-6cb779f875-pfsr5" Jul 29 22:47:00.108890 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:00.108811 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/cb3010ad-c703-4901-979e-88a3e1b07ec3-metrics-tls\") pod \"dns-default-ncs4k\" (UID: \"cb3010ad-c703-4901-979e-88a3e1b07ec3\") " pod="openshift-dns/dns-default-ncs4k" Jul 29 22:47:00.108890 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:00.108837 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cb3010ad-c703-4901-979e-88a3e1b07ec3-tmp-dir\") pod \"dns-default-ncs4k\" (UID: \"cb3010ad-c703-4901-979e-88a3e1b07ec3\") " pod="openshift-dns/dns-default-ncs4k" Jul 29 22:47:00.109211 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:00.108897 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r8v8p\" (UniqueName: \"kubernetes.io/projected/9b18dc65-d191-4eef-b23e-14561cbe95f1-kube-api-access-r8v8p\") pod \"ingress-canary-c5tdl\" (UID: \"9b18dc65-d191-4eef-b23e-14561cbe95f1\") " pod="openshift-ingress-canary/ingress-canary-c5tdl" Jul 29 22:47:00.210101 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:00.210006 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qnk7m\" (UniqueName: \"kubernetes.io/projected/f05d8f86-4adc-47ed-9a27-1e9a00f68a69-kube-api-access-qnk7m\") pod \"image-registry-6cb779f875-pfsr5\" (UID: \"f05d8f86-4adc-47ed-9a27-1e9a00f68a69\") " pod="openshift-image-registry/image-registry-6cb779f875-pfsr5" Jul 29 22:47:00.210101 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:00.210059 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/f05d8f86-4adc-47ed-9a27-1e9a00f68a69-registry-certificates\") pod \"image-registry-6cb779f875-pfsr5\" (UID: \"f05d8f86-4adc-47ed-9a27-1e9a00f68a69\") " pod="openshift-image-registry/image-registry-6cb779f875-pfsr5" Jul 29 22:47:00.210101 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:00.210084 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f05d8f86-4adc-47ed-9a27-1e9a00f68a69-registry-tls\") pod \"image-registry-6cb779f875-pfsr5\" (UID: \"f05d8f86-4adc-47ed-9a27-1e9a00f68a69\") " pod="openshift-image-registry/image-registry-6cb779f875-pfsr5" Jul 29 22:47:00.210764 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:00.210113 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/f05d8f86-4adc-47ed-9a27-1e9a00f68a69-ca-trust-extracted\") pod \"image-registry-6cb779f875-pfsr5\" (UID: \"f05d8f86-4adc-47ed-9a27-1e9a00f68a69\") " pod="openshift-image-registry/image-registry-6cb779f875-pfsr5" Jul 29 22:47:00.210764 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:00.210147 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/cb3010ad-c703-4901-979e-88a3e1b07ec3-config-volume\") pod \"dns-default-ncs4k\" (UID: \"cb3010ad-c703-4901-979e-88a3e1b07ec3\") " pod="openshift-dns/dns-default-ncs4k" Jul 29 22:47:00.210764 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:00.210171 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mgxxz\" (UniqueName: \"kubernetes.io/projected/cb3010ad-c703-4901-979e-88a3e1b07ec3-kube-api-access-mgxxz\") pod \"dns-default-ncs4k\" (UID: \"cb3010ad-c703-4901-979e-88a3e1b07ec3\") " pod="openshift-dns/dns-default-ncs4k" Jul 29 22:47:00.210764 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:00.210210 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9b18dc65-d191-4eef-b23e-14561cbe95f1-cert\") pod \"ingress-canary-c5tdl\" (UID: \"9b18dc65-d191-4eef-b23e-14561cbe95f1\") " pod="openshift-ingress-canary/ingress-canary-c5tdl" Jul 29 22:47:00.210764 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:00.210235 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/f05d8f86-4adc-47ed-9a27-1e9a00f68a69-image-registry-private-configuration\") pod \"image-registry-6cb779f875-pfsr5\" (UID: \"f05d8f86-4adc-47ed-9a27-1e9a00f68a69\") " pod="openshift-image-registry/image-registry-6cb779f875-pfsr5" Jul 29 22:47:00.210764 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:00.210258 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f05d8f86-4adc-47ed-9a27-1e9a00f68a69-trusted-ca\") pod \"image-registry-6cb779f875-pfsr5\" (UID: \"f05d8f86-4adc-47ed-9a27-1e9a00f68a69\") " pod="openshift-image-registry/image-registry-6cb779f875-pfsr5" Jul 29 22:47:00.210764 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:00.210281 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/f05d8f86-4adc-47ed-9a27-1e9a00f68a69-installation-pull-secrets\") pod \"image-registry-6cb779f875-pfsr5\" (UID: \"f05d8f86-4adc-47ed-9a27-1e9a00f68a69\") " pod="openshift-image-registry/image-registry-6cb779f875-pfsr5" Jul 29 22:47:00.210764 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:00.210304 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f05d8f86-4adc-47ed-9a27-1e9a00f68a69-bound-sa-token\") pod \"image-registry-6cb779f875-pfsr5\" (UID: \"f05d8f86-4adc-47ed-9a27-1e9a00f68a69\") " pod="openshift-image-registry/image-registry-6cb779f875-pfsr5" Jul 29 22:47:00.210764 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:00.210327 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/cb3010ad-c703-4901-979e-88a3e1b07ec3-metrics-tls\") pod \"dns-default-ncs4k\" (UID: \"cb3010ad-c703-4901-979e-88a3e1b07ec3\") " pod="openshift-dns/dns-default-ncs4k" Jul 29 22:47:00.210764 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:00.210348 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cb3010ad-c703-4901-979e-88a3e1b07ec3-tmp-dir\") pod \"dns-default-ncs4k\" (UID: \"cb3010ad-c703-4901-979e-88a3e1b07ec3\") " pod="openshift-dns/dns-default-ncs4k" Jul 29 22:47:00.210764 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:00.210374 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r8v8p\" (UniqueName: \"kubernetes.io/projected/9b18dc65-d191-4eef-b23e-14561cbe95f1-kube-api-access-r8v8p\") pod \"ingress-canary-c5tdl\" (UID: \"9b18dc65-d191-4eef-b23e-14561cbe95f1\") " pod="openshift-ingress-canary/ingress-canary-c5tdl" Jul 29 22:47:00.210764 ip-10-0-138-112 kubenswrapper[2562]: E0729 22:47:00.210708 2562 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 29 22:47:00.210764 ip-10-0-138-112 kubenswrapper[2562]: E0729 22:47:00.210730 2562 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6cb779f875-pfsr5: secret "image-registry-tls" not found Jul 29 22:47:00.210764 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:00.210754 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/f05d8f86-4adc-47ed-9a27-1e9a00f68a69-registry-certificates\") pod \"image-registry-6cb779f875-pfsr5\" (UID: \"f05d8f86-4adc-47ed-9a27-1e9a00f68a69\") " pod="openshift-image-registry/image-registry-6cb779f875-pfsr5" Jul 29 22:47:00.211439 ip-10-0-138-112 kubenswrapper[2562]: E0729 22:47:00.210793 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f05d8f86-4adc-47ed-9a27-1e9a00f68a69-registry-tls podName:f05d8f86-4adc-47ed-9a27-1e9a00f68a69 nodeName:}" failed. No retries permitted until 2026-07-29 22:47:00.710775077 +0000 UTC m=+35.622600742 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/f05d8f86-4adc-47ed-9a27-1e9a00f68a69-registry-tls") pod "image-registry-6cb779f875-pfsr5" (UID: "f05d8f86-4adc-47ed-9a27-1e9a00f68a69") : secret "image-registry-tls" not found Jul 29 22:47:00.211439 ip-10-0-138-112 kubenswrapper[2562]: E0729 22:47:00.210840 2562 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 29 22:47:00.211439 ip-10-0-138-112 kubenswrapper[2562]: E0729 22:47:00.210896 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cb3010ad-c703-4901-979e-88a3e1b07ec3-metrics-tls podName:cb3010ad-c703-4901-979e-88a3e1b07ec3 nodeName:}" failed. No retries permitted until 2026-07-29 22:47:00.710881599 +0000 UTC m=+35.622707271 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/cb3010ad-c703-4901-979e-88a3e1b07ec3-metrics-tls") pod "dns-default-ncs4k" (UID: "cb3010ad-c703-4901-979e-88a3e1b07ec3") : secret "dns-default-metrics-tls" not found Jul 29 22:47:00.211439 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:00.211064 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cb3010ad-c703-4901-979e-88a3e1b07ec3-tmp-dir\") pod \"dns-default-ncs4k\" (UID: \"cb3010ad-c703-4901-979e-88a3e1b07ec3\") " pod="openshift-dns/dns-default-ncs4k" Jul 29 22:47:00.211439 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:00.211088 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/f05d8f86-4adc-47ed-9a27-1e9a00f68a69-ca-trust-extracted\") pod \"image-registry-6cb779f875-pfsr5\" (UID: \"f05d8f86-4adc-47ed-9a27-1e9a00f68a69\") " pod="openshift-image-registry/image-registry-6cb779f875-pfsr5" Jul 29 22:47:00.211713 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:00.211645 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/cb3010ad-c703-4901-979e-88a3e1b07ec3-config-volume\") pod \"dns-default-ncs4k\" (UID: \"cb3010ad-c703-4901-979e-88a3e1b07ec3\") " pod="openshift-dns/dns-default-ncs4k" Jul 29 22:47:00.211983 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:00.211929 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f05d8f86-4adc-47ed-9a27-1e9a00f68a69-trusted-ca\") pod \"image-registry-6cb779f875-pfsr5\" (UID: \"f05d8f86-4adc-47ed-9a27-1e9a00f68a69\") " pod="openshift-image-registry/image-registry-6cb779f875-pfsr5" Jul 29 22:47:00.215500 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:00.215435 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/f05d8f86-4adc-47ed-9a27-1e9a00f68a69-image-registry-private-configuration\") pod \"image-registry-6cb779f875-pfsr5\" (UID: \"f05d8f86-4adc-47ed-9a27-1e9a00f68a69\") " pod="openshift-image-registry/image-registry-6cb779f875-pfsr5" Jul 29 22:47:00.215500 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:00.215478 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9b18dc65-d191-4eef-b23e-14561cbe95f1-cert\") pod \"ingress-canary-c5tdl\" (UID: \"9b18dc65-d191-4eef-b23e-14561cbe95f1\") " pod="openshift-ingress-canary/ingress-canary-c5tdl" Jul 29 22:47:00.215681 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:00.215479 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/f05d8f86-4adc-47ed-9a27-1e9a00f68a69-installation-pull-secrets\") pod \"image-registry-6cb779f875-pfsr5\" (UID: \"f05d8f86-4adc-47ed-9a27-1e9a00f68a69\") " pod="openshift-image-registry/image-registry-6cb779f875-pfsr5" Jul 29 22:47:00.219575 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:00.219545 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mgxxz\" (UniqueName: \"kubernetes.io/projected/cb3010ad-c703-4901-979e-88a3e1b07ec3-kube-api-access-mgxxz\") pod \"dns-default-ncs4k\" (UID: \"cb3010ad-c703-4901-979e-88a3e1b07ec3\") " pod="openshift-dns/dns-default-ncs4k" Jul 29 22:47:00.220120 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:00.220095 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qnk7m\" (UniqueName: \"kubernetes.io/projected/f05d8f86-4adc-47ed-9a27-1e9a00f68a69-kube-api-access-qnk7m\") pod \"image-registry-6cb779f875-pfsr5\" (UID: \"f05d8f86-4adc-47ed-9a27-1e9a00f68a69\") " pod="openshift-image-registry/image-registry-6cb779f875-pfsr5" Jul 29 22:47:00.220768 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:00.220697 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r8v8p\" (UniqueName: \"kubernetes.io/projected/9b18dc65-d191-4eef-b23e-14561cbe95f1-kube-api-access-r8v8p\") pod \"ingress-canary-c5tdl\" (UID: \"9b18dc65-d191-4eef-b23e-14561cbe95f1\") " pod="openshift-ingress-canary/ingress-canary-c5tdl" Jul 29 22:47:00.221029 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:00.220979 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f05d8f86-4adc-47ed-9a27-1e9a00f68a69-bound-sa-token\") pod \"image-registry-6cb779f875-pfsr5\" (UID: \"f05d8f86-4adc-47ed-9a27-1e9a00f68a69\") " pod="openshift-image-registry/image-registry-6cb779f875-pfsr5" Jul 29 22:47:00.271289 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:00.271253 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-c5tdl" Jul 29 22:47:00.665372 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:00.665081 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-c5tdl"] Jul 29 22:47:00.671477 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:47:00.671449 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9b18dc65_d191_4eef_b23e_14561cbe95f1.slice/crio-eed8e70859e2c33ab3292cab60104c3b5a2296d4880965334006a1304ccc713f WatchSource:0}: Error finding container eed8e70859e2c33ab3292cab60104c3b5a2296d4880965334006a1304ccc713f: Status 404 returned error can't find the container with id eed8e70859e2c33ab3292cab60104c3b5a2296d4880965334006a1304ccc713f Jul 29 22:47:00.715278 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:00.715245 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f05d8f86-4adc-47ed-9a27-1e9a00f68a69-registry-tls\") pod \"image-registry-6cb779f875-pfsr5\" (UID: \"f05d8f86-4adc-47ed-9a27-1e9a00f68a69\") " pod="openshift-image-registry/image-registry-6cb779f875-pfsr5" Jul 29 22:47:00.715478 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:00.715316 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/cb3010ad-c703-4901-979e-88a3e1b07ec3-metrics-tls\") pod \"dns-default-ncs4k\" (UID: \"cb3010ad-c703-4901-979e-88a3e1b07ec3\") " pod="openshift-dns/dns-default-ncs4k" Jul 29 22:47:00.715478 ip-10-0-138-112 kubenswrapper[2562]: E0729 22:47:00.715414 2562 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 29 22:47:00.715478 ip-10-0-138-112 kubenswrapper[2562]: E0729 22:47:00.715439 2562 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6cb779f875-pfsr5: secret "image-registry-tls" not found Jul 29 22:47:00.715478 ip-10-0-138-112 kubenswrapper[2562]: E0729 22:47:00.715451 2562 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 29 22:47:00.715621 ip-10-0-138-112 kubenswrapper[2562]: E0729 22:47:00.715496 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f05d8f86-4adc-47ed-9a27-1e9a00f68a69-registry-tls podName:f05d8f86-4adc-47ed-9a27-1e9a00f68a69 nodeName:}" failed. No retries permitted until 2026-07-29 22:47:01.71548 +0000 UTC m=+36.627305665 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/f05d8f86-4adc-47ed-9a27-1e9a00f68a69-registry-tls") pod "image-registry-6cb779f875-pfsr5" (UID: "f05d8f86-4adc-47ed-9a27-1e9a00f68a69") : secret "image-registry-tls" not found Jul 29 22:47:00.715621 ip-10-0-138-112 kubenswrapper[2562]: E0729 22:47:00.715510 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cb3010ad-c703-4901-979e-88a3e1b07ec3-metrics-tls podName:cb3010ad-c703-4901-979e-88a3e1b07ec3 nodeName:}" failed. No retries permitted until 2026-07-29 22:47:01.715503683 +0000 UTC m=+36.627329348 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/cb3010ad-c703-4901-979e-88a3e1b07ec3-metrics-tls") pod "dns-default-ncs4k" (UID: "cb3010ad-c703-4901-979e-88a3e1b07ec3") : secret "dns-default-metrics-tls" not found Jul 29 22:47:00.724737 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:00.724695 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-c5tdl" event={"ID":"9b18dc65-d191-4eef-b23e-14561cbe95f1","Type":"ContainerStarted","Data":"eed8e70859e2c33ab3292cab60104c3b5a2296d4880965334006a1304ccc713f"} Jul 29 22:47:00.727598 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:00.727572 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-h6flc" event={"ID":"d40a4cdf-f141-46d3-8159-986f680b2dec","Type":"ContainerStarted","Data":"29f725be2fb8edf43df6e3e51dbcc1ae6d6b01240776dcff06622363b395cb67"} Jul 29 22:47:01.570509 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:01.570421 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fxb52" Jul 29 22:47:01.571138 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:01.570432 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z92kj" Jul 29 22:47:01.571138 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:01.570571 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nkzq5" Jul 29 22:47:01.574243 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:01.574222 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 29 22:47:01.574374 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:01.574282 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 29 22:47:01.575569 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:01.575548 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 29 22:47:01.575673 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:01.575581 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 29 22:47:01.575673 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:01.575583 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-hgxlh\"" Jul 29 22:47:01.575775 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:01.575679 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-s6c2g\"" Jul 29 22:47:01.724588 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:01.724548 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/cb3010ad-c703-4901-979e-88a3e1b07ec3-metrics-tls\") pod \"dns-default-ncs4k\" (UID: \"cb3010ad-c703-4901-979e-88a3e1b07ec3\") " pod="openshift-dns/dns-default-ncs4k" Jul 29 22:47:01.724778 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:01.724625 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f05d8f86-4adc-47ed-9a27-1e9a00f68a69-registry-tls\") pod \"image-registry-6cb779f875-pfsr5\" (UID: \"f05d8f86-4adc-47ed-9a27-1e9a00f68a69\") " pod="openshift-image-registry/image-registry-6cb779f875-pfsr5" Jul 29 22:47:01.724778 ip-10-0-138-112 kubenswrapper[2562]: E0729 22:47:01.724734 2562 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 29 22:47:01.724890 ip-10-0-138-112 kubenswrapper[2562]: E0729 22:47:01.724803 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cb3010ad-c703-4901-979e-88a3e1b07ec3-metrics-tls podName:cb3010ad-c703-4901-979e-88a3e1b07ec3 nodeName:}" failed. No retries permitted until 2026-07-29 22:47:03.724783533 +0000 UTC m=+38.636609198 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/cb3010ad-c703-4901-979e-88a3e1b07ec3-metrics-tls") pod "dns-default-ncs4k" (UID: "cb3010ad-c703-4901-979e-88a3e1b07ec3") : secret "dns-default-metrics-tls" not found Jul 29 22:47:01.724890 ip-10-0-138-112 kubenswrapper[2562]: E0729 22:47:01.724738 2562 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 29 22:47:01.724890 ip-10-0-138-112 kubenswrapper[2562]: E0729 22:47:01.724831 2562 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6cb779f875-pfsr5: secret "image-registry-tls" not found Jul 29 22:47:01.724890 ip-10-0-138-112 kubenswrapper[2562]: E0729 22:47:01.724864 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f05d8f86-4adc-47ed-9a27-1e9a00f68a69-registry-tls podName:f05d8f86-4adc-47ed-9a27-1e9a00f68a69 nodeName:}" failed. No retries permitted until 2026-07-29 22:47:03.724854968 +0000 UTC m=+38.636680633 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/f05d8f86-4adc-47ed-9a27-1e9a00f68a69-registry-tls") pod "image-registry-6cb779f875-pfsr5" (UID: "f05d8f86-4adc-47ed-9a27-1e9a00f68a69") : secret "image-registry-tls" not found Jul 29 22:47:01.733573 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:01.733538 2562 generic.go:356] "Generic (PLEG): container finished" podID="d40a4cdf-f141-46d3-8159-986f680b2dec" containerID="29f725be2fb8edf43df6e3e51dbcc1ae6d6b01240776dcff06622363b395cb67" exitCode=0 Jul 29 22:47:01.733747 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:01.733580 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-h6flc" event={"ID":"d40a4cdf-f141-46d3-8159-986f680b2dec","Type":"ContainerDied","Data":"29f725be2fb8edf43df6e3e51dbcc1ae6d6b01240776dcff06622363b395cb67"} Jul 29 22:47:02.740435 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:02.740386 2562 generic.go:356] "Generic (PLEG): container finished" podID="d40a4cdf-f141-46d3-8159-986f680b2dec" containerID="f1bccac35dfc39ff6b12cedf269afc3749598ae80fb368d8e7fe1ab49ffc3f58" exitCode=0 Jul 29 22:47:02.740833 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:02.740440 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-h6flc" event={"ID":"d40a4cdf-f141-46d3-8159-986f680b2dec","Type":"ContainerDied","Data":"f1bccac35dfc39ff6b12cedf269afc3749598ae80fb368d8e7fe1ab49ffc3f58"} Jul 29 22:47:03.740903 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:03.740643 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/cb3010ad-c703-4901-979e-88a3e1b07ec3-metrics-tls\") pod \"dns-default-ncs4k\" (UID: \"cb3010ad-c703-4901-979e-88a3e1b07ec3\") " pod="openshift-dns/dns-default-ncs4k" Jul 29 22:47:03.741369 ip-10-0-138-112 kubenswrapper[2562]: E0729 22:47:03.740780 2562 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 29 22:47:03.741369 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:03.740984 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f05d8f86-4adc-47ed-9a27-1e9a00f68a69-registry-tls\") pod \"image-registry-6cb779f875-pfsr5\" (UID: \"f05d8f86-4adc-47ed-9a27-1e9a00f68a69\") " pod="openshift-image-registry/image-registry-6cb779f875-pfsr5" Jul 29 22:47:03.741369 ip-10-0-138-112 kubenswrapper[2562]: E0729 22:47:03.741015 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cb3010ad-c703-4901-979e-88a3e1b07ec3-metrics-tls podName:cb3010ad-c703-4901-979e-88a3e1b07ec3 nodeName:}" failed. No retries permitted until 2026-07-29 22:47:07.740999115 +0000 UTC m=+42.652824784 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/cb3010ad-c703-4901-979e-88a3e1b07ec3-metrics-tls") pod "dns-default-ncs4k" (UID: "cb3010ad-c703-4901-979e-88a3e1b07ec3") : secret "dns-default-metrics-tls" not found Jul 29 22:47:03.741369 ip-10-0-138-112 kubenswrapper[2562]: E0729 22:47:03.741076 2562 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 29 22:47:03.741369 ip-10-0-138-112 kubenswrapper[2562]: E0729 22:47:03.741089 2562 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6cb779f875-pfsr5: secret "image-registry-tls" not found Jul 29 22:47:03.741369 ip-10-0-138-112 kubenswrapper[2562]: E0729 22:47:03.741141 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f05d8f86-4adc-47ed-9a27-1e9a00f68a69-registry-tls podName:f05d8f86-4adc-47ed-9a27-1e9a00f68a69 nodeName:}" failed. No retries permitted until 2026-07-29 22:47:07.741128524 +0000 UTC m=+42.652954193 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/f05d8f86-4adc-47ed-9a27-1e9a00f68a69-registry-tls") pod "image-registry-6cb779f875-pfsr5" (UID: "f05d8f86-4adc-47ed-9a27-1e9a00f68a69") : secret "image-registry-tls" not found Jul 29 22:47:03.745969 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:03.745931 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-h6flc" event={"ID":"d40a4cdf-f141-46d3-8159-986f680b2dec","Type":"ContainerStarted","Data":"938495835b64a79dd326b31dc6ead28b4d9fe840e0c18b61e7e2154e7bccb025"} Jul 29 22:47:03.747286 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:03.747262 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-c5tdl" event={"ID":"9b18dc65-d191-4eef-b23e-14561cbe95f1","Type":"ContainerStarted","Data":"c12dd2b639f298c6dade8c5030c508df28c8e4e9a6af4491970fda6470dbbe24"} Jul 29 22:47:03.769219 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:03.769164 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-h6flc" podStartSLOduration=9.976521591000001 podStartE2EDuration="38.769148594s" totalImagesPullingTime="28.792627003s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 22:46:25 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 22:47:03.768882445 +0000 UTC m=+38.680708132" watchObservedRunningTime="2026-07-29 22:47:03.769148594 +0000 UTC m=+38.680974281" Jul 29 22:47:03.785815 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:03.785755 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-c5tdl" podStartSLOduration=2.22241529 podStartE2EDuration="4.785739522s" totalImagesPullingTime="2.563324232s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 22:46:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 22:47:03.784248732 +0000 UTC m=+38.696074419" watchObservedRunningTime="2026-07-29 22:47:03.785739522 +0000 UTC m=+38.697565210" Jul 29 22:47:07.771659 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:07.771617 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f05d8f86-4adc-47ed-9a27-1e9a00f68a69-registry-tls\") pod \"image-registry-6cb779f875-pfsr5\" (UID: \"f05d8f86-4adc-47ed-9a27-1e9a00f68a69\") " pod="openshift-image-registry/image-registry-6cb779f875-pfsr5" Jul 29 22:47:07.772137 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:07.771697 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/cb3010ad-c703-4901-979e-88a3e1b07ec3-metrics-tls\") pod \"dns-default-ncs4k\" (UID: \"cb3010ad-c703-4901-979e-88a3e1b07ec3\") " pod="openshift-dns/dns-default-ncs4k" Jul 29 22:47:07.772137 ip-10-0-138-112 kubenswrapper[2562]: E0729 22:47:07.771776 2562 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 29 22:47:07.772137 ip-10-0-138-112 kubenswrapper[2562]: E0729 22:47:07.771791 2562 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6cb779f875-pfsr5: secret "image-registry-tls" not found Jul 29 22:47:07.772137 ip-10-0-138-112 kubenswrapper[2562]: E0729 22:47:07.771838 2562 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 29 22:47:07.772137 ip-10-0-138-112 kubenswrapper[2562]: E0729 22:47:07.771845 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f05d8f86-4adc-47ed-9a27-1e9a00f68a69-registry-tls podName:f05d8f86-4adc-47ed-9a27-1e9a00f68a69 nodeName:}" failed. No retries permitted until 2026-07-29 22:47:15.77183113 +0000 UTC m=+50.683656798 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/f05d8f86-4adc-47ed-9a27-1e9a00f68a69-registry-tls") pod "image-registry-6cb779f875-pfsr5" (UID: "f05d8f86-4adc-47ed-9a27-1e9a00f68a69") : secret "image-registry-tls" not found Jul 29 22:47:07.772137 ip-10-0-138-112 kubenswrapper[2562]: E0729 22:47:07.771910 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cb3010ad-c703-4901-979e-88a3e1b07ec3-metrics-tls podName:cb3010ad-c703-4901-979e-88a3e1b07ec3 nodeName:}" failed. No retries permitted until 2026-07-29 22:47:15.771896367 +0000 UTC m=+50.683722032 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/cb3010ad-c703-4901-979e-88a3e1b07ec3-metrics-tls") pod "dns-default-ncs4k" (UID: "cb3010ad-c703-4901-979e-88a3e1b07ec3") : secret "dns-default-metrics-tls" not found Jul 29 22:47:13.363116 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:13.363070 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-rhx9q"] Jul 29 22:47:13.415317 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:13.415271 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-rhx9q"] Jul 29 22:47:13.415543 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:13.415328 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-rhx9q" Jul 29 22:47:13.419621 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:13.419594 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 29 22:47:13.419621 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:13.419614 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-m9mf7\"" Jul 29 22:47:13.419808 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:13.419631 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 29 22:47:13.420856 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:13.420821 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 29 22:47:13.421011 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:13.420895 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 29 22:47:13.509496 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:13.509449 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/ad207e3b-5129-47d5-9151-d39346f17cdd-data-volume\") pod \"insights-runtime-extractor-rhx9q\" (UID: \"ad207e3b-5129-47d5-9151-d39346f17cdd\") " pod="openshift-insights/insights-runtime-extractor-rhx9q" Jul 29 22:47:13.509699 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:13.509528 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ad207e3b-5129-47d5-9151-d39346f17cdd-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-rhx9q\" (UID: \"ad207e3b-5129-47d5-9151-d39346f17cdd\") " pod="openshift-insights/insights-runtime-extractor-rhx9q" Jul 29 22:47:13.509699 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:13.509558 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n8s7w\" (UniqueName: \"kubernetes.io/projected/ad207e3b-5129-47d5-9151-d39346f17cdd-kube-api-access-n8s7w\") pod \"insights-runtime-extractor-rhx9q\" (UID: \"ad207e3b-5129-47d5-9151-d39346f17cdd\") " pod="openshift-insights/insights-runtime-extractor-rhx9q" Jul 29 22:47:13.509699 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:13.509626 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ad207e3b-5129-47d5-9151-d39346f17cdd-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-rhx9q\" (UID: \"ad207e3b-5129-47d5-9151-d39346f17cdd\") " pod="openshift-insights/insights-runtime-extractor-rhx9q" Jul 29 22:47:13.509826 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:13.509707 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/ad207e3b-5129-47d5-9151-d39346f17cdd-crio-socket\") pod \"insights-runtime-extractor-rhx9q\" (UID: \"ad207e3b-5129-47d5-9151-d39346f17cdd\") " pod="openshift-insights/insights-runtime-extractor-rhx9q" Jul 29 22:47:13.610806 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:13.610763 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ad207e3b-5129-47d5-9151-d39346f17cdd-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-rhx9q\" (UID: \"ad207e3b-5129-47d5-9151-d39346f17cdd\") " pod="openshift-insights/insights-runtime-extractor-rhx9q" Jul 29 22:47:13.610983 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:13.610840 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/ad207e3b-5129-47d5-9151-d39346f17cdd-crio-socket\") pod \"insights-runtime-extractor-rhx9q\" (UID: \"ad207e3b-5129-47d5-9151-d39346f17cdd\") " pod="openshift-insights/insights-runtime-extractor-rhx9q" Jul 29 22:47:13.610983 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:13.610882 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/ad207e3b-5129-47d5-9151-d39346f17cdd-data-volume\") pod \"insights-runtime-extractor-rhx9q\" (UID: \"ad207e3b-5129-47d5-9151-d39346f17cdd\") " pod="openshift-insights/insights-runtime-extractor-rhx9q" Jul 29 22:47:13.610983 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:13.610928 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ad207e3b-5129-47d5-9151-d39346f17cdd-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-rhx9q\" (UID: \"ad207e3b-5129-47d5-9151-d39346f17cdd\") " pod="openshift-insights/insights-runtime-extractor-rhx9q" Jul 29 22:47:13.610983 ip-10-0-138-112 kubenswrapper[2562]: E0729 22:47:13.610934 2562 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 29 22:47:13.610983 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:13.610959 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n8s7w\" (UniqueName: \"kubernetes.io/projected/ad207e3b-5129-47d5-9151-d39346f17cdd-kube-api-access-n8s7w\") pod \"insights-runtime-extractor-rhx9q\" (UID: \"ad207e3b-5129-47d5-9151-d39346f17cdd\") " pod="openshift-insights/insights-runtime-extractor-rhx9q" Jul 29 22:47:13.611174 ip-10-0-138-112 kubenswrapper[2562]: E0729 22:47:13.611003 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ad207e3b-5129-47d5-9151-d39346f17cdd-insights-runtime-extractor-tls podName:ad207e3b-5129-47d5-9151-d39346f17cdd nodeName:}" failed. No retries permitted until 2026-07-29 22:47:14.110983594 +0000 UTC m=+49.022809321 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/ad207e3b-5129-47d5-9151-d39346f17cdd-insights-runtime-extractor-tls") pod "insights-runtime-extractor-rhx9q" (UID: "ad207e3b-5129-47d5-9151-d39346f17cdd") : secret "insights-runtime-extractor-tls" not found Jul 29 22:47:13.611174 ip-10-0-138-112 kubenswrapper[2562]: E0729 22:47:13.611035 2562 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 29 22:47:13.611174 ip-10-0-138-112 kubenswrapper[2562]: E0729 22:47:13.611110 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ad207e3b-5129-47d5-9151-d39346f17cdd-kube-rbac-proxy-cm podName:ad207e3b-5129-47d5-9151-d39346f17cdd nodeName:}" failed. No retries permitted until 2026-07-29 22:47:14.111095531 +0000 UTC m=+49.022921195 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/ad207e3b-5129-47d5-9151-d39346f17cdd-kube-rbac-proxy-cm") pod "insights-runtime-extractor-rhx9q" (UID: "ad207e3b-5129-47d5-9151-d39346f17cdd") : configmap "kube-rbac-proxy" not found Jul 29 22:47:13.611174 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:13.611038 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/ad207e3b-5129-47d5-9151-d39346f17cdd-crio-socket\") pod \"insights-runtime-extractor-rhx9q\" (UID: \"ad207e3b-5129-47d5-9151-d39346f17cdd\") " pod="openshift-insights/insights-runtime-extractor-rhx9q" Jul 29 22:47:13.611308 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:13.611237 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/ad207e3b-5129-47d5-9151-d39346f17cdd-data-volume\") pod \"insights-runtime-extractor-rhx9q\" (UID: \"ad207e3b-5129-47d5-9151-d39346f17cdd\") " pod="openshift-insights/insights-runtime-extractor-rhx9q" Jul 29 22:47:13.621615 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:13.621547 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n8s7w\" (UniqueName: \"kubernetes.io/projected/ad207e3b-5129-47d5-9151-d39346f17cdd-kube-api-access-n8s7w\") pod \"insights-runtime-extractor-rhx9q\" (UID: \"ad207e3b-5129-47d5-9151-d39346f17cdd\") " pod="openshift-insights/insights-runtime-extractor-rhx9q" Jul 29 22:47:14.115423 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:14.115367 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ad207e3b-5129-47d5-9151-d39346f17cdd-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-rhx9q\" (UID: \"ad207e3b-5129-47d5-9151-d39346f17cdd\") " pod="openshift-insights/insights-runtime-extractor-rhx9q" Jul 29 22:47:14.115636 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:14.115463 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ad207e3b-5129-47d5-9151-d39346f17cdd-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-rhx9q\" (UID: \"ad207e3b-5129-47d5-9151-d39346f17cdd\") " pod="openshift-insights/insights-runtime-extractor-rhx9q" Jul 29 22:47:14.115636 ip-10-0-138-112 kubenswrapper[2562]: E0729 22:47:14.115491 2562 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 29 22:47:14.115636 ip-10-0-138-112 kubenswrapper[2562]: E0729 22:47:14.115576 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ad207e3b-5129-47d5-9151-d39346f17cdd-kube-rbac-proxy-cm podName:ad207e3b-5129-47d5-9151-d39346f17cdd nodeName:}" failed. No retries permitted until 2026-07-29 22:47:15.115558042 +0000 UTC m=+50.027383707 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/ad207e3b-5129-47d5-9151-d39346f17cdd-kube-rbac-proxy-cm") pod "insights-runtime-extractor-rhx9q" (UID: "ad207e3b-5129-47d5-9151-d39346f17cdd") : configmap "kube-rbac-proxy" not found Jul 29 22:47:14.115636 ip-10-0-138-112 kubenswrapper[2562]: E0729 22:47:14.115587 2562 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 29 22:47:14.115636 ip-10-0-138-112 kubenswrapper[2562]: E0729 22:47:14.115638 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ad207e3b-5129-47d5-9151-d39346f17cdd-insights-runtime-extractor-tls podName:ad207e3b-5129-47d5-9151-d39346f17cdd nodeName:}" failed. No retries permitted until 2026-07-29 22:47:15.115622083 +0000 UTC m=+50.027447748 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/ad207e3b-5129-47d5-9151-d39346f17cdd-insights-runtime-extractor-tls") pod "insights-runtime-extractor-rhx9q" (UID: "ad207e3b-5129-47d5-9151-d39346f17cdd") : secret "insights-runtime-extractor-tls" not found Jul 29 22:47:15.125393 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:15.125324 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ad207e3b-5129-47d5-9151-d39346f17cdd-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-rhx9q\" (UID: \"ad207e3b-5129-47d5-9151-d39346f17cdd\") " pod="openshift-insights/insights-runtime-extractor-rhx9q" Jul 29 22:47:15.125832 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:15.125454 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ad207e3b-5129-47d5-9151-d39346f17cdd-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-rhx9q\" (UID: \"ad207e3b-5129-47d5-9151-d39346f17cdd\") " pod="openshift-insights/insights-runtime-extractor-rhx9q" Jul 29 22:47:15.125832 ip-10-0-138-112 kubenswrapper[2562]: E0729 22:47:15.125504 2562 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 29 22:47:15.125832 ip-10-0-138-112 kubenswrapper[2562]: E0729 22:47:15.125563 2562 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 29 22:47:15.125832 ip-10-0-138-112 kubenswrapper[2562]: E0729 22:47:15.125578 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ad207e3b-5129-47d5-9151-d39346f17cdd-kube-rbac-proxy-cm podName:ad207e3b-5129-47d5-9151-d39346f17cdd nodeName:}" failed. No retries permitted until 2026-07-29 22:47:17.125561497 +0000 UTC m=+52.037387162 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/ad207e3b-5129-47d5-9151-d39346f17cdd-kube-rbac-proxy-cm") pod "insights-runtime-extractor-rhx9q" (UID: "ad207e3b-5129-47d5-9151-d39346f17cdd") : configmap "kube-rbac-proxy" not found Jul 29 22:47:15.125832 ip-10-0-138-112 kubenswrapper[2562]: E0729 22:47:15.125619 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ad207e3b-5129-47d5-9151-d39346f17cdd-insights-runtime-extractor-tls podName:ad207e3b-5129-47d5-9151-d39346f17cdd nodeName:}" failed. No retries permitted until 2026-07-29 22:47:17.125606121 +0000 UTC m=+52.037431785 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/ad207e3b-5129-47d5-9151-d39346f17cdd-insights-runtime-extractor-tls") pod "insights-runtime-extractor-rhx9q" (UID: "ad207e3b-5129-47d5-9151-d39346f17cdd") : secret "insights-runtime-extractor-tls" not found Jul 29 22:47:15.266771 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:15.266747 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-k8fl9_51d8cffe-16b4-4bab-9193-953163e45417/dns-node-resolver/0.log" Jul 29 22:47:15.831328 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:15.831290 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/cb3010ad-c703-4901-979e-88a3e1b07ec3-metrics-tls\") pod \"dns-default-ncs4k\" (UID: \"cb3010ad-c703-4901-979e-88a3e1b07ec3\") " pod="openshift-dns/dns-default-ncs4k" Jul 29 22:47:15.831532 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:15.831367 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f05d8f86-4adc-47ed-9a27-1e9a00f68a69-registry-tls\") pod \"image-registry-6cb779f875-pfsr5\" (UID: \"f05d8f86-4adc-47ed-9a27-1e9a00f68a69\") " pod="openshift-image-registry/image-registry-6cb779f875-pfsr5" Jul 29 22:47:15.831532 ip-10-0-138-112 kubenswrapper[2562]: E0729 22:47:15.831480 2562 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 29 22:47:15.831611 ip-10-0-138-112 kubenswrapper[2562]: E0729 22:47:15.831569 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cb3010ad-c703-4901-979e-88a3e1b07ec3-metrics-tls podName:cb3010ad-c703-4901-979e-88a3e1b07ec3 nodeName:}" failed. No retries permitted until 2026-07-29 22:47:31.831549613 +0000 UTC m=+66.743375283 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/cb3010ad-c703-4901-979e-88a3e1b07ec3-metrics-tls") pod "dns-default-ncs4k" (UID: "cb3010ad-c703-4901-979e-88a3e1b07ec3") : secret "dns-default-metrics-tls" not found Jul 29 22:47:15.831611 ip-10-0-138-112 kubenswrapper[2562]: E0729 22:47:15.831482 2562 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 29 22:47:15.831611 ip-10-0-138-112 kubenswrapper[2562]: E0729 22:47:15.831594 2562 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6cb779f875-pfsr5: secret "image-registry-tls" not found Jul 29 22:47:15.831709 ip-10-0-138-112 kubenswrapper[2562]: E0729 22:47:15.831636 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f05d8f86-4adc-47ed-9a27-1e9a00f68a69-registry-tls podName:f05d8f86-4adc-47ed-9a27-1e9a00f68a69 nodeName:}" failed. No retries permitted until 2026-07-29 22:47:31.831623664 +0000 UTC m=+66.743449329 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/f05d8f86-4adc-47ed-9a27-1e9a00f68a69-registry-tls") pod "image-registry-6cb779f875-pfsr5" (UID: "f05d8f86-4adc-47ed-9a27-1e9a00f68a69") : secret "image-registry-tls" not found Jul 29 22:47:16.466114 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:16.466085 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-vwtj2_f6d8c777-cb9a-4505-8e30-ffaa01e53239/node-ca/0.log" Jul 29 22:47:16.867471 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:16.867439 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-c5tdl_9b18dc65-d191-4eef-b23e-14561cbe95f1/serve-healthcheck-canary/0.log" Jul 29 22:47:17.141416 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:17.141316 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ad207e3b-5129-47d5-9151-d39346f17cdd-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-rhx9q\" (UID: \"ad207e3b-5129-47d5-9151-d39346f17cdd\") " pod="openshift-insights/insights-runtime-extractor-rhx9q" Jul 29 22:47:17.141416 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:17.141377 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ad207e3b-5129-47d5-9151-d39346f17cdd-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-rhx9q\" (UID: \"ad207e3b-5129-47d5-9151-d39346f17cdd\") " pod="openshift-insights/insights-runtime-extractor-rhx9q" Jul 29 22:47:17.141605 ip-10-0-138-112 kubenswrapper[2562]: E0729 22:47:17.141486 2562 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 29 22:47:17.141605 ip-10-0-138-112 kubenswrapper[2562]: E0729 22:47:17.141541 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ad207e3b-5129-47d5-9151-d39346f17cdd-insights-runtime-extractor-tls podName:ad207e3b-5129-47d5-9151-d39346f17cdd nodeName:}" failed. No retries permitted until 2026-07-29 22:47:21.141527568 +0000 UTC m=+56.053353233 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/ad207e3b-5129-47d5-9151-d39346f17cdd-insights-runtime-extractor-tls") pod "insights-runtime-extractor-rhx9q" (UID: "ad207e3b-5129-47d5-9151-d39346f17cdd") : secret "insights-runtime-extractor-tls" not found Jul 29 22:47:17.141605 ip-10-0-138-112 kubenswrapper[2562]: E0729 22:47:17.141558 2562 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 29 22:47:17.141702 ip-10-0-138-112 kubenswrapper[2562]: E0729 22:47:17.141611 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ad207e3b-5129-47d5-9151-d39346f17cdd-kube-rbac-proxy-cm podName:ad207e3b-5129-47d5-9151-d39346f17cdd nodeName:}" failed. No retries permitted until 2026-07-29 22:47:21.141599108 +0000 UTC m=+56.053424773 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/ad207e3b-5129-47d5-9151-d39346f17cdd-kube-rbac-proxy-cm") pod "insights-runtime-extractor-rhx9q" (UID: "ad207e3b-5129-47d5-9151-d39346f17cdd") : configmap "kube-rbac-proxy" not found Jul 29 22:47:21.167664 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:21.167616 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ad207e3b-5129-47d5-9151-d39346f17cdd-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-rhx9q\" (UID: \"ad207e3b-5129-47d5-9151-d39346f17cdd\") " pod="openshift-insights/insights-runtime-extractor-rhx9q" Jul 29 22:47:21.168174 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:21.167732 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ad207e3b-5129-47d5-9151-d39346f17cdd-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-rhx9q\" (UID: \"ad207e3b-5129-47d5-9151-d39346f17cdd\") " pod="openshift-insights/insights-runtime-extractor-rhx9q" Jul 29 22:47:21.168174 ip-10-0-138-112 kubenswrapper[2562]: E0729 22:47:21.167764 2562 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 29 22:47:21.168174 ip-10-0-138-112 kubenswrapper[2562]: E0729 22:47:21.167828 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ad207e3b-5129-47d5-9151-d39346f17cdd-insights-runtime-extractor-tls podName:ad207e3b-5129-47d5-9151-d39346f17cdd nodeName:}" failed. No retries permitted until 2026-07-29 22:47:29.167813106 +0000 UTC m=+64.079638771 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/ad207e3b-5129-47d5-9151-d39346f17cdd-insights-runtime-extractor-tls") pod "insights-runtime-extractor-rhx9q" (UID: "ad207e3b-5129-47d5-9151-d39346f17cdd") : secret "insights-runtime-extractor-tls" not found Jul 29 22:47:21.168174 ip-10-0-138-112 kubenswrapper[2562]: E0729 22:47:21.167842 2562 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 29 22:47:21.168174 ip-10-0-138-112 kubenswrapper[2562]: E0729 22:47:21.167894 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ad207e3b-5129-47d5-9151-d39346f17cdd-kube-rbac-proxy-cm podName:ad207e3b-5129-47d5-9151-d39346f17cdd nodeName:}" failed. No retries permitted until 2026-07-29 22:47:29.167879642 +0000 UTC m=+64.079705307 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/ad207e3b-5129-47d5-9151-d39346f17cdd-kube-rbac-proxy-cm") pod "insights-runtime-extractor-rhx9q" (UID: "ad207e3b-5129-47d5-9151-d39346f17cdd") : configmap "kube-rbac-proxy" not found Jul 29 22:47:24.719291 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:24.719260 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-9g28r" Jul 29 22:47:29.232545 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:29.232484 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ad207e3b-5129-47d5-9151-d39346f17cdd-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-rhx9q\" (UID: \"ad207e3b-5129-47d5-9151-d39346f17cdd\") " pod="openshift-insights/insights-runtime-extractor-rhx9q" Jul 29 22:47:29.232942 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:29.232595 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ad207e3b-5129-47d5-9151-d39346f17cdd-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-rhx9q\" (UID: \"ad207e3b-5129-47d5-9151-d39346f17cdd\") " pod="openshift-insights/insights-runtime-extractor-rhx9q" Jul 29 22:47:29.232942 ip-10-0-138-112 kubenswrapper[2562]: E0729 22:47:29.232648 2562 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 29 22:47:29.232942 ip-10-0-138-112 kubenswrapper[2562]: E0729 22:47:29.232730 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ad207e3b-5129-47d5-9151-d39346f17cdd-insights-runtime-extractor-tls podName:ad207e3b-5129-47d5-9151-d39346f17cdd nodeName:}" failed. No retries permitted until 2026-07-29 22:47:45.232711866 +0000 UTC m=+80.144537535 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/ad207e3b-5129-47d5-9151-d39346f17cdd-insights-runtime-extractor-tls") pod "insights-runtime-extractor-rhx9q" (UID: "ad207e3b-5129-47d5-9151-d39346f17cdd") : secret "insights-runtime-extractor-tls" not found Jul 29 22:47:29.233082 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:29.233065 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ad207e3b-5129-47d5-9151-d39346f17cdd-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-rhx9q\" (UID: \"ad207e3b-5129-47d5-9151-d39346f17cdd\") " pod="openshift-insights/insights-runtime-extractor-rhx9q" Jul 29 22:47:31.247596 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:31.247545 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/916b3077-78fa-4054-921c-2e7124dc35f8-original-pull-secret\") pod \"global-pull-secret-syncer-nkzq5\" (UID: \"916b3077-78fa-4054-921c-2e7124dc35f8\") " pod="kube-system/global-pull-secret-syncer-nkzq5" Jul 29 22:47:31.250840 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:31.250815 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 29 22:47:31.260707 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:31.260681 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/916b3077-78fa-4054-921c-2e7124dc35f8-original-pull-secret\") pod \"global-pull-secret-syncer-nkzq5\" (UID: \"916b3077-78fa-4054-921c-2e7124dc35f8\") " pod="kube-system/global-pull-secret-syncer-nkzq5" Jul 29 22:47:31.296154 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:31.296115 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nkzq5" Jul 29 22:47:31.349106 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:31.349028 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f0a6baed-8bea-4992-8852-0b84808a78d1-metrics-certs\") pod \"network-metrics-daemon-fxb52\" (UID: \"f0a6baed-8bea-4992-8852-0b84808a78d1\") " pod="openshift-multus/network-metrics-daemon-fxb52" Jul 29 22:47:31.352218 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:31.352193 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 29 22:47:31.362016 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:31.361962 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f0a6baed-8bea-4992-8852-0b84808a78d1-metrics-certs\") pod \"network-metrics-daemon-fxb52\" (UID: \"f0a6baed-8bea-4992-8852-0b84808a78d1\") " pod="openshift-multus/network-metrics-daemon-fxb52" Jul 29 22:47:31.428848 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:31.428814 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-nkzq5"] Jul 29 22:47:31.432437 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:47:31.432394 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod916b3077_78fa_4054_921c_2e7124dc35f8.slice/crio-e84e962f986b1b0d92157761db94abbdccd28e53d5459090241a6fcbbdfba558 WatchSource:0}: Error finding container e84e962f986b1b0d92157761db94abbdccd28e53d5459090241a6fcbbdfba558: Status 404 returned error can't find the container with id e84e962f986b1b0d92157761db94abbdccd28e53d5459090241a6fcbbdfba558 Jul 29 22:47:31.450140 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:31.450106 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5kv55\" (UniqueName: \"kubernetes.io/projected/2771bdbc-d054-464b-a01f-82344308451f-kube-api-access-5kv55\") pod \"network-check-target-z92kj\" (UID: \"2771bdbc-d054-464b-a01f-82344308451f\") " pod="openshift-network-diagnostics/network-check-target-z92kj" Jul 29 22:47:31.453518 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:31.453490 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 29 22:47:31.463327 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:31.463299 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 29 22:47:31.473477 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:31.473450 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5kv55\" (UniqueName: \"kubernetes.io/projected/2771bdbc-d054-464b-a01f-82344308451f-kube-api-access-5kv55\") pod \"network-check-target-z92kj\" (UID: \"2771bdbc-d054-464b-a01f-82344308451f\") " pod="openshift-network-diagnostics/network-check-target-z92kj" Jul 29 22:47:31.585705 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:31.585678 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-hgxlh\"" Jul 29 22:47:31.591943 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:31.591924 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-s6c2g\"" Jul 29 22:47:31.592937 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:31.592921 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fxb52" Jul 29 22:47:31.599494 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:31.599472 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z92kj" Jul 29 22:47:31.756115 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:31.756082 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-fxb52"] Jul 29 22:47:31.760319 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:47:31.760293 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf0a6baed_8bea_4992_8852_0b84808a78d1.slice/crio-61adf6fff48c14e3208ad01dc375b7d21d992507dd6e048226e9cc4e171ef8b7 WatchSource:0}: Error finding container 61adf6fff48c14e3208ad01dc375b7d21d992507dd6e048226e9cc4e171ef8b7: Status 404 returned error can't find the container with id 61adf6fff48c14e3208ad01dc375b7d21d992507dd6e048226e9cc4e171ef8b7 Jul 29 22:47:31.776261 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:31.776236 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-z92kj"] Jul 29 22:47:31.779819 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:47:31.779794 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2771bdbc_d054_464b_a01f_82344308451f.slice/crio-d510afbd01543b5eaabca13f7e1d1bad7541ec8e773686b23f8fad9c908d10d5 WatchSource:0}: Error finding container d510afbd01543b5eaabca13f7e1d1bad7541ec8e773686b23f8fad9c908d10d5: Status 404 returned error can't find the container with id d510afbd01543b5eaabca13f7e1d1bad7541ec8e773686b23f8fad9c908d10d5 Jul 29 22:47:31.813034 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:31.812991 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-fxb52" event={"ID":"f0a6baed-8bea-4992-8852-0b84808a78d1","Type":"ContainerStarted","Data":"61adf6fff48c14e3208ad01dc375b7d21d992507dd6e048226e9cc4e171ef8b7"} Jul 29 22:47:31.814045 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:31.814019 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-z92kj" event={"ID":"2771bdbc-d054-464b-a01f-82344308451f","Type":"ContainerStarted","Data":"d510afbd01543b5eaabca13f7e1d1bad7541ec8e773686b23f8fad9c908d10d5"} Jul 29 22:47:31.814994 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:31.814975 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-nkzq5" event={"ID":"916b3077-78fa-4054-921c-2e7124dc35f8","Type":"ContainerStarted","Data":"e84e962f986b1b0d92157761db94abbdccd28e53d5459090241a6fcbbdfba558"} Jul 29 22:47:31.852521 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:31.852443 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f05d8f86-4adc-47ed-9a27-1e9a00f68a69-registry-tls\") pod \"image-registry-6cb779f875-pfsr5\" (UID: \"f05d8f86-4adc-47ed-9a27-1e9a00f68a69\") " pod="openshift-image-registry/image-registry-6cb779f875-pfsr5" Jul 29 22:47:31.852521 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:31.852499 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/cb3010ad-c703-4901-979e-88a3e1b07ec3-metrics-tls\") pod \"dns-default-ncs4k\" (UID: \"cb3010ad-c703-4901-979e-88a3e1b07ec3\") " pod="openshift-dns/dns-default-ncs4k" Jul 29 22:47:31.854711 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:31.854688 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/cb3010ad-c703-4901-979e-88a3e1b07ec3-metrics-tls\") pod \"dns-default-ncs4k\" (UID: \"cb3010ad-c703-4901-979e-88a3e1b07ec3\") " pod="openshift-dns/dns-default-ncs4k" Jul 29 22:47:31.854822 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:31.854804 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f05d8f86-4adc-47ed-9a27-1e9a00f68a69-registry-tls\") pod \"image-registry-6cb779f875-pfsr5\" (UID: \"f05d8f86-4adc-47ed-9a27-1e9a00f68a69\") " pod="openshift-image-registry/image-registry-6cb779f875-pfsr5" Jul 29 22:47:32.056684 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:32.056646 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-vqjws\"" Jul 29 22:47:32.064212 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:32.064184 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6cb779f875-pfsr5" Jul 29 22:47:32.068054 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:32.068027 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-tpckt\"" Jul 29 22:47:32.075629 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:32.075604 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-ncs4k" Jul 29 22:47:32.235080 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:32.235020 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6cb779f875-pfsr5"] Jul 29 22:47:32.239171 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:47:32.239139 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf05d8f86_4adc_47ed_9a27_1e9a00f68a69.slice/crio-ad449c565a8e65dce7fbc8deef182a3bbf59c1ba370beb2b4f9ab59b1c81a1bb WatchSource:0}: Error finding container ad449c565a8e65dce7fbc8deef182a3bbf59c1ba370beb2b4f9ab59b1c81a1bb: Status 404 returned error can't find the container with id ad449c565a8e65dce7fbc8deef182a3bbf59c1ba370beb2b4f9ab59b1c81a1bb Jul 29 22:47:32.248069 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:32.248035 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-ncs4k"] Jul 29 22:47:32.253602 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:47:32.253571 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcb3010ad_c703_4901_979e_88a3e1b07ec3.slice/crio-aaa5d67f7e6e4b3ca87d33fc2472fcd065d4a4f540ec49e885ca0e95aaf7b72b WatchSource:0}: Error finding container aaa5d67f7e6e4b3ca87d33fc2472fcd065d4a4f540ec49e885ca0e95aaf7b72b: Status 404 returned error can't find the container with id aaa5d67f7e6e4b3ca87d33fc2472fcd065d4a4f540ec49e885ca0e95aaf7b72b Jul 29 22:47:32.821861 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:32.820963 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6cb779f875-pfsr5" event={"ID":"f05d8f86-4adc-47ed-9a27-1e9a00f68a69","Type":"ContainerStarted","Data":"96d24c293c0e84201a05eadb06c379ef2d497547bd40b26565540383ee0e9c47"} Jul 29 22:47:32.821861 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:32.821023 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6cb779f875-pfsr5" event={"ID":"f05d8f86-4adc-47ed-9a27-1e9a00f68a69","Type":"ContainerStarted","Data":"ad449c565a8e65dce7fbc8deef182a3bbf59c1ba370beb2b4f9ab59b1c81a1bb"} Jul 29 22:47:32.821861 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:32.821796 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-6cb779f875-pfsr5" Jul 29 22:47:32.824468 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:32.824391 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-ncs4k" event={"ID":"cb3010ad-c703-4901-979e-88a3e1b07ec3","Type":"ContainerStarted","Data":"aaa5d67f7e6e4b3ca87d33fc2472fcd065d4a4f540ec49e885ca0e95aaf7b72b"} Jul 29 22:47:33.831762 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:33.831101 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-fxb52" event={"ID":"f0a6baed-8bea-4992-8852-0b84808a78d1","Type":"ContainerStarted","Data":"a229eaafc7959d2e27a249b6f026f43fb8d9fc25b6f1a33798d28fee60dcd4a7"} Jul 29 22:47:33.831762 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:33.831207 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-fxb52" event={"ID":"f0a6baed-8bea-4992-8852-0b84808a78d1","Type":"ContainerStarted","Data":"965b3643f7ccf198b0cd83292dc446fb74e67e842bee31ad397ba53a6029ca61"} Jul 29 22:47:33.852969 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:33.852475 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-fxb52" podStartSLOduration=67.318574614 podStartE2EDuration="1m8.852453867s" totalImagesPullingTime="1.533879253s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 22:46:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 22:47:33.850791053 +0000 UTC m=+68.762616740" watchObservedRunningTime="2026-07-29 22:47:33.852453867 +0000 UTC m=+68.764279555" Jul 29 22:47:33.852969 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:33.852782 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-6cb779f875-pfsr5" podStartSLOduration=67.852770681 podStartE2EDuration="1m7.852770681s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 22:46:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 22:47:32.843116904 +0000 UTC m=+67.754942595" watchObservedRunningTime="2026-07-29 22:47:33.852770681 +0000 UTC m=+68.764596369" Jul 29 22:47:36.842742 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:36.842698 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-z92kj" event={"ID":"2771bdbc-d054-464b-a01f-82344308451f","Type":"ContainerStarted","Data":"1232d1582a59e823aa14932c68d9ba507685464d5f7cafb27cbbc03dff8995c4"} Jul 29 22:47:36.843222 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:36.842802 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-z92kj" Jul 29 22:47:36.844164 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:36.844137 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-nkzq5" event={"ID":"916b3077-78fa-4054-921c-2e7124dc35f8","Type":"ContainerStarted","Data":"297a89763211963bbaccf1f72ea5c32a8234e12022f8d932ba024f63c7780e09"} Jul 29 22:47:36.845441 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:36.845421 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-ncs4k" event={"ID":"cb3010ad-c703-4901-979e-88a3e1b07ec3","Type":"ContainerStarted","Data":"68a62d1c1a3aabb3270106fcd34c7806fd74764f55bc5c0a64c36c8343cabc93"} Jul 29 22:47:36.862226 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:36.862167 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-z92kj" podStartSLOduration=66.94154734 podStartE2EDuration="1m11.862147469s" totalImagesPullingTime="4.920600129s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 22:46:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 22:47:36.861191123 +0000 UTC m=+71.773016812" watchObservedRunningTime="2026-07-29 22:47:36.862147469 +0000 UTC m=+71.773973156" Jul 29 22:47:36.877234 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:36.876948 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-nkzq5" podStartSLOduration=66.605335643 podStartE2EDuration="1m11.876932607s" totalImagesPullingTime="5.271596964s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 22:46:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 22:47:36.876649585 +0000 UTC m=+71.788475284" watchObservedRunningTime="2026-07-29 22:47:36.876932607 +0000 UTC m=+71.788758294" Jul 29 22:47:37.851163 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:37.851119 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-ncs4k" event={"ID":"cb3010ad-c703-4901-979e-88a3e1b07ec3","Type":"ContainerStarted","Data":"b9ffd73e23511990e1800af7e4e99a85c8db0dd7773bab88bbcdf93145c71123"} Jul 29 22:47:37.851580 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:37.851419 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-ncs4k" Jul 29 22:47:37.869662 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:37.869608 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-ncs4k" podStartSLOduration=34.436688595 podStartE2EDuration="38.869589623s" totalImagesPullingTime="4.432901028s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 22:46:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 22:47:37.86879624 +0000 UTC m=+72.780621926" watchObservedRunningTime="2026-07-29 22:47:37.869589623 +0000 UTC m=+72.781415311" Jul 29 22:47:40.577247 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:40.577213 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-c4zbl"] Jul 29 22:47:40.580140 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:40.580118 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-c4zbl" Jul 29 22:47:40.583024 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:40.582943 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-496wv\"" Jul 29 22:47:40.584376 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:40.584354 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\"" Jul 29 22:47:40.600116 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:40.600087 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-c4zbl"] Jul 29 22:47:40.603129 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:40.603103 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-6cb779f875-pfsr5"] Jul 29 22:47:40.619222 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:40.619185 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/d8ed6b79-82fb-46b7-bf41-a68dd856fe24-tls-certificates\") pod \"prometheus-operator-admission-webhook-7f8d78f54b-c4zbl\" (UID: \"d8ed6b79-82fb-46b7-bf41-a68dd856fe24\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-c4zbl" Jul 29 22:47:40.712327 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:40.712293 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-7f7448c9db-mqm8x"] Jul 29 22:47:40.714482 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:40.714465 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7f7448c9db-mqm8x" Jul 29 22:47:40.719969 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:40.719946 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/d8ed6b79-82fb-46b7-bf41-a68dd856fe24-tls-certificates\") pod \"prometheus-operator-admission-webhook-7f8d78f54b-c4zbl\" (UID: \"d8ed6b79-82fb-46b7-bf41-a68dd856fe24\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-c4zbl" Jul 29 22:47:40.722551 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:40.722526 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/d8ed6b79-82fb-46b7-bf41-a68dd856fe24-tls-certificates\") pod \"prometheus-operator-admission-webhook-7f8d78f54b-c4zbl\" (UID: \"d8ed6b79-82fb-46b7-bf41-a68dd856fe24\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-c4zbl" Jul 29 22:47:40.734684 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:40.734637 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7f7448c9db-mqm8x"] Jul 29 22:47:40.820900 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:40.820861 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e40c1be2-77bb-4d00-971f-a99e666826eb-registry-tls\") pod \"image-registry-7f7448c9db-mqm8x\" (UID: \"e40c1be2-77bb-4d00-971f-a99e666826eb\") " pod="openshift-image-registry/image-registry-7f7448c9db-mqm8x" Jul 29 22:47:40.820900 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:40.820901 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/e40c1be2-77bb-4d00-971f-a99e666826eb-ca-trust-extracted\") pod \"image-registry-7f7448c9db-mqm8x\" (UID: \"e40c1be2-77bb-4d00-971f-a99e666826eb\") " pod="openshift-image-registry/image-registry-7f7448c9db-mqm8x" Jul 29 22:47:40.821108 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:40.820950 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bdkr5\" (UniqueName: \"kubernetes.io/projected/e40c1be2-77bb-4d00-971f-a99e666826eb-kube-api-access-bdkr5\") pod \"image-registry-7f7448c9db-mqm8x\" (UID: \"e40c1be2-77bb-4d00-971f-a99e666826eb\") " pod="openshift-image-registry/image-registry-7f7448c9db-mqm8x" Jul 29 22:47:40.821108 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:40.820981 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e40c1be2-77bb-4d00-971f-a99e666826eb-bound-sa-token\") pod \"image-registry-7f7448c9db-mqm8x\" (UID: \"e40c1be2-77bb-4d00-971f-a99e666826eb\") " pod="openshift-image-registry/image-registry-7f7448c9db-mqm8x" Jul 29 22:47:40.821108 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:40.821011 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/e40c1be2-77bb-4d00-971f-a99e666826eb-registry-certificates\") pod \"image-registry-7f7448c9db-mqm8x\" (UID: \"e40c1be2-77bb-4d00-971f-a99e666826eb\") " pod="openshift-image-registry/image-registry-7f7448c9db-mqm8x" Jul 29 22:47:40.821108 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:40.821043 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/e40c1be2-77bb-4d00-971f-a99e666826eb-installation-pull-secrets\") pod \"image-registry-7f7448c9db-mqm8x\" (UID: \"e40c1be2-77bb-4d00-971f-a99e666826eb\") " pod="openshift-image-registry/image-registry-7f7448c9db-mqm8x" Jul 29 22:47:40.821108 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:40.821066 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/e40c1be2-77bb-4d00-971f-a99e666826eb-image-registry-private-configuration\") pod \"image-registry-7f7448c9db-mqm8x\" (UID: \"e40c1be2-77bb-4d00-971f-a99e666826eb\") " pod="openshift-image-registry/image-registry-7f7448c9db-mqm8x" Jul 29 22:47:40.821303 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:40.821114 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/e40c1be2-77bb-4d00-971f-a99e666826eb-trusted-ca\") pod \"image-registry-7f7448c9db-mqm8x\" (UID: \"e40c1be2-77bb-4d00-971f-a99e666826eb\") " pod="openshift-image-registry/image-registry-7f7448c9db-mqm8x" Jul 29 22:47:40.888962 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:40.888866 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-c4zbl" Jul 29 22:47:40.921537 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:40.921500 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/e40c1be2-77bb-4d00-971f-a99e666826eb-registry-certificates\") pod \"image-registry-7f7448c9db-mqm8x\" (UID: \"e40c1be2-77bb-4d00-971f-a99e666826eb\") " pod="openshift-image-registry/image-registry-7f7448c9db-mqm8x" Jul 29 22:47:40.921715 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:40.921544 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/e40c1be2-77bb-4d00-971f-a99e666826eb-installation-pull-secrets\") pod \"image-registry-7f7448c9db-mqm8x\" (UID: \"e40c1be2-77bb-4d00-971f-a99e666826eb\") " pod="openshift-image-registry/image-registry-7f7448c9db-mqm8x" Jul 29 22:47:40.921715 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:40.921574 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/e40c1be2-77bb-4d00-971f-a99e666826eb-image-registry-private-configuration\") pod \"image-registry-7f7448c9db-mqm8x\" (UID: \"e40c1be2-77bb-4d00-971f-a99e666826eb\") " pod="openshift-image-registry/image-registry-7f7448c9db-mqm8x" Jul 29 22:47:40.921715 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:40.921660 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/e40c1be2-77bb-4d00-971f-a99e666826eb-trusted-ca\") pod \"image-registry-7f7448c9db-mqm8x\" (UID: \"e40c1be2-77bb-4d00-971f-a99e666826eb\") " pod="openshift-image-registry/image-registry-7f7448c9db-mqm8x" Jul 29 22:47:40.921715 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:40.921691 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e40c1be2-77bb-4d00-971f-a99e666826eb-registry-tls\") pod \"image-registry-7f7448c9db-mqm8x\" (UID: \"e40c1be2-77bb-4d00-971f-a99e666826eb\") " pod="openshift-image-registry/image-registry-7f7448c9db-mqm8x" Jul 29 22:47:40.921935 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:40.921717 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/e40c1be2-77bb-4d00-971f-a99e666826eb-ca-trust-extracted\") pod \"image-registry-7f7448c9db-mqm8x\" (UID: \"e40c1be2-77bb-4d00-971f-a99e666826eb\") " pod="openshift-image-registry/image-registry-7f7448c9db-mqm8x" Jul 29 22:47:40.921935 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:40.921759 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bdkr5\" (UniqueName: \"kubernetes.io/projected/e40c1be2-77bb-4d00-971f-a99e666826eb-kube-api-access-bdkr5\") pod \"image-registry-7f7448c9db-mqm8x\" (UID: \"e40c1be2-77bb-4d00-971f-a99e666826eb\") " pod="openshift-image-registry/image-registry-7f7448c9db-mqm8x" Jul 29 22:47:40.921935 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:40.921805 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e40c1be2-77bb-4d00-971f-a99e666826eb-bound-sa-token\") pod \"image-registry-7f7448c9db-mqm8x\" (UID: \"e40c1be2-77bb-4d00-971f-a99e666826eb\") " pod="openshift-image-registry/image-registry-7f7448c9db-mqm8x" Jul 29 22:47:40.922462 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:40.922430 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/e40c1be2-77bb-4d00-971f-a99e666826eb-ca-trust-extracted\") pod \"image-registry-7f7448c9db-mqm8x\" (UID: \"e40c1be2-77bb-4d00-971f-a99e666826eb\") " pod="openshift-image-registry/image-registry-7f7448c9db-mqm8x" Jul 29 22:47:40.922600 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:40.922506 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/e40c1be2-77bb-4d00-971f-a99e666826eb-registry-certificates\") pod \"image-registry-7f7448c9db-mqm8x\" (UID: \"e40c1be2-77bb-4d00-971f-a99e666826eb\") " pod="openshift-image-registry/image-registry-7f7448c9db-mqm8x" Jul 29 22:47:40.923127 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:40.923102 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/e40c1be2-77bb-4d00-971f-a99e666826eb-trusted-ca\") pod \"image-registry-7f7448c9db-mqm8x\" (UID: \"e40c1be2-77bb-4d00-971f-a99e666826eb\") " pod="openshift-image-registry/image-registry-7f7448c9db-mqm8x" Jul 29 22:47:40.925125 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:40.924967 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e40c1be2-77bb-4d00-971f-a99e666826eb-registry-tls\") pod \"image-registry-7f7448c9db-mqm8x\" (UID: \"e40c1be2-77bb-4d00-971f-a99e666826eb\") " pod="openshift-image-registry/image-registry-7f7448c9db-mqm8x" Jul 29 22:47:40.925225 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:40.925122 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/e40c1be2-77bb-4d00-971f-a99e666826eb-installation-pull-secrets\") pod \"image-registry-7f7448c9db-mqm8x\" (UID: \"e40c1be2-77bb-4d00-971f-a99e666826eb\") " pod="openshift-image-registry/image-registry-7f7448c9db-mqm8x" Jul 29 22:47:40.925805 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:40.925785 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/e40c1be2-77bb-4d00-971f-a99e666826eb-image-registry-private-configuration\") pod \"image-registry-7f7448c9db-mqm8x\" (UID: \"e40c1be2-77bb-4d00-971f-a99e666826eb\") " pod="openshift-image-registry/image-registry-7f7448c9db-mqm8x" Jul 29 22:47:40.935996 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:40.935372 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e40c1be2-77bb-4d00-971f-a99e666826eb-bound-sa-token\") pod \"image-registry-7f7448c9db-mqm8x\" (UID: \"e40c1be2-77bb-4d00-971f-a99e666826eb\") " pod="openshift-image-registry/image-registry-7f7448c9db-mqm8x" Jul 29 22:47:40.935996 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:40.935983 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bdkr5\" (UniqueName: \"kubernetes.io/projected/e40c1be2-77bb-4d00-971f-a99e666826eb-kube-api-access-bdkr5\") pod \"image-registry-7f7448c9db-mqm8x\" (UID: \"e40c1be2-77bb-4d00-971f-a99e666826eb\") " pod="openshift-image-registry/image-registry-7f7448c9db-mqm8x" Jul 29 22:47:41.014462 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:41.014431 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-c4zbl"] Jul 29 22:47:41.017305 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:47:41.017275 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd8ed6b79_82fb_46b7_bf41_a68dd856fe24.slice/crio-3090391dd82fbc4ca28e001d099b97bf336689389fe7974cf1202f5b500ab24d WatchSource:0}: Error finding container 3090391dd82fbc4ca28e001d099b97bf336689389fe7974cf1202f5b500ab24d: Status 404 returned error can't find the container with id 3090391dd82fbc4ca28e001d099b97bf336689389fe7974cf1202f5b500ab24d Jul 29 22:47:41.022731 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:41.022710 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7f7448c9db-mqm8x" Jul 29 22:47:41.156190 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:41.156113 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7f7448c9db-mqm8x"] Jul 29 22:47:41.159112 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:47:41.159078 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode40c1be2_77bb_4d00_971f_a99e666826eb.slice/crio-28bcbadfd05a3f66d53a864df73e7e53e710d9a2a4bc079744414e09359763d5 WatchSource:0}: Error finding container 28bcbadfd05a3f66d53a864df73e7e53e710d9a2a4bc079744414e09359763d5: Status 404 returned error can't find the container with id 28bcbadfd05a3f66d53a864df73e7e53e710d9a2a4bc079744414e09359763d5 Jul 29 22:47:41.867745 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:41.867704 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7f7448c9db-mqm8x" event={"ID":"e40c1be2-77bb-4d00-971f-a99e666826eb","Type":"ContainerStarted","Data":"dc42b0ef0196b9ebe0a275867e958fb3d66899c6eac411080ac69a0d20d06c45"} Jul 29 22:47:41.868224 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:41.867756 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7f7448c9db-mqm8x" event={"ID":"e40c1be2-77bb-4d00-971f-a99e666826eb","Type":"ContainerStarted","Data":"28bcbadfd05a3f66d53a864df73e7e53e710d9a2a4bc079744414e09359763d5"} Jul 29 22:47:41.868224 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:41.867779 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-7f7448c9db-mqm8x" Jul 29 22:47:41.869031 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:41.869000 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-c4zbl" event={"ID":"d8ed6b79-82fb-46b7-bf41-a68dd856fe24","Type":"ContainerStarted","Data":"3090391dd82fbc4ca28e001d099b97bf336689389fe7974cf1202f5b500ab24d"} Jul 29 22:47:41.894336 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:41.894264 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-7f7448c9db-mqm8x" podStartSLOduration=1.894242061 podStartE2EDuration="1.894242061s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 22:47:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 22:47:41.89208671 +0000 UTC m=+76.803912397" watchObservedRunningTime="2026-07-29 22:47:41.894242061 +0000 UTC m=+76.806067747" Jul 29 22:47:42.873726 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:42.873683 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-c4zbl" event={"ID":"d8ed6b79-82fb-46b7-bf41-a68dd856fe24","Type":"ContainerStarted","Data":"f4af0bbf9e1db14ea4124d2cd195597fa73353d7ddc01537520d9e31f8c88788"} Jul 29 22:47:42.874130 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:42.873766 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-c4zbl" Jul 29 22:47:42.878308 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:42.878280 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-c4zbl" Jul 29 22:47:42.891953 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:42.891899 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-c4zbl" podStartSLOduration=1.891709673 podStartE2EDuration="2.891884835s" totalImagesPullingTime="1.000175162s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 22:47:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 22:47:42.890904517 +0000 UTC m=+77.802730206" watchObservedRunningTime="2026-07-29 22:47:42.891884835 +0000 UTC m=+77.803710532" Jul 29 22:47:43.180414 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:43.180315 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-84d69f794d-sw54f"] Jul 29 22:47:43.182487 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:43.182469 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-84d69f794d-sw54f" Jul 29 22:47:43.186085 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:43.186060 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\"" Jul 29 22:47:43.186851 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:43.186832 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 29 22:47:43.186923 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:43.186895 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\"" Jul 29 22:47:43.187569 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:43.187549 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 29 22:47:43.188229 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:43.188205 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-tcldc\"" Jul 29 22:47:43.188322 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:43.188240 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 29 22:47:43.198763 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:43.198742 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-84d69f794d-sw54f"] Jul 29 22:47:43.243565 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:43.243538 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2tvqr\" (UniqueName: \"kubernetes.io/projected/1f5b7f19-7d60-4f37-8943-f4f9f4507ac5-kube-api-access-2tvqr\") pod \"prometheus-operator-84d69f794d-sw54f\" (UID: \"1f5b7f19-7d60-4f37-8943-f4f9f4507ac5\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-sw54f" Jul 29 22:47:43.243729 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:43.243589 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/1f5b7f19-7d60-4f37-8943-f4f9f4507ac5-prometheus-operator-tls\") pod \"prometheus-operator-84d69f794d-sw54f\" (UID: \"1f5b7f19-7d60-4f37-8943-f4f9f4507ac5\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-sw54f" Jul 29 22:47:43.243729 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:43.243629 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1f5b7f19-7d60-4f37-8943-f4f9f4507ac5-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-84d69f794d-sw54f\" (UID: \"1f5b7f19-7d60-4f37-8943-f4f9f4507ac5\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-sw54f" Jul 29 22:47:43.243729 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:43.243646 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1f5b7f19-7d60-4f37-8943-f4f9f4507ac5-metrics-client-ca\") pod \"prometheus-operator-84d69f794d-sw54f\" (UID: \"1f5b7f19-7d60-4f37-8943-f4f9f4507ac5\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-sw54f" Jul 29 22:47:43.344595 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:43.344542 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/1f5b7f19-7d60-4f37-8943-f4f9f4507ac5-prometheus-operator-tls\") pod \"prometheus-operator-84d69f794d-sw54f\" (UID: \"1f5b7f19-7d60-4f37-8943-f4f9f4507ac5\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-sw54f" Jul 29 22:47:43.344595 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:43.344610 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1f5b7f19-7d60-4f37-8943-f4f9f4507ac5-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-84d69f794d-sw54f\" (UID: \"1f5b7f19-7d60-4f37-8943-f4f9f4507ac5\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-sw54f" Jul 29 22:47:43.344819 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:43.344636 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1f5b7f19-7d60-4f37-8943-f4f9f4507ac5-metrics-client-ca\") pod \"prometheus-operator-84d69f794d-sw54f\" (UID: \"1f5b7f19-7d60-4f37-8943-f4f9f4507ac5\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-sw54f" Jul 29 22:47:43.344819 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:43.344713 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2tvqr\" (UniqueName: \"kubernetes.io/projected/1f5b7f19-7d60-4f37-8943-f4f9f4507ac5-kube-api-access-2tvqr\") pod \"prometheus-operator-84d69f794d-sw54f\" (UID: \"1f5b7f19-7d60-4f37-8943-f4f9f4507ac5\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-sw54f" Jul 29 22:47:43.345449 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:43.345424 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1f5b7f19-7d60-4f37-8943-f4f9f4507ac5-metrics-client-ca\") pod \"prometheus-operator-84d69f794d-sw54f\" (UID: \"1f5b7f19-7d60-4f37-8943-f4f9f4507ac5\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-sw54f" Jul 29 22:47:43.346999 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:43.346969 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1f5b7f19-7d60-4f37-8943-f4f9f4507ac5-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-84d69f794d-sw54f\" (UID: \"1f5b7f19-7d60-4f37-8943-f4f9f4507ac5\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-sw54f" Jul 29 22:47:43.347147 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:43.347128 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/1f5b7f19-7d60-4f37-8943-f4f9f4507ac5-prometheus-operator-tls\") pod \"prometheus-operator-84d69f794d-sw54f\" (UID: \"1f5b7f19-7d60-4f37-8943-f4f9f4507ac5\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-sw54f" Jul 29 22:47:43.354357 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:43.354334 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2tvqr\" (UniqueName: \"kubernetes.io/projected/1f5b7f19-7d60-4f37-8943-f4f9f4507ac5-kube-api-access-2tvqr\") pod \"prometheus-operator-84d69f794d-sw54f\" (UID: \"1f5b7f19-7d60-4f37-8943-f4f9f4507ac5\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-sw54f" Jul 29 22:47:43.491661 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:43.491555 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-84d69f794d-sw54f" Jul 29 22:47:43.613311 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:43.613279 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-84d69f794d-sw54f"] Jul 29 22:47:43.616107 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:47:43.616082 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1f5b7f19_7d60_4f37_8943_f4f9f4507ac5.slice/crio-433abb1eb7b0ef6dfe58988fc852d425f200e278cb2554a6f363cc8cd105b67d WatchSource:0}: Error finding container 433abb1eb7b0ef6dfe58988fc852d425f200e278cb2554a6f363cc8cd105b67d: Status 404 returned error can't find the container with id 433abb1eb7b0ef6dfe58988fc852d425f200e278cb2554a6f363cc8cd105b67d Jul 29 22:47:43.877497 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:43.877461 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-84d69f794d-sw54f" event={"ID":"1f5b7f19-7d60-4f37-8943-f4f9f4507ac5","Type":"ContainerStarted","Data":"433abb1eb7b0ef6dfe58988fc852d425f200e278cb2554a6f363cc8cd105b67d"} Jul 29 22:47:44.882353 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:44.882315 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-84d69f794d-sw54f" event={"ID":"1f5b7f19-7d60-4f37-8943-f4f9f4507ac5","Type":"ContainerStarted","Data":"29ec3afdafa928837d69356bad31cf3973e94ac284ce531331aeadde674102f7"} Jul 29 22:47:44.882748 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:44.882432 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-84d69f794d-sw54f" event={"ID":"1f5b7f19-7d60-4f37-8943-f4f9f4507ac5","Type":"ContainerStarted","Data":"b8661a16a9bfddad5b091eee36e5f4c30c8dd4b2f1e27cd2b9a99da09e3c68a1"} Jul 29 22:47:44.904788 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:44.904740 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-84d69f794d-sw54f" podStartSLOduration=0.791975838 podStartE2EDuration="1.904726237s" totalImagesPullingTime="1.112750399s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 22:47:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 22:47:44.904173811 +0000 UTC m=+79.815999497" watchObservedRunningTime="2026-07-29 22:47:44.904726237 +0000 UTC m=+79.816551923" Jul 29 22:47:45.261874 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:45.261827 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ad207e3b-5129-47d5-9151-d39346f17cdd-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-rhx9q\" (UID: \"ad207e3b-5129-47d5-9151-d39346f17cdd\") " pod="openshift-insights/insights-runtime-extractor-rhx9q" Jul 29 22:47:45.264127 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:45.264095 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ad207e3b-5129-47d5-9151-d39346f17cdd-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-rhx9q\" (UID: \"ad207e3b-5129-47d5-9151-d39346f17cdd\") " pod="openshift-insights/insights-runtime-extractor-rhx9q" Jul 29 22:47:45.527315 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:45.527289 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-m9mf7\"" Jul 29 22:47:45.534701 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:45.534675 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-rhx9q" Jul 29 22:47:45.664227 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:45.664190 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-rhx9q"] Jul 29 22:47:45.669153 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:47:45.669124 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podad207e3b_5129_47d5_9151_d39346f17cdd.slice/crio-e73f1a74f8f3f74e9c1bddb9f27750ef472c8d5d184880cb08b2c0517649cabc WatchSource:0}: Error finding container e73f1a74f8f3f74e9c1bddb9f27750ef472c8d5d184880cb08b2c0517649cabc: Status 404 returned error can't find the container with id e73f1a74f8f3f74e9c1bddb9f27750ef472c8d5d184880cb08b2c0517649cabc Jul 29 22:47:45.887526 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:45.887434 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-rhx9q" event={"ID":"ad207e3b-5129-47d5-9151-d39346f17cdd","Type":"ContainerStarted","Data":"e73f1a74f8f3f74e9c1bddb9f27750ef472c8d5d184880cb08b2c0517649cabc"} Jul 29 22:47:46.581572 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:46.581533 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-56d9f944b4-brv22"] Jul 29 22:47:46.584043 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:46.583915 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-zvrsv"] Jul 29 22:47:46.584223 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:46.584203 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-brv22" Jul 29 22:47:46.586383 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:46.586354 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-zvrsv" Jul 29 22:47:46.588111 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:46.588082 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\"" Jul 29 22:47:46.588232 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:46.588142 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\"" Jul 29 22:47:46.588752 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:46.588380 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-zdkdn\"" Jul 29 22:47:46.588752 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:46.588579 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\"" Jul 29 22:47:46.589534 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:46.589515 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 29 22:47:46.589792 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:46.589767 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 29 22:47:46.589878 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:46.589769 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 29 22:47:46.590097 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:46.590075 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-sjfmc\"" Jul 29 22:47:46.601599 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:46.601525 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-56d9f944b4-brv22"] Jul 29 22:47:46.675802 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:46.675755 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/347281f0-0a1e-4710-9d6b-8e81fe4e7613-node-exporter-textfile\") pod \"node-exporter-zvrsv\" (UID: \"347281f0-0a1e-4710-9d6b-8e81fe4e7613\") " pod="openshift-monitoring/node-exporter-zvrsv" Jul 29 22:47:46.675802 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:46.675796 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/347281f0-0a1e-4710-9d6b-8e81fe4e7613-root\") pod \"node-exporter-zvrsv\" (UID: \"347281f0-0a1e-4710-9d6b-8e81fe4e7613\") " pod="openshift-monitoring/node-exporter-zvrsv" Jul 29 22:47:46.676049 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:46.675843 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/347281f0-0a1e-4710-9d6b-8e81fe4e7613-node-exporter-accelerators-collector-config\") pod \"node-exporter-zvrsv\" (UID: \"347281f0-0a1e-4710-9d6b-8e81fe4e7613\") " pod="openshift-monitoring/node-exporter-zvrsv" Jul 29 22:47:46.676049 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:46.675906 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/347281f0-0a1e-4710-9d6b-8e81fe4e7613-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-zvrsv\" (UID: \"347281f0-0a1e-4710-9d6b-8e81fe4e7613\") " pod="openshift-monitoring/node-exporter-zvrsv" Jul 29 22:47:46.676049 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:46.675931 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/347281f0-0a1e-4710-9d6b-8e81fe4e7613-sys\") pod \"node-exporter-zvrsv\" (UID: \"347281f0-0a1e-4710-9d6b-8e81fe4e7613\") " pod="openshift-monitoring/node-exporter-zvrsv" Jul 29 22:47:46.676049 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:46.675967 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/83c2796a-d9a0-45be-a1dd-d9e3e3d11108-volume-directive-shadow\") pod \"kube-state-metrics-56d9f944b4-brv22\" (UID: \"83c2796a-d9a0-45be-a1dd-d9e3e3d11108\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-brv22" Jul 29 22:47:46.676049 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:46.675991 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gmdpk\" (UniqueName: \"kubernetes.io/projected/83c2796a-d9a0-45be-a1dd-d9e3e3d11108-kube-api-access-gmdpk\") pod \"kube-state-metrics-56d9f944b4-brv22\" (UID: \"83c2796a-d9a0-45be-a1dd-d9e3e3d11108\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-brv22" Jul 29 22:47:46.676049 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:46.676017 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/347281f0-0a1e-4710-9d6b-8e81fe4e7613-node-exporter-wtmp\") pod \"node-exporter-zvrsv\" (UID: \"347281f0-0a1e-4710-9d6b-8e81fe4e7613\") " pod="openshift-monitoring/node-exporter-zvrsv" Jul 29 22:47:46.676305 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:46.676108 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2t7xx\" (UniqueName: \"kubernetes.io/projected/347281f0-0a1e-4710-9d6b-8e81fe4e7613-kube-api-access-2t7xx\") pod \"node-exporter-zvrsv\" (UID: \"347281f0-0a1e-4710-9d6b-8e81fe4e7613\") " pod="openshift-monitoring/node-exporter-zvrsv" Jul 29 22:47:46.676305 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:46.676174 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/83c2796a-d9a0-45be-a1dd-d9e3e3d11108-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56d9f944b4-brv22\" (UID: \"83c2796a-d9a0-45be-a1dd-d9e3e3d11108\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-brv22" Jul 29 22:47:46.676305 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:46.676209 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/83c2796a-d9a0-45be-a1dd-d9e3e3d11108-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56d9f944b4-brv22\" (UID: \"83c2796a-d9a0-45be-a1dd-d9e3e3d11108\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-brv22" Jul 29 22:47:46.676305 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:46.676239 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/347281f0-0a1e-4710-9d6b-8e81fe4e7613-node-exporter-tls\") pod \"node-exporter-zvrsv\" (UID: \"347281f0-0a1e-4710-9d6b-8e81fe4e7613\") " pod="openshift-monitoring/node-exporter-zvrsv" Jul 29 22:47:46.676305 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:46.676254 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/347281f0-0a1e-4710-9d6b-8e81fe4e7613-metrics-client-ca\") pod \"node-exporter-zvrsv\" (UID: \"347281f0-0a1e-4710-9d6b-8e81fe4e7613\") " pod="openshift-monitoring/node-exporter-zvrsv" Jul 29 22:47:46.676305 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:46.676290 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/83c2796a-d9a0-45be-a1dd-d9e3e3d11108-metrics-client-ca\") pod \"kube-state-metrics-56d9f944b4-brv22\" (UID: \"83c2796a-d9a0-45be-a1dd-d9e3e3d11108\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-brv22" Jul 29 22:47:46.676570 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:46.676309 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/83c2796a-d9a0-45be-a1dd-d9e3e3d11108-kube-state-metrics-tls\") pod \"kube-state-metrics-56d9f944b4-brv22\" (UID: \"83c2796a-d9a0-45be-a1dd-d9e3e3d11108\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-brv22" Jul 29 22:47:46.777717 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:46.777675 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/347281f0-0a1e-4710-9d6b-8e81fe4e7613-node-exporter-textfile\") pod \"node-exporter-zvrsv\" (UID: \"347281f0-0a1e-4710-9d6b-8e81fe4e7613\") " pod="openshift-monitoring/node-exporter-zvrsv" Jul 29 22:47:46.777897 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:46.777729 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/347281f0-0a1e-4710-9d6b-8e81fe4e7613-root\") pod \"node-exporter-zvrsv\" (UID: \"347281f0-0a1e-4710-9d6b-8e81fe4e7613\") " pod="openshift-monitoring/node-exporter-zvrsv" Jul 29 22:47:46.777897 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:46.777774 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/347281f0-0a1e-4710-9d6b-8e81fe4e7613-node-exporter-accelerators-collector-config\") pod \"node-exporter-zvrsv\" (UID: \"347281f0-0a1e-4710-9d6b-8e81fe4e7613\") " pod="openshift-monitoring/node-exporter-zvrsv" Jul 29 22:47:46.777897 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:46.777810 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/347281f0-0a1e-4710-9d6b-8e81fe4e7613-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-zvrsv\" (UID: \"347281f0-0a1e-4710-9d6b-8e81fe4e7613\") " pod="openshift-monitoring/node-exporter-zvrsv" Jul 29 22:47:46.777897 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:46.777868 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/347281f0-0a1e-4710-9d6b-8e81fe4e7613-sys\") pod \"node-exporter-zvrsv\" (UID: \"347281f0-0a1e-4710-9d6b-8e81fe4e7613\") " pod="openshift-monitoring/node-exporter-zvrsv" Jul 29 22:47:46.778110 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:46.777940 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/83c2796a-d9a0-45be-a1dd-d9e3e3d11108-volume-directive-shadow\") pod \"kube-state-metrics-56d9f944b4-brv22\" (UID: \"83c2796a-d9a0-45be-a1dd-d9e3e3d11108\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-brv22" Jul 29 22:47:46.778110 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:46.777967 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gmdpk\" (UniqueName: \"kubernetes.io/projected/83c2796a-d9a0-45be-a1dd-d9e3e3d11108-kube-api-access-gmdpk\") pod \"kube-state-metrics-56d9f944b4-brv22\" (UID: \"83c2796a-d9a0-45be-a1dd-d9e3e3d11108\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-brv22" Jul 29 22:47:46.778110 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:46.777994 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/347281f0-0a1e-4710-9d6b-8e81fe4e7613-node-exporter-wtmp\") pod \"node-exporter-zvrsv\" (UID: \"347281f0-0a1e-4710-9d6b-8e81fe4e7613\") " pod="openshift-monitoring/node-exporter-zvrsv" Jul 29 22:47:46.778110 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:46.778004 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/347281f0-0a1e-4710-9d6b-8e81fe4e7613-sys\") pod \"node-exporter-zvrsv\" (UID: \"347281f0-0a1e-4710-9d6b-8e81fe4e7613\") " pod="openshift-monitoring/node-exporter-zvrsv" Jul 29 22:47:46.778110 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:46.778025 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2t7xx\" (UniqueName: \"kubernetes.io/projected/347281f0-0a1e-4710-9d6b-8e81fe4e7613-kube-api-access-2t7xx\") pod \"node-exporter-zvrsv\" (UID: \"347281f0-0a1e-4710-9d6b-8e81fe4e7613\") " pod="openshift-monitoring/node-exporter-zvrsv" Jul 29 22:47:46.778110 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:46.778079 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/83c2796a-d9a0-45be-a1dd-d9e3e3d11108-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56d9f944b4-brv22\" (UID: \"83c2796a-d9a0-45be-a1dd-d9e3e3d11108\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-brv22" Jul 29 22:47:46.778373 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:46.778111 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/83c2796a-d9a0-45be-a1dd-d9e3e3d11108-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56d9f944b4-brv22\" (UID: \"83c2796a-d9a0-45be-a1dd-d9e3e3d11108\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-brv22" Jul 29 22:47:46.778373 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:46.778125 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/347281f0-0a1e-4710-9d6b-8e81fe4e7613-node-exporter-textfile\") pod \"node-exporter-zvrsv\" (UID: \"347281f0-0a1e-4710-9d6b-8e81fe4e7613\") " pod="openshift-monitoring/node-exporter-zvrsv" Jul 29 22:47:46.778373 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:46.778145 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/347281f0-0a1e-4710-9d6b-8e81fe4e7613-node-exporter-tls\") pod \"node-exporter-zvrsv\" (UID: \"347281f0-0a1e-4710-9d6b-8e81fe4e7613\") " pod="openshift-monitoring/node-exporter-zvrsv" Jul 29 22:47:46.778373 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:46.778172 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/347281f0-0a1e-4710-9d6b-8e81fe4e7613-metrics-client-ca\") pod \"node-exporter-zvrsv\" (UID: \"347281f0-0a1e-4710-9d6b-8e81fe4e7613\") " pod="openshift-monitoring/node-exporter-zvrsv" Jul 29 22:47:46.778373 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:46.778220 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/83c2796a-d9a0-45be-a1dd-d9e3e3d11108-metrics-client-ca\") pod \"kube-state-metrics-56d9f944b4-brv22\" (UID: \"83c2796a-d9a0-45be-a1dd-d9e3e3d11108\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-brv22" Jul 29 22:47:46.778373 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:46.778248 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/83c2796a-d9a0-45be-a1dd-d9e3e3d11108-kube-state-metrics-tls\") pod \"kube-state-metrics-56d9f944b4-brv22\" (UID: \"83c2796a-d9a0-45be-a1dd-d9e3e3d11108\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-brv22" Jul 29 22:47:46.778677 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:46.778469 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/347281f0-0a1e-4710-9d6b-8e81fe4e7613-node-exporter-accelerators-collector-config\") pod \"node-exporter-zvrsv\" (UID: \"347281f0-0a1e-4710-9d6b-8e81fe4e7613\") " pod="openshift-monitoring/node-exporter-zvrsv" Jul 29 22:47:46.778677 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:46.778502 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/347281f0-0a1e-4710-9d6b-8e81fe4e7613-node-exporter-wtmp\") pod \"node-exporter-zvrsv\" (UID: \"347281f0-0a1e-4710-9d6b-8e81fe4e7613\") " pod="openshift-monitoring/node-exporter-zvrsv" Jul 29 22:47:46.778677 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:46.778530 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/347281f0-0a1e-4710-9d6b-8e81fe4e7613-root\") pod \"node-exporter-zvrsv\" (UID: \"347281f0-0a1e-4710-9d6b-8e81fe4e7613\") " pod="openshift-monitoring/node-exporter-zvrsv" Jul 29 22:47:46.778677 ip-10-0-138-112 kubenswrapper[2562]: E0729 22:47:46.778630 2562 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Jul 29 22:47:46.778677 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:46.778637 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/83c2796a-d9a0-45be-a1dd-d9e3e3d11108-volume-directive-shadow\") pod \"kube-state-metrics-56d9f944b4-brv22\" (UID: \"83c2796a-d9a0-45be-a1dd-d9e3e3d11108\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-brv22" Jul 29 22:47:46.778910 ip-10-0-138-112 kubenswrapper[2562]: E0729 22:47:46.778686 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/347281f0-0a1e-4710-9d6b-8e81fe4e7613-node-exporter-tls podName:347281f0-0a1e-4710-9d6b-8e81fe4e7613 nodeName:}" failed. No retries permitted until 2026-07-29 22:47:47.278668226 +0000 UTC m=+82.190493897 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/347281f0-0a1e-4710-9d6b-8e81fe4e7613-node-exporter-tls") pod "node-exporter-zvrsv" (UID: "347281f0-0a1e-4710-9d6b-8e81fe4e7613") : secret "node-exporter-tls" not found Jul 29 22:47:46.778910 ip-10-0-138-112 kubenswrapper[2562]: E0729 22:47:46.778725 2562 secret.go:189] Couldn't get secret openshift-monitoring/kube-state-metrics-tls: secret "kube-state-metrics-tls" not found Jul 29 22:47:46.778910 ip-10-0-138-112 kubenswrapper[2562]: E0729 22:47:46.778780 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/83c2796a-d9a0-45be-a1dd-d9e3e3d11108-kube-state-metrics-tls podName:83c2796a-d9a0-45be-a1dd-d9e3e3d11108 nodeName:}" failed. No retries permitted until 2026-07-29 22:47:47.278761767 +0000 UTC m=+82.190587433 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-state-metrics-tls" (UniqueName: "kubernetes.io/secret/83c2796a-d9a0-45be-a1dd-d9e3e3d11108-kube-state-metrics-tls") pod "kube-state-metrics-56d9f944b4-brv22" (UID: "83c2796a-d9a0-45be-a1dd-d9e3e3d11108") : secret "kube-state-metrics-tls" not found Jul 29 22:47:46.779484 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:46.779457 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/83c2796a-d9a0-45be-a1dd-d9e3e3d11108-metrics-client-ca\") pod \"kube-state-metrics-56d9f944b4-brv22\" (UID: \"83c2796a-d9a0-45be-a1dd-d9e3e3d11108\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-brv22" Jul 29 22:47:46.779578 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:46.779497 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/83c2796a-d9a0-45be-a1dd-d9e3e3d11108-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56d9f944b4-brv22\" (UID: \"83c2796a-d9a0-45be-a1dd-d9e3e3d11108\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-brv22" Jul 29 22:47:46.779993 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:46.779969 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/347281f0-0a1e-4710-9d6b-8e81fe4e7613-metrics-client-ca\") pod \"node-exporter-zvrsv\" (UID: \"347281f0-0a1e-4710-9d6b-8e81fe4e7613\") " pod="openshift-monitoring/node-exporter-zvrsv" Jul 29 22:47:46.780695 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:46.780677 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/347281f0-0a1e-4710-9d6b-8e81fe4e7613-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-zvrsv\" (UID: \"347281f0-0a1e-4710-9d6b-8e81fe4e7613\") " pod="openshift-monitoring/node-exporter-zvrsv" Jul 29 22:47:46.781271 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:46.781250 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/83c2796a-d9a0-45be-a1dd-d9e3e3d11108-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56d9f944b4-brv22\" (UID: \"83c2796a-d9a0-45be-a1dd-d9e3e3d11108\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-brv22" Jul 29 22:47:46.788722 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:46.787353 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2t7xx\" (UniqueName: \"kubernetes.io/projected/347281f0-0a1e-4710-9d6b-8e81fe4e7613-kube-api-access-2t7xx\") pod \"node-exporter-zvrsv\" (UID: \"347281f0-0a1e-4710-9d6b-8e81fe4e7613\") " pod="openshift-monitoring/node-exporter-zvrsv" Jul 29 22:47:46.788816 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:46.788785 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gmdpk\" (UniqueName: \"kubernetes.io/projected/83c2796a-d9a0-45be-a1dd-d9e3e3d11108-kube-api-access-gmdpk\") pod \"kube-state-metrics-56d9f944b4-brv22\" (UID: \"83c2796a-d9a0-45be-a1dd-d9e3e3d11108\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-brv22" Jul 29 22:47:47.283928 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:47.283867 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/347281f0-0a1e-4710-9d6b-8e81fe4e7613-node-exporter-tls\") pod \"node-exporter-zvrsv\" (UID: \"347281f0-0a1e-4710-9d6b-8e81fe4e7613\") " pod="openshift-monitoring/node-exporter-zvrsv" Jul 29 22:47:47.283928 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:47.283931 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/83c2796a-d9a0-45be-a1dd-d9e3e3d11108-kube-state-metrics-tls\") pod \"kube-state-metrics-56d9f944b4-brv22\" (UID: \"83c2796a-d9a0-45be-a1dd-d9e3e3d11108\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-brv22" Jul 29 22:47:47.284640 ip-10-0-138-112 kubenswrapper[2562]: E0729 22:47:47.284075 2562 secret.go:189] Couldn't get secret openshift-monitoring/kube-state-metrics-tls: secret "kube-state-metrics-tls" not found Jul 29 22:47:47.284640 ip-10-0-138-112 kubenswrapper[2562]: E0729 22:47:47.284130 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/83c2796a-d9a0-45be-a1dd-d9e3e3d11108-kube-state-metrics-tls podName:83c2796a-d9a0-45be-a1dd-d9e3e3d11108 nodeName:}" failed. No retries permitted until 2026-07-29 22:47:48.284116284 +0000 UTC m=+83.195941948 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-state-metrics-tls" (UniqueName: "kubernetes.io/secret/83c2796a-d9a0-45be-a1dd-d9e3e3d11108-kube-state-metrics-tls") pod "kube-state-metrics-56d9f944b4-brv22" (UID: "83c2796a-d9a0-45be-a1dd-d9e3e3d11108") : secret "kube-state-metrics-tls" not found Jul 29 22:47:47.286588 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:47.286555 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/347281f0-0a1e-4710-9d6b-8e81fe4e7613-node-exporter-tls\") pod \"node-exporter-zvrsv\" (UID: \"347281f0-0a1e-4710-9d6b-8e81fe4e7613\") " pod="openshift-monitoring/node-exporter-zvrsv" Jul 29 22:47:47.507657 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:47.507620 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-zvrsv" Jul 29 22:47:47.521316 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:47:47.521277 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod347281f0_0a1e_4710_9d6b_8e81fe4e7613.slice/crio-ce223c999e514b7740973002f52fdd56510b197680f422f024ff0bccca6d6759 WatchSource:0}: Error finding container ce223c999e514b7740973002f52fdd56510b197680f422f024ff0bccca6d6759: Status 404 returned error can't find the container with id ce223c999e514b7740973002f52fdd56510b197680f422f024ff0bccca6d6759 Jul 29 22:47:47.858552 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:47.858386 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-ncs4k" Jul 29 22:47:47.897209 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:47.897160 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-zvrsv" event={"ID":"347281f0-0a1e-4710-9d6b-8e81fe4e7613","Type":"ContainerStarted","Data":"ce223c999e514b7740973002f52fdd56510b197680f422f024ff0bccca6d6759"} Jul 29 22:47:48.292083 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:48.292042 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/83c2796a-d9a0-45be-a1dd-d9e3e3d11108-kube-state-metrics-tls\") pod \"kube-state-metrics-56d9f944b4-brv22\" (UID: \"83c2796a-d9a0-45be-a1dd-d9e3e3d11108\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-brv22" Jul 29 22:47:48.294870 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:48.294840 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/83c2796a-d9a0-45be-a1dd-d9e3e3d11108-kube-state-metrics-tls\") pod \"kube-state-metrics-56d9f944b4-brv22\" (UID: \"83c2796a-d9a0-45be-a1dd-d9e3e3d11108\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-brv22" Jul 29 22:47:48.398263 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:48.398227 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-brv22" Jul 29 22:47:50.609771 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:50.609737 2562 patch_prober.go:28] interesting pod/image-registry-6cb779f875-pfsr5 container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jul 29 22:47:50.610205 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:50.609796 2562 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-6cb779f875-pfsr5" podUID="f05d8f86-4adc-47ed-9a27-1e9a00f68a69" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 29 22:47:50.847524 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:50.847483 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-69bb45d4cd-b5lk4"] Jul 29 22:47:50.851083 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:50.851058 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-69bb45d4cd-b5lk4" Jul 29 22:47:50.854153 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:50.854128 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\"" Jul 29 22:47:50.855445 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:50.855415 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-vkxv5\"" Jul 29 22:47:50.855692 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:50.855673 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jul 29 22:47:50.855773 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:50.855703 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\"" Jul 29 22:47:50.855773 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:50.855708 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\"" Jul 29 22:47:50.855973 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:50.855945 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-7ei5oagk1rk0f\"" Jul 29 22:47:50.861243 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:50.861172 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-69bb45d4cd-b5lk4"] Jul 29 22:47:50.916384 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:50.916352 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/75075b31-e2a5-4667-87fe-dfcd79e196e0-audit-log\") pod \"metrics-server-69bb45d4cd-b5lk4\" (UID: \"75075b31-e2a5-4667-87fe-dfcd79e196e0\") " pod="openshift-monitoring/metrics-server-69bb45d4cd-b5lk4" Jul 29 22:47:50.916570 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:50.916425 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/75075b31-e2a5-4667-87fe-dfcd79e196e0-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-69bb45d4cd-b5lk4\" (UID: \"75075b31-e2a5-4667-87fe-dfcd79e196e0\") " pod="openshift-monitoring/metrics-server-69bb45d4cd-b5lk4" Jul 29 22:47:50.916570 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:50.916467 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/75075b31-e2a5-4667-87fe-dfcd79e196e0-secret-metrics-server-tls\") pod \"metrics-server-69bb45d4cd-b5lk4\" (UID: \"75075b31-e2a5-4667-87fe-dfcd79e196e0\") " pod="openshift-monitoring/metrics-server-69bb45d4cd-b5lk4" Jul 29 22:47:50.916570 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:50.916490 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/75075b31-e2a5-4667-87fe-dfcd79e196e0-secret-metrics-server-client-certs\") pod \"metrics-server-69bb45d4cd-b5lk4\" (UID: \"75075b31-e2a5-4667-87fe-dfcd79e196e0\") " pod="openshift-monitoring/metrics-server-69bb45d4cd-b5lk4" Jul 29 22:47:50.916570 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:50.916539 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m5w4b\" (UniqueName: \"kubernetes.io/projected/75075b31-e2a5-4667-87fe-dfcd79e196e0-kube-api-access-m5w4b\") pod \"metrics-server-69bb45d4cd-b5lk4\" (UID: \"75075b31-e2a5-4667-87fe-dfcd79e196e0\") " pod="openshift-monitoring/metrics-server-69bb45d4cd-b5lk4" Jul 29 22:47:50.916714 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:50.916571 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/75075b31-e2a5-4667-87fe-dfcd79e196e0-client-ca-bundle\") pod \"metrics-server-69bb45d4cd-b5lk4\" (UID: \"75075b31-e2a5-4667-87fe-dfcd79e196e0\") " pod="openshift-monitoring/metrics-server-69bb45d4cd-b5lk4" Jul 29 22:47:50.916714 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:50.916620 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/75075b31-e2a5-4667-87fe-dfcd79e196e0-metrics-server-audit-profiles\") pod \"metrics-server-69bb45d4cd-b5lk4\" (UID: \"75075b31-e2a5-4667-87fe-dfcd79e196e0\") " pod="openshift-monitoring/metrics-server-69bb45d4cd-b5lk4" Jul 29 22:47:51.018113 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:51.018066 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/75075b31-e2a5-4667-87fe-dfcd79e196e0-secret-metrics-server-tls\") pod \"metrics-server-69bb45d4cd-b5lk4\" (UID: \"75075b31-e2a5-4667-87fe-dfcd79e196e0\") " pod="openshift-monitoring/metrics-server-69bb45d4cd-b5lk4" Jul 29 22:47:51.018232 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:51.018132 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/75075b31-e2a5-4667-87fe-dfcd79e196e0-secret-metrics-server-client-certs\") pod \"metrics-server-69bb45d4cd-b5lk4\" (UID: \"75075b31-e2a5-4667-87fe-dfcd79e196e0\") " pod="openshift-monitoring/metrics-server-69bb45d4cd-b5lk4" Jul 29 22:47:51.018232 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:51.018160 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m5w4b\" (UniqueName: \"kubernetes.io/projected/75075b31-e2a5-4667-87fe-dfcd79e196e0-kube-api-access-m5w4b\") pod \"metrics-server-69bb45d4cd-b5lk4\" (UID: \"75075b31-e2a5-4667-87fe-dfcd79e196e0\") " pod="openshift-monitoring/metrics-server-69bb45d4cd-b5lk4" Jul 29 22:47:51.018232 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:51.018201 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/75075b31-e2a5-4667-87fe-dfcd79e196e0-client-ca-bundle\") pod \"metrics-server-69bb45d4cd-b5lk4\" (UID: \"75075b31-e2a5-4667-87fe-dfcd79e196e0\") " pod="openshift-monitoring/metrics-server-69bb45d4cd-b5lk4" Jul 29 22:47:51.018390 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:51.018270 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/75075b31-e2a5-4667-87fe-dfcd79e196e0-metrics-server-audit-profiles\") pod \"metrics-server-69bb45d4cd-b5lk4\" (UID: \"75075b31-e2a5-4667-87fe-dfcd79e196e0\") " pod="openshift-monitoring/metrics-server-69bb45d4cd-b5lk4" Jul 29 22:47:51.018390 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:51.018325 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/75075b31-e2a5-4667-87fe-dfcd79e196e0-audit-log\") pod \"metrics-server-69bb45d4cd-b5lk4\" (UID: \"75075b31-e2a5-4667-87fe-dfcd79e196e0\") " pod="openshift-monitoring/metrics-server-69bb45d4cd-b5lk4" Jul 29 22:47:51.018509 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:51.018393 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/75075b31-e2a5-4667-87fe-dfcd79e196e0-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-69bb45d4cd-b5lk4\" (UID: \"75075b31-e2a5-4667-87fe-dfcd79e196e0\") " pod="openshift-monitoring/metrics-server-69bb45d4cd-b5lk4" Jul 29 22:47:51.019551 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:51.019312 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/75075b31-e2a5-4667-87fe-dfcd79e196e0-metrics-server-audit-profiles\") pod \"metrics-server-69bb45d4cd-b5lk4\" (UID: \"75075b31-e2a5-4667-87fe-dfcd79e196e0\") " pod="openshift-monitoring/metrics-server-69bb45d4cd-b5lk4" Jul 29 22:47:51.019551 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:51.019373 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/75075b31-e2a5-4667-87fe-dfcd79e196e0-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-69bb45d4cd-b5lk4\" (UID: \"75075b31-e2a5-4667-87fe-dfcd79e196e0\") " pod="openshift-monitoring/metrics-server-69bb45d4cd-b5lk4" Jul 29 22:47:51.020061 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:51.019731 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/75075b31-e2a5-4667-87fe-dfcd79e196e0-audit-log\") pod \"metrics-server-69bb45d4cd-b5lk4\" (UID: \"75075b31-e2a5-4667-87fe-dfcd79e196e0\") " pod="openshift-monitoring/metrics-server-69bb45d4cd-b5lk4" Jul 29 22:47:51.021634 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:51.021592 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/75075b31-e2a5-4667-87fe-dfcd79e196e0-secret-metrics-server-client-certs\") pod \"metrics-server-69bb45d4cd-b5lk4\" (UID: \"75075b31-e2a5-4667-87fe-dfcd79e196e0\") " pod="openshift-monitoring/metrics-server-69bb45d4cd-b5lk4" Jul 29 22:47:51.022762 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:51.022738 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/75075b31-e2a5-4667-87fe-dfcd79e196e0-secret-metrics-server-tls\") pod \"metrics-server-69bb45d4cd-b5lk4\" (UID: \"75075b31-e2a5-4667-87fe-dfcd79e196e0\") " pod="openshift-monitoring/metrics-server-69bb45d4cd-b5lk4" Jul 29 22:47:51.023481 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:51.023448 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/75075b31-e2a5-4667-87fe-dfcd79e196e0-client-ca-bundle\") pod \"metrics-server-69bb45d4cd-b5lk4\" (UID: \"75075b31-e2a5-4667-87fe-dfcd79e196e0\") " pod="openshift-monitoring/metrics-server-69bb45d4cd-b5lk4" Jul 29 22:47:51.027378 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:51.027351 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m5w4b\" (UniqueName: \"kubernetes.io/projected/75075b31-e2a5-4667-87fe-dfcd79e196e0-kube-api-access-m5w4b\") pod \"metrics-server-69bb45d4cd-b5lk4\" (UID: \"75075b31-e2a5-4667-87fe-dfcd79e196e0\") " pod="openshift-monitoring/metrics-server-69bb45d4cd-b5lk4" Jul 29 22:47:51.072079 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:51.072047 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-56d9f944b4-brv22"] Jul 29 22:47:51.075835 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:47:51.075782 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod83c2796a_d9a0_45be_a1dd_d9e3e3d11108.slice/crio-eb604379b6ae6f4e5e328615021cf35927f9182d4c09daa2c753b20b20d64529 WatchSource:0}: Error finding container eb604379b6ae6f4e5e328615021cf35927f9182d4c09daa2c753b20b20d64529: Status 404 returned error can't find the container with id eb604379b6ae6f4e5e328615021cf35927f9182d4c09daa2c753b20b20d64529 Jul 29 22:47:51.163583 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:51.163536 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-69bb45d4cd-b5lk4" Jul 29 22:47:51.294709 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:51.294677 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-69bb45d4cd-b5lk4"] Jul 29 22:47:51.296639 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:47:51.296613 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod75075b31_e2a5_4667_87fe_dfcd79e196e0.slice/crio-8efdb65308d0bf44f921db230a96bf41badeb02715ed7635b842079f8f1ac258 WatchSource:0}: Error finding container 8efdb65308d0bf44f921db230a96bf41badeb02715ed7635b842079f8f1ac258: Status 404 returned error can't find the container with id 8efdb65308d0bf44f921db230a96bf41badeb02715ed7635b842079f8f1ac258 Jul 29 22:47:51.915786 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:51.915743 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-rhx9q" event={"ID":"ad207e3b-5129-47d5-9151-d39346f17cdd","Type":"ContainerStarted","Data":"9519b2ebe768ad338a947dfe4341780f496a1fbe5680e614a421f88a920b7ce9"} Jul 29 22:47:51.917698 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:51.917657 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-69bb45d4cd-b5lk4" event={"ID":"75075b31-e2a5-4667-87fe-dfcd79e196e0","Type":"ContainerStarted","Data":"8efdb65308d0bf44f921db230a96bf41badeb02715ed7635b842079f8f1ac258"} Jul 29 22:47:51.920539 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:51.920498 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-brv22" event={"ID":"83c2796a-d9a0-45be-a1dd-d9e3e3d11108","Type":"ContainerStarted","Data":"eb604379b6ae6f4e5e328615021cf35927f9182d4c09daa2c753b20b20d64529"} Jul 29 22:47:51.925326 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:51.924604 2562 generic.go:356] "Generic (PLEG): container finished" podID="347281f0-0a1e-4710-9d6b-8e81fe4e7613" containerID="dcd30e3852e20c7bb96238f90413b14686ac32bf11501657e27a77377a4e15f5" exitCode=0 Jul 29 22:47:51.925326 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:51.925035 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-zvrsv" event={"ID":"347281f0-0a1e-4710-9d6b-8e81fe4e7613","Type":"ContainerDied","Data":"dcd30e3852e20c7bb96238f90413b14686ac32bf11501657e27a77377a4e15f5"} Jul 29 22:47:52.931967 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:52.931925 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-zvrsv" event={"ID":"347281f0-0a1e-4710-9d6b-8e81fe4e7613","Type":"ContainerStarted","Data":"b5b646dbc0ab7a357a89dde15f1a202e845a4fb760b59e7fb9a68b8e702b3dda"} Jul 29 22:47:52.932438 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:52.931991 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-zvrsv" event={"ID":"347281f0-0a1e-4710-9d6b-8e81fe4e7613","Type":"ContainerStarted","Data":"bbbdbc7609367f1165f1086cc01e2e5f5db3873368cfb9523045622c43427613"} Jul 29 22:47:52.958478 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:52.958385 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-zvrsv" podStartSLOduration=3.481982759 podStartE2EDuration="6.958362876s" totalImagesPullingTime="3.40660162s" totalInitContainerRuntime="69.778497ms" isStatefulPod=true podCreationTimestamp="2026-07-29 22:47:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 22:47:52.956700563 +0000 UTC m=+87.868526249" watchObservedRunningTime="2026-07-29 22:47:52.958362876 +0000 UTC m=+87.870188563" Jul 29 22:47:55.946744 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:55.946700 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-69bb45d4cd-b5lk4" event={"ID":"75075b31-e2a5-4667-87fe-dfcd79e196e0","Type":"ContainerStarted","Data":"29e8b6ed3dc41c345e286174f75770e016183613e8a24ac806fef978f24b99fe"} Jul 29 22:47:55.949671 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:55.949644 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-brv22" event={"ID":"83c2796a-d9a0-45be-a1dd-d9e3e3d11108","Type":"ContainerStarted","Data":"905a7273adcfa95caf27444050f26f16807a23ba1de8629e0fc78401c24d7c15"} Jul 29 22:47:55.949822 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:55.949702 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-brv22" event={"ID":"83c2796a-d9a0-45be-a1dd-d9e3e3d11108","Type":"ContainerStarted","Data":"0414c8f428a7ea4a5e7703b6f9b610683c10d6be11d89c953a1ace9516069329"} Jul 29 22:47:55.949822 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:55.949730 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-brv22" event={"ID":"83c2796a-d9a0-45be-a1dd-d9e3e3d11108","Type":"ContainerStarted","Data":"6ddee1b58da18246ec725508155a0163e9e5b1747a7bb64897a412e6d4e426d9"} Jul 29 22:47:55.951587 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:55.951562 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-rhx9q" event={"ID":"ad207e3b-5129-47d5-9151-d39346f17cdd","Type":"ContainerStarted","Data":"32bb49678eeb1eb306f0cb9b009bfcc585a312413ed9a9435475bd0d47faf0e6"} Jul 29 22:47:55.968227 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:55.968161 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-69bb45d4cd-b5lk4" podStartSLOduration=2.055108739 podStartE2EDuration="5.968140834s" totalImagesPullingTime="3.913032095s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 22:47:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 22:47:55.966787572 +0000 UTC m=+90.878613259" watchObservedRunningTime="2026-07-29 22:47:55.968140834 +0000 UTC m=+90.879966521" Jul 29 22:47:55.989381 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:55.989335 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-brv22" podStartSLOduration=5.854476755 podStartE2EDuration="9.989318096s" totalImagesPullingTime="4.134841341s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 22:47:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 22:47:55.989031647 +0000 UTC m=+90.900857346" watchObservedRunningTime="2026-07-29 22:47:55.989318096 +0000 UTC m=+90.901143783" Jul 29 22:47:56.839684 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:56.839387 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-ccfdc4d97-5fvkk"] Jul 29 22:47:56.843332 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:56.843308 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-ccfdc4d97-5fvkk" Jul 29 22:47:56.846544 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:56.846518 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 29 22:47:56.849331 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:56.847323 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 29 22:47:56.849331 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:56.847536 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-fv65f\"" Jul 29 22:47:56.849331 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:56.848193 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 29 22:47:56.851669 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:56.850984 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 29 22:47:56.851669 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:56.851043 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 29 22:47:56.854235 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:56.852534 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 29 22:47:56.854235 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:56.853654 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 29 22:47:56.860277 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:56.860239 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-ccfdc4d97-5fvkk"] Jul 29 22:47:56.863746 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:56.863717 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jul 29 22:47:56.966344 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:56.966305 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1e31854a-9461-47db-a6be-75a543ef426e-console-serving-cert\") pod \"console-ccfdc4d97-5fvkk\" (UID: \"1e31854a-9461-47db-a6be-75a543ef426e\") " pod="openshift-console/console-ccfdc4d97-5fvkk" Jul 29 22:47:56.966810 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:56.966354 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1e31854a-9461-47db-a6be-75a543ef426e-console-oauth-config\") pod \"console-ccfdc4d97-5fvkk\" (UID: \"1e31854a-9461-47db-a6be-75a543ef426e\") " pod="openshift-console/console-ccfdc4d97-5fvkk" Jul 29 22:47:56.966810 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:56.966420 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-shd68\" (UniqueName: \"kubernetes.io/projected/1e31854a-9461-47db-a6be-75a543ef426e-kube-api-access-shd68\") pod \"console-ccfdc4d97-5fvkk\" (UID: \"1e31854a-9461-47db-a6be-75a543ef426e\") " pod="openshift-console/console-ccfdc4d97-5fvkk" Jul 29 22:47:56.966810 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:56.966454 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1e31854a-9461-47db-a6be-75a543ef426e-service-ca\") pod \"console-ccfdc4d97-5fvkk\" (UID: \"1e31854a-9461-47db-a6be-75a543ef426e\") " pod="openshift-console/console-ccfdc4d97-5fvkk" Jul 29 22:47:56.966810 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:56.966487 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1e31854a-9461-47db-a6be-75a543ef426e-oauth-serving-cert\") pod \"console-ccfdc4d97-5fvkk\" (UID: \"1e31854a-9461-47db-a6be-75a543ef426e\") " pod="openshift-console/console-ccfdc4d97-5fvkk" Jul 29 22:47:56.966810 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:56.966568 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1e31854a-9461-47db-a6be-75a543ef426e-console-config\") pod \"console-ccfdc4d97-5fvkk\" (UID: \"1e31854a-9461-47db-a6be-75a543ef426e\") " pod="openshift-console/console-ccfdc4d97-5fvkk" Jul 29 22:47:56.966810 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:56.966611 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1e31854a-9461-47db-a6be-75a543ef426e-trusted-ca-bundle\") pod \"console-ccfdc4d97-5fvkk\" (UID: \"1e31854a-9461-47db-a6be-75a543ef426e\") " pod="openshift-console/console-ccfdc4d97-5fvkk" Jul 29 22:47:57.067684 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:57.067645 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-shd68\" (UniqueName: \"kubernetes.io/projected/1e31854a-9461-47db-a6be-75a543ef426e-kube-api-access-shd68\") pod \"console-ccfdc4d97-5fvkk\" (UID: \"1e31854a-9461-47db-a6be-75a543ef426e\") " pod="openshift-console/console-ccfdc4d97-5fvkk" Jul 29 22:47:57.067879 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:57.067746 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1e31854a-9461-47db-a6be-75a543ef426e-service-ca\") pod \"console-ccfdc4d97-5fvkk\" (UID: \"1e31854a-9461-47db-a6be-75a543ef426e\") " pod="openshift-console/console-ccfdc4d97-5fvkk" Jul 29 22:47:57.067879 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:57.067779 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1e31854a-9461-47db-a6be-75a543ef426e-oauth-serving-cert\") pod \"console-ccfdc4d97-5fvkk\" (UID: \"1e31854a-9461-47db-a6be-75a543ef426e\") " pod="openshift-console/console-ccfdc4d97-5fvkk" Jul 29 22:47:57.067879 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:57.067835 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1e31854a-9461-47db-a6be-75a543ef426e-console-config\") pod \"console-ccfdc4d97-5fvkk\" (UID: \"1e31854a-9461-47db-a6be-75a543ef426e\") " pod="openshift-console/console-ccfdc4d97-5fvkk" Jul 29 22:47:57.067879 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:57.067860 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1e31854a-9461-47db-a6be-75a543ef426e-trusted-ca-bundle\") pod \"console-ccfdc4d97-5fvkk\" (UID: \"1e31854a-9461-47db-a6be-75a543ef426e\") " pod="openshift-console/console-ccfdc4d97-5fvkk" Jul 29 22:47:57.068188 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:57.068162 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1e31854a-9461-47db-a6be-75a543ef426e-console-serving-cert\") pod \"console-ccfdc4d97-5fvkk\" (UID: \"1e31854a-9461-47db-a6be-75a543ef426e\") " pod="openshift-console/console-ccfdc4d97-5fvkk" Jul 29 22:47:57.068362 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:57.068197 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1e31854a-9461-47db-a6be-75a543ef426e-console-oauth-config\") pod \"console-ccfdc4d97-5fvkk\" (UID: \"1e31854a-9461-47db-a6be-75a543ef426e\") " pod="openshift-console/console-ccfdc4d97-5fvkk" Jul 29 22:47:57.069075 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:57.069055 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1e31854a-9461-47db-a6be-75a543ef426e-oauth-serving-cert\") pod \"console-ccfdc4d97-5fvkk\" (UID: \"1e31854a-9461-47db-a6be-75a543ef426e\") " pod="openshift-console/console-ccfdc4d97-5fvkk" Jul 29 22:47:57.069815 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:57.069786 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1e31854a-9461-47db-a6be-75a543ef426e-service-ca\") pod \"console-ccfdc4d97-5fvkk\" (UID: \"1e31854a-9461-47db-a6be-75a543ef426e\") " pod="openshift-console/console-ccfdc4d97-5fvkk" Jul 29 22:47:57.069911 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:57.069859 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1e31854a-9461-47db-a6be-75a543ef426e-trusted-ca-bundle\") pod \"console-ccfdc4d97-5fvkk\" (UID: \"1e31854a-9461-47db-a6be-75a543ef426e\") " pod="openshift-console/console-ccfdc4d97-5fvkk" Jul 29 22:47:57.070253 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:57.070230 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1e31854a-9461-47db-a6be-75a543ef426e-console-config\") pod \"console-ccfdc4d97-5fvkk\" (UID: \"1e31854a-9461-47db-a6be-75a543ef426e\") " pod="openshift-console/console-ccfdc4d97-5fvkk" Jul 29 22:47:57.070945 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:57.070921 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1e31854a-9461-47db-a6be-75a543ef426e-console-serving-cert\") pod \"console-ccfdc4d97-5fvkk\" (UID: \"1e31854a-9461-47db-a6be-75a543ef426e\") " pod="openshift-console/console-ccfdc4d97-5fvkk" Jul 29 22:47:57.071027 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:57.071000 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1e31854a-9461-47db-a6be-75a543ef426e-console-oauth-config\") pod \"console-ccfdc4d97-5fvkk\" (UID: \"1e31854a-9461-47db-a6be-75a543ef426e\") " pod="openshift-console/console-ccfdc4d97-5fvkk" Jul 29 22:47:57.078247 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:57.078220 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-shd68\" (UniqueName: \"kubernetes.io/projected/1e31854a-9461-47db-a6be-75a543ef426e-kube-api-access-shd68\") pod \"console-ccfdc4d97-5fvkk\" (UID: \"1e31854a-9461-47db-a6be-75a543ef426e\") " pod="openshift-console/console-ccfdc4d97-5fvkk" Jul 29 22:47:57.161178 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:57.161055 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-ccfdc4d97-5fvkk" Jul 29 22:47:57.305635 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:57.305602 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-ccfdc4d97-5fvkk"] Jul 29 22:47:58.892547 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:47:58.892515 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1e31854a_9461_47db_a6be_75a543ef426e.slice/crio-71a7783082151a8118eed9a1f60e8e564918eadbc9b5ce17c3286c18a41c4c7b WatchSource:0}: Error finding container 71a7783082151a8118eed9a1f60e8e564918eadbc9b5ce17c3286c18a41c4c7b: Status 404 returned error can't find the container with id 71a7783082151a8118eed9a1f60e8e564918eadbc9b5ce17c3286c18a41c4c7b Jul 29 22:47:58.965595 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:58.965566 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-ccfdc4d97-5fvkk" event={"ID":"1e31854a-9461-47db-a6be-75a543ef426e","Type":"ContainerStarted","Data":"71a7783082151a8118eed9a1f60e8e564918eadbc9b5ce17c3286c18a41c4c7b"} Jul 29 22:47:59.972919 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:59.972882 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-rhx9q" event={"ID":"ad207e3b-5129-47d5-9151-d39346f17cdd","Type":"ContainerStarted","Data":"ffbc22c23ab0ba767eb18beb3760ad0a0a26b4bfbc47b92df4ca489e510332cd"} Jul 29 22:47:59.997634 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:47:59.997572 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-rhx9q" podStartSLOduration=33.859823411 podStartE2EDuration="46.997553012s" totalImagesPullingTime="13.137729601s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 22:47:13 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 22:47:59.996393408 +0000 UTC m=+94.908219097" watchObservedRunningTime="2026-07-29 22:47:59.997553012 +0000 UTC m=+94.909378700" Jul 29 22:48:00.608629 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:48:00.608600 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-6cb779f875-pfsr5" Jul 29 22:48:01.982676 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:48:01.982638 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-ccfdc4d97-5fvkk" event={"ID":"1e31854a-9461-47db-a6be-75a543ef426e","Type":"ContainerStarted","Data":"e7f38d468a956f7fc7bf4b2b3b85853cc997f04e211cd8f6af99cef5124bd91e"} Jul 29 22:48:02.003838 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:48:02.003770 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-ccfdc4d97-5fvkk" podStartSLOduration=3.07056021 podStartE2EDuration="6.003748044s" totalImagesPullingTime="2.933187834s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 22:47:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 22:48:02.002766431 +0000 UTC m=+96.914592133" watchObservedRunningTime="2026-07-29 22:48:02.003748044 +0000 UTC m=+96.915573732" Jul 29 22:48:02.878313 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:48:02.878280 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-7f7448c9db-mqm8x" Jul 29 22:48:05.621334 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:48:05.621290 2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-6cb779f875-pfsr5" podUID="f05d8f86-4adc-47ed-9a27-1e9a00f68a69" containerName="registry" containerID="cri-o://96d24c293c0e84201a05eadb06c379ef2d497547bd40b26565540383ee0e9c47" gracePeriod=30 Jul 29 22:48:05.861046 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:48:05.861021 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6cb779f875-pfsr5" Jul 29 22:48:05.949684 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:48:05.949592 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/f05d8f86-4adc-47ed-9a27-1e9a00f68a69-installation-pull-secrets\") pod \"f05d8f86-4adc-47ed-9a27-1e9a00f68a69\" (UID: \"f05d8f86-4adc-47ed-9a27-1e9a00f68a69\") " Jul 29 22:48:05.949684 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:48:05.949640 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/f05d8f86-4adc-47ed-9a27-1e9a00f68a69-registry-certificates\") pod \"f05d8f86-4adc-47ed-9a27-1e9a00f68a69\" (UID: \"f05d8f86-4adc-47ed-9a27-1e9a00f68a69\") " Jul 29 22:48:05.949684 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:48:05.949669 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f05d8f86-4adc-47ed-9a27-1e9a00f68a69-registry-tls\") pod \"f05d8f86-4adc-47ed-9a27-1e9a00f68a69\" (UID: \"f05d8f86-4adc-47ed-9a27-1e9a00f68a69\") " Jul 29 22:48:05.949953 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:48:05.949714 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/f05d8f86-4adc-47ed-9a27-1e9a00f68a69-image-registry-private-configuration\") pod \"f05d8f86-4adc-47ed-9a27-1e9a00f68a69\" (UID: \"f05d8f86-4adc-47ed-9a27-1e9a00f68a69\") " Jul 29 22:48:05.949953 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:48:05.949743 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qnk7m\" (UniqueName: \"kubernetes.io/projected/f05d8f86-4adc-47ed-9a27-1e9a00f68a69-kube-api-access-qnk7m\") pod \"f05d8f86-4adc-47ed-9a27-1e9a00f68a69\" (UID: \"f05d8f86-4adc-47ed-9a27-1e9a00f68a69\") " Jul 29 22:48:05.950064 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:48:05.949942 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/f05d8f86-4adc-47ed-9a27-1e9a00f68a69-ca-trust-extracted\") pod \"f05d8f86-4adc-47ed-9a27-1e9a00f68a69\" (UID: \"f05d8f86-4adc-47ed-9a27-1e9a00f68a69\") " Jul 29 22:48:05.950064 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:48:05.950032 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f05d8f86-4adc-47ed-9a27-1e9a00f68a69-trusted-ca\") pod \"f05d8f86-4adc-47ed-9a27-1e9a00f68a69\" (UID: \"f05d8f86-4adc-47ed-9a27-1e9a00f68a69\") " Jul 29 22:48:05.950172 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:48:05.950079 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f05d8f86-4adc-47ed-9a27-1e9a00f68a69-bound-sa-token\") pod \"f05d8f86-4adc-47ed-9a27-1e9a00f68a69\" (UID: \"f05d8f86-4adc-47ed-9a27-1e9a00f68a69\") " Jul 29 22:48:05.950293 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:48:05.950260 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f05d8f86-4adc-47ed-9a27-1e9a00f68a69-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "f05d8f86-4adc-47ed-9a27-1e9a00f68a69" (UID: "f05d8f86-4adc-47ed-9a27-1e9a00f68a69"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 29 22:48:05.950746 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:48:05.950715 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f05d8f86-4adc-47ed-9a27-1e9a00f68a69-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "f05d8f86-4adc-47ed-9a27-1e9a00f68a69" (UID: "f05d8f86-4adc-47ed-9a27-1e9a00f68a69"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 29 22:48:05.952476 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:48:05.952424 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f05d8f86-4adc-47ed-9a27-1e9a00f68a69-kube-api-access-qnk7m" (OuterVolumeSpecName: "kube-api-access-qnk7m") pod "f05d8f86-4adc-47ed-9a27-1e9a00f68a69" (UID: "f05d8f86-4adc-47ed-9a27-1e9a00f68a69"). InnerVolumeSpecName "kube-api-access-qnk7m". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 29 22:48:05.952476 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:48:05.952444 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f05d8f86-4adc-47ed-9a27-1e9a00f68a69-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "f05d8f86-4adc-47ed-9a27-1e9a00f68a69" (UID: "f05d8f86-4adc-47ed-9a27-1e9a00f68a69"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 29 22:48:05.952656 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:48:05.952506 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f05d8f86-4adc-47ed-9a27-1e9a00f68a69-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "f05d8f86-4adc-47ed-9a27-1e9a00f68a69" (UID: "f05d8f86-4adc-47ed-9a27-1e9a00f68a69"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 29 22:48:05.952656 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:48:05.952585 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f05d8f86-4adc-47ed-9a27-1e9a00f68a69-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "f05d8f86-4adc-47ed-9a27-1e9a00f68a69" (UID: "f05d8f86-4adc-47ed-9a27-1e9a00f68a69"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 29 22:48:05.952763 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:48:05.952694 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f05d8f86-4adc-47ed-9a27-1e9a00f68a69-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "f05d8f86-4adc-47ed-9a27-1e9a00f68a69" (UID: "f05d8f86-4adc-47ed-9a27-1e9a00f68a69"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 29 22:48:05.960197 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:48:05.960165 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f05d8f86-4adc-47ed-9a27-1e9a00f68a69-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "f05d8f86-4adc-47ed-9a27-1e9a00f68a69" (UID: "f05d8f86-4adc-47ed-9a27-1e9a00f68a69"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 29 22:48:05.996854 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:48:05.996818 2562 generic.go:356] "Generic (PLEG): container finished" podID="f05d8f86-4adc-47ed-9a27-1e9a00f68a69" containerID="96d24c293c0e84201a05eadb06c379ef2d497547bd40b26565540383ee0e9c47" exitCode=0 Jul 29 22:48:05.997096 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:48:05.996863 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6cb779f875-pfsr5" event={"ID":"f05d8f86-4adc-47ed-9a27-1e9a00f68a69","Type":"ContainerDied","Data":"96d24c293c0e84201a05eadb06c379ef2d497547bd40b26565540383ee0e9c47"} Jul 29 22:48:05.997096 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:48:05.996869 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6cb779f875-pfsr5" Jul 29 22:48:05.997096 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:48:05.996904 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6cb779f875-pfsr5" event={"ID":"f05d8f86-4adc-47ed-9a27-1e9a00f68a69","Type":"ContainerDied","Data":"ad449c565a8e65dce7fbc8deef182a3bbf59c1ba370beb2b4f9ab59b1c81a1bb"} Jul 29 22:48:05.997096 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:48:05.996925 2562 scope.go:117] "RemoveContainer" containerID="96d24c293c0e84201a05eadb06c379ef2d497547bd40b26565540383ee0e9c47" Jul 29 22:48:06.005472 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:48:06.005452 2562 scope.go:117] "RemoveContainer" containerID="96d24c293c0e84201a05eadb06c379ef2d497547bd40b26565540383ee0e9c47" Jul 29 22:48:06.005752 ip-10-0-138-112 kubenswrapper[2562]: E0729 22:48:06.005733 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"96d24c293c0e84201a05eadb06c379ef2d497547bd40b26565540383ee0e9c47\": container with ID starting with 96d24c293c0e84201a05eadb06c379ef2d497547bd40b26565540383ee0e9c47 not found: ID does not exist" containerID="96d24c293c0e84201a05eadb06c379ef2d497547bd40b26565540383ee0e9c47" Jul 29 22:48:06.005805 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:48:06.005762 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"96d24c293c0e84201a05eadb06c379ef2d497547bd40b26565540383ee0e9c47"} err="failed to get container status \"96d24c293c0e84201a05eadb06c379ef2d497547bd40b26565540383ee0e9c47\": rpc error: code = NotFound desc = could not find container \"96d24c293c0e84201a05eadb06c379ef2d497547bd40b26565540383ee0e9c47\": container with ID starting with 96d24c293c0e84201a05eadb06c379ef2d497547bd40b26565540383ee0e9c47 not found: ID does not exist" Jul 29 22:48:06.022444 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:48:06.022382 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-6cb779f875-pfsr5"] Jul 29 22:48:06.028408 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:48:06.028366 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-6cb779f875-pfsr5"] Jul 29 22:48:06.051321 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:48:06.051286 2562 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/f05d8f86-4adc-47ed-9a27-1e9a00f68a69-installation-pull-secrets\") on node \"ip-10-0-138-112.ec2.internal\" DevicePath \"\"" Jul 29 22:48:06.051321 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:48:06.051319 2562 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/f05d8f86-4adc-47ed-9a27-1e9a00f68a69-registry-certificates\") on node \"ip-10-0-138-112.ec2.internal\" DevicePath \"\"" Jul 29 22:48:06.051321 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:48:06.051332 2562 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f05d8f86-4adc-47ed-9a27-1e9a00f68a69-registry-tls\") on node \"ip-10-0-138-112.ec2.internal\" DevicePath \"\"" Jul 29 22:48:06.051553 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:48:06.051341 2562 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/f05d8f86-4adc-47ed-9a27-1e9a00f68a69-image-registry-private-configuration\") on node \"ip-10-0-138-112.ec2.internal\" DevicePath \"\"" Jul 29 22:48:06.051553 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:48:06.051350 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qnk7m\" (UniqueName: \"kubernetes.io/projected/f05d8f86-4adc-47ed-9a27-1e9a00f68a69-kube-api-access-qnk7m\") on node \"ip-10-0-138-112.ec2.internal\" DevicePath \"\"" Jul 29 22:48:06.051553 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:48:06.051358 2562 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/f05d8f86-4adc-47ed-9a27-1e9a00f68a69-ca-trust-extracted\") on node \"ip-10-0-138-112.ec2.internal\" DevicePath \"\"" Jul 29 22:48:06.051553 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:48:06.051369 2562 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f05d8f86-4adc-47ed-9a27-1e9a00f68a69-trusted-ca\") on node \"ip-10-0-138-112.ec2.internal\" DevicePath \"\"" Jul 29 22:48:06.051553 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:48:06.051377 2562 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f05d8f86-4adc-47ed-9a27-1e9a00f68a69-bound-sa-token\") on node \"ip-10-0-138-112.ec2.internal\" DevicePath \"\"" Jul 29 22:48:07.162170 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:48:07.162126 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-ccfdc4d97-5fvkk" Jul 29 22:48:07.162635 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:48:07.162199 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-ccfdc4d97-5fvkk" Jul 29 22:48:07.167038 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:48:07.167012 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-ccfdc4d97-5fvkk" Jul 29 22:48:07.574207 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:48:07.574175 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f05d8f86-4adc-47ed-9a27-1e9a00f68a69" path="/var/lib/kubelet/pods/f05d8f86-4adc-47ed-9a27-1e9a00f68a69/volumes" Jul 29 22:48:07.854029 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:48:07.853944 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-z92kj" Jul 29 22:48:08.008177 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:48:08.008143 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-ccfdc4d97-5fvkk" Jul 29 22:48:11.163919 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:48:11.163884 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-69bb45d4cd-b5lk4" Jul 29 22:48:11.164392 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:48:11.163948 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-69bb45d4cd-b5lk4" Jul 29 22:48:24.441258 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:48:24.441229 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-c5tdl_9b18dc65-d191-4eef-b23e-14561cbe95f1/serve-healthcheck-canary/0.log" Jul 29 22:48:31.169730 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:48:31.169700 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-69bb45d4cd-b5lk4" Jul 29 22:48:31.173923 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:48:31.173898 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-69bb45d4cd-b5lk4" Jul 29 22:49:10.996107 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:49:10.996067 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-65fbcc5b55-l9zgn"] Jul 29 22:49:10.996756 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:49:10.996443 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f05d8f86-4adc-47ed-9a27-1e9a00f68a69" containerName="registry" Jul 29 22:49:10.996756 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:49:10.996516 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="f05d8f86-4adc-47ed-9a27-1e9a00f68a69" containerName="registry" Jul 29 22:49:10.996756 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:49:10.996626 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="f05d8f86-4adc-47ed-9a27-1e9a00f68a69" containerName="registry" Jul 29 22:49:10.999633 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:49:10.999609 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-65fbcc5b55-l9zgn" Jul 29 22:49:11.002521 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:49:11.002495 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\"" Jul 29 22:49:11.002663 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:49:11.002626 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-44h86\"" Jul 29 22:49:11.002730 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:49:11.002713 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\"" Jul 29 22:49:11.002788 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:49:11.002722 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\"" Jul 29 22:49:11.002788 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:49:11.002717 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\"" Jul 29 22:49:11.003038 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:49:11.002999 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\"" Jul 29 22:49:11.008185 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:49:11.008160 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\"" Jul 29 22:49:11.011545 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:49:11.011518 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-65fbcc5b55-l9zgn"] Jul 29 22:49:11.136370 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:49:11.136322 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/30052ca6-a70f-44b3-abae-4374772320ef-metrics-client-ca\") pod \"telemeter-client-65fbcc5b55-l9zgn\" (UID: \"30052ca6-a70f-44b3-abae-4374772320ef\") " pod="openshift-monitoring/telemeter-client-65fbcc5b55-l9zgn" Jul 29 22:49:11.136370 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:49:11.136359 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/30052ca6-a70f-44b3-abae-4374772320ef-telemeter-trusted-ca-bundle\") pod \"telemeter-client-65fbcc5b55-l9zgn\" (UID: \"30052ca6-a70f-44b3-abae-4374772320ef\") " pod="openshift-monitoring/telemeter-client-65fbcc5b55-l9zgn" Jul 29 22:49:11.136370 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:49:11.136378 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/30052ca6-a70f-44b3-abae-4374772320ef-federate-client-tls\") pod \"telemeter-client-65fbcc5b55-l9zgn\" (UID: \"30052ca6-a70f-44b3-abae-4374772320ef\") " pod="openshift-monitoring/telemeter-client-65fbcc5b55-l9zgn" Jul 29 22:49:11.136698 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:49:11.136439 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/30052ca6-a70f-44b3-abae-4374772320ef-secret-telemeter-client\") pod \"telemeter-client-65fbcc5b55-l9zgn\" (UID: \"30052ca6-a70f-44b3-abae-4374772320ef\") " pod="openshift-monitoring/telemeter-client-65fbcc5b55-l9zgn" Jul 29 22:49:11.136698 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:49:11.136503 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/30052ca6-a70f-44b3-abae-4374772320ef-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-65fbcc5b55-l9zgn\" (UID: \"30052ca6-a70f-44b3-abae-4374772320ef\") " pod="openshift-monitoring/telemeter-client-65fbcc5b55-l9zgn" Jul 29 22:49:11.136698 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:49:11.136545 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/30052ca6-a70f-44b3-abae-4374772320ef-telemeter-client-tls\") pod \"telemeter-client-65fbcc5b55-l9zgn\" (UID: \"30052ca6-a70f-44b3-abae-4374772320ef\") " pod="openshift-monitoring/telemeter-client-65fbcc5b55-l9zgn" Jul 29 22:49:11.136698 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:49:11.136575 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/30052ca6-a70f-44b3-abae-4374772320ef-serving-certs-ca-bundle\") pod \"telemeter-client-65fbcc5b55-l9zgn\" (UID: \"30052ca6-a70f-44b3-abae-4374772320ef\") " pod="openshift-monitoring/telemeter-client-65fbcc5b55-l9zgn" Jul 29 22:49:11.136698 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:49:11.136594 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z8gb5\" (UniqueName: \"kubernetes.io/projected/30052ca6-a70f-44b3-abae-4374772320ef-kube-api-access-z8gb5\") pod \"telemeter-client-65fbcc5b55-l9zgn\" (UID: \"30052ca6-a70f-44b3-abae-4374772320ef\") " pod="openshift-monitoring/telemeter-client-65fbcc5b55-l9zgn" Jul 29 22:49:11.237635 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:49:11.237599 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/30052ca6-a70f-44b3-abae-4374772320ef-metrics-client-ca\") pod \"telemeter-client-65fbcc5b55-l9zgn\" (UID: \"30052ca6-a70f-44b3-abae-4374772320ef\") " pod="openshift-monitoring/telemeter-client-65fbcc5b55-l9zgn" Jul 29 22:49:11.237635 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:49:11.237635 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/30052ca6-a70f-44b3-abae-4374772320ef-telemeter-trusted-ca-bundle\") pod \"telemeter-client-65fbcc5b55-l9zgn\" (UID: \"30052ca6-a70f-44b3-abae-4374772320ef\") " pod="openshift-monitoring/telemeter-client-65fbcc5b55-l9zgn" Jul 29 22:49:11.237890 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:49:11.237653 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/30052ca6-a70f-44b3-abae-4374772320ef-federate-client-tls\") pod \"telemeter-client-65fbcc5b55-l9zgn\" (UID: \"30052ca6-a70f-44b3-abae-4374772320ef\") " pod="openshift-monitoring/telemeter-client-65fbcc5b55-l9zgn" Jul 29 22:49:11.237890 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:49:11.237669 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/30052ca6-a70f-44b3-abae-4374772320ef-secret-telemeter-client\") pod \"telemeter-client-65fbcc5b55-l9zgn\" (UID: \"30052ca6-a70f-44b3-abae-4374772320ef\") " pod="openshift-monitoring/telemeter-client-65fbcc5b55-l9zgn" Jul 29 22:49:11.237890 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:49:11.237703 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/30052ca6-a70f-44b3-abae-4374772320ef-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-65fbcc5b55-l9zgn\" (UID: \"30052ca6-a70f-44b3-abae-4374772320ef\") " pod="openshift-monitoring/telemeter-client-65fbcc5b55-l9zgn" Jul 29 22:49:11.237890 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:49:11.237739 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/30052ca6-a70f-44b3-abae-4374772320ef-telemeter-client-tls\") pod \"telemeter-client-65fbcc5b55-l9zgn\" (UID: \"30052ca6-a70f-44b3-abae-4374772320ef\") " pod="openshift-monitoring/telemeter-client-65fbcc5b55-l9zgn" Jul 29 22:49:11.237890 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:49:11.237765 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/30052ca6-a70f-44b3-abae-4374772320ef-serving-certs-ca-bundle\") pod \"telemeter-client-65fbcc5b55-l9zgn\" (UID: \"30052ca6-a70f-44b3-abae-4374772320ef\") " pod="openshift-monitoring/telemeter-client-65fbcc5b55-l9zgn" Jul 29 22:49:11.237890 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:49:11.237789 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z8gb5\" (UniqueName: \"kubernetes.io/projected/30052ca6-a70f-44b3-abae-4374772320ef-kube-api-access-z8gb5\") pod \"telemeter-client-65fbcc5b55-l9zgn\" (UID: \"30052ca6-a70f-44b3-abae-4374772320ef\") " pod="openshift-monitoring/telemeter-client-65fbcc5b55-l9zgn" Jul 29 22:49:11.238496 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:49:11.238468 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/30052ca6-a70f-44b3-abae-4374772320ef-metrics-client-ca\") pod \"telemeter-client-65fbcc5b55-l9zgn\" (UID: \"30052ca6-a70f-44b3-abae-4374772320ef\") " pod="openshift-monitoring/telemeter-client-65fbcc5b55-l9zgn" Jul 29 22:49:11.238754 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:49:11.238731 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/30052ca6-a70f-44b3-abae-4374772320ef-serving-certs-ca-bundle\") pod \"telemeter-client-65fbcc5b55-l9zgn\" (UID: \"30052ca6-a70f-44b3-abae-4374772320ef\") " pod="openshift-monitoring/telemeter-client-65fbcc5b55-l9zgn" Jul 29 22:49:11.238802 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:49:11.238731 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/30052ca6-a70f-44b3-abae-4374772320ef-telemeter-trusted-ca-bundle\") pod \"telemeter-client-65fbcc5b55-l9zgn\" (UID: \"30052ca6-a70f-44b3-abae-4374772320ef\") " pod="openshift-monitoring/telemeter-client-65fbcc5b55-l9zgn" Jul 29 22:49:11.240766 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:49:11.240740 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/30052ca6-a70f-44b3-abae-4374772320ef-telemeter-client-tls\") pod \"telemeter-client-65fbcc5b55-l9zgn\" (UID: \"30052ca6-a70f-44b3-abae-4374772320ef\") " pod="openshift-monitoring/telemeter-client-65fbcc5b55-l9zgn" Jul 29 22:49:11.240875 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:49:11.240740 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/30052ca6-a70f-44b3-abae-4374772320ef-federate-client-tls\") pod \"telemeter-client-65fbcc5b55-l9zgn\" (UID: \"30052ca6-a70f-44b3-abae-4374772320ef\") " pod="openshift-monitoring/telemeter-client-65fbcc5b55-l9zgn" Jul 29 22:49:11.240875 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:49:11.240826 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/30052ca6-a70f-44b3-abae-4374772320ef-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-65fbcc5b55-l9zgn\" (UID: \"30052ca6-a70f-44b3-abae-4374772320ef\") " pod="openshift-monitoring/telemeter-client-65fbcc5b55-l9zgn" Jul 29 22:49:11.240875 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:49:11.240839 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/30052ca6-a70f-44b3-abae-4374772320ef-secret-telemeter-client\") pod \"telemeter-client-65fbcc5b55-l9zgn\" (UID: \"30052ca6-a70f-44b3-abae-4374772320ef\") " pod="openshift-monitoring/telemeter-client-65fbcc5b55-l9zgn" Jul 29 22:49:11.246170 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:49:11.246115 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z8gb5\" (UniqueName: \"kubernetes.io/projected/30052ca6-a70f-44b3-abae-4374772320ef-kube-api-access-z8gb5\") pod \"telemeter-client-65fbcc5b55-l9zgn\" (UID: \"30052ca6-a70f-44b3-abae-4374772320ef\") " pod="openshift-monitoring/telemeter-client-65fbcc5b55-l9zgn" Jul 29 22:49:11.311726 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:49:11.311674 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-65fbcc5b55-l9zgn" Jul 29 22:49:11.455753 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:49:11.455719 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-65fbcc5b55-l9zgn"] Jul 29 22:49:11.460411 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:49:11.460366 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod30052ca6_a70f_44b3_abae_4374772320ef.slice/crio-5fe976aced1949c1238b3fc81c7acdf2ff09361825834594c132e88689448bbd WatchSource:0}: Error finding container 5fe976aced1949c1238b3fc81c7acdf2ff09361825834594c132e88689448bbd: Status 404 returned error can't find the container with id 5fe976aced1949c1238b3fc81c7acdf2ff09361825834594c132e88689448bbd Jul 29 22:49:12.222343 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:49:12.222302 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-65fbcc5b55-l9zgn" event={"ID":"30052ca6-a70f-44b3-abae-4374772320ef","Type":"ContainerStarted","Data":"5fe976aced1949c1238b3fc81c7acdf2ff09361825834594c132e88689448bbd"} Jul 29 22:49:13.227244 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:49:13.227209 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-65fbcc5b55-l9zgn" event={"ID":"30052ca6-a70f-44b3-abae-4374772320ef","Type":"ContainerStarted","Data":"31b15d49a9be400824c6809a9cf773a17d9e12cc8b9617630b6cb036dc98777f"} Jul 29 22:49:14.238121 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:49:14.238083 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-65fbcc5b55-l9zgn" event={"ID":"30052ca6-a70f-44b3-abae-4374772320ef","Type":"ContainerStarted","Data":"26e7f2f35723f8dbc76004d7ee674d680394a43645061e7125c50db7556318b2"} Jul 29 22:49:14.238743 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:49:14.238148 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-65fbcc5b55-l9zgn" event={"ID":"30052ca6-a70f-44b3-abae-4374772320ef","Type":"ContainerStarted","Data":"f70c9d26c8750df299cbff163336f294bc238785518aeb2f6556ab8a5a67f766"} Jul 29 22:49:14.262696 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:49:14.262603 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-65fbcc5b55-l9zgn" podStartSLOduration=1.681419969 podStartE2EDuration="4.262580062s" totalImagesPullingTime="2.581160093s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 22:49:10 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 22:49:14.262286961 +0000 UTC m=+169.174112642" watchObservedRunningTime="2026-07-29 22:49:14.262580062 +0000 UTC m=+169.174405751" Jul 29 22:49:15.124161 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:49:15.124125 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-7c6944f9c8-x5ctb"] Jul 29 22:49:15.126354 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:49:15.126329 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7c6944f9c8-x5ctb" Jul 29 22:49:15.138085 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:49:15.138057 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7c6944f9c8-x5ctb"] Jul 29 22:49:15.270500 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:49:15.270466 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f06420cf-e94f-4dc8-8954-15764efed001-console-config\") pod \"console-7c6944f9c8-x5ctb\" (UID: \"f06420cf-e94f-4dc8-8954-15764efed001\") " pod="openshift-console/console-7c6944f9c8-x5ctb" Jul 29 22:49:15.270912 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:49:15.270531 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f06420cf-e94f-4dc8-8954-15764efed001-oauth-serving-cert\") pod \"console-7c6944f9c8-x5ctb\" (UID: \"f06420cf-e94f-4dc8-8954-15764efed001\") " pod="openshift-console/console-7c6944f9c8-x5ctb" Jul 29 22:49:15.270912 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:49:15.270559 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-plpmv\" (UniqueName: \"kubernetes.io/projected/f06420cf-e94f-4dc8-8954-15764efed001-kube-api-access-plpmv\") pod \"console-7c6944f9c8-x5ctb\" (UID: \"f06420cf-e94f-4dc8-8954-15764efed001\") " pod="openshift-console/console-7c6944f9c8-x5ctb" Jul 29 22:49:15.270912 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:49:15.270651 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f06420cf-e94f-4dc8-8954-15764efed001-service-ca\") pod \"console-7c6944f9c8-x5ctb\" (UID: \"f06420cf-e94f-4dc8-8954-15764efed001\") " pod="openshift-console/console-7c6944f9c8-x5ctb" Jul 29 22:49:15.270912 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:49:15.270719 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f06420cf-e94f-4dc8-8954-15764efed001-console-oauth-config\") pod \"console-7c6944f9c8-x5ctb\" (UID: \"f06420cf-e94f-4dc8-8954-15764efed001\") " pod="openshift-console/console-7c6944f9c8-x5ctb" Jul 29 22:49:15.270912 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:49:15.270756 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f06420cf-e94f-4dc8-8954-15764efed001-console-serving-cert\") pod \"console-7c6944f9c8-x5ctb\" (UID: \"f06420cf-e94f-4dc8-8954-15764efed001\") " pod="openshift-console/console-7c6944f9c8-x5ctb" Jul 29 22:49:15.270912 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:49:15.270780 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f06420cf-e94f-4dc8-8954-15764efed001-trusted-ca-bundle\") pod \"console-7c6944f9c8-x5ctb\" (UID: \"f06420cf-e94f-4dc8-8954-15764efed001\") " pod="openshift-console/console-7c6944f9c8-x5ctb" Jul 29 22:49:15.372184 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:49:15.372127 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f06420cf-e94f-4dc8-8954-15764efed001-console-oauth-config\") pod \"console-7c6944f9c8-x5ctb\" (UID: \"f06420cf-e94f-4dc8-8954-15764efed001\") " pod="openshift-console/console-7c6944f9c8-x5ctb" Jul 29 22:49:15.372184 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:49:15.372187 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f06420cf-e94f-4dc8-8954-15764efed001-console-serving-cert\") pod \"console-7c6944f9c8-x5ctb\" (UID: \"f06420cf-e94f-4dc8-8954-15764efed001\") " pod="openshift-console/console-7c6944f9c8-x5ctb" Jul 29 22:49:15.372478 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:49:15.372216 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f06420cf-e94f-4dc8-8954-15764efed001-trusted-ca-bundle\") pod \"console-7c6944f9c8-x5ctb\" (UID: \"f06420cf-e94f-4dc8-8954-15764efed001\") " pod="openshift-console/console-7c6944f9c8-x5ctb" Jul 29 22:49:15.372539 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:49:15.372491 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f06420cf-e94f-4dc8-8954-15764efed001-console-config\") pod \"console-7c6944f9c8-x5ctb\" (UID: \"f06420cf-e94f-4dc8-8954-15764efed001\") " pod="openshift-console/console-7c6944f9c8-x5ctb" Jul 29 22:49:15.372927 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:49:15.372873 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f06420cf-e94f-4dc8-8954-15764efed001-oauth-serving-cert\") pod \"console-7c6944f9c8-x5ctb\" (UID: \"f06420cf-e94f-4dc8-8954-15764efed001\") " pod="openshift-console/console-7c6944f9c8-x5ctb" Jul 29 22:49:15.373051 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:49:15.372939 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-plpmv\" (UniqueName: \"kubernetes.io/projected/f06420cf-e94f-4dc8-8954-15764efed001-kube-api-access-plpmv\") pod \"console-7c6944f9c8-x5ctb\" (UID: \"f06420cf-e94f-4dc8-8954-15764efed001\") " pod="openshift-console/console-7c6944f9c8-x5ctb" Jul 29 22:49:15.373114 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:49:15.373090 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f06420cf-e94f-4dc8-8954-15764efed001-service-ca\") pod \"console-7c6944f9c8-x5ctb\" (UID: \"f06420cf-e94f-4dc8-8954-15764efed001\") " pod="openshift-console/console-7c6944f9c8-x5ctb" Jul 29 22:49:15.373343 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:49:15.373307 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f06420cf-e94f-4dc8-8954-15764efed001-trusted-ca-bundle\") pod \"console-7c6944f9c8-x5ctb\" (UID: \"f06420cf-e94f-4dc8-8954-15764efed001\") " pod="openshift-console/console-7c6944f9c8-x5ctb" Jul 29 22:49:15.373731 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:49:15.373709 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f06420cf-e94f-4dc8-8954-15764efed001-console-config\") pod \"console-7c6944f9c8-x5ctb\" (UID: \"f06420cf-e94f-4dc8-8954-15764efed001\") " pod="openshift-console/console-7c6944f9c8-x5ctb" Jul 29 22:49:15.373883 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:49:15.373786 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f06420cf-e94f-4dc8-8954-15764efed001-service-ca\") pod \"console-7c6944f9c8-x5ctb\" (UID: \"f06420cf-e94f-4dc8-8954-15764efed001\") " pod="openshift-console/console-7c6944f9c8-x5ctb" Jul 29 22:49:15.374000 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:49:15.373786 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f06420cf-e94f-4dc8-8954-15764efed001-oauth-serving-cert\") pod \"console-7c6944f9c8-x5ctb\" (UID: \"f06420cf-e94f-4dc8-8954-15764efed001\") " pod="openshift-console/console-7c6944f9c8-x5ctb" Jul 29 22:49:15.375473 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:49:15.375417 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f06420cf-e94f-4dc8-8954-15764efed001-console-oauth-config\") pod \"console-7c6944f9c8-x5ctb\" (UID: \"f06420cf-e94f-4dc8-8954-15764efed001\") " pod="openshift-console/console-7c6944f9c8-x5ctb" Jul 29 22:49:15.375624 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:49:15.375422 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f06420cf-e94f-4dc8-8954-15764efed001-console-serving-cert\") pod \"console-7c6944f9c8-x5ctb\" (UID: \"f06420cf-e94f-4dc8-8954-15764efed001\") " pod="openshift-console/console-7c6944f9c8-x5ctb" Jul 29 22:49:15.387120 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:49:15.387085 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-plpmv\" (UniqueName: \"kubernetes.io/projected/f06420cf-e94f-4dc8-8954-15764efed001-kube-api-access-plpmv\") pod \"console-7c6944f9c8-x5ctb\" (UID: \"f06420cf-e94f-4dc8-8954-15764efed001\") " pod="openshift-console/console-7c6944f9c8-x5ctb" Jul 29 22:49:15.439281 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:49:15.439241 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7c6944f9c8-x5ctb" Jul 29 22:49:15.611915 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:49:15.611888 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7c6944f9c8-x5ctb"] Jul 29 22:49:15.614365 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:49:15.614336 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf06420cf_e94f_4dc8_8954_15764efed001.slice/crio-a944b12ef8c69b0afe0e073a9d1d37d812295dbb4f600d14d62ce16f771e4d3d WatchSource:0}: Error finding container a944b12ef8c69b0afe0e073a9d1d37d812295dbb4f600d14d62ce16f771e4d3d: Status 404 returned error can't find the container with id a944b12ef8c69b0afe0e073a9d1d37d812295dbb4f600d14d62ce16f771e4d3d Jul 29 22:49:16.247027 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:49:16.246994 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7c6944f9c8-x5ctb" event={"ID":"f06420cf-e94f-4dc8-8954-15764efed001","Type":"ContainerStarted","Data":"50fd0542a982264a2e6cad7336a1073822505e8ab7a801818b7c8de0e15b1bd9"} Jul 29 22:49:16.247296 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:49:16.247044 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7c6944f9c8-x5ctb" event={"ID":"f06420cf-e94f-4dc8-8954-15764efed001","Type":"ContainerStarted","Data":"a944b12ef8c69b0afe0e073a9d1d37d812295dbb4f600d14d62ce16f771e4d3d"} Jul 29 22:49:16.267632 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:49:16.267576 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-7c6944f9c8-x5ctb" podStartSLOduration=1.2675572640000001 podStartE2EDuration="1.267557264s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 22:49:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 22:49:16.26677063 +0000 UTC m=+171.178596318" watchObservedRunningTime="2026-07-29 22:49:16.267557264 +0000 UTC m=+171.179382951" Jul 29 22:49:25.440072 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:49:25.439974 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7c6944f9c8-x5ctb" Jul 29 22:49:25.440072 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:49:25.440058 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-7c6944f9c8-x5ctb" Jul 29 22:49:25.444926 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:49:25.444901 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-7c6944f9c8-x5ctb" Jul 29 22:49:26.285095 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:49:26.285061 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-7c6944f9c8-x5ctb" Jul 29 22:49:26.334843 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:49:26.334803 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-ccfdc4d97-5fvkk"] Jul 29 22:49:51.354133 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:49:51.354074 2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-ccfdc4d97-5fvkk" podUID="1e31854a-9461-47db-a6be-75a543ef426e" containerName="console" containerID="cri-o://e7f38d468a956f7fc7bf4b2b3b85853cc997f04e211cd8f6af99cef5124bd91e" gracePeriod=15 Jul 29 22:49:51.589219 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:49:51.589190 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-ccfdc4d97-5fvkk" Jul 29 22:49:51.672790 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:49:51.672698 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1e31854a-9461-47db-a6be-75a543ef426e-console-oauth-config\") pod \"1e31854a-9461-47db-a6be-75a543ef426e\" (UID: \"1e31854a-9461-47db-a6be-75a543ef426e\") " Jul 29 22:49:51.672790 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:49:51.672752 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1e31854a-9461-47db-a6be-75a543ef426e-service-ca\") pod \"1e31854a-9461-47db-a6be-75a543ef426e\" (UID: \"1e31854a-9461-47db-a6be-75a543ef426e\") " Jul 29 22:49:51.672790 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:49:51.672777 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-shd68\" (UniqueName: \"kubernetes.io/projected/1e31854a-9461-47db-a6be-75a543ef426e-kube-api-access-shd68\") pod \"1e31854a-9461-47db-a6be-75a543ef426e\" (UID: \"1e31854a-9461-47db-a6be-75a543ef426e\") " Jul 29 22:49:51.673072 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:49:51.672800 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1e31854a-9461-47db-a6be-75a543ef426e-trusted-ca-bundle\") pod \"1e31854a-9461-47db-a6be-75a543ef426e\" (UID: \"1e31854a-9461-47db-a6be-75a543ef426e\") " Jul 29 22:49:51.673072 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:49:51.672925 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1e31854a-9461-47db-a6be-75a543ef426e-console-serving-cert\") pod \"1e31854a-9461-47db-a6be-75a543ef426e\" (UID: \"1e31854a-9461-47db-a6be-75a543ef426e\") " Jul 29 22:49:51.673072 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:49:51.673011 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1e31854a-9461-47db-a6be-75a543ef426e-oauth-serving-cert\") pod \"1e31854a-9461-47db-a6be-75a543ef426e\" (UID: \"1e31854a-9461-47db-a6be-75a543ef426e\") " Jul 29 22:49:51.673072 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:49:51.673044 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1e31854a-9461-47db-a6be-75a543ef426e-console-config\") pod \"1e31854a-9461-47db-a6be-75a543ef426e\" (UID: \"1e31854a-9461-47db-a6be-75a543ef426e\") " Jul 29 22:49:51.673252 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:49:51.673177 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1e31854a-9461-47db-a6be-75a543ef426e-service-ca" (OuterVolumeSpecName: "service-ca") pod "1e31854a-9461-47db-a6be-75a543ef426e" (UID: "1e31854a-9461-47db-a6be-75a543ef426e"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 29 22:49:51.673252 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:49:51.673202 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1e31854a-9461-47db-a6be-75a543ef426e-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "1e31854a-9461-47db-a6be-75a543ef426e" (UID: "1e31854a-9461-47db-a6be-75a543ef426e"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 29 22:49:51.673460 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:49:51.673331 2562 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1e31854a-9461-47db-a6be-75a543ef426e-service-ca\") on node \"ip-10-0-138-112.ec2.internal\" DevicePath \"\"" Jul 29 22:49:51.673589 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:49:51.673465 2562 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1e31854a-9461-47db-a6be-75a543ef426e-trusted-ca-bundle\") on node \"ip-10-0-138-112.ec2.internal\" DevicePath \"\"" Jul 29 22:49:51.673589 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:49:51.673540 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1e31854a-9461-47db-a6be-75a543ef426e-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "1e31854a-9461-47db-a6be-75a543ef426e" (UID: "1e31854a-9461-47db-a6be-75a543ef426e"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 29 22:49:51.673589 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:49:51.673547 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1e31854a-9461-47db-a6be-75a543ef426e-console-config" (OuterVolumeSpecName: "console-config") pod "1e31854a-9461-47db-a6be-75a543ef426e" (UID: "1e31854a-9461-47db-a6be-75a543ef426e"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 29 22:49:51.675086 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:49:51.675062 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1e31854a-9461-47db-a6be-75a543ef426e-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "1e31854a-9461-47db-a6be-75a543ef426e" (UID: "1e31854a-9461-47db-a6be-75a543ef426e"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 29 22:49:51.675342 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:49:51.675327 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1e31854a-9461-47db-a6be-75a543ef426e-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "1e31854a-9461-47db-a6be-75a543ef426e" (UID: "1e31854a-9461-47db-a6be-75a543ef426e"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 29 22:49:51.675387 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:49:51.675359 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1e31854a-9461-47db-a6be-75a543ef426e-kube-api-access-shd68" (OuterVolumeSpecName: "kube-api-access-shd68") pod "1e31854a-9461-47db-a6be-75a543ef426e" (UID: "1e31854a-9461-47db-a6be-75a543ef426e"). InnerVolumeSpecName "kube-api-access-shd68". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 29 22:49:51.774822 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:49:51.774780 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-shd68\" (UniqueName: \"kubernetes.io/projected/1e31854a-9461-47db-a6be-75a543ef426e-kube-api-access-shd68\") on node \"ip-10-0-138-112.ec2.internal\" DevicePath \"\"" Jul 29 22:49:51.774822 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:49:51.774811 2562 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1e31854a-9461-47db-a6be-75a543ef426e-console-serving-cert\") on node \"ip-10-0-138-112.ec2.internal\" DevicePath \"\"" Jul 29 22:49:51.774822 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:49:51.774821 2562 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1e31854a-9461-47db-a6be-75a543ef426e-oauth-serving-cert\") on node \"ip-10-0-138-112.ec2.internal\" DevicePath \"\"" Jul 29 22:49:51.774822 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:49:51.774832 2562 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1e31854a-9461-47db-a6be-75a543ef426e-console-config\") on node \"ip-10-0-138-112.ec2.internal\" DevicePath \"\"" Jul 29 22:49:51.775079 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:49:51.774840 2562 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1e31854a-9461-47db-a6be-75a543ef426e-console-oauth-config\") on node \"ip-10-0-138-112.ec2.internal\" DevicePath \"\"" Jul 29 22:49:52.373927 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:49:52.373887 2562 generic.go:356] "Generic (PLEG): container finished" podID="1e31854a-9461-47db-a6be-75a543ef426e" containerID="e7f38d468a956f7fc7bf4b2b3b85853cc997f04e211cd8f6af99cef5124bd91e" exitCode=0 Jul 29 22:49:52.374422 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:49:52.373937 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-ccfdc4d97-5fvkk" Jul 29 22:49:52.374422 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:49:52.373977 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-ccfdc4d97-5fvkk" event={"ID":"1e31854a-9461-47db-a6be-75a543ef426e","Type":"ContainerDied","Data":"e7f38d468a956f7fc7bf4b2b3b85853cc997f04e211cd8f6af99cef5124bd91e"} Jul 29 22:49:52.374422 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:49:52.374047 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-ccfdc4d97-5fvkk" event={"ID":"1e31854a-9461-47db-a6be-75a543ef426e","Type":"ContainerDied","Data":"71a7783082151a8118eed9a1f60e8e564918eadbc9b5ce17c3286c18a41c4c7b"} Jul 29 22:49:52.374422 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:49:52.374080 2562 scope.go:117] "RemoveContainer" containerID="e7f38d468a956f7fc7bf4b2b3b85853cc997f04e211cd8f6af99cef5124bd91e" Jul 29 22:49:52.383043 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:49:52.383024 2562 scope.go:117] "RemoveContainer" containerID="e7f38d468a956f7fc7bf4b2b3b85853cc997f04e211cd8f6af99cef5124bd91e" Jul 29 22:49:52.383352 ip-10-0-138-112 kubenswrapper[2562]: E0729 22:49:52.383329 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e7f38d468a956f7fc7bf4b2b3b85853cc997f04e211cd8f6af99cef5124bd91e\": container with ID starting with e7f38d468a956f7fc7bf4b2b3b85853cc997f04e211cd8f6af99cef5124bd91e not found: ID does not exist" containerID="e7f38d468a956f7fc7bf4b2b3b85853cc997f04e211cd8f6af99cef5124bd91e" Jul 29 22:49:52.383451 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:49:52.383360 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e7f38d468a956f7fc7bf4b2b3b85853cc997f04e211cd8f6af99cef5124bd91e"} err="failed to get container status \"e7f38d468a956f7fc7bf4b2b3b85853cc997f04e211cd8f6af99cef5124bd91e\": rpc error: code = NotFound desc = could not find container \"e7f38d468a956f7fc7bf4b2b3b85853cc997f04e211cd8f6af99cef5124bd91e\": container with ID starting with e7f38d468a956f7fc7bf4b2b3b85853cc997f04e211cd8f6af99cef5124bd91e not found: ID does not exist" Jul 29 22:49:52.396575 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:49:52.396542 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-ccfdc4d97-5fvkk"] Jul 29 22:49:52.399444 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:49:52.399420 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-ccfdc4d97-5fvkk"] Jul 29 22:49:53.574292 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:49:53.574259 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1e31854a-9461-47db-a6be-75a543ef426e" path="/var/lib/kubelet/pods/1e31854a-9461-47db-a6be-75a543ef426e/volumes" Jul 29 22:50:34.463975 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:50:34.463940 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-6d665b88d6-z7blc"] Jul 29 22:50:34.464467 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:50:34.464191 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1e31854a-9461-47db-a6be-75a543ef426e" containerName="console" Jul 29 22:50:34.464467 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:50:34.464201 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="1e31854a-9461-47db-a6be-75a543ef426e" containerName="console" Jul 29 22:50:34.464467 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:50:34.464252 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="1e31854a-9461-47db-a6be-75a543ef426e" containerName="console" Jul 29 22:50:34.468186 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:50:34.468166 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6d665b88d6-z7blc" Jul 29 22:50:34.479612 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:50:34.479587 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6d665b88d6-z7blc"] Jul 29 22:50:34.601350 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:50:34.601311 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a50338ed-263f-4855-aa0a-53fb0cd72ed7-oauth-serving-cert\") pod \"console-6d665b88d6-z7blc\" (UID: \"a50338ed-263f-4855-aa0a-53fb0cd72ed7\") " pod="openshift-console/console-6d665b88d6-z7blc" Jul 29 22:50:34.601350 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:50:34.601356 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h2lkn\" (UniqueName: \"kubernetes.io/projected/a50338ed-263f-4855-aa0a-53fb0cd72ed7-kube-api-access-h2lkn\") pod \"console-6d665b88d6-z7blc\" (UID: \"a50338ed-263f-4855-aa0a-53fb0cd72ed7\") " pod="openshift-console/console-6d665b88d6-z7blc" Jul 29 22:50:34.601638 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:50:34.601391 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a50338ed-263f-4855-aa0a-53fb0cd72ed7-service-ca\") pod \"console-6d665b88d6-z7blc\" (UID: \"a50338ed-263f-4855-aa0a-53fb0cd72ed7\") " pod="openshift-console/console-6d665b88d6-z7blc" Jul 29 22:50:34.601638 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:50:34.601472 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a50338ed-263f-4855-aa0a-53fb0cd72ed7-console-config\") pod \"console-6d665b88d6-z7blc\" (UID: \"a50338ed-263f-4855-aa0a-53fb0cd72ed7\") " pod="openshift-console/console-6d665b88d6-z7blc" Jul 29 22:50:34.601638 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:50:34.601530 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a50338ed-263f-4855-aa0a-53fb0cd72ed7-trusted-ca-bundle\") pod \"console-6d665b88d6-z7blc\" (UID: \"a50338ed-263f-4855-aa0a-53fb0cd72ed7\") " pod="openshift-console/console-6d665b88d6-z7blc" Jul 29 22:50:34.601638 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:50:34.601566 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a50338ed-263f-4855-aa0a-53fb0cd72ed7-console-oauth-config\") pod \"console-6d665b88d6-z7blc\" (UID: \"a50338ed-263f-4855-aa0a-53fb0cd72ed7\") " pod="openshift-console/console-6d665b88d6-z7blc" Jul 29 22:50:34.601638 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:50:34.601590 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a50338ed-263f-4855-aa0a-53fb0cd72ed7-console-serving-cert\") pod \"console-6d665b88d6-z7blc\" (UID: \"a50338ed-263f-4855-aa0a-53fb0cd72ed7\") " pod="openshift-console/console-6d665b88d6-z7blc" Jul 29 22:50:34.702011 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:50:34.701974 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a50338ed-263f-4855-aa0a-53fb0cd72ed7-oauth-serving-cert\") pod \"console-6d665b88d6-z7blc\" (UID: \"a50338ed-263f-4855-aa0a-53fb0cd72ed7\") " pod="openshift-console/console-6d665b88d6-z7blc" Jul 29 22:50:34.702011 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:50:34.702020 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h2lkn\" (UniqueName: \"kubernetes.io/projected/a50338ed-263f-4855-aa0a-53fb0cd72ed7-kube-api-access-h2lkn\") pod \"console-6d665b88d6-z7blc\" (UID: \"a50338ed-263f-4855-aa0a-53fb0cd72ed7\") " pod="openshift-console/console-6d665b88d6-z7blc" Jul 29 22:50:34.702267 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:50:34.702053 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a50338ed-263f-4855-aa0a-53fb0cd72ed7-service-ca\") pod \"console-6d665b88d6-z7blc\" (UID: \"a50338ed-263f-4855-aa0a-53fb0cd72ed7\") " pod="openshift-console/console-6d665b88d6-z7blc" Jul 29 22:50:34.702267 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:50:34.702074 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a50338ed-263f-4855-aa0a-53fb0cd72ed7-console-config\") pod \"console-6d665b88d6-z7blc\" (UID: \"a50338ed-263f-4855-aa0a-53fb0cd72ed7\") " pod="openshift-console/console-6d665b88d6-z7blc" Jul 29 22:50:34.702267 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:50:34.702116 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a50338ed-263f-4855-aa0a-53fb0cd72ed7-trusted-ca-bundle\") pod \"console-6d665b88d6-z7blc\" (UID: \"a50338ed-263f-4855-aa0a-53fb0cd72ed7\") " pod="openshift-console/console-6d665b88d6-z7blc" Jul 29 22:50:34.702267 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:50:34.702140 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a50338ed-263f-4855-aa0a-53fb0cd72ed7-console-oauth-config\") pod \"console-6d665b88d6-z7blc\" (UID: \"a50338ed-263f-4855-aa0a-53fb0cd72ed7\") " pod="openshift-console/console-6d665b88d6-z7blc" Jul 29 22:50:34.702267 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:50:34.702166 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a50338ed-263f-4855-aa0a-53fb0cd72ed7-console-serving-cert\") pod \"console-6d665b88d6-z7blc\" (UID: \"a50338ed-263f-4855-aa0a-53fb0cd72ed7\") " pod="openshift-console/console-6d665b88d6-z7blc" Jul 29 22:50:34.702844 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:50:34.702819 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a50338ed-263f-4855-aa0a-53fb0cd72ed7-oauth-serving-cert\") pod \"console-6d665b88d6-z7blc\" (UID: \"a50338ed-263f-4855-aa0a-53fb0cd72ed7\") " pod="openshift-console/console-6d665b88d6-z7blc" Jul 29 22:50:34.702844 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:50:34.702840 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a50338ed-263f-4855-aa0a-53fb0cd72ed7-service-ca\") pod \"console-6d665b88d6-z7blc\" (UID: \"a50338ed-263f-4855-aa0a-53fb0cd72ed7\") " pod="openshift-console/console-6d665b88d6-z7blc" Jul 29 22:50:34.703000 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:50:34.702976 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a50338ed-263f-4855-aa0a-53fb0cd72ed7-console-config\") pod \"console-6d665b88d6-z7blc\" (UID: \"a50338ed-263f-4855-aa0a-53fb0cd72ed7\") " pod="openshift-console/console-6d665b88d6-z7blc" Jul 29 22:50:34.703039 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:50:34.703026 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a50338ed-263f-4855-aa0a-53fb0cd72ed7-trusted-ca-bundle\") pod \"console-6d665b88d6-z7blc\" (UID: \"a50338ed-263f-4855-aa0a-53fb0cd72ed7\") " pod="openshift-console/console-6d665b88d6-z7blc" Jul 29 22:50:34.705090 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:50:34.705062 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a50338ed-263f-4855-aa0a-53fb0cd72ed7-console-oauth-config\") pod \"console-6d665b88d6-z7blc\" (UID: \"a50338ed-263f-4855-aa0a-53fb0cd72ed7\") " pod="openshift-console/console-6d665b88d6-z7blc" Jul 29 22:50:34.705181 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:50:34.705145 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a50338ed-263f-4855-aa0a-53fb0cd72ed7-console-serving-cert\") pod \"console-6d665b88d6-z7blc\" (UID: \"a50338ed-263f-4855-aa0a-53fb0cd72ed7\") " pod="openshift-console/console-6d665b88d6-z7blc" Jul 29 22:50:34.710270 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:50:34.710242 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h2lkn\" (UniqueName: \"kubernetes.io/projected/a50338ed-263f-4855-aa0a-53fb0cd72ed7-kube-api-access-h2lkn\") pod \"console-6d665b88d6-z7blc\" (UID: \"a50338ed-263f-4855-aa0a-53fb0cd72ed7\") " pod="openshift-console/console-6d665b88d6-z7blc" Jul 29 22:50:34.777247 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:50:34.777211 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6d665b88d6-z7blc" Jul 29 22:50:34.910640 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:50:34.910608 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6d665b88d6-z7blc"] Jul 29 22:50:34.913535 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:50:34.913504 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda50338ed_263f_4855_aa0a_53fb0cd72ed7.slice/crio-7246d302d3704cfa6d59be3ef53a29ed893091abacb0257d20b1c06f01c7c39b WatchSource:0}: Error finding container 7246d302d3704cfa6d59be3ef53a29ed893091abacb0257d20b1c06f01c7c39b: Status 404 returned error can't find the container with id 7246d302d3704cfa6d59be3ef53a29ed893091abacb0257d20b1c06f01c7c39b Jul 29 22:50:35.526652 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:50:35.526615 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6d665b88d6-z7blc" event={"ID":"a50338ed-263f-4855-aa0a-53fb0cd72ed7","Type":"ContainerStarted","Data":"0ae94cf3f0392475058f0c2f35ba14a5b8394d0f4989823b149f103ea2a3b7a2"} Jul 29 22:50:35.527062 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:50:35.526669 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6d665b88d6-z7blc" event={"ID":"a50338ed-263f-4855-aa0a-53fb0cd72ed7","Type":"ContainerStarted","Data":"7246d302d3704cfa6d59be3ef53a29ed893091abacb0257d20b1c06f01c7c39b"} Jul 29 22:50:35.547306 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:50:35.547250 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-6d665b88d6-z7blc" podStartSLOduration=1.547228491 podStartE2EDuration="1.547228491s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 22:50:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 22:50:35.545320489 +0000 UTC m=+250.457146177" watchObservedRunningTime="2026-07-29 22:50:35.547228491 +0000 UTC m=+250.459054178" Jul 29 22:50:44.777317 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:50:44.777280 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6d665b88d6-z7blc" Jul 29 22:50:44.777807 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:50:44.777343 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-6d665b88d6-z7blc" Jul 29 22:50:44.782124 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:50:44.782103 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-6d665b88d6-z7blc" Jul 29 22:50:45.574098 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:50:45.574073 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-6d665b88d6-z7blc" Jul 29 22:50:45.624596 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:50:45.624559 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7c6944f9c8-x5ctb"] Jul 29 22:51:10.643298 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:51:10.643253 2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-7c6944f9c8-x5ctb" podUID="f06420cf-e94f-4dc8-8954-15764efed001" containerName="console" containerID="cri-o://50fd0542a982264a2e6cad7336a1073822505e8ab7a801818b7c8de0e15b1bd9" gracePeriod=15 Jul 29 22:51:10.883334 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:51:10.883306 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7c6944f9c8-x5ctb" Jul 29 22:51:10.983877 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:51:10.983783 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-plpmv\" (UniqueName: \"kubernetes.io/projected/f06420cf-e94f-4dc8-8954-15764efed001-kube-api-access-plpmv\") pod \"f06420cf-e94f-4dc8-8954-15764efed001\" (UID: \"f06420cf-e94f-4dc8-8954-15764efed001\") " Jul 29 22:51:10.983877 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:51:10.983833 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f06420cf-e94f-4dc8-8954-15764efed001-console-oauth-config\") pod \"f06420cf-e94f-4dc8-8954-15764efed001\" (UID: \"f06420cf-e94f-4dc8-8954-15764efed001\") " Jul 29 22:51:10.983877 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:51:10.983857 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f06420cf-e94f-4dc8-8954-15764efed001-service-ca\") pod \"f06420cf-e94f-4dc8-8954-15764efed001\" (UID: \"f06420cf-e94f-4dc8-8954-15764efed001\") " Jul 29 22:51:10.984165 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:51:10.983885 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f06420cf-e94f-4dc8-8954-15764efed001-trusted-ca-bundle\") pod \"f06420cf-e94f-4dc8-8954-15764efed001\" (UID: \"f06420cf-e94f-4dc8-8954-15764efed001\") " Jul 29 22:51:10.984165 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:51:10.983989 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f06420cf-e94f-4dc8-8954-15764efed001-console-serving-cert\") pod \"f06420cf-e94f-4dc8-8954-15764efed001\" (UID: \"f06420cf-e94f-4dc8-8954-15764efed001\") " Jul 29 22:51:10.984165 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:51:10.984034 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f06420cf-e94f-4dc8-8954-15764efed001-console-config\") pod \"f06420cf-e94f-4dc8-8954-15764efed001\" (UID: \"f06420cf-e94f-4dc8-8954-15764efed001\") " Jul 29 22:51:10.984165 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:51:10.984100 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f06420cf-e94f-4dc8-8954-15764efed001-oauth-serving-cert\") pod \"f06420cf-e94f-4dc8-8954-15764efed001\" (UID: \"f06420cf-e94f-4dc8-8954-15764efed001\") " Jul 29 22:51:10.984353 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:51:10.984257 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f06420cf-e94f-4dc8-8954-15764efed001-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "f06420cf-e94f-4dc8-8954-15764efed001" (UID: "f06420cf-e94f-4dc8-8954-15764efed001"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 29 22:51:10.984353 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:51:10.984337 2562 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f06420cf-e94f-4dc8-8954-15764efed001-trusted-ca-bundle\") on node \"ip-10-0-138-112.ec2.internal\" DevicePath \"\"" Jul 29 22:51:10.984504 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:51:10.984420 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f06420cf-e94f-4dc8-8954-15764efed001-service-ca" (OuterVolumeSpecName: "service-ca") pod "f06420cf-e94f-4dc8-8954-15764efed001" (UID: "f06420cf-e94f-4dc8-8954-15764efed001"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 29 22:51:10.984504 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:51:10.984492 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f06420cf-e94f-4dc8-8954-15764efed001-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "f06420cf-e94f-4dc8-8954-15764efed001" (UID: "f06420cf-e94f-4dc8-8954-15764efed001"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 29 22:51:10.984711 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:51:10.984685 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f06420cf-e94f-4dc8-8954-15764efed001-console-config" (OuterVolumeSpecName: "console-config") pod "f06420cf-e94f-4dc8-8954-15764efed001" (UID: "f06420cf-e94f-4dc8-8954-15764efed001"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 29 22:51:10.986087 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:51:10.986056 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f06420cf-e94f-4dc8-8954-15764efed001-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "f06420cf-e94f-4dc8-8954-15764efed001" (UID: "f06420cf-e94f-4dc8-8954-15764efed001"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 29 22:51:10.986192 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:51:10.986087 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f06420cf-e94f-4dc8-8954-15764efed001-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "f06420cf-e94f-4dc8-8954-15764efed001" (UID: "f06420cf-e94f-4dc8-8954-15764efed001"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 29 22:51:10.986192 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:51:10.986114 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f06420cf-e94f-4dc8-8954-15764efed001-kube-api-access-plpmv" (OuterVolumeSpecName: "kube-api-access-plpmv") pod "f06420cf-e94f-4dc8-8954-15764efed001" (UID: "f06420cf-e94f-4dc8-8954-15764efed001"). InnerVolumeSpecName "kube-api-access-plpmv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 29 22:51:11.085570 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:51:11.085530 2562 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f06420cf-e94f-4dc8-8954-15764efed001-oauth-serving-cert\") on node \"ip-10-0-138-112.ec2.internal\" DevicePath \"\"" Jul 29 22:51:11.085570 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:51:11.085560 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-plpmv\" (UniqueName: \"kubernetes.io/projected/f06420cf-e94f-4dc8-8954-15764efed001-kube-api-access-plpmv\") on node \"ip-10-0-138-112.ec2.internal\" DevicePath \"\"" Jul 29 22:51:11.085570 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:51:11.085570 2562 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f06420cf-e94f-4dc8-8954-15764efed001-console-oauth-config\") on node \"ip-10-0-138-112.ec2.internal\" DevicePath \"\"" Jul 29 22:51:11.085570 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:51:11.085579 2562 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f06420cf-e94f-4dc8-8954-15764efed001-service-ca\") on node \"ip-10-0-138-112.ec2.internal\" DevicePath \"\"" Jul 29 22:51:11.085824 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:51:11.085588 2562 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f06420cf-e94f-4dc8-8954-15764efed001-console-serving-cert\") on node \"ip-10-0-138-112.ec2.internal\" DevicePath \"\"" Jul 29 22:51:11.085824 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:51:11.085596 2562 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f06420cf-e94f-4dc8-8954-15764efed001-console-config\") on node \"ip-10-0-138-112.ec2.internal\" DevicePath \"\"" Jul 29 22:51:11.653458 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:51:11.653419 2562 generic.go:356] "Generic (PLEG): container finished" podID="f06420cf-e94f-4dc8-8954-15764efed001" containerID="50fd0542a982264a2e6cad7336a1073822505e8ab7a801818b7c8de0e15b1bd9" exitCode=0 Jul 29 22:51:11.653943 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:51:11.653470 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7c6944f9c8-x5ctb" event={"ID":"f06420cf-e94f-4dc8-8954-15764efed001","Type":"ContainerDied","Data":"50fd0542a982264a2e6cad7336a1073822505e8ab7a801818b7c8de0e15b1bd9"} Jul 29 22:51:11.653943 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:51:11.653487 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7c6944f9c8-x5ctb" Jul 29 22:51:11.653943 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:51:11.653513 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7c6944f9c8-x5ctb" event={"ID":"f06420cf-e94f-4dc8-8954-15764efed001","Type":"ContainerDied","Data":"a944b12ef8c69b0afe0e073a9d1d37d812295dbb4f600d14d62ce16f771e4d3d"} Jul 29 22:51:11.653943 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:51:11.653535 2562 scope.go:117] "RemoveContainer" containerID="50fd0542a982264a2e6cad7336a1073822505e8ab7a801818b7c8de0e15b1bd9" Jul 29 22:51:11.661462 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:51:11.661445 2562 scope.go:117] "RemoveContainer" containerID="50fd0542a982264a2e6cad7336a1073822505e8ab7a801818b7c8de0e15b1bd9" Jul 29 22:51:11.661775 ip-10-0-138-112 kubenswrapper[2562]: E0729 22:51:11.661751 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"50fd0542a982264a2e6cad7336a1073822505e8ab7a801818b7c8de0e15b1bd9\": container with ID starting with 50fd0542a982264a2e6cad7336a1073822505e8ab7a801818b7c8de0e15b1bd9 not found: ID does not exist" containerID="50fd0542a982264a2e6cad7336a1073822505e8ab7a801818b7c8de0e15b1bd9" Jul 29 22:51:11.661865 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:51:11.661782 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"50fd0542a982264a2e6cad7336a1073822505e8ab7a801818b7c8de0e15b1bd9"} err="failed to get container status \"50fd0542a982264a2e6cad7336a1073822505e8ab7a801818b7c8de0e15b1bd9\": rpc error: code = NotFound desc = could not find container \"50fd0542a982264a2e6cad7336a1073822505e8ab7a801818b7c8de0e15b1bd9\": container with ID starting with 50fd0542a982264a2e6cad7336a1073822505e8ab7a801818b7c8de0e15b1bd9 not found: ID does not exist" Jul 29 22:51:11.670324 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:51:11.670275 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7c6944f9c8-x5ctb"] Jul 29 22:51:11.673895 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:51:11.673871 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-7c6944f9c8-x5ctb"] Jul 29 22:51:13.574472 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:51:13.574437 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f06420cf-e94f-4dc8-8954-15764efed001" path="/var/lib/kubelet/pods/f06420cf-e94f-4dc8-8954-15764efed001/volumes" Jul 29 22:51:25.452334 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:51:25.452300 2562 kubelet.go:1648] "Image garbage collection succeeded" Jul 29 22:51:49.363158 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:51:49.363119 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5tl4wd"] Jul 29 22:51:49.365593 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:51:49.363372 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f06420cf-e94f-4dc8-8954-15764efed001" containerName="console" Jul 29 22:51:49.365593 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:51:49.363382 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="f06420cf-e94f-4dc8-8954-15764efed001" containerName="console" Jul 29 22:51:49.365593 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:51:49.363448 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="f06420cf-e94f-4dc8-8954-15764efed001" containerName="console" Jul 29 22:51:49.366487 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:51:49.366469 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5tl4wd" Jul 29 22:51:49.369799 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:51:49.369773 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 29 22:51:49.369942 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:51:49.369838 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 29 22:51:49.369942 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:51:49.369914 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-jzs2h\"" Jul 29 22:51:49.375052 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:51:49.375029 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5tl4wd"] Jul 29 22:51:49.486649 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:51:49.486607 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/cd1d526b-9639-4b5f-82d4-a29f7bc2f0dd-util\") pod \"320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5tl4wd\" (UID: \"cd1d526b-9639-4b5f-82d4-a29f7bc2f0dd\") " pod="openshift-marketplace/320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5tl4wd" Jul 29 22:51:49.486649 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:51:49.486655 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ntd8k\" (UniqueName: \"kubernetes.io/projected/cd1d526b-9639-4b5f-82d4-a29f7bc2f0dd-kube-api-access-ntd8k\") pod \"320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5tl4wd\" (UID: \"cd1d526b-9639-4b5f-82d4-a29f7bc2f0dd\") " pod="openshift-marketplace/320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5tl4wd" Jul 29 22:51:49.486869 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:51:49.486732 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/cd1d526b-9639-4b5f-82d4-a29f7bc2f0dd-bundle\") pod \"320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5tl4wd\" (UID: \"cd1d526b-9639-4b5f-82d4-a29f7bc2f0dd\") " pod="openshift-marketplace/320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5tl4wd" Jul 29 22:51:49.587618 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:51:49.587576 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/cd1d526b-9639-4b5f-82d4-a29f7bc2f0dd-bundle\") pod \"320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5tl4wd\" (UID: \"cd1d526b-9639-4b5f-82d4-a29f7bc2f0dd\") " pod="openshift-marketplace/320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5tl4wd" Jul 29 22:51:49.587788 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:51:49.587635 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/cd1d526b-9639-4b5f-82d4-a29f7bc2f0dd-util\") pod \"320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5tl4wd\" (UID: \"cd1d526b-9639-4b5f-82d4-a29f7bc2f0dd\") " pod="openshift-marketplace/320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5tl4wd" Jul 29 22:51:49.587788 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:51:49.587661 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ntd8k\" (UniqueName: \"kubernetes.io/projected/cd1d526b-9639-4b5f-82d4-a29f7bc2f0dd-kube-api-access-ntd8k\") pod \"320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5tl4wd\" (UID: \"cd1d526b-9639-4b5f-82d4-a29f7bc2f0dd\") " pod="openshift-marketplace/320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5tl4wd" Jul 29 22:51:49.587979 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:51:49.587959 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/cd1d526b-9639-4b5f-82d4-a29f7bc2f0dd-bundle\") pod \"320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5tl4wd\" (UID: \"cd1d526b-9639-4b5f-82d4-a29f7bc2f0dd\") " pod="openshift-marketplace/320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5tl4wd" Jul 29 22:51:49.588019 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:51:49.587994 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/cd1d526b-9639-4b5f-82d4-a29f7bc2f0dd-util\") pod \"320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5tl4wd\" (UID: \"cd1d526b-9639-4b5f-82d4-a29f7bc2f0dd\") " pod="openshift-marketplace/320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5tl4wd" Jul 29 22:51:49.596747 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:51:49.596724 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ntd8k\" (UniqueName: \"kubernetes.io/projected/cd1d526b-9639-4b5f-82d4-a29f7bc2f0dd-kube-api-access-ntd8k\") pod \"320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5tl4wd\" (UID: \"cd1d526b-9639-4b5f-82d4-a29f7bc2f0dd\") " pod="openshift-marketplace/320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5tl4wd" Jul 29 22:51:49.676963 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:51:49.676872 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5tl4wd" Jul 29 22:51:49.805487 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:51:49.805453 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5tl4wd"] Jul 29 22:51:49.811139 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:51:49.811108 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcd1d526b_9639_4b5f_82d4_a29f7bc2f0dd.slice/crio-15416d6a02e1b9e904783166b9a320b7dcfd0acc222582a836828bca00565da2 WatchSource:0}: Error finding container 15416d6a02e1b9e904783166b9a320b7dcfd0acc222582a836828bca00565da2: Status 404 returned error can't find the container with id 15416d6a02e1b9e904783166b9a320b7dcfd0acc222582a836828bca00565da2 Jul 29 22:51:49.813477 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:51:49.813457 2562 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 29 22:51:50.788826 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:51:50.788787 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5tl4wd" event={"ID":"cd1d526b-9639-4b5f-82d4-a29f7bc2f0dd","Type":"ContainerStarted","Data":"15416d6a02e1b9e904783166b9a320b7dcfd0acc222582a836828bca00565da2"} Jul 29 22:51:55.811090 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:51:55.811052 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5tl4wd" event={"ID":"cd1d526b-9639-4b5f-82d4-a29f7bc2f0dd","Type":"ContainerStarted","Data":"ae57e0f147b7171b3b2728e884d565160340a4ecfb51abc2facde1350db65351"} Jul 29 22:51:56.815806 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:51:56.815766 2562 generic.go:356] "Generic (PLEG): container finished" podID="cd1d526b-9639-4b5f-82d4-a29f7bc2f0dd" containerID="ae57e0f147b7171b3b2728e884d565160340a4ecfb51abc2facde1350db65351" exitCode=0 Jul 29 22:51:56.816202 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:51:56.815841 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5tl4wd" event={"ID":"cd1d526b-9639-4b5f-82d4-a29f7bc2f0dd","Type":"ContainerDied","Data":"ae57e0f147b7171b3b2728e884d565160340a4ecfb51abc2facde1350db65351"} Jul 29 22:51:59.828601 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:51:59.828562 2562 generic.go:356] "Generic (PLEG): container finished" podID="cd1d526b-9639-4b5f-82d4-a29f7bc2f0dd" containerID="c309b598260cc202ff735505a57d9a6cd1161cbd21e99c30f0e576c358c125bb" exitCode=0 Jul 29 22:51:59.828990 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:51:59.828652 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5tl4wd" event={"ID":"cd1d526b-9639-4b5f-82d4-a29f7bc2f0dd","Type":"ContainerDied","Data":"c309b598260cc202ff735505a57d9a6cd1161cbd21e99c30f0e576c358c125bb"} Jul 29 22:52:06.855744 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:52:06.855706 2562 generic.go:356] "Generic (PLEG): container finished" podID="cd1d526b-9639-4b5f-82d4-a29f7bc2f0dd" containerID="22dad93f14024aaa33ac2fb22c80e40f5a18ddd433f046cb9e77041c00cdeb3e" exitCode=0 Jul 29 22:52:06.856179 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:52:06.855795 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5tl4wd" event={"ID":"cd1d526b-9639-4b5f-82d4-a29f7bc2f0dd","Type":"ContainerDied","Data":"22dad93f14024aaa33ac2fb22c80e40f5a18ddd433f046cb9e77041c00cdeb3e"} Jul 29 22:52:07.983835 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:52:07.983809 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5tl4wd" Jul 29 22:52:08.045237 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:52:08.045192 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ntd8k\" (UniqueName: \"kubernetes.io/projected/cd1d526b-9639-4b5f-82d4-a29f7bc2f0dd-kube-api-access-ntd8k\") pod \"cd1d526b-9639-4b5f-82d4-a29f7bc2f0dd\" (UID: \"cd1d526b-9639-4b5f-82d4-a29f7bc2f0dd\") " Jul 29 22:52:08.045442 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:52:08.045258 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/cd1d526b-9639-4b5f-82d4-a29f7bc2f0dd-bundle\") pod \"cd1d526b-9639-4b5f-82d4-a29f7bc2f0dd\" (UID: \"cd1d526b-9639-4b5f-82d4-a29f7bc2f0dd\") " Jul 29 22:52:08.045442 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:52:08.045308 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/cd1d526b-9639-4b5f-82d4-a29f7bc2f0dd-util\") pod \"cd1d526b-9639-4b5f-82d4-a29f7bc2f0dd\" (UID: \"cd1d526b-9639-4b5f-82d4-a29f7bc2f0dd\") " Jul 29 22:52:08.046086 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:52:08.046057 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cd1d526b-9639-4b5f-82d4-a29f7bc2f0dd-bundle" (OuterVolumeSpecName: "bundle") pod "cd1d526b-9639-4b5f-82d4-a29f7bc2f0dd" (UID: "cd1d526b-9639-4b5f-82d4-a29f7bc2f0dd"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 29 22:52:08.047593 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:52:08.047566 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cd1d526b-9639-4b5f-82d4-a29f7bc2f0dd-kube-api-access-ntd8k" (OuterVolumeSpecName: "kube-api-access-ntd8k") pod "cd1d526b-9639-4b5f-82d4-a29f7bc2f0dd" (UID: "cd1d526b-9639-4b5f-82d4-a29f7bc2f0dd"). InnerVolumeSpecName "kube-api-access-ntd8k". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 29 22:52:08.049620 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:52:08.049594 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cd1d526b-9639-4b5f-82d4-a29f7bc2f0dd-util" (OuterVolumeSpecName: "util") pod "cd1d526b-9639-4b5f-82d4-a29f7bc2f0dd" (UID: "cd1d526b-9639-4b5f-82d4-a29f7bc2f0dd"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 29 22:52:08.145943 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:52:08.145843 2562 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/cd1d526b-9639-4b5f-82d4-a29f7bc2f0dd-util\") on node \"ip-10-0-138-112.ec2.internal\" DevicePath \"\"" Jul 29 22:52:08.145943 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:52:08.145885 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ntd8k\" (UniqueName: \"kubernetes.io/projected/cd1d526b-9639-4b5f-82d4-a29f7bc2f0dd-kube-api-access-ntd8k\") on node \"ip-10-0-138-112.ec2.internal\" DevicePath \"\"" Jul 29 22:52:08.145943 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:52:08.145894 2562 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/cd1d526b-9639-4b5f-82d4-a29f7bc2f0dd-bundle\") on node \"ip-10-0-138-112.ec2.internal\" DevicePath \"\"" Jul 29 22:52:08.864831 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:52:08.864799 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5tl4wd" event={"ID":"cd1d526b-9639-4b5f-82d4-a29f7bc2f0dd","Type":"ContainerDied","Data":"15416d6a02e1b9e904783166b9a320b7dcfd0acc222582a836828bca00565da2"} Jul 29 22:52:08.865033 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:52:08.864856 2562 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="15416d6a02e1b9e904783166b9a320b7dcfd0acc222582a836828bca00565da2" Jul 29 22:52:08.865033 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:52:08.864813 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5tl4wd" Jul 29 22:52:11.492252 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:52:11.492218 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-keda/custom-metrics-autoscaler-operator-6969cb7dfb-fcss7"] Jul 29 22:52:11.492731 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:52:11.492505 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cd1d526b-9639-4b5f-82d4-a29f7bc2f0dd" containerName="extract" Jul 29 22:52:11.492731 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:52:11.492518 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="cd1d526b-9639-4b5f-82d4-a29f7bc2f0dd" containerName="extract" Jul 29 22:52:11.492731 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:52:11.492532 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cd1d526b-9639-4b5f-82d4-a29f7bc2f0dd" containerName="util" Jul 29 22:52:11.492731 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:52:11.492538 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="cd1d526b-9639-4b5f-82d4-a29f7bc2f0dd" containerName="util" Jul 29 22:52:11.492731 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:52:11.492548 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cd1d526b-9639-4b5f-82d4-a29f7bc2f0dd" containerName="pull" Jul 29 22:52:11.492731 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:52:11.492553 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="cd1d526b-9639-4b5f-82d4-a29f7bc2f0dd" containerName="pull" Jul 29 22:52:11.492731 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:52:11.492592 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="cd1d526b-9639-4b5f-82d4-a29f7bc2f0dd" containerName="extract" Jul 29 22:52:11.496007 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:52:11.495988 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/custom-metrics-autoscaler-operator-6969cb7dfb-fcss7" Jul 29 22:52:11.498974 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:52:11.498953 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"kedaorg-certs\"" Jul 29 22:52:11.499141 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:52:11.499122 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"custom-metrics-autoscaler-operator-dockercfg-fcc7l\"" Jul 29 22:52:11.499218 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:52:11.499174 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-keda\"/\"openshift-service-ca.crt\"" Jul 29 22:52:11.499275 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:52:11.499175 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-keda\"/\"kube-root-ca.crt\"" Jul 29 22:52:11.508223 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:52:11.508197 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/custom-metrics-autoscaler-operator-6969cb7dfb-fcss7"] Jul 29 22:52:11.574259 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:52:11.574219 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pqkgj\" (UniqueName: \"kubernetes.io/projected/d2c71659-b44a-41bb-b082-a9aa3368e44c-kube-api-access-pqkgj\") pod \"custom-metrics-autoscaler-operator-6969cb7dfb-fcss7\" (UID: \"d2c71659-b44a-41bb-b082-a9aa3368e44c\") " pod="openshift-keda/custom-metrics-autoscaler-operator-6969cb7dfb-fcss7" Jul 29 22:52:11.574450 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:52:11.574351 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/secret/d2c71659-b44a-41bb-b082-a9aa3368e44c-certificates\") pod \"custom-metrics-autoscaler-operator-6969cb7dfb-fcss7\" (UID: \"d2c71659-b44a-41bb-b082-a9aa3368e44c\") " pod="openshift-keda/custom-metrics-autoscaler-operator-6969cb7dfb-fcss7" Jul 29 22:52:11.675351 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:52:11.675311 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/secret/d2c71659-b44a-41bb-b082-a9aa3368e44c-certificates\") pod \"custom-metrics-autoscaler-operator-6969cb7dfb-fcss7\" (UID: \"d2c71659-b44a-41bb-b082-a9aa3368e44c\") " pod="openshift-keda/custom-metrics-autoscaler-operator-6969cb7dfb-fcss7" Jul 29 22:52:11.675351 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:52:11.675359 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pqkgj\" (UniqueName: \"kubernetes.io/projected/d2c71659-b44a-41bb-b082-a9aa3368e44c-kube-api-access-pqkgj\") pod \"custom-metrics-autoscaler-operator-6969cb7dfb-fcss7\" (UID: \"d2c71659-b44a-41bb-b082-a9aa3368e44c\") " pod="openshift-keda/custom-metrics-autoscaler-operator-6969cb7dfb-fcss7" Jul 29 22:52:11.677664 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:52:11.677640 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"certificates\" (UniqueName: \"kubernetes.io/secret/d2c71659-b44a-41bb-b082-a9aa3368e44c-certificates\") pod \"custom-metrics-autoscaler-operator-6969cb7dfb-fcss7\" (UID: \"d2c71659-b44a-41bb-b082-a9aa3368e44c\") " pod="openshift-keda/custom-metrics-autoscaler-operator-6969cb7dfb-fcss7" Jul 29 22:52:11.683879 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:52:11.683851 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pqkgj\" (UniqueName: \"kubernetes.io/projected/d2c71659-b44a-41bb-b082-a9aa3368e44c-kube-api-access-pqkgj\") pod \"custom-metrics-autoscaler-operator-6969cb7dfb-fcss7\" (UID: \"d2c71659-b44a-41bb-b082-a9aa3368e44c\") " pod="openshift-keda/custom-metrics-autoscaler-operator-6969cb7dfb-fcss7" Jul 29 22:52:11.806338 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:52:11.806299 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/custom-metrics-autoscaler-operator-6969cb7dfb-fcss7" Jul 29 22:52:11.939219 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:52:11.939190 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/custom-metrics-autoscaler-operator-6969cb7dfb-fcss7"] Jul 29 22:52:11.941855 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:52:11.941828 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd2c71659_b44a_41bb_b082_a9aa3368e44c.slice/crio-3fbd2f2d7079609204c0ba5595b3fbe9c142f95eea6be4766908699213a55e97 WatchSource:0}: Error finding container 3fbd2f2d7079609204c0ba5595b3fbe9c142f95eea6be4766908699213a55e97: Status 404 returned error can't find the container with id 3fbd2f2d7079609204c0ba5595b3fbe9c142f95eea6be4766908699213a55e97 Jul 29 22:52:12.882712 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:52:12.882677 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/custom-metrics-autoscaler-operator-6969cb7dfb-fcss7" event={"ID":"d2c71659-b44a-41bb-b082-a9aa3368e44c","Type":"ContainerStarted","Data":"3fbd2f2d7079609204c0ba5595b3fbe9c142f95eea6be4766908699213a55e97"} Jul 29 22:52:14.892387 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:52:14.892290 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/custom-metrics-autoscaler-operator-6969cb7dfb-fcss7" event={"ID":"d2c71659-b44a-41bb-b082-a9aa3368e44c","Type":"ContainerStarted","Data":"047ae4fff5d6f6f1092aeb6a8415ebcb42922d5de300eadacda28e854f5c3bce"} Jul 29 22:52:14.892794 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:52:14.892441 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-keda/custom-metrics-autoscaler-operator-6969cb7dfb-fcss7" Jul 29 22:52:14.918195 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:52:14.918135 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-keda/custom-metrics-autoscaler-operator-6969cb7dfb-fcss7" podStartSLOduration=1.326792513 podStartE2EDuration="3.91811732s" totalImagesPullingTime="2.591324807s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 22:52:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 22:52:14.915278669 +0000 UTC m=+349.827104358" watchObservedRunningTime="2026-07-29 22:52:14.91811732 +0000 UTC m=+349.829943007" Jul 29 22:52:15.287248 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:52:15.287209 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-keda/keda-operator-77f648d694-gtrvm"] Jul 29 22:52:15.289435 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:52:15.289414 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-operator-77f648d694-gtrvm" Jul 29 22:52:15.292332 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:52:15.292313 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"keda-operator-dockercfg-wsnwn\"" Jul 29 22:52:15.292332 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:52:15.292324 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"keda-operator-certs\"" Jul 29 22:52:15.292518 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:52:15.292317 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-keda\"/\"keda-ocp-cabundle\"" Jul 29 22:52:15.300707 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:52:15.300678 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-operator-77f648d694-gtrvm"] Jul 29 22:52:15.407137 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:52:15.407102 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle0\" (UniqueName: \"kubernetes.io/configmap/7fda632c-282e-4baf-b731-92d4d2fe5ad6-cabundle0\") pod \"keda-operator-77f648d694-gtrvm\" (UID: \"7fda632c-282e-4baf-b731-92d4d2fe5ad6\") " pod="openshift-keda/keda-operator-77f648d694-gtrvm" Jul 29 22:52:15.407340 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:52:15.407164 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/7fda632c-282e-4baf-b731-92d4d2fe5ad6-certificates\") pod \"keda-operator-77f648d694-gtrvm\" (UID: \"7fda632c-282e-4baf-b731-92d4d2fe5ad6\") " pod="openshift-keda/keda-operator-77f648d694-gtrvm" Jul 29 22:52:15.407340 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:52:15.407192 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2n8l5\" (UniqueName: \"kubernetes.io/projected/7fda632c-282e-4baf-b731-92d4d2fe5ad6-kube-api-access-2n8l5\") pod \"keda-operator-77f648d694-gtrvm\" (UID: \"7fda632c-282e-4baf-b731-92d4d2fe5ad6\") " pod="openshift-keda/keda-operator-77f648d694-gtrvm" Jul 29 22:52:15.508475 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:52:15.508438 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle0\" (UniqueName: \"kubernetes.io/configmap/7fda632c-282e-4baf-b731-92d4d2fe5ad6-cabundle0\") pod \"keda-operator-77f648d694-gtrvm\" (UID: \"7fda632c-282e-4baf-b731-92d4d2fe5ad6\") " pod="openshift-keda/keda-operator-77f648d694-gtrvm" Jul 29 22:52:15.508657 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:52:15.508544 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/7fda632c-282e-4baf-b731-92d4d2fe5ad6-certificates\") pod \"keda-operator-77f648d694-gtrvm\" (UID: \"7fda632c-282e-4baf-b731-92d4d2fe5ad6\") " pod="openshift-keda/keda-operator-77f648d694-gtrvm" Jul 29 22:52:15.508657 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:52:15.508585 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2n8l5\" (UniqueName: \"kubernetes.io/projected/7fda632c-282e-4baf-b731-92d4d2fe5ad6-kube-api-access-2n8l5\") pod \"keda-operator-77f648d694-gtrvm\" (UID: \"7fda632c-282e-4baf-b731-92d4d2fe5ad6\") " pod="openshift-keda/keda-operator-77f648d694-gtrvm" Jul 29 22:52:15.508742 ip-10-0-138-112 kubenswrapper[2562]: E0729 22:52:15.508693 2562 projected.go:266] Couldn't get secret openshift-keda/keda-operator-certs: secret "keda-operator-certs" not found Jul 29 22:52:15.508742 ip-10-0-138-112 kubenswrapper[2562]: E0729 22:52:15.508714 2562 secret.go:281] references non-existent secret key: ca.crt Jul 29 22:52:15.508742 ip-10-0-138-112 kubenswrapper[2562]: E0729 22:52:15.508722 2562 projected.go:279] Couldn't get secret payload openshift-keda/kedaorg-certs: references non-existent secret key: ca.crt Jul 29 22:52:15.508742 ip-10-0-138-112 kubenswrapper[2562]: E0729 22:52:15.508734 2562 projected.go:196] Error preparing data for projected volume certificates for pod openshift-keda/keda-operator-77f648d694-gtrvm: [secret "keda-operator-certs" not found, references non-existent secret key: ca.crt] Jul 29 22:52:15.508893 ip-10-0-138-112 kubenswrapper[2562]: E0729 22:52:15.508785 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7fda632c-282e-4baf-b731-92d4d2fe5ad6-certificates podName:7fda632c-282e-4baf-b731-92d4d2fe5ad6 nodeName:}" failed. No retries permitted until 2026-07-29 22:52:16.008768597 +0000 UTC m=+350.920594262 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "certificates" (UniqueName: "kubernetes.io/projected/7fda632c-282e-4baf-b731-92d4d2fe5ad6-certificates") pod "keda-operator-77f648d694-gtrvm" (UID: "7fda632c-282e-4baf-b731-92d4d2fe5ad6") : [secret "keda-operator-certs" not found, references non-existent secret key: ca.crt] Jul 29 22:52:15.508893 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:52:15.508808 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-keda/keda-metrics-apiserver-55557df8c-vkgvs"] Jul 29 22:52:15.509091 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:52:15.509073 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cabundle0\" (UniqueName: \"kubernetes.io/configmap/7fda632c-282e-4baf-b731-92d4d2fe5ad6-cabundle0\") pod \"keda-operator-77f648d694-gtrvm\" (UID: \"7fda632c-282e-4baf-b731-92d4d2fe5ad6\") " pod="openshift-keda/keda-operator-77f648d694-gtrvm" Jul 29 22:52:15.510945 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:52:15.510931 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-metrics-apiserver-55557df8c-vkgvs" Jul 29 22:52:15.513606 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:52:15.513589 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"keda-metrics-apiserver-certs\"" Jul 29 22:52:15.520708 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:52:15.520678 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-metrics-apiserver-55557df8c-vkgvs"] Jul 29 22:52:15.522672 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:52:15.522643 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2n8l5\" (UniqueName: \"kubernetes.io/projected/7fda632c-282e-4baf-b731-92d4d2fe5ad6-kube-api-access-2n8l5\") pod \"keda-operator-77f648d694-gtrvm\" (UID: \"7fda632c-282e-4baf-b731-92d4d2fe5ad6\") " pod="openshift-keda/keda-operator-77f648d694-gtrvm" Jul 29 22:52:15.609458 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:52:15.609355 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/c19f96e7-e682-490e-b9d1-934b6c1c4e96-certificates\") pod \"keda-metrics-apiserver-55557df8c-vkgvs\" (UID: \"c19f96e7-e682-490e-b9d1-934b6c1c4e96\") " pod="openshift-keda/keda-metrics-apiserver-55557df8c-vkgvs" Jul 29 22:52:15.609604 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:52:15.609461 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hp4lq\" (UniqueName: \"kubernetes.io/projected/c19f96e7-e682-490e-b9d1-934b6c1c4e96-kube-api-access-hp4lq\") pod \"keda-metrics-apiserver-55557df8c-vkgvs\" (UID: \"c19f96e7-e682-490e-b9d1-934b6c1c4e96\") " pod="openshift-keda/keda-metrics-apiserver-55557df8c-vkgvs" Jul 29 22:52:15.609604 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:52:15.609495 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"temp-vol\" (UniqueName: \"kubernetes.io/empty-dir/c19f96e7-e682-490e-b9d1-934b6c1c4e96-temp-vol\") pod \"keda-metrics-apiserver-55557df8c-vkgvs\" (UID: \"c19f96e7-e682-490e-b9d1-934b6c1c4e96\") " pod="openshift-keda/keda-metrics-apiserver-55557df8c-vkgvs" Jul 29 22:52:15.710926 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:52:15.710885 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hp4lq\" (UniqueName: \"kubernetes.io/projected/c19f96e7-e682-490e-b9d1-934b6c1c4e96-kube-api-access-hp4lq\") pod \"keda-metrics-apiserver-55557df8c-vkgvs\" (UID: \"c19f96e7-e682-490e-b9d1-934b6c1c4e96\") " pod="openshift-keda/keda-metrics-apiserver-55557df8c-vkgvs" Jul 29 22:52:15.711101 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:52:15.710940 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"temp-vol\" (UniqueName: \"kubernetes.io/empty-dir/c19f96e7-e682-490e-b9d1-934b6c1c4e96-temp-vol\") pod \"keda-metrics-apiserver-55557df8c-vkgvs\" (UID: \"c19f96e7-e682-490e-b9d1-934b6c1c4e96\") " pod="openshift-keda/keda-metrics-apiserver-55557df8c-vkgvs" Jul 29 22:52:15.711101 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:52:15.710968 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/c19f96e7-e682-490e-b9d1-934b6c1c4e96-certificates\") pod \"keda-metrics-apiserver-55557df8c-vkgvs\" (UID: \"c19f96e7-e682-490e-b9d1-934b6c1c4e96\") " pod="openshift-keda/keda-metrics-apiserver-55557df8c-vkgvs" Jul 29 22:52:15.711171 ip-10-0-138-112 kubenswrapper[2562]: E0729 22:52:15.711111 2562 secret.go:281] references non-existent secret key: tls.crt Jul 29 22:52:15.711171 ip-10-0-138-112 kubenswrapper[2562]: E0729 22:52:15.711128 2562 projected.go:279] Couldn't get secret payload openshift-keda/kedaorg-certs: references non-existent secret key: tls.crt Jul 29 22:52:15.711171 ip-10-0-138-112 kubenswrapper[2562]: E0729 22:52:15.711146 2562 projected.go:266] Couldn't get secret openshift-keda/keda-metrics-apiserver-certs: secret "keda-metrics-apiserver-certs" not found Jul 29 22:52:15.711171 ip-10-0-138-112 kubenswrapper[2562]: E0729 22:52:15.711166 2562 projected.go:196] Error preparing data for projected volume certificates for pod openshift-keda/keda-metrics-apiserver-55557df8c-vkgvs: [references non-existent secret key: tls.crt, secret "keda-metrics-apiserver-certs" not found] Jul 29 22:52:15.711312 ip-10-0-138-112 kubenswrapper[2562]: E0729 22:52:15.711233 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c19f96e7-e682-490e-b9d1-934b6c1c4e96-certificates podName:c19f96e7-e682-490e-b9d1-934b6c1c4e96 nodeName:}" failed. No retries permitted until 2026-07-29 22:52:16.211213737 +0000 UTC m=+351.123039403 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "certificates" (UniqueName: "kubernetes.io/projected/c19f96e7-e682-490e-b9d1-934b6c1c4e96-certificates") pod "keda-metrics-apiserver-55557df8c-vkgvs" (UID: "c19f96e7-e682-490e-b9d1-934b6c1c4e96") : [references non-existent secret key: tls.crt, secret "keda-metrics-apiserver-certs" not found] Jul 29 22:52:15.711460 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:52:15.711438 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"temp-vol\" (UniqueName: \"kubernetes.io/empty-dir/c19f96e7-e682-490e-b9d1-934b6c1c4e96-temp-vol\") pod \"keda-metrics-apiserver-55557df8c-vkgvs\" (UID: \"c19f96e7-e682-490e-b9d1-934b6c1c4e96\") " pod="openshift-keda/keda-metrics-apiserver-55557df8c-vkgvs" Jul 29 22:52:15.722288 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:52:15.722246 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hp4lq\" (UniqueName: \"kubernetes.io/projected/c19f96e7-e682-490e-b9d1-934b6c1c4e96-kube-api-access-hp4lq\") pod \"keda-metrics-apiserver-55557df8c-vkgvs\" (UID: \"c19f96e7-e682-490e-b9d1-934b6c1c4e96\") " pod="openshift-keda/keda-metrics-apiserver-55557df8c-vkgvs" Jul 29 22:52:15.993571 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:52:15.993483 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-keda/keda-admission-56f9798bb4-w6t4r"] Jul 29 22:52:15.995813 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:52:15.995787 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-admission-56f9798bb4-w6t4r" Jul 29 22:52:15.999345 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:52:15.999321 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"keda-admission-webhooks-certs\"" Jul 29 22:52:16.008798 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:52:16.008771 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-admission-56f9798bb4-w6t4r"] Jul 29 22:52:16.013639 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:52:16.013603 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/7fda632c-282e-4baf-b731-92d4d2fe5ad6-certificates\") pod \"keda-operator-77f648d694-gtrvm\" (UID: \"7fda632c-282e-4baf-b731-92d4d2fe5ad6\") " pod="openshift-keda/keda-operator-77f648d694-gtrvm" Jul 29 22:52:16.013800 ip-10-0-138-112 kubenswrapper[2562]: E0729 22:52:16.013713 2562 projected.go:266] Couldn't get secret openshift-keda/keda-operator-certs: secret "keda-operator-certs" not found Jul 29 22:52:16.013800 ip-10-0-138-112 kubenswrapper[2562]: E0729 22:52:16.013735 2562 projected.go:196] Error preparing data for projected volume certificates for pod openshift-keda/keda-operator-77f648d694-gtrvm: secret "keda-operator-certs" not found Jul 29 22:52:16.013800 ip-10-0-138-112 kubenswrapper[2562]: E0729 22:52:16.013799 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7fda632c-282e-4baf-b731-92d4d2fe5ad6-certificates podName:7fda632c-282e-4baf-b731-92d4d2fe5ad6 nodeName:}" failed. No retries permitted until 2026-07-29 22:52:17.013779741 +0000 UTC m=+351.925605427 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "certificates" (UniqueName: "kubernetes.io/projected/7fda632c-282e-4baf-b731-92d4d2fe5ad6-certificates") pod "keda-operator-77f648d694-gtrvm" (UID: "7fda632c-282e-4baf-b731-92d4d2fe5ad6") : secret "keda-operator-certs" not found Jul 29 22:52:16.114890 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:52:16.114843 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/ca951017-2bfc-4535-be1a-9f81738de091-certificates\") pod \"keda-admission-56f9798bb4-w6t4r\" (UID: \"ca951017-2bfc-4535-be1a-9f81738de091\") " pod="openshift-keda/keda-admission-56f9798bb4-w6t4r" Jul 29 22:52:16.115107 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:52:16.114930 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t62g2\" (UniqueName: \"kubernetes.io/projected/ca951017-2bfc-4535-be1a-9f81738de091-kube-api-access-t62g2\") pod \"keda-admission-56f9798bb4-w6t4r\" (UID: \"ca951017-2bfc-4535-be1a-9f81738de091\") " pod="openshift-keda/keda-admission-56f9798bb4-w6t4r" Jul 29 22:52:16.215345 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:52:16.215304 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/c19f96e7-e682-490e-b9d1-934b6c1c4e96-certificates\") pod \"keda-metrics-apiserver-55557df8c-vkgvs\" (UID: \"c19f96e7-e682-490e-b9d1-934b6c1c4e96\") " pod="openshift-keda/keda-metrics-apiserver-55557df8c-vkgvs" Jul 29 22:52:16.215553 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:52:16.215501 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/ca951017-2bfc-4535-be1a-9f81738de091-certificates\") pod \"keda-admission-56f9798bb4-w6t4r\" (UID: \"ca951017-2bfc-4535-be1a-9f81738de091\") " pod="openshift-keda/keda-admission-56f9798bb4-w6t4r" Jul 29 22:52:16.215553 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:52:16.215541 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t62g2\" (UniqueName: \"kubernetes.io/projected/ca951017-2bfc-4535-be1a-9f81738de091-kube-api-access-t62g2\") pod \"keda-admission-56f9798bb4-w6t4r\" (UID: \"ca951017-2bfc-4535-be1a-9f81738de091\") " pod="openshift-keda/keda-admission-56f9798bb4-w6t4r" Jul 29 22:52:16.215652 ip-10-0-138-112 kubenswrapper[2562]: E0729 22:52:16.215631 2562 projected.go:266] Couldn't get secret openshift-keda/keda-admission-webhooks-certs: secret "keda-admission-webhooks-certs" not found Jul 29 22:52:16.215707 ip-10-0-138-112 kubenswrapper[2562]: E0729 22:52:16.215658 2562 projected.go:196] Error preparing data for projected volume certificates for pod openshift-keda/keda-admission-56f9798bb4-w6t4r: secret "keda-admission-webhooks-certs" not found Jul 29 22:52:16.215750 ip-10-0-138-112 kubenswrapper[2562]: E0729 22:52:16.215727 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ca951017-2bfc-4535-be1a-9f81738de091-certificates podName:ca951017-2bfc-4535-be1a-9f81738de091 nodeName:}" failed. No retries permitted until 2026-07-29 22:52:16.715708977 +0000 UTC m=+351.627534653 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "certificates" (UniqueName: "kubernetes.io/projected/ca951017-2bfc-4535-be1a-9f81738de091-certificates") pod "keda-admission-56f9798bb4-w6t4r" (UID: "ca951017-2bfc-4535-be1a-9f81738de091") : secret "keda-admission-webhooks-certs" not found Jul 29 22:52:16.218031 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:52:16.218005 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/c19f96e7-e682-490e-b9d1-934b6c1c4e96-certificates\") pod \"keda-metrics-apiserver-55557df8c-vkgvs\" (UID: \"c19f96e7-e682-490e-b9d1-934b6c1c4e96\") " pod="openshift-keda/keda-metrics-apiserver-55557df8c-vkgvs" Jul 29 22:52:16.225206 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:52:16.225176 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t62g2\" (UniqueName: \"kubernetes.io/projected/ca951017-2bfc-4535-be1a-9f81738de091-kube-api-access-t62g2\") pod \"keda-admission-56f9798bb4-w6t4r\" (UID: \"ca951017-2bfc-4535-be1a-9f81738de091\") " pod="openshift-keda/keda-admission-56f9798bb4-w6t4r" Jul 29 22:52:16.424330 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:52:16.424285 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-metrics-apiserver-55557df8c-vkgvs" Jul 29 22:52:16.578143 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:52:16.578110 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-metrics-apiserver-55557df8c-vkgvs"] Jul 29 22:52:16.581465 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:52:16.581434 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc19f96e7_e682_490e_b9d1_934b6c1c4e96.slice/crio-b99425ed048a85d89bc4937a9921a93c4ae2c389983363dcf9ba1e8139f11f1d WatchSource:0}: Error finding container b99425ed048a85d89bc4937a9921a93c4ae2c389983363dcf9ba1e8139f11f1d: Status 404 returned error can't find the container with id b99425ed048a85d89bc4937a9921a93c4ae2c389983363dcf9ba1e8139f11f1d Jul 29 22:52:16.722040 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:52:16.721930 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/ca951017-2bfc-4535-be1a-9f81738de091-certificates\") pod \"keda-admission-56f9798bb4-w6t4r\" (UID: \"ca951017-2bfc-4535-be1a-9f81738de091\") " pod="openshift-keda/keda-admission-56f9798bb4-w6t4r" Jul 29 22:52:16.724429 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:52:16.724376 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/ca951017-2bfc-4535-be1a-9f81738de091-certificates\") pod \"keda-admission-56f9798bb4-w6t4r\" (UID: \"ca951017-2bfc-4535-be1a-9f81738de091\") " pod="openshift-keda/keda-admission-56f9798bb4-w6t4r" Jul 29 22:52:16.900675 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:52:16.900636 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-metrics-apiserver-55557df8c-vkgvs" event={"ID":"c19f96e7-e682-490e-b9d1-934b6c1c4e96","Type":"ContainerStarted","Data":"b99425ed048a85d89bc4937a9921a93c4ae2c389983363dcf9ba1e8139f11f1d"} Jul 29 22:52:16.905891 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:52:16.905865 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-admission-56f9798bb4-w6t4r" Jul 29 22:52:17.023989 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:52:17.023953 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/7fda632c-282e-4baf-b731-92d4d2fe5ad6-certificates\") pod \"keda-operator-77f648d694-gtrvm\" (UID: \"7fda632c-282e-4baf-b731-92d4d2fe5ad6\") " pod="openshift-keda/keda-operator-77f648d694-gtrvm" Jul 29 22:52:17.026504 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:52:17.026479 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/7fda632c-282e-4baf-b731-92d4d2fe5ad6-certificates\") pod \"keda-operator-77f648d694-gtrvm\" (UID: \"7fda632c-282e-4baf-b731-92d4d2fe5ad6\") " pod="openshift-keda/keda-operator-77f648d694-gtrvm" Jul 29 22:52:17.040876 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:52:17.040842 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-admission-56f9798bb4-w6t4r"] Jul 29 22:52:17.043933 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:52:17.043905 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podca951017_2bfc_4535_be1a_9f81738de091.slice/crio-e8cf30eada8cd7e90f4aca5911ea2c18040d3ed15b047ac1730d2adbf8828364 WatchSource:0}: Error finding container e8cf30eada8cd7e90f4aca5911ea2c18040d3ed15b047ac1730d2adbf8828364: Status 404 returned error can't find the container with id e8cf30eada8cd7e90f4aca5911ea2c18040d3ed15b047ac1730d2adbf8828364 Jul 29 22:52:17.100024 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:52:17.099981 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-operator-77f648d694-gtrvm" Jul 29 22:52:17.233462 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:52:17.233429 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-operator-77f648d694-gtrvm"] Jul 29 22:52:17.237258 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:52:17.237226 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7fda632c_282e_4baf_b731_92d4d2fe5ad6.slice/crio-83be23144e6df51a8d08c25dd425adb4857036cfcffea9228be7977043c78382 WatchSource:0}: Error finding container 83be23144e6df51a8d08c25dd425adb4857036cfcffea9228be7977043c78382: Status 404 returned error can't find the container with id 83be23144e6df51a8d08c25dd425adb4857036cfcffea9228be7977043c78382 Jul 29 22:52:17.911803 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:52:17.911698 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-admission-56f9798bb4-w6t4r" event={"ID":"ca951017-2bfc-4535-be1a-9f81738de091","Type":"ContainerStarted","Data":"e8cf30eada8cd7e90f4aca5911ea2c18040d3ed15b047ac1730d2adbf8828364"} Jul 29 22:52:17.913043 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:52:17.913004 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-operator-77f648d694-gtrvm" event={"ID":"7fda632c-282e-4baf-b731-92d4d2fe5ad6","Type":"ContainerStarted","Data":"83be23144e6df51a8d08c25dd425adb4857036cfcffea9228be7977043c78382"} Jul 29 22:52:20.929411 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:52:20.929350 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-admission-56f9798bb4-w6t4r" event={"ID":"ca951017-2bfc-4535-be1a-9f81738de091","Type":"ContainerStarted","Data":"746d0e76fc9930e9ef6eb1babfcd870265e4ae430bb3d9e8a58fad36051ea3ea"} Jul 29 22:52:20.929909 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:52:20.929595 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-keda/keda-admission-56f9798bb4-w6t4r" Jul 29 22:52:20.930953 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:52:20.930929 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-metrics-apiserver-55557df8c-vkgvs" event={"ID":"c19f96e7-e682-490e-b9d1-934b6c1c4e96","Type":"ContainerStarted","Data":"032f1a3244fcb30645bc90ed52bd27090a4eec2563a392dd654f799e388940f3"} Jul 29 22:52:20.931071 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:52:20.930988 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-keda/keda-metrics-apiserver-55557df8c-vkgvs" Jul 29 22:52:20.947006 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:52:20.946950 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-keda/keda-admission-56f9798bb4-w6t4r" podStartSLOduration=3.074709928 podStartE2EDuration="5.946934752s" totalImagesPullingTime="2.872224824s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 22:52:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 22:52:20.945033172 +0000 UTC m=+355.856858895" watchObservedRunningTime="2026-07-29 22:52:20.946934752 +0000 UTC m=+355.858760417" Jul 29 22:52:20.962555 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:52:20.962491 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-keda/keda-metrics-apiserver-55557df8c-vkgvs" podStartSLOduration=2.620499897 podStartE2EDuration="5.962472738s" totalImagesPullingTime="3.341972841s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 22:52:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 22:52:20.960881918 +0000 UTC m=+355.872707611" watchObservedRunningTime="2026-07-29 22:52:20.962472738 +0000 UTC m=+355.874298424" Jul 29 22:52:21.936301 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:52:21.936257 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-operator-77f648d694-gtrvm" event={"ID":"7fda632c-282e-4baf-b731-92d4d2fe5ad6","Type":"ContainerStarted","Data":"84324fd1dc31265fccb8deb247c05bd507ff4a6d89cce1a1807dcec918ebc5c9"} Jul 29 22:52:21.955860 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:52:21.955809 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-keda/keda-operator-77f648d694-gtrvm" podStartSLOduration=2.723593292 podStartE2EDuration="6.955791026s" totalImagesPullingTime="4.232197734s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 22:52:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 22:52:21.953978613 +0000 UTC m=+356.865804313" watchObservedRunningTime="2026-07-29 22:52:21.955791026 +0000 UTC m=+356.867616712" Jul 29 22:52:22.940743 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:52:22.940705 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-keda/keda-operator-77f648d694-gtrvm" Jul 29 22:52:31.940625 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:52:31.940589 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-keda/keda-metrics-apiserver-55557df8c-vkgvs" Jul 29 22:52:35.898728 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:52:35.898694 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-keda/custom-metrics-autoscaler-operator-6969cb7dfb-fcss7" Jul 29 22:52:41.938493 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:52:41.938461 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-keda/keda-admission-56f9798bb4-w6t4r" Jul 29 22:52:43.947376 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:52:43.947346 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-keda/keda-operator-77f648d694-gtrvm" Jul 29 22:53:35.078049 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:53:35.078013 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve/kserve-controller-manager-844cf74686-242x8"] Jul 29 22:53:35.080558 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:53:35.080538 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/kserve-controller-manager-844cf74686-242x8" Jul 29 22:53:35.083782 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:53:35.083762 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"kserve-webhook-server-cert\"" Jul 29 22:53:35.083782 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:53:35.083779 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"kserve-controller-manager-dockercfg-9ddct\"" Jul 29 22:53:35.083939 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:53:35.083779 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve\"/\"openshift-service-ca.crt\"" Jul 29 22:53:35.084835 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:53:35.084817 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve\"/\"kube-root-ca.crt\"" Jul 29 22:53:35.091186 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:53:35.091168 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve/llmisvc-controller-manager-654c557856-k2ld6"] Jul 29 22:53:35.093234 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:53:35.093216 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/kserve-controller-manager-844cf74686-242x8"] Jul 29 22:53:35.093349 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:53:35.093314 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/llmisvc-controller-manager-654c557856-k2ld6" Jul 29 22:53:35.096053 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:53:35.096035 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"llmisvc-webhook-server-cert\"" Jul 29 22:53:35.096155 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:53:35.096067 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"llmisvc-controller-manager-dockercfg-6rxx4\"" Jul 29 22:53:35.104790 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:53:35.104751 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/llmisvc-controller-manager-654c557856-k2ld6"] Jul 29 22:53:35.119529 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:53:35.119501 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve/seaweedfs-64568bcd49-6wt5l"] Jul 29 22:53:35.121808 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:53:35.121788 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/seaweedfs-64568bcd49-6wt5l" Jul 29 22:53:35.124640 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:53:35.124617 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"mlpipeline-s3-artifact\"" Jul 29 22:53:35.124755 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:53:35.124617 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"default-dockercfg-9nznz\"" Jul 29 22:53:35.132768 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:53:35.132742 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/seaweedfs-64568bcd49-6wt5l"] Jul 29 22:53:35.189701 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:53:35.189662 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/43ca558c-d85d-426c-9560-c71ccdefcae1-cert\") pod \"llmisvc-controller-manager-654c557856-k2ld6\" (UID: \"43ca558c-d85d-426c-9560-c71ccdefcae1\") " pod="kserve/llmisvc-controller-manager-654c557856-k2ld6" Jul 29 22:53:35.189872 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:53:35.189706 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zbtmr\" (UniqueName: \"kubernetes.io/projected/23059c0f-1b26-41dc-9031-e96ddce46577-kube-api-access-zbtmr\") pod \"kserve-controller-manager-844cf74686-242x8\" (UID: \"23059c0f-1b26-41dc-9031-e96ddce46577\") " pod="kserve/kserve-controller-manager-844cf74686-242x8" Jul 29 22:53:35.189872 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:53:35.189732 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/23059c0f-1b26-41dc-9031-e96ddce46577-cert\") pod \"kserve-controller-manager-844cf74686-242x8\" (UID: \"23059c0f-1b26-41dc-9031-e96ddce46577\") " pod="kserve/kserve-controller-manager-844cf74686-242x8" Jul 29 22:53:35.189872 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:53:35.189750 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nhgnb\" (UniqueName: \"kubernetes.io/projected/43ca558c-d85d-426c-9560-c71ccdefcae1-kube-api-access-nhgnb\") pod \"llmisvc-controller-manager-654c557856-k2ld6\" (UID: \"43ca558c-d85d-426c-9560-c71ccdefcae1\") " pod="kserve/llmisvc-controller-manager-654c557856-k2ld6" Jul 29 22:53:35.290964 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:53:35.290922 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/a02e53c9-9d46-4f90-a406-0fbb66307fa8-data\") pod \"seaweedfs-64568bcd49-6wt5l\" (UID: \"a02e53c9-9d46-4f90-a406-0fbb66307fa8\") " pod="kserve/seaweedfs-64568bcd49-6wt5l" Jul 29 22:53:35.291145 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:53:35.290973 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rqbpv\" (UniqueName: \"kubernetes.io/projected/a02e53c9-9d46-4f90-a406-0fbb66307fa8-kube-api-access-rqbpv\") pod \"seaweedfs-64568bcd49-6wt5l\" (UID: \"a02e53c9-9d46-4f90-a406-0fbb66307fa8\") " pod="kserve/seaweedfs-64568bcd49-6wt5l" Jul 29 22:53:35.291145 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:53:35.291068 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/43ca558c-d85d-426c-9560-c71ccdefcae1-cert\") pod \"llmisvc-controller-manager-654c557856-k2ld6\" (UID: \"43ca558c-d85d-426c-9560-c71ccdefcae1\") " pod="kserve/llmisvc-controller-manager-654c557856-k2ld6" Jul 29 22:53:35.291145 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:53:35.291112 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zbtmr\" (UniqueName: \"kubernetes.io/projected/23059c0f-1b26-41dc-9031-e96ddce46577-kube-api-access-zbtmr\") pod \"kserve-controller-manager-844cf74686-242x8\" (UID: \"23059c0f-1b26-41dc-9031-e96ddce46577\") " pod="kserve/kserve-controller-manager-844cf74686-242x8" Jul 29 22:53:35.291145 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:53:35.291137 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/23059c0f-1b26-41dc-9031-e96ddce46577-cert\") pod \"kserve-controller-manager-844cf74686-242x8\" (UID: \"23059c0f-1b26-41dc-9031-e96ddce46577\") " pod="kserve/kserve-controller-manager-844cf74686-242x8" Jul 29 22:53:35.291342 ip-10-0-138-112 kubenswrapper[2562]: E0729 22:53:35.291219 2562 secret.go:189] Couldn't get secret kserve/kserve-webhook-server-cert: secret "kserve-webhook-server-cert" not found Jul 29 22:53:35.291342 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:53:35.291249 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nhgnb\" (UniqueName: \"kubernetes.io/projected/43ca558c-d85d-426c-9560-c71ccdefcae1-kube-api-access-nhgnb\") pod \"llmisvc-controller-manager-654c557856-k2ld6\" (UID: \"43ca558c-d85d-426c-9560-c71ccdefcae1\") " pod="kserve/llmisvc-controller-manager-654c557856-k2ld6" Jul 29 22:53:35.291342 ip-10-0-138-112 kubenswrapper[2562]: E0729 22:53:35.291273 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/23059c0f-1b26-41dc-9031-e96ddce46577-cert podName:23059c0f-1b26-41dc-9031-e96ddce46577 nodeName:}" failed. No retries permitted until 2026-07-29 22:53:35.791256022 +0000 UTC m=+430.703081687 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/23059c0f-1b26-41dc-9031-e96ddce46577-cert") pod "kserve-controller-manager-844cf74686-242x8" (UID: "23059c0f-1b26-41dc-9031-e96ddce46577") : secret "kserve-webhook-server-cert" not found Jul 29 22:53:35.293646 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:53:35.293624 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/43ca558c-d85d-426c-9560-c71ccdefcae1-cert\") pod \"llmisvc-controller-manager-654c557856-k2ld6\" (UID: \"43ca558c-d85d-426c-9560-c71ccdefcae1\") " pod="kserve/llmisvc-controller-manager-654c557856-k2ld6" Jul 29 22:53:35.300131 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:53:35.300107 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zbtmr\" (UniqueName: \"kubernetes.io/projected/23059c0f-1b26-41dc-9031-e96ddce46577-kube-api-access-zbtmr\") pod \"kserve-controller-manager-844cf74686-242x8\" (UID: \"23059c0f-1b26-41dc-9031-e96ddce46577\") " pod="kserve/kserve-controller-manager-844cf74686-242x8" Jul 29 22:53:35.301440 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:53:35.301130 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nhgnb\" (UniqueName: \"kubernetes.io/projected/43ca558c-d85d-426c-9560-c71ccdefcae1-kube-api-access-nhgnb\") pod \"llmisvc-controller-manager-654c557856-k2ld6\" (UID: \"43ca558c-d85d-426c-9560-c71ccdefcae1\") " pod="kserve/llmisvc-controller-manager-654c557856-k2ld6" Jul 29 22:53:35.391759 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:53:35.391647 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/a02e53c9-9d46-4f90-a406-0fbb66307fa8-data\") pod \"seaweedfs-64568bcd49-6wt5l\" (UID: \"a02e53c9-9d46-4f90-a406-0fbb66307fa8\") " pod="kserve/seaweedfs-64568bcd49-6wt5l" Jul 29 22:53:35.391759 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:53:35.391708 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rqbpv\" (UniqueName: \"kubernetes.io/projected/a02e53c9-9d46-4f90-a406-0fbb66307fa8-kube-api-access-rqbpv\") pod \"seaweedfs-64568bcd49-6wt5l\" (UID: \"a02e53c9-9d46-4f90-a406-0fbb66307fa8\") " pod="kserve/seaweedfs-64568bcd49-6wt5l" Jul 29 22:53:35.392064 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:53:35.392043 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/a02e53c9-9d46-4f90-a406-0fbb66307fa8-data\") pod \"seaweedfs-64568bcd49-6wt5l\" (UID: \"a02e53c9-9d46-4f90-a406-0fbb66307fa8\") " pod="kserve/seaweedfs-64568bcd49-6wt5l" Jul 29 22:53:35.400541 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:53:35.400508 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rqbpv\" (UniqueName: \"kubernetes.io/projected/a02e53c9-9d46-4f90-a406-0fbb66307fa8-kube-api-access-rqbpv\") pod \"seaweedfs-64568bcd49-6wt5l\" (UID: \"a02e53c9-9d46-4f90-a406-0fbb66307fa8\") " pod="kserve/seaweedfs-64568bcd49-6wt5l" Jul 29 22:53:35.405180 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:53:35.405156 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/llmisvc-controller-manager-654c557856-k2ld6" Jul 29 22:53:35.430545 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:53:35.430510 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/seaweedfs-64568bcd49-6wt5l" Jul 29 22:53:35.545435 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:53:35.545377 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod43ca558c_d85d_426c_9560_c71ccdefcae1.slice/crio-f26f7a79bd23d31af3fb51d84e791b4017a0e166df45e4b3d4e3cb03bcb966d5 WatchSource:0}: Error finding container f26f7a79bd23d31af3fb51d84e791b4017a0e166df45e4b3d4e3cb03bcb966d5: Status 404 returned error can't find the container with id f26f7a79bd23d31af3fb51d84e791b4017a0e166df45e4b3d4e3cb03bcb966d5 Jul 29 22:53:35.545751 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:53:35.545732 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/llmisvc-controller-manager-654c557856-k2ld6"] Jul 29 22:53:35.572772 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:53:35.572735 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda02e53c9_9d46_4f90_a406_0fbb66307fa8.slice/crio-19364c569b425a46af461a15e1f91d73bbd7f34d851d21685a87f635ec07ba77 WatchSource:0}: Error finding container 19364c569b425a46af461a15e1f91d73bbd7f34d851d21685a87f635ec07ba77: Status 404 returned error can't find the container with id 19364c569b425a46af461a15e1f91d73bbd7f34d851d21685a87f635ec07ba77 Jul 29 22:53:35.577216 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:53:35.577197 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/seaweedfs-64568bcd49-6wt5l"] Jul 29 22:53:35.795239 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:53:35.795194 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/23059c0f-1b26-41dc-9031-e96ddce46577-cert\") pod \"kserve-controller-manager-844cf74686-242x8\" (UID: \"23059c0f-1b26-41dc-9031-e96ddce46577\") " pod="kserve/kserve-controller-manager-844cf74686-242x8" Jul 29 22:53:35.797734 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:53:35.797710 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/23059c0f-1b26-41dc-9031-e96ddce46577-cert\") pod \"kserve-controller-manager-844cf74686-242x8\" (UID: \"23059c0f-1b26-41dc-9031-e96ddce46577\") " pod="kserve/kserve-controller-manager-844cf74686-242x8" Jul 29 22:53:35.991100 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:53:35.991027 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/kserve-controller-manager-844cf74686-242x8" Jul 29 22:53:36.244622 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:53:36.244543 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/llmisvc-controller-manager-654c557856-k2ld6" event={"ID":"43ca558c-d85d-426c-9560-c71ccdefcae1","Type":"ContainerStarted","Data":"f26f7a79bd23d31af3fb51d84e791b4017a0e166df45e4b3d4e3cb03bcb966d5"} Jul 29 22:53:36.245979 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:53:36.245947 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/seaweedfs-64568bcd49-6wt5l" event={"ID":"a02e53c9-9d46-4f90-a406-0fbb66307fa8","Type":"ContainerStarted","Data":"19364c569b425a46af461a15e1f91d73bbd7f34d851d21685a87f635ec07ba77"} Jul 29 22:53:36.372318 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:53:36.372273 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/kserve-controller-manager-844cf74686-242x8"] Jul 29 22:53:36.397894 ip-10-0-138-112 kubenswrapper[2562]: W0729 22:53:36.397856 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod23059c0f_1b26_41dc_9031_e96ddce46577.slice/crio-052a258c2868695fb254dd6d1335f92d71e386b7464c502e332ec2ed68f08cff WatchSource:0}: Error finding container 052a258c2868695fb254dd6d1335f92d71e386b7464c502e332ec2ed68f08cff: Status 404 returned error can't find the container with id 052a258c2868695fb254dd6d1335f92d71e386b7464c502e332ec2ed68f08cff Jul 29 22:53:37.253177 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:53:37.253133 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/kserve-controller-manager-844cf74686-242x8" event={"ID":"23059c0f-1b26-41dc-9031-e96ddce46577","Type":"ContainerStarted","Data":"052a258c2868695fb254dd6d1335f92d71e386b7464c502e332ec2ed68f08cff"} Jul 29 22:53:41.277765 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:53:41.277721 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/seaweedfs-64568bcd49-6wt5l" event={"ID":"a02e53c9-9d46-4f90-a406-0fbb66307fa8","Type":"ContainerStarted","Data":"7c141fbb24c312a310abe6eb15c1ef961f5ddafa7a48b14139540cefe4027159"} Jul 29 22:53:41.278211 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:53:41.277889 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve/seaweedfs-64568bcd49-6wt5l" Jul 29 22:53:41.279360 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:53:41.279336 2562 generic.go:356] "Generic (PLEG): container finished" podID="43ca558c-d85d-426c-9560-c71ccdefcae1" containerID="a1c7020380011d52d6b21774a6129c4e6c8710175fa2536e84648fae053bb546" exitCode=1 Jul 29 22:53:41.279471 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:53:41.279426 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/llmisvc-controller-manager-654c557856-k2ld6" event={"ID":"43ca558c-d85d-426c-9560-c71ccdefcae1","Type":"ContainerDied","Data":"a1c7020380011d52d6b21774a6129c4e6c8710175fa2536e84648fae053bb546"} Jul 29 22:53:41.279600 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:53:41.279586 2562 scope.go:117] "RemoveContainer" containerID="a1c7020380011d52d6b21774a6129c4e6c8710175fa2536e84648fae053bb546" Jul 29 22:53:41.280897 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:53:41.280875 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/kserve-controller-manager-844cf74686-242x8" event={"ID":"23059c0f-1b26-41dc-9031-e96ddce46577","Type":"ContainerStarted","Data":"1a301829e836fecb532ab0803096ced51621b88098e32e5f982ae6ac88dd6e91"} Jul 29 22:53:41.280995 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:53:41.280983 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve/kserve-controller-manager-844cf74686-242x8" Jul 29 22:53:41.294622 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:53:41.294576 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve/seaweedfs-64568bcd49-6wt5l" podStartSLOduration=1.5191546200000001 podStartE2EDuration="6.294562925s" totalImagesPullingTime="4.775408305s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 22:53:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 22:53:41.293509399 +0000 UTC m=+436.205335098" watchObservedRunningTime="2026-07-29 22:53:41.294562925 +0000 UTC m=+436.206388612" Jul 29 22:53:41.328901 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:53:41.328842 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve/kserve-controller-manager-844cf74686-242x8" podStartSLOduration=2.418425669 podStartE2EDuration="6.328823828s" totalImagesPullingTime="3.910398159s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 22:53:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 22:53:41.327776786 +0000 UTC m=+436.239602474" watchObservedRunningTime="2026-07-29 22:53:41.328823828 +0000 UTC m=+436.240649514" Jul 29 22:53:42.286921 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:53:42.286883 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/llmisvc-controller-manager-654c557856-k2ld6" event={"ID":"43ca558c-d85d-426c-9560-c71ccdefcae1","Type":"ContainerStarted","Data":"b0a72181cecf9c84516f3188ef6af68d0eea8b6d1aabc62561304c877fc415ad"} Jul 29 22:53:42.287365 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:53:42.287270 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve/llmisvc-controller-manager-654c557856-k2ld6" Jul 29 22:53:42.305434 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:53:42.305364 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve/llmisvc-controller-manager-654c557856-k2ld6" podStartSLOduration=2.232170136 podStartE2EDuration="7.305344659s" totalImagesPullingTime="5.073174523s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 22:53:35 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 22:53:42.304731125 +0000 UTC m=+437.216556815" watchObservedRunningTime="2026-07-29 22:53:42.305344659 +0000 UTC m=+437.217170366" Jul 29 22:53:43.292629 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:53:43.292596 2562 generic.go:356] "Generic (PLEG): container finished" podID="43ca558c-d85d-426c-9560-c71ccdefcae1" containerID="b0a72181cecf9c84516f3188ef6af68d0eea8b6d1aabc62561304c877fc415ad" exitCode=1 Jul 29 22:53:43.293046 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:53:43.292659 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/llmisvc-controller-manager-654c557856-k2ld6" event={"ID":"43ca558c-d85d-426c-9560-c71ccdefcae1","Type":"ContainerDied","Data":"b0a72181cecf9c84516f3188ef6af68d0eea8b6d1aabc62561304c877fc415ad"} Jul 29 22:53:43.293046 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:53:43.292708 2562 scope.go:117] "RemoveContainer" containerID="a1c7020380011d52d6b21774a6129c4e6c8710175fa2536e84648fae053bb546" Jul 29 22:53:43.293046 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:53:43.292878 2562 scope.go:117] "RemoveContainer" containerID="b0a72181cecf9c84516f3188ef6af68d0eea8b6d1aabc62561304c877fc415ad" Jul 29 22:53:43.293202 ip-10-0-138-112 kubenswrapper[2562]: E0729 22:53:43.293111 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 10s restarting failed container=manager pod=llmisvc-controller-manager-654c557856-k2ld6_kserve(43ca558c-d85d-426c-9560-c71ccdefcae1)\"" pod="kserve/llmisvc-controller-manager-654c557856-k2ld6" podUID="43ca558c-d85d-426c-9560-c71ccdefcae1" Jul 29 22:53:44.298699 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:53:44.298671 2562 scope.go:117] "RemoveContainer" containerID="b0a72181cecf9c84516f3188ef6af68d0eea8b6d1aabc62561304c877fc415ad" Jul 29 22:53:44.299089 ip-10-0-138-112 kubenswrapper[2562]: E0729 22:53:44.298843 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 10s restarting failed container=manager pod=llmisvc-controller-manager-654c557856-k2ld6_kserve(43ca558c-d85d-426c-9560-c71ccdefcae1)\"" pod="kserve/llmisvc-controller-manager-654c557856-k2ld6" podUID="43ca558c-d85d-426c-9560-c71ccdefcae1" Jul 29 22:53:45.406194 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:53:45.406154 2562 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="kserve/llmisvc-controller-manager-654c557856-k2ld6" Jul 29 22:53:45.406602 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:53:45.406559 2562 scope.go:117] "RemoveContainer" containerID="b0a72181cecf9c84516f3188ef6af68d0eea8b6d1aabc62561304c877fc415ad" Jul 29 22:53:45.406743 ip-10-0-138-112 kubenswrapper[2562]: E0729 22:53:45.406721 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 10s restarting failed container=manager pod=llmisvc-controller-manager-654c557856-k2ld6_kserve(43ca558c-d85d-426c-9560-c71ccdefcae1)\"" pod="kserve/llmisvc-controller-manager-654c557856-k2ld6" podUID="43ca558c-d85d-426c-9560-c71ccdefcae1" Jul 29 22:53:47.289698 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:53:47.289672 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve/seaweedfs-64568bcd49-6wt5l" Jul 29 22:53:59.570945 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:53:59.570858 2562 scope.go:117] "RemoveContainer" containerID="b0a72181cecf9c84516f3188ef6af68d0eea8b6d1aabc62561304c877fc415ad" Jul 29 22:54:00.364857 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:54:00.364823 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/llmisvc-controller-manager-654c557856-k2ld6" event={"ID":"43ca558c-d85d-426c-9560-c71ccdefcae1","Type":"ContainerStarted","Data":"e659847c06c30a4c04917f4b939ce9439f3a584a56432af277ec7abe1c60ec1f"} Jul 29 22:54:00.365040 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:54:00.365025 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve/llmisvc-controller-manager-654c557856-k2ld6" Jul 29 22:54:01.370247 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:54:01.370209 2562 generic.go:356] "Generic (PLEG): container finished" podID="43ca558c-d85d-426c-9560-c71ccdefcae1" containerID="e659847c06c30a4c04917f4b939ce9439f3a584a56432af277ec7abe1c60ec1f" exitCode=1 Jul 29 22:54:01.370736 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:54:01.370311 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/llmisvc-controller-manager-654c557856-k2ld6" event={"ID":"43ca558c-d85d-426c-9560-c71ccdefcae1","Type":"ContainerDied","Data":"e659847c06c30a4c04917f4b939ce9439f3a584a56432af277ec7abe1c60ec1f"} Jul 29 22:54:01.370736 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:54:01.370371 2562 scope.go:117] "RemoveContainer" containerID="b0a72181cecf9c84516f3188ef6af68d0eea8b6d1aabc62561304c877fc415ad" Jul 29 22:54:01.370736 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:54:01.370640 2562 scope.go:117] "RemoveContainer" containerID="e659847c06c30a4c04917f4b939ce9439f3a584a56432af277ec7abe1c60ec1f" Jul 29 22:54:01.370857 ip-10-0-138-112 kubenswrapper[2562]: E0729 22:54:01.370833 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 20s restarting failed container=manager pod=llmisvc-controller-manager-654c557856-k2ld6_kserve(43ca558c-d85d-426c-9560-c71ccdefcae1)\"" pod="kserve/llmisvc-controller-manager-654c557856-k2ld6" podUID="43ca558c-d85d-426c-9560-c71ccdefcae1" Jul 29 22:54:02.376817 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:54:02.376791 2562 scope.go:117] "RemoveContainer" containerID="e659847c06c30a4c04917f4b939ce9439f3a584a56432af277ec7abe1c60ec1f" Jul 29 22:54:02.377186 ip-10-0-138-112 kubenswrapper[2562]: E0729 22:54:02.376955 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 20s restarting failed container=manager pod=llmisvc-controller-manager-654c557856-k2ld6_kserve(43ca558c-d85d-426c-9560-c71ccdefcae1)\"" pod="kserve/llmisvc-controller-manager-654c557856-k2ld6" podUID="43ca558c-d85d-426c-9560-c71ccdefcae1" Jul 29 22:54:05.406188 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:54:05.406145 2562 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="kserve/llmisvc-controller-manager-654c557856-k2ld6" Jul 29 22:54:05.406660 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:54:05.406577 2562 scope.go:117] "RemoveContainer" containerID="e659847c06c30a4c04917f4b939ce9439f3a584a56432af277ec7abe1c60ec1f" Jul 29 22:54:05.406799 ip-10-0-138-112 kubenswrapper[2562]: E0729 22:54:05.406778 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 20s restarting failed container=manager pod=llmisvc-controller-manager-654c557856-k2ld6_kserve(43ca558c-d85d-426c-9560-c71ccdefcae1)\"" pod="kserve/llmisvc-controller-manager-654c557856-k2ld6" podUID="43ca558c-d85d-426c-9560-c71ccdefcae1" Jul 29 22:54:12.292463 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:54:12.292427 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve/kserve-controller-manager-844cf74686-242x8" Jul 29 22:54:16.570306 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:54:16.570276 2562 scope.go:117] "RemoveContainer" containerID="e659847c06c30a4c04917f4b939ce9439f3a584a56432af277ec7abe1c60ec1f" Jul 29 22:54:16.570756 ip-10-0-138-112 kubenswrapper[2562]: E0729 22:54:16.570477 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 20s restarting failed container=manager pod=llmisvc-controller-manager-654c557856-k2ld6_kserve(43ca558c-d85d-426c-9560-c71ccdefcae1)\"" pod="kserve/llmisvc-controller-manager-654c557856-k2ld6" podUID="43ca558c-d85d-426c-9560-c71ccdefcae1" Jul 29 22:54:28.570335 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:54:28.570298 2562 scope.go:117] "RemoveContainer" containerID="e659847c06c30a4c04917f4b939ce9439f3a584a56432af277ec7abe1c60ec1f" Jul 29 22:54:29.490446 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:54:29.490389 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/llmisvc-controller-manager-654c557856-k2ld6" event={"ID":"43ca558c-d85d-426c-9560-c71ccdefcae1","Type":"ContainerStarted","Data":"56a225700d3ad5172ab883e219775f8b74b1c72b7685a54f2894e588616ecb48"} Jul 29 22:54:29.490737 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:54:29.490716 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve/llmisvc-controller-manager-654c557856-k2ld6" Jul 29 22:54:30.496027 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:54:30.495993 2562 generic.go:356] "Generic (PLEG): container finished" podID="43ca558c-d85d-426c-9560-c71ccdefcae1" containerID="56a225700d3ad5172ab883e219775f8b74b1c72b7685a54f2894e588616ecb48" exitCode=1 Jul 29 22:54:30.496513 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:54:30.496062 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/llmisvc-controller-manager-654c557856-k2ld6" event={"ID":"43ca558c-d85d-426c-9560-c71ccdefcae1","Type":"ContainerDied","Data":"56a225700d3ad5172ab883e219775f8b74b1c72b7685a54f2894e588616ecb48"} Jul 29 22:54:30.496513 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:54:30.496105 2562 scope.go:117] "RemoveContainer" containerID="e659847c06c30a4c04917f4b939ce9439f3a584a56432af277ec7abe1c60ec1f" Jul 29 22:54:30.496513 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:54:30.496425 2562 scope.go:117] "RemoveContainer" containerID="56a225700d3ad5172ab883e219775f8b74b1c72b7685a54f2894e588616ecb48" Jul 29 22:54:30.496685 ip-10-0-138-112 kubenswrapper[2562]: E0729 22:54:30.496656 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 40s restarting failed container=manager pod=llmisvc-controller-manager-654c557856-k2ld6_kserve(43ca558c-d85d-426c-9560-c71ccdefcae1)\"" pod="kserve/llmisvc-controller-manager-654c557856-k2ld6" podUID="43ca558c-d85d-426c-9560-c71ccdefcae1" Jul 29 22:54:31.501818 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:54:31.501784 2562 scope.go:117] "RemoveContainer" containerID="56a225700d3ad5172ab883e219775f8b74b1c72b7685a54f2894e588616ecb48" Jul 29 22:54:31.502218 ip-10-0-138-112 kubenswrapper[2562]: E0729 22:54:31.501953 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 40s restarting failed container=manager pod=llmisvc-controller-manager-654c557856-k2ld6_kserve(43ca558c-d85d-426c-9560-c71ccdefcae1)\"" pod="kserve/llmisvc-controller-manager-654c557856-k2ld6" podUID="43ca558c-d85d-426c-9560-c71ccdefcae1" Jul 29 22:54:35.405530 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:54:35.405484 2562 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="kserve/llmisvc-controller-manager-654c557856-k2ld6" Jul 29 22:54:35.406006 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:54:35.405900 2562 scope.go:117] "RemoveContainer" containerID="56a225700d3ad5172ab883e219775f8b74b1c72b7685a54f2894e588616ecb48" Jul 29 22:54:35.406096 ip-10-0-138-112 kubenswrapper[2562]: E0729 22:54:35.406075 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 40s restarting failed container=manager pod=llmisvc-controller-manager-654c557856-k2ld6_kserve(43ca558c-d85d-426c-9560-c71ccdefcae1)\"" pod="kserve/llmisvc-controller-manager-654c557856-k2ld6" podUID="43ca558c-d85d-426c-9560-c71ccdefcae1" Jul 29 22:54:45.738629 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:54:45.738595 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6d665b88d6-z7blc"] Jul 29 22:54:47.571084 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:54:47.571054 2562 scope.go:117] "RemoveContainer" containerID="56a225700d3ad5172ab883e219775f8b74b1c72b7685a54f2894e588616ecb48" Jul 29 22:54:47.571483 ip-10-0-138-112 kubenswrapper[2562]: E0729 22:54:47.571242 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 40s restarting failed container=manager pod=llmisvc-controller-manager-654c557856-k2ld6_kserve(43ca558c-d85d-426c-9560-c71ccdefcae1)\"" pod="kserve/llmisvc-controller-manager-654c557856-k2ld6" podUID="43ca558c-d85d-426c-9560-c71ccdefcae1" Jul 29 22:55:00.570992 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:55:00.570948 2562 scope.go:117] "RemoveContainer" containerID="56a225700d3ad5172ab883e219775f8b74b1c72b7685a54f2894e588616ecb48" Jul 29 22:55:00.571510 ip-10-0-138-112 kubenswrapper[2562]: E0729 22:55:00.571189 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 40s restarting failed container=manager pod=llmisvc-controller-manager-654c557856-k2ld6_kserve(43ca558c-d85d-426c-9560-c71ccdefcae1)\"" pod="kserve/llmisvc-controller-manager-654c557856-k2ld6" podUID="43ca558c-d85d-426c-9560-c71ccdefcae1" Jul 29 22:55:10.764707 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:55:10.764648 2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-6d665b88d6-z7blc" podUID="a50338ed-263f-4855-aa0a-53fb0cd72ed7" containerName="console" containerID="cri-o://0ae94cf3f0392475058f0c2f35ba14a5b8394d0f4989823b149f103ea2a3b7a2" gracePeriod=15 Jul 29 22:55:11.007935 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:55:11.007909 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6d665b88d6-z7blc" Jul 29 22:55:11.105030 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:55:11.104991 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a50338ed-263f-4855-aa0a-53fb0cd72ed7-console-serving-cert\") pod \"a50338ed-263f-4855-aa0a-53fb0cd72ed7\" (UID: \"a50338ed-263f-4855-aa0a-53fb0cd72ed7\") " Jul 29 22:55:11.105236 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:55:11.105042 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a50338ed-263f-4855-aa0a-53fb0cd72ed7-trusted-ca-bundle\") pod \"a50338ed-263f-4855-aa0a-53fb0cd72ed7\" (UID: \"a50338ed-263f-4855-aa0a-53fb0cd72ed7\") " Jul 29 22:55:11.105236 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:55:11.105068 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a50338ed-263f-4855-aa0a-53fb0cd72ed7-console-config\") pod \"a50338ed-263f-4855-aa0a-53fb0cd72ed7\" (UID: \"a50338ed-263f-4855-aa0a-53fb0cd72ed7\") " Jul 29 22:55:11.105236 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:55:11.105092 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a50338ed-263f-4855-aa0a-53fb0cd72ed7-console-oauth-config\") pod \"a50338ed-263f-4855-aa0a-53fb0cd72ed7\" (UID: \"a50338ed-263f-4855-aa0a-53fb0cd72ed7\") " Jul 29 22:55:11.105236 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:55:11.105120 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a50338ed-263f-4855-aa0a-53fb0cd72ed7-oauth-serving-cert\") pod \"a50338ed-263f-4855-aa0a-53fb0cd72ed7\" (UID: \"a50338ed-263f-4855-aa0a-53fb0cd72ed7\") " Jul 29 22:55:11.105236 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:55:11.105141 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a50338ed-263f-4855-aa0a-53fb0cd72ed7-service-ca\") pod \"a50338ed-263f-4855-aa0a-53fb0cd72ed7\" (UID: \"a50338ed-263f-4855-aa0a-53fb0cd72ed7\") " Jul 29 22:55:11.105236 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:55:11.105188 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h2lkn\" (UniqueName: \"kubernetes.io/projected/a50338ed-263f-4855-aa0a-53fb0cd72ed7-kube-api-access-h2lkn\") pod \"a50338ed-263f-4855-aa0a-53fb0cd72ed7\" (UID: \"a50338ed-263f-4855-aa0a-53fb0cd72ed7\") " Jul 29 22:55:11.105589 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:55:11.105560 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a50338ed-263f-4855-aa0a-53fb0cd72ed7-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "a50338ed-263f-4855-aa0a-53fb0cd72ed7" (UID: "a50338ed-263f-4855-aa0a-53fb0cd72ed7"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 29 22:55:11.105709 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:55:11.105682 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a50338ed-263f-4855-aa0a-53fb0cd72ed7-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "a50338ed-263f-4855-aa0a-53fb0cd72ed7" (UID: "a50338ed-263f-4855-aa0a-53fb0cd72ed7"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 29 22:55:11.105709 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:55:11.105693 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a50338ed-263f-4855-aa0a-53fb0cd72ed7-console-config" (OuterVolumeSpecName: "console-config") pod "a50338ed-263f-4855-aa0a-53fb0cd72ed7" (UID: "a50338ed-263f-4855-aa0a-53fb0cd72ed7"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 29 22:55:11.105801 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:55:11.105702 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a50338ed-263f-4855-aa0a-53fb0cd72ed7-service-ca" (OuterVolumeSpecName: "service-ca") pod "a50338ed-263f-4855-aa0a-53fb0cd72ed7" (UID: "a50338ed-263f-4855-aa0a-53fb0cd72ed7"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 29 22:55:11.107427 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:55:11.107385 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a50338ed-263f-4855-aa0a-53fb0cd72ed7-kube-api-access-h2lkn" (OuterVolumeSpecName: "kube-api-access-h2lkn") pod "a50338ed-263f-4855-aa0a-53fb0cd72ed7" (UID: "a50338ed-263f-4855-aa0a-53fb0cd72ed7"). InnerVolumeSpecName "kube-api-access-h2lkn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 29 22:55:11.107658 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:55:11.107631 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a50338ed-263f-4855-aa0a-53fb0cd72ed7-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "a50338ed-263f-4855-aa0a-53fb0cd72ed7" (UID: "a50338ed-263f-4855-aa0a-53fb0cd72ed7"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 29 22:55:11.107714 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:55:11.107670 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a50338ed-263f-4855-aa0a-53fb0cd72ed7-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "a50338ed-263f-4855-aa0a-53fb0cd72ed7" (UID: "a50338ed-263f-4855-aa0a-53fb0cd72ed7"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 29 22:55:11.206466 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:55:11.206429 2562 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a50338ed-263f-4855-aa0a-53fb0cd72ed7-trusted-ca-bundle\") on node \"ip-10-0-138-112.ec2.internal\" DevicePath \"\"" Jul 29 22:55:11.206466 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:55:11.206458 2562 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a50338ed-263f-4855-aa0a-53fb0cd72ed7-console-config\") on node \"ip-10-0-138-112.ec2.internal\" DevicePath \"\"" Jul 29 22:55:11.206466 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:55:11.206470 2562 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a50338ed-263f-4855-aa0a-53fb0cd72ed7-console-oauth-config\") on node \"ip-10-0-138-112.ec2.internal\" DevicePath \"\"" Jul 29 22:55:11.206699 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:55:11.206479 2562 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a50338ed-263f-4855-aa0a-53fb0cd72ed7-oauth-serving-cert\") on node \"ip-10-0-138-112.ec2.internal\" DevicePath \"\"" Jul 29 22:55:11.206699 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:55:11.206487 2562 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a50338ed-263f-4855-aa0a-53fb0cd72ed7-service-ca\") on node \"ip-10-0-138-112.ec2.internal\" DevicePath \"\"" Jul 29 22:55:11.206699 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:55:11.206495 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h2lkn\" (UniqueName: \"kubernetes.io/projected/a50338ed-263f-4855-aa0a-53fb0cd72ed7-kube-api-access-h2lkn\") on node \"ip-10-0-138-112.ec2.internal\" DevicePath \"\"" Jul 29 22:55:11.206699 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:55:11.206503 2562 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a50338ed-263f-4855-aa0a-53fb0cd72ed7-console-serving-cert\") on node \"ip-10-0-138-112.ec2.internal\" DevicePath \"\"" Jul 29 22:55:11.665017 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:55:11.664981 2562 generic.go:356] "Generic (PLEG): container finished" podID="a50338ed-263f-4855-aa0a-53fb0cd72ed7" containerID="0ae94cf3f0392475058f0c2f35ba14a5b8394d0f4989823b149f103ea2a3b7a2" exitCode=0 Jul 29 22:55:11.665183 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:55:11.665026 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6d665b88d6-z7blc" Jul 29 22:55:11.665183 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:55:11.665070 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6d665b88d6-z7blc" event={"ID":"a50338ed-263f-4855-aa0a-53fb0cd72ed7","Type":"ContainerDied","Data":"0ae94cf3f0392475058f0c2f35ba14a5b8394d0f4989823b149f103ea2a3b7a2"} Jul 29 22:55:11.665183 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:55:11.665123 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6d665b88d6-z7blc" event={"ID":"a50338ed-263f-4855-aa0a-53fb0cd72ed7","Type":"ContainerDied","Data":"7246d302d3704cfa6d59be3ef53a29ed893091abacb0257d20b1c06f01c7c39b"} Jul 29 22:55:11.665183 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:55:11.665148 2562 scope.go:117] "RemoveContainer" containerID="0ae94cf3f0392475058f0c2f35ba14a5b8394d0f4989823b149f103ea2a3b7a2" Jul 29 22:55:11.673310 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:55:11.673294 2562 scope.go:117] "RemoveContainer" containerID="0ae94cf3f0392475058f0c2f35ba14a5b8394d0f4989823b149f103ea2a3b7a2" Jul 29 22:55:11.673605 ip-10-0-138-112 kubenswrapper[2562]: E0729 22:55:11.673584 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0ae94cf3f0392475058f0c2f35ba14a5b8394d0f4989823b149f103ea2a3b7a2\": container with ID starting with 0ae94cf3f0392475058f0c2f35ba14a5b8394d0f4989823b149f103ea2a3b7a2 not found: ID does not exist" containerID="0ae94cf3f0392475058f0c2f35ba14a5b8394d0f4989823b149f103ea2a3b7a2" Jul 29 22:55:11.673664 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:55:11.673614 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0ae94cf3f0392475058f0c2f35ba14a5b8394d0f4989823b149f103ea2a3b7a2"} err="failed to get container status \"0ae94cf3f0392475058f0c2f35ba14a5b8394d0f4989823b149f103ea2a3b7a2\": rpc error: code = NotFound desc = could not find container \"0ae94cf3f0392475058f0c2f35ba14a5b8394d0f4989823b149f103ea2a3b7a2\": container with ID starting with 0ae94cf3f0392475058f0c2f35ba14a5b8394d0f4989823b149f103ea2a3b7a2 not found: ID does not exist" Jul 29 22:55:11.686170 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:55:11.686140 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6d665b88d6-z7blc"] Jul 29 22:55:11.690662 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:55:11.690637 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-6d665b88d6-z7blc"] Jul 29 22:55:12.570929 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:55:12.570896 2562 scope.go:117] "RemoveContainer" containerID="56a225700d3ad5172ab883e219775f8b74b1c72b7685a54f2894e588616ecb48" Jul 29 22:55:13.580210 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:55:13.580165 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a50338ed-263f-4855-aa0a-53fb0cd72ed7" path="/var/lib/kubelet/pods/a50338ed-263f-4855-aa0a-53fb0cd72ed7/volumes" Jul 29 22:55:13.675470 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:55:13.675431 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/llmisvc-controller-manager-654c557856-k2ld6" event={"ID":"43ca558c-d85d-426c-9560-c71ccdefcae1","Type":"ContainerStarted","Data":"af97aaeea9b2b46ebce43388a74342db4b112ed556e6527c0a9ec140527ed5d0"} Jul 29 22:55:13.675683 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:55:13.675664 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve/llmisvc-controller-manager-654c557856-k2ld6" Jul 29 22:55:14.680888 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:55:14.680856 2562 generic.go:356] "Generic (PLEG): container finished" podID="43ca558c-d85d-426c-9560-c71ccdefcae1" containerID="af97aaeea9b2b46ebce43388a74342db4b112ed556e6527c0a9ec140527ed5d0" exitCode=1 Jul 29 22:55:14.681308 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:55:14.680903 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/llmisvc-controller-manager-654c557856-k2ld6" event={"ID":"43ca558c-d85d-426c-9560-c71ccdefcae1","Type":"ContainerDied","Data":"af97aaeea9b2b46ebce43388a74342db4b112ed556e6527c0a9ec140527ed5d0"} Jul 29 22:55:14.681308 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:55:14.680951 2562 scope.go:117] "RemoveContainer" containerID="56a225700d3ad5172ab883e219775f8b74b1c72b7685a54f2894e588616ecb48" Jul 29 22:55:14.681308 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:55:14.681098 2562 scope.go:117] "RemoveContainer" containerID="af97aaeea9b2b46ebce43388a74342db4b112ed556e6527c0a9ec140527ed5d0" Jul 29 22:55:14.681425 ip-10-0-138-112 kubenswrapper[2562]: E0729 22:55:14.681339 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=manager pod=llmisvc-controller-manager-654c557856-k2ld6_kserve(43ca558c-d85d-426c-9560-c71ccdefcae1)\"" pod="kserve/llmisvc-controller-manager-654c557856-k2ld6" podUID="43ca558c-d85d-426c-9560-c71ccdefcae1" Jul 29 22:55:15.405884 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:55:15.405842 2562 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="kserve/llmisvc-controller-manager-654c557856-k2ld6" Jul 29 22:55:15.686811 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:55:15.686728 2562 scope.go:117] "RemoveContainer" containerID="af97aaeea9b2b46ebce43388a74342db4b112ed556e6527c0a9ec140527ed5d0" Jul 29 22:55:15.687164 ip-10-0-138-112 kubenswrapper[2562]: E0729 22:55:15.686912 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=manager pod=llmisvc-controller-manager-654c557856-k2ld6_kserve(43ca558c-d85d-426c-9560-c71ccdefcae1)\"" pod="kserve/llmisvc-controller-manager-654c557856-k2ld6" podUID="43ca558c-d85d-426c-9560-c71ccdefcae1" Jul 29 22:55:16.690383 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:55:16.690358 2562 scope.go:117] "RemoveContainer" containerID="af97aaeea9b2b46ebce43388a74342db4b112ed556e6527c0a9ec140527ed5d0" Jul 29 22:55:16.690870 ip-10-0-138-112 kubenswrapper[2562]: E0729 22:55:16.690575 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=manager pod=llmisvc-controller-manager-654c557856-k2ld6_kserve(43ca558c-d85d-426c-9560-c71ccdefcae1)\"" pod="kserve/llmisvc-controller-manager-654c557856-k2ld6" podUID="43ca558c-d85d-426c-9560-c71ccdefcae1" Jul 29 22:55:29.570490 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:55:29.570459 2562 scope.go:117] "RemoveContainer" containerID="af97aaeea9b2b46ebce43388a74342db4b112ed556e6527c0a9ec140527ed5d0" Jul 29 22:55:29.570898 ip-10-0-138-112 kubenswrapper[2562]: E0729 22:55:29.570646 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=manager pod=llmisvc-controller-manager-654c557856-k2ld6_kserve(43ca558c-d85d-426c-9560-c71ccdefcae1)\"" pod="kserve/llmisvc-controller-manager-654c557856-k2ld6" podUID="43ca558c-d85d-426c-9560-c71ccdefcae1" Jul 29 22:55:44.570589 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:55:44.570554 2562 scope.go:117] "RemoveContainer" containerID="af97aaeea9b2b46ebce43388a74342db4b112ed556e6527c0a9ec140527ed5d0" Jul 29 22:55:44.571057 ip-10-0-138-112 kubenswrapper[2562]: E0729 22:55:44.570738 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=manager pod=llmisvc-controller-manager-654c557856-k2ld6_kserve(43ca558c-d85d-426c-9560-c71ccdefcae1)\"" pod="kserve/llmisvc-controller-manager-654c557856-k2ld6" podUID="43ca558c-d85d-426c-9560-c71ccdefcae1" Jul 29 22:55:59.570757 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:55:59.570711 2562 scope.go:117] "RemoveContainer" containerID="af97aaeea9b2b46ebce43388a74342db4b112ed556e6527c0a9ec140527ed5d0" Jul 29 22:55:59.571182 ip-10-0-138-112 kubenswrapper[2562]: E0729 22:55:59.570874 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=manager pod=llmisvc-controller-manager-654c557856-k2ld6_kserve(43ca558c-d85d-426c-9560-c71ccdefcae1)\"" pod="kserve/llmisvc-controller-manager-654c557856-k2ld6" podUID="43ca558c-d85d-426c-9560-c71ccdefcae1" Jul 29 22:56:10.570474 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:56:10.570438 2562 scope.go:117] "RemoveContainer" containerID="af97aaeea9b2b46ebce43388a74342db4b112ed556e6527c0a9ec140527ed5d0" Jul 29 22:56:10.570884 ip-10-0-138-112 kubenswrapper[2562]: E0729 22:56:10.570640 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=manager pod=llmisvc-controller-manager-654c557856-k2ld6_kserve(43ca558c-d85d-426c-9560-c71ccdefcae1)\"" pod="kserve/llmisvc-controller-manager-654c557856-k2ld6" podUID="43ca558c-d85d-426c-9560-c71ccdefcae1" Jul 29 22:56:25.578604 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:56:25.578571 2562 scope.go:117] "RemoveContainer" containerID="af97aaeea9b2b46ebce43388a74342db4b112ed556e6527c0a9ec140527ed5d0" Jul 29 22:56:25.579203 ip-10-0-138-112 kubenswrapper[2562]: E0729 22:56:25.578755 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=manager pod=llmisvc-controller-manager-654c557856-k2ld6_kserve(43ca558c-d85d-426c-9560-c71ccdefcae1)\"" pod="kserve/llmisvc-controller-manager-654c557856-k2ld6" podUID="43ca558c-d85d-426c-9560-c71ccdefcae1" Jul 29 22:56:38.570367 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:56:38.570335 2562 scope.go:117] "RemoveContainer" containerID="af97aaeea9b2b46ebce43388a74342db4b112ed556e6527c0a9ec140527ed5d0" Jul 29 22:56:39.034575 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:56:39.034548 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/llmisvc-controller-manager-654c557856-k2ld6" event={"ID":"43ca558c-d85d-426c-9560-c71ccdefcae1","Type":"ContainerStarted","Data":"d3899610188ebde4da9a4d33e76675927d39a5a18c744fdb0215d74c159f8250"} Jul 29 22:56:40.039867 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:56:40.039833 2562 generic.go:356] "Generic (PLEG): container finished" podID="43ca558c-d85d-426c-9560-c71ccdefcae1" containerID="d3899610188ebde4da9a4d33e76675927d39a5a18c744fdb0215d74c159f8250" exitCode=1 Jul 29 22:56:40.040350 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:56:40.039877 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/llmisvc-controller-manager-654c557856-k2ld6" event={"ID":"43ca558c-d85d-426c-9560-c71ccdefcae1","Type":"ContainerDied","Data":"d3899610188ebde4da9a4d33e76675927d39a5a18c744fdb0215d74c159f8250"} Jul 29 22:56:40.040350 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:56:40.039926 2562 scope.go:117] "RemoveContainer" containerID="af97aaeea9b2b46ebce43388a74342db4b112ed556e6527c0a9ec140527ed5d0" Jul 29 22:56:40.040350 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:56:40.040193 2562 scope.go:117] "RemoveContainer" containerID="d3899610188ebde4da9a4d33e76675927d39a5a18c744fdb0215d74c159f8250" Jul 29 22:56:40.040532 ip-10-0-138-112 kubenswrapper[2562]: E0729 22:56:40.040422 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=manager pod=llmisvc-controller-manager-654c557856-k2ld6_kserve(43ca558c-d85d-426c-9560-c71ccdefcae1)\"" pod="kserve/llmisvc-controller-manager-654c557856-k2ld6" podUID="43ca558c-d85d-426c-9560-c71ccdefcae1" Jul 29 22:56:41.047188 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:56:41.047158 2562 scope.go:117] "RemoveContainer" containerID="d3899610188ebde4da9a4d33e76675927d39a5a18c744fdb0215d74c159f8250" Jul 29 22:56:41.047588 ip-10-0-138-112 kubenswrapper[2562]: E0729 22:56:41.047352 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=manager pod=llmisvc-controller-manager-654c557856-k2ld6_kserve(43ca558c-d85d-426c-9560-c71ccdefcae1)\"" pod="kserve/llmisvc-controller-manager-654c557856-k2ld6" podUID="43ca558c-d85d-426c-9560-c71ccdefcae1" Jul 29 22:56:43.676071 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:56:43.676029 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve/llmisvc-controller-manager-654c557856-k2ld6" Jul 29 22:56:43.676627 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:56:43.676391 2562 scope.go:117] "RemoveContainer" containerID="d3899610188ebde4da9a4d33e76675927d39a5a18c744fdb0215d74c159f8250" Jul 29 22:56:43.676686 ip-10-0-138-112 kubenswrapper[2562]: E0729 22:56:43.676622 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=manager pod=llmisvc-controller-manager-654c557856-k2ld6_kserve(43ca558c-d85d-426c-9560-c71ccdefcae1)\"" pod="kserve/llmisvc-controller-manager-654c557856-k2ld6" podUID="43ca558c-d85d-426c-9560-c71ccdefcae1" Jul 29 22:56:45.406154 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:56:45.406104 2562 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="kserve/llmisvc-controller-manager-654c557856-k2ld6" Jul 29 22:56:45.406661 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:56:45.406497 2562 scope.go:117] "RemoveContainer" containerID="d3899610188ebde4da9a4d33e76675927d39a5a18c744fdb0215d74c159f8250" Jul 29 22:56:45.406661 ip-10-0-138-112 kubenswrapper[2562]: E0729 22:56:45.406653 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=manager pod=llmisvc-controller-manager-654c557856-k2ld6_kserve(43ca558c-d85d-426c-9560-c71ccdefcae1)\"" pod="kserve/llmisvc-controller-manager-654c557856-k2ld6" podUID="43ca558c-d85d-426c-9560-c71ccdefcae1" Jul 29 22:56:58.570630 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:56:58.570597 2562 scope.go:117] "RemoveContainer" containerID="d3899610188ebde4da9a4d33e76675927d39a5a18c744fdb0215d74c159f8250" Jul 29 22:56:58.572917 ip-10-0-138-112 kubenswrapper[2562]: E0729 22:56:58.570757 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=manager pod=llmisvc-controller-manager-654c557856-k2ld6_kserve(43ca558c-d85d-426c-9560-c71ccdefcae1)\"" pod="kserve/llmisvc-controller-manager-654c557856-k2ld6" podUID="43ca558c-d85d-426c-9560-c71ccdefcae1" Jul 29 22:57:13.570501 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:57:13.570465 2562 scope.go:117] "RemoveContainer" containerID="d3899610188ebde4da9a4d33e76675927d39a5a18c744fdb0215d74c159f8250" Jul 29 22:57:13.571026 ip-10-0-138-112 kubenswrapper[2562]: E0729 22:57:13.570647 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=manager pod=llmisvc-controller-manager-654c557856-k2ld6_kserve(43ca558c-d85d-426c-9560-c71ccdefcae1)\"" pod="kserve/llmisvc-controller-manager-654c557856-k2ld6" podUID="43ca558c-d85d-426c-9560-c71ccdefcae1" Jul 29 22:57:24.570371 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:57:24.570343 2562 scope.go:117] "RemoveContainer" containerID="d3899610188ebde4da9a4d33e76675927d39a5a18c744fdb0215d74c159f8250" Jul 29 22:57:24.570855 ip-10-0-138-112 kubenswrapper[2562]: E0729 22:57:24.570550 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=manager pod=llmisvc-controller-manager-654c557856-k2ld6_kserve(43ca558c-d85d-426c-9560-c71ccdefcae1)\"" pod="kserve/llmisvc-controller-manager-654c557856-k2ld6" podUID="43ca558c-d85d-426c-9560-c71ccdefcae1" Jul 29 22:57:38.571006 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:57:38.570968 2562 scope.go:117] "RemoveContainer" containerID="d3899610188ebde4da9a4d33e76675927d39a5a18c744fdb0215d74c159f8250" Jul 29 22:57:38.571501 ip-10-0-138-112 kubenswrapper[2562]: E0729 22:57:38.571145 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=manager pod=llmisvc-controller-manager-654c557856-k2ld6_kserve(43ca558c-d85d-426c-9560-c71ccdefcae1)\"" pod="kserve/llmisvc-controller-manager-654c557856-k2ld6" podUID="43ca558c-d85d-426c-9560-c71ccdefcae1" Jul 29 22:57:49.570512 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:57:49.570477 2562 scope.go:117] "RemoveContainer" containerID="d3899610188ebde4da9a4d33e76675927d39a5a18c744fdb0215d74c159f8250" Jul 29 22:57:49.570998 ip-10-0-138-112 kubenswrapper[2562]: E0729 22:57:49.570664 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=manager pod=llmisvc-controller-manager-654c557856-k2ld6_kserve(43ca558c-d85d-426c-9560-c71ccdefcae1)\"" pod="kserve/llmisvc-controller-manager-654c557856-k2ld6" podUID="43ca558c-d85d-426c-9560-c71ccdefcae1" Jul 29 22:58:02.570460 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:58:02.570424 2562 scope.go:117] "RemoveContainer" containerID="d3899610188ebde4da9a4d33e76675927d39a5a18c744fdb0215d74c159f8250" Jul 29 22:58:02.570935 ip-10-0-138-112 kubenswrapper[2562]: E0729 22:58:02.570638 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=manager pod=llmisvc-controller-manager-654c557856-k2ld6_kserve(43ca558c-d85d-426c-9560-c71ccdefcae1)\"" pod="kserve/llmisvc-controller-manager-654c557856-k2ld6" podUID="43ca558c-d85d-426c-9560-c71ccdefcae1" Jul 29 22:58:13.570818 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:58:13.570783 2562 scope.go:117] "RemoveContainer" containerID="d3899610188ebde4da9a4d33e76675927d39a5a18c744fdb0215d74c159f8250" Jul 29 22:58:13.571214 ip-10-0-138-112 kubenswrapper[2562]: E0729 22:58:13.570965 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=manager pod=llmisvc-controller-manager-654c557856-k2ld6_kserve(43ca558c-d85d-426c-9560-c71ccdefcae1)\"" pod="kserve/llmisvc-controller-manager-654c557856-k2ld6" podUID="43ca558c-d85d-426c-9560-c71ccdefcae1" Jul 29 22:58:25.572590 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:58:25.572512 2562 scope.go:117] "RemoveContainer" containerID="d3899610188ebde4da9a4d33e76675927d39a5a18c744fdb0215d74c159f8250" Jul 29 22:58:25.573058 ip-10-0-138-112 kubenswrapper[2562]: E0729 22:58:25.572789 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=manager pod=llmisvc-controller-manager-654c557856-k2ld6_kserve(43ca558c-d85d-426c-9560-c71ccdefcae1)\"" pod="kserve/llmisvc-controller-manager-654c557856-k2ld6" podUID="43ca558c-d85d-426c-9560-c71ccdefcae1" Jul 29 22:58:38.571009 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:58:38.570972 2562 scope.go:117] "RemoveContainer" containerID="d3899610188ebde4da9a4d33e76675927d39a5a18c744fdb0215d74c159f8250" Jul 29 22:58:38.571567 ip-10-0-138-112 kubenswrapper[2562]: E0729 22:58:38.571144 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=manager pod=llmisvc-controller-manager-654c557856-k2ld6_kserve(43ca558c-d85d-426c-9560-c71ccdefcae1)\"" pod="kserve/llmisvc-controller-manager-654c557856-k2ld6" podUID="43ca558c-d85d-426c-9560-c71ccdefcae1" Jul 29 22:58:49.571166 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:58:49.571119 2562 scope.go:117] "RemoveContainer" containerID="d3899610188ebde4da9a4d33e76675927d39a5a18c744fdb0215d74c159f8250" Jul 29 22:58:49.571875 ip-10-0-138-112 kubenswrapper[2562]: E0729 22:58:49.571365 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=manager pod=llmisvc-controller-manager-654c557856-k2ld6_kserve(43ca558c-d85d-426c-9560-c71ccdefcae1)\"" pod="kserve/llmisvc-controller-manager-654c557856-k2ld6" podUID="43ca558c-d85d-426c-9560-c71ccdefcae1" Jul 29 22:59:03.570721 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:59:03.570683 2562 scope.go:117] "RemoveContainer" containerID="d3899610188ebde4da9a4d33e76675927d39a5a18c744fdb0215d74c159f8250" Jul 29 22:59:03.571289 ip-10-0-138-112 kubenswrapper[2562]: E0729 22:59:03.570930 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=manager pod=llmisvc-controller-manager-654c557856-k2ld6_kserve(43ca558c-d85d-426c-9560-c71ccdefcae1)\"" pod="kserve/llmisvc-controller-manager-654c557856-k2ld6" podUID="43ca558c-d85d-426c-9560-c71ccdefcae1" Jul 29 22:59:14.570578 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:59:14.570540 2562 scope.go:117] "RemoveContainer" containerID="d3899610188ebde4da9a4d33e76675927d39a5a18c744fdb0215d74c159f8250" Jul 29 22:59:14.571072 ip-10-0-138-112 kubenswrapper[2562]: E0729 22:59:14.570766 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 2m40s restarting failed container=manager pod=llmisvc-controller-manager-654c557856-k2ld6_kserve(43ca558c-d85d-426c-9560-c71ccdefcae1)\"" pod="kserve/llmisvc-controller-manager-654c557856-k2ld6" podUID="43ca558c-d85d-426c-9560-c71ccdefcae1" Jul 29 22:59:28.570619 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:59:28.570583 2562 scope.go:117] "RemoveContainer" containerID="d3899610188ebde4da9a4d33e76675927d39a5a18c744fdb0215d74c159f8250" Jul 29 22:59:28.571091 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:59:28.570866 2562 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 29 22:59:29.744044 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:59:29.744006 2562 generic.go:356] "Generic (PLEG): container finished" podID="43ca558c-d85d-426c-9560-c71ccdefcae1" containerID="2312686273fc0156575f3b8d3483858b96661665b16c04c93364738d690cdeb9" exitCode=1 Jul 29 22:59:29.744538 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:59:29.744056 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/llmisvc-controller-manager-654c557856-k2ld6" event={"ID":"43ca558c-d85d-426c-9560-c71ccdefcae1","Type":"ContainerDied","Data":"2312686273fc0156575f3b8d3483858b96661665b16c04c93364738d690cdeb9"} Jul 29 22:59:29.744538 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:59:29.744117 2562 scope.go:117] "RemoveContainer" containerID="d3899610188ebde4da9a4d33e76675927d39a5a18c744fdb0215d74c159f8250" Jul 29 22:59:29.744538 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:59:29.744482 2562 scope.go:117] "RemoveContainer" containerID="2312686273fc0156575f3b8d3483858b96661665b16c04c93364738d690cdeb9" Jul 29 22:59:29.744700 ip-10-0-138-112 kubenswrapper[2562]: E0729 22:59:29.744679 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=manager pod=llmisvc-controller-manager-654c557856-k2ld6_kserve(43ca558c-d85d-426c-9560-c71ccdefcae1)\"" pod="kserve/llmisvc-controller-manager-654c557856-k2ld6" podUID="43ca558c-d85d-426c-9560-c71ccdefcae1" Jul 29 22:59:33.676306 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:59:33.676262 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve/llmisvc-controller-manager-654c557856-k2ld6" Jul 29 22:59:33.676758 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:59:33.676611 2562 scope.go:117] "RemoveContainer" containerID="2312686273fc0156575f3b8d3483858b96661665b16c04c93364738d690cdeb9" Jul 29 22:59:33.676808 ip-10-0-138-112 kubenswrapper[2562]: E0729 22:59:33.676787 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=manager pod=llmisvc-controller-manager-654c557856-k2ld6_kserve(43ca558c-d85d-426c-9560-c71ccdefcae1)\"" pod="kserve/llmisvc-controller-manager-654c557856-k2ld6" podUID="43ca558c-d85d-426c-9560-c71ccdefcae1" Jul 29 22:59:35.406214 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:59:35.406176 2562 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="kserve/llmisvc-controller-manager-654c557856-k2ld6" Jul 29 22:59:35.406640 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:59:35.406584 2562 scope.go:117] "RemoveContainer" containerID="2312686273fc0156575f3b8d3483858b96661665b16c04c93364738d690cdeb9" Jul 29 22:59:35.407416 ip-10-0-138-112 kubenswrapper[2562]: E0729 22:59:35.407384 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=manager pod=llmisvc-controller-manager-654c557856-k2ld6_kserve(43ca558c-d85d-426c-9560-c71ccdefcae1)\"" pod="kserve/llmisvc-controller-manager-654c557856-k2ld6" podUID="43ca558c-d85d-426c-9560-c71ccdefcae1" Jul 29 22:59:48.570421 ip-10-0-138-112 kubenswrapper[2562]: I0729 22:59:48.570321 2562 scope.go:117] "RemoveContainer" containerID="2312686273fc0156575f3b8d3483858b96661665b16c04c93364738d690cdeb9" Jul 29 22:59:48.570856 ip-10-0-138-112 kubenswrapper[2562]: E0729 22:59:48.570597 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=manager pod=llmisvc-controller-manager-654c557856-k2ld6_kserve(43ca558c-d85d-426c-9560-c71ccdefcae1)\"" pod="kserve/llmisvc-controller-manager-654c557856-k2ld6" podUID="43ca558c-d85d-426c-9560-c71ccdefcae1" Jul 29 23:00:03.570930 ip-10-0-138-112 kubenswrapper[2562]: I0729 23:00:03.570899 2562 scope.go:117] "RemoveContainer" containerID="2312686273fc0156575f3b8d3483858b96661665b16c04c93364738d690cdeb9" Jul 29 23:00:03.571390 ip-10-0-138-112 kubenswrapper[2562]: E0729 23:00:03.571059 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=manager pod=llmisvc-controller-manager-654c557856-k2ld6_kserve(43ca558c-d85d-426c-9560-c71ccdefcae1)\"" pod="kserve/llmisvc-controller-manager-654c557856-k2ld6" podUID="43ca558c-d85d-426c-9560-c71ccdefcae1" Jul 29 23:00:17.570309 ip-10-0-138-112 kubenswrapper[2562]: I0729 23:00:17.570273 2562 scope.go:117] "RemoveContainer" containerID="2312686273fc0156575f3b8d3483858b96661665b16c04c93364738d690cdeb9" Jul 29 23:00:17.570741 ip-10-0-138-112 kubenswrapper[2562]: E0729 23:00:17.570538 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=manager pod=llmisvc-controller-manager-654c557856-k2ld6_kserve(43ca558c-d85d-426c-9560-c71ccdefcae1)\"" pod="kserve/llmisvc-controller-manager-654c557856-k2ld6" podUID="43ca558c-d85d-426c-9560-c71ccdefcae1" Jul 29 23:00:28.570536 ip-10-0-138-112 kubenswrapper[2562]: I0729 23:00:28.570502 2562 scope.go:117] "RemoveContainer" containerID="2312686273fc0156575f3b8d3483858b96661665b16c04c93364738d690cdeb9" Jul 29 23:00:28.570917 ip-10-0-138-112 kubenswrapper[2562]: E0729 23:00:28.570656 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=manager pod=llmisvc-controller-manager-654c557856-k2ld6_kserve(43ca558c-d85d-426c-9560-c71ccdefcae1)\"" pod="kserve/llmisvc-controller-manager-654c557856-k2ld6" podUID="43ca558c-d85d-426c-9560-c71ccdefcae1" Jul 29 23:00:40.571250 ip-10-0-138-112 kubenswrapper[2562]: I0729 23:00:40.571207 2562 scope.go:117] "RemoveContainer" containerID="2312686273fc0156575f3b8d3483858b96661665b16c04c93364738d690cdeb9" Jul 29 23:00:40.571785 ip-10-0-138-112 kubenswrapper[2562]: E0729 23:00:40.571455 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=manager pod=llmisvc-controller-manager-654c557856-k2ld6_kserve(43ca558c-d85d-426c-9560-c71ccdefcae1)\"" pod="kserve/llmisvc-controller-manager-654c557856-k2ld6" podUID="43ca558c-d85d-426c-9560-c71ccdefcae1" Jul 29 23:00:55.572012 ip-10-0-138-112 kubenswrapper[2562]: I0729 23:00:55.571985 2562 scope.go:117] "RemoveContainer" containerID="2312686273fc0156575f3b8d3483858b96661665b16c04c93364738d690cdeb9" Jul 29 23:00:55.572522 ip-10-0-138-112 kubenswrapper[2562]: E0729 23:00:55.572153 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=manager pod=llmisvc-controller-manager-654c557856-k2ld6_kserve(43ca558c-d85d-426c-9560-c71ccdefcae1)\"" pod="kserve/llmisvc-controller-manager-654c557856-k2ld6" podUID="43ca558c-d85d-426c-9560-c71ccdefcae1" Jul 29 23:01:07.570412 ip-10-0-138-112 kubenswrapper[2562]: I0729 23:01:07.570376 2562 scope.go:117] "RemoveContainer" containerID="2312686273fc0156575f3b8d3483858b96661665b16c04c93364738d690cdeb9" Jul 29 23:01:07.570873 ip-10-0-138-112 kubenswrapper[2562]: E0729 23:01:07.570596 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=manager pod=llmisvc-controller-manager-654c557856-k2ld6_kserve(43ca558c-d85d-426c-9560-c71ccdefcae1)\"" pod="kserve/llmisvc-controller-manager-654c557856-k2ld6" podUID="43ca558c-d85d-426c-9560-c71ccdefcae1" Jul 29 23:01:22.570740 ip-10-0-138-112 kubenswrapper[2562]: I0729 23:01:22.570659 2562 scope.go:117] "RemoveContainer" containerID="2312686273fc0156575f3b8d3483858b96661665b16c04c93364738d690cdeb9" Jul 29 23:01:22.571177 ip-10-0-138-112 kubenswrapper[2562]: E0729 23:01:22.570822 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=manager pod=llmisvc-controller-manager-654c557856-k2ld6_kserve(43ca558c-d85d-426c-9560-c71ccdefcae1)\"" pod="kserve/llmisvc-controller-manager-654c557856-k2ld6" podUID="43ca558c-d85d-426c-9560-c71ccdefcae1" Jul 29 23:01:34.570489 ip-10-0-138-112 kubenswrapper[2562]: I0729 23:01:34.570451 2562 scope.go:117] "RemoveContainer" containerID="2312686273fc0156575f3b8d3483858b96661665b16c04c93364738d690cdeb9" Jul 29 23:01:34.570926 ip-10-0-138-112 kubenswrapper[2562]: E0729 23:01:34.570632 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=manager pod=llmisvc-controller-manager-654c557856-k2ld6_kserve(43ca558c-d85d-426c-9560-c71ccdefcae1)\"" pod="kserve/llmisvc-controller-manager-654c557856-k2ld6" podUID="43ca558c-d85d-426c-9560-c71ccdefcae1" Jul 29 23:01:47.570876 ip-10-0-138-112 kubenswrapper[2562]: I0729 23:01:47.570847 2562 scope.go:117] "RemoveContainer" containerID="2312686273fc0156575f3b8d3483858b96661665b16c04c93364738d690cdeb9" Jul 29 23:01:47.571327 ip-10-0-138-112 kubenswrapper[2562]: E0729 23:01:47.571028 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=manager pod=llmisvc-controller-manager-654c557856-k2ld6_kserve(43ca558c-d85d-426c-9560-c71ccdefcae1)\"" pod="kserve/llmisvc-controller-manager-654c557856-k2ld6" podUID="43ca558c-d85d-426c-9560-c71ccdefcae1" Jul 29 23:01:58.570920 ip-10-0-138-112 kubenswrapper[2562]: I0729 23:01:58.570804 2562 scope.go:117] "RemoveContainer" containerID="2312686273fc0156575f3b8d3483858b96661665b16c04c93364738d690cdeb9" Jul 29 23:01:58.574934 ip-10-0-138-112 kubenswrapper[2562]: E0729 23:01:58.570995 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=manager pod=llmisvc-controller-manager-654c557856-k2ld6_kserve(43ca558c-d85d-426c-9560-c71ccdefcae1)\"" pod="kserve/llmisvc-controller-manager-654c557856-k2ld6" podUID="43ca558c-d85d-426c-9560-c71ccdefcae1" Jul 29 23:02:11.570948 ip-10-0-138-112 kubenswrapper[2562]: I0729 23:02:11.570912 2562 scope.go:117] "RemoveContainer" containerID="2312686273fc0156575f3b8d3483858b96661665b16c04c93364738d690cdeb9" Jul 29 23:02:11.571460 ip-10-0-138-112 kubenswrapper[2562]: E0729 23:02:11.571125 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=manager pod=llmisvc-controller-manager-654c557856-k2ld6_kserve(43ca558c-d85d-426c-9560-c71ccdefcae1)\"" pod="kserve/llmisvc-controller-manager-654c557856-k2ld6" podUID="43ca558c-d85d-426c-9560-c71ccdefcae1" Jul 29 23:02:22.570542 ip-10-0-138-112 kubenswrapper[2562]: I0729 23:02:22.570509 2562 scope.go:117] "RemoveContainer" containerID="2312686273fc0156575f3b8d3483858b96661665b16c04c93364738d690cdeb9" Jul 29 23:02:22.571025 ip-10-0-138-112 kubenswrapper[2562]: E0729 23:02:22.570703 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=manager pod=llmisvc-controller-manager-654c557856-k2ld6_kserve(43ca558c-d85d-426c-9560-c71ccdefcae1)\"" pod="kserve/llmisvc-controller-manager-654c557856-k2ld6" podUID="43ca558c-d85d-426c-9560-c71ccdefcae1" Jul 29 23:02:33.570208 ip-10-0-138-112 kubenswrapper[2562]: I0729 23:02:33.570168 2562 scope.go:117] "RemoveContainer" containerID="2312686273fc0156575f3b8d3483858b96661665b16c04c93364738d690cdeb9" Jul 29 23:02:33.570760 ip-10-0-138-112 kubenswrapper[2562]: E0729 23:02:33.570428 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=manager pod=llmisvc-controller-manager-654c557856-k2ld6_kserve(43ca558c-d85d-426c-9560-c71ccdefcae1)\"" pod="kserve/llmisvc-controller-manager-654c557856-k2ld6" podUID="43ca558c-d85d-426c-9560-c71ccdefcae1" Jul 29 23:02:44.570661 ip-10-0-138-112 kubenswrapper[2562]: I0729 23:02:44.570622 2562 scope.go:117] "RemoveContainer" containerID="2312686273fc0156575f3b8d3483858b96661665b16c04c93364738d690cdeb9" Jul 29 23:02:44.571150 ip-10-0-138-112 kubenswrapper[2562]: E0729 23:02:44.570828 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=manager pod=llmisvc-controller-manager-654c557856-k2ld6_kserve(43ca558c-d85d-426c-9560-c71ccdefcae1)\"" pod="kserve/llmisvc-controller-manager-654c557856-k2ld6" podUID="43ca558c-d85d-426c-9560-c71ccdefcae1" Jul 29 23:02:55.574091 ip-10-0-138-112 kubenswrapper[2562]: I0729 23:02:55.574001 2562 scope.go:117] "RemoveContainer" containerID="2312686273fc0156575f3b8d3483858b96661665b16c04c93364738d690cdeb9" Jul 29 23:02:55.574635 ip-10-0-138-112 kubenswrapper[2562]: E0729 23:02:55.574204 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=manager pod=llmisvc-controller-manager-654c557856-k2ld6_kserve(43ca558c-d85d-426c-9560-c71ccdefcae1)\"" pod="kserve/llmisvc-controller-manager-654c557856-k2ld6" podUID="43ca558c-d85d-426c-9560-c71ccdefcae1" Jul 29 23:03:10.570681 ip-10-0-138-112 kubenswrapper[2562]: I0729 23:03:10.570646 2562 scope.go:117] "RemoveContainer" containerID="2312686273fc0156575f3b8d3483858b96661665b16c04c93364738d690cdeb9" Jul 29 23:03:10.571069 ip-10-0-138-112 kubenswrapper[2562]: E0729 23:03:10.570834 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=manager pod=llmisvc-controller-manager-654c557856-k2ld6_kserve(43ca558c-d85d-426c-9560-c71ccdefcae1)\"" pod="kserve/llmisvc-controller-manager-654c557856-k2ld6" podUID="43ca558c-d85d-426c-9560-c71ccdefcae1" Jul 29 23:03:21.570452 ip-10-0-138-112 kubenswrapper[2562]: I0729 23:03:21.570394 2562 scope.go:117] "RemoveContainer" containerID="2312686273fc0156575f3b8d3483858b96661665b16c04c93364738d690cdeb9" Jul 29 23:03:21.570974 ip-10-0-138-112 kubenswrapper[2562]: E0729 23:03:21.570626 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=manager pod=llmisvc-controller-manager-654c557856-k2ld6_kserve(43ca558c-d85d-426c-9560-c71ccdefcae1)\"" pod="kserve/llmisvc-controller-manager-654c557856-k2ld6" podUID="43ca558c-d85d-426c-9560-c71ccdefcae1" Jul 29 23:03:36.570995 ip-10-0-138-112 kubenswrapper[2562]: I0729 23:03:36.570962 2562 scope.go:117] "RemoveContainer" containerID="2312686273fc0156575f3b8d3483858b96661665b16c04c93364738d690cdeb9" Jul 29 23:03:36.571504 ip-10-0-138-112 kubenswrapper[2562]: E0729 23:03:36.571151 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=manager pod=llmisvc-controller-manager-654c557856-k2ld6_kserve(43ca558c-d85d-426c-9560-c71ccdefcae1)\"" pod="kserve/llmisvc-controller-manager-654c557856-k2ld6" podUID="43ca558c-d85d-426c-9560-c71ccdefcae1" Jul 29 23:03:41.711673 ip-10-0-138-112 kubenswrapper[2562]: I0729 23:03:41.711633 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve_llmisvc-controller-manager-654c557856-k2ld6_43ca558c-d85d-426c-9560-c71ccdefcae1/manager/6.log" Jul 29 23:03:50.571189 ip-10-0-138-112 kubenswrapper[2562]: I0729 23:03:50.571153 2562 scope.go:117] "RemoveContainer" containerID="2312686273fc0156575f3b8d3483858b96661665b16c04c93364738d690cdeb9" Jul 29 23:03:50.571714 ip-10-0-138-112 kubenswrapper[2562]: E0729 23:03:50.571357 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=manager pod=llmisvc-controller-manager-654c557856-k2ld6_kserve(43ca558c-d85d-426c-9560-c71ccdefcae1)\"" pod="kserve/llmisvc-controller-manager-654c557856-k2ld6" podUID="43ca558c-d85d-426c-9560-c71ccdefcae1" Jul 29 23:04:05.571964 ip-10-0-138-112 kubenswrapper[2562]: I0729 23:04:05.571937 2562 scope.go:117] "RemoveContainer" containerID="2312686273fc0156575f3b8d3483858b96661665b16c04c93364738d690cdeb9" Jul 29 23:04:05.572362 ip-10-0-138-112 kubenswrapper[2562]: E0729 23:04:05.572102 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=manager pod=llmisvc-controller-manager-654c557856-k2ld6_kserve(43ca558c-d85d-426c-9560-c71ccdefcae1)\"" pod="kserve/llmisvc-controller-manager-654c557856-k2ld6" podUID="43ca558c-d85d-426c-9560-c71ccdefcae1" Jul 29 23:04:09.495267 ip-10-0-138-112 kubenswrapper[2562]: I0729 23:04:09.495230 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-nkzq5_916b3077-78fa-4054-921c-2e7124dc35f8/global-pull-secret-syncer/0.log" Jul 29 23:04:09.600270 ip-10-0-138-112 kubenswrapper[2562]: I0729 23:04:09.600240 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-k9nmc_ece97cfa-80dc-4fa8-ad31-61ba5c38aee3/konnectivity-agent/0.log" Jul 29 23:04:09.671447 ip-10-0-138-112 kubenswrapper[2562]: I0729 23:04:09.671414 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-138-112.ec2.internal_5376269631e55085670dee0434ce97e1/haproxy/0.log" Jul 29 23:04:14.443115 ip-10-0-138-112 kubenswrapper[2562]: I0729 23:04:14.443081 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56d9f944b4-brv22_83c2796a-d9a0-45be-a1dd-d9e3e3d11108/kube-state-metrics/0.log" Jul 29 23:04:14.466315 ip-10-0-138-112 kubenswrapper[2562]: I0729 23:04:14.466285 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56d9f944b4-brv22_83c2796a-d9a0-45be-a1dd-d9e3e3d11108/kube-rbac-proxy-main/0.log" Jul 29 23:04:14.486893 ip-10-0-138-112 kubenswrapper[2562]: I0729 23:04:14.486865 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56d9f944b4-brv22_83c2796a-d9a0-45be-a1dd-d9e3e3d11108/kube-rbac-proxy-self/0.log" Jul 29 23:04:14.516381 ip-10-0-138-112 kubenswrapper[2562]: I0729 23:04:14.516350 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-69bb45d4cd-b5lk4_75075b31-e2a5-4667-87fe-dfcd79e196e0/metrics-server/0.log" Jul 29 23:04:14.735309 ip-10-0-138-112 kubenswrapper[2562]: I0729 23:04:14.735230 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-zvrsv_347281f0-0a1e-4710-9d6b-8e81fe4e7613/node-exporter/0.log" Jul 29 23:04:14.758779 ip-10-0-138-112 kubenswrapper[2562]: I0729 23:04:14.758750 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-zvrsv_347281f0-0a1e-4710-9d6b-8e81fe4e7613/kube-rbac-proxy/0.log" Jul 29 23:04:14.781128 ip-10-0-138-112 kubenswrapper[2562]: I0729 23:04:14.781099 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-zvrsv_347281f0-0a1e-4710-9d6b-8e81fe4e7613/init-textfile/0.log" Jul 29 23:04:15.063262 ip-10-0-138-112 kubenswrapper[2562]: I0729 23:04:15.063236 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-84d69f794d-sw54f_1f5b7f19-7d60-4f37-8943-f4f9f4507ac5/prometheus-operator/0.log" Jul 29 23:04:15.083552 ip-10-0-138-112 kubenswrapper[2562]: I0729 23:04:15.083529 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-84d69f794d-sw54f_1f5b7f19-7d60-4f37-8943-f4f9f4507ac5/kube-rbac-proxy/0.log" Jul 29 23:04:15.114874 ip-10-0-138-112 kubenswrapper[2562]: I0729 23:04:15.114847 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-7f8d78f54b-c4zbl_d8ed6b79-82fb-46b7-bf41-a68dd856fe24/prometheus-operator-admission-webhook/0.log" Jul 29 23:04:15.145078 ip-10-0-138-112 kubenswrapper[2562]: I0729 23:04:15.145053 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-65fbcc5b55-l9zgn_30052ca6-a70f-44b3-abae-4374772320ef/telemeter-client/0.log" Jul 29 23:04:15.166357 ip-10-0-138-112 kubenswrapper[2562]: I0729 23:04:15.166331 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-65fbcc5b55-l9zgn_30052ca6-a70f-44b3-abae-4374772320ef/reload/0.log" Jul 29 23:04:15.186962 ip-10-0-138-112 kubenswrapper[2562]: I0729 23:04:15.186934 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-65fbcc5b55-l9zgn_30052ca6-a70f-44b3-abae-4374772320ef/kube-rbac-proxy/0.log" Jul 29 23:04:18.356155 ip-10-0-138-112 kubenswrapper[2562]: I0729 23:04:18.356071 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-z2kfm/perf-node-gather-daemonset-z4v2h"] Jul 29 23:04:18.356534 ip-10-0-138-112 kubenswrapper[2562]: I0729 23:04:18.356380 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a50338ed-263f-4855-aa0a-53fb0cd72ed7" containerName="console" Jul 29 23:04:18.356534 ip-10-0-138-112 kubenswrapper[2562]: I0729 23:04:18.356390 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="a50338ed-263f-4855-aa0a-53fb0cd72ed7" containerName="console" Jul 29 23:04:18.356534 ip-10-0-138-112 kubenswrapper[2562]: I0729 23:04:18.356464 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="a50338ed-263f-4855-aa0a-53fb0cd72ed7" containerName="console" Jul 29 23:04:18.359414 ip-10-0-138-112 kubenswrapper[2562]: I0729 23:04:18.359377 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-z2kfm/perf-node-gather-daemonset-z4v2h" Jul 29 23:04:18.362003 ip-10-0-138-112 kubenswrapper[2562]: I0729 23:04:18.361975 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-z2kfm\"/\"kube-root-ca.crt\"" Jul 29 23:04:18.363359 ip-10-0-138-112 kubenswrapper[2562]: I0729 23:04:18.363344 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-z2kfm\"/\"default-dockercfg-m4vvg\"" Jul 29 23:04:18.363433 ip-10-0-138-112 kubenswrapper[2562]: I0729 23:04:18.363381 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-z2kfm\"/\"openshift-service-ca.crt\"" Jul 29 23:04:18.368113 ip-10-0-138-112 kubenswrapper[2562]: I0729 23:04:18.368065 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-z2kfm/perf-node-gather-daemonset-z4v2h"] Jul 29 23:04:18.515755 ip-10-0-138-112 kubenswrapper[2562]: I0729 23:04:18.515709 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/5d57895d-f637-452a-888e-7eeb1e05cdd0-proc\") pod \"perf-node-gather-daemonset-z4v2h\" (UID: \"5d57895d-f637-452a-888e-7eeb1e05cdd0\") " pod="openshift-must-gather-z2kfm/perf-node-gather-daemonset-z4v2h" Jul 29 23:04:18.515755 ip-10-0-138-112 kubenswrapper[2562]: I0729 23:04:18.515752 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-clcll\" (UniqueName: \"kubernetes.io/projected/5d57895d-f637-452a-888e-7eeb1e05cdd0-kube-api-access-clcll\") pod \"perf-node-gather-daemonset-z4v2h\" (UID: \"5d57895d-f637-452a-888e-7eeb1e05cdd0\") " pod="openshift-must-gather-z2kfm/perf-node-gather-daemonset-z4v2h" Jul 29 23:04:18.515978 ip-10-0-138-112 kubenswrapper[2562]: I0729 23:04:18.515782 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/5d57895d-f637-452a-888e-7eeb1e05cdd0-podres\") pod \"perf-node-gather-daemonset-z4v2h\" (UID: \"5d57895d-f637-452a-888e-7eeb1e05cdd0\") " pod="openshift-must-gather-z2kfm/perf-node-gather-daemonset-z4v2h" Jul 29 23:04:18.515978 ip-10-0-138-112 kubenswrapper[2562]: I0729 23:04:18.515819 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5d57895d-f637-452a-888e-7eeb1e05cdd0-sys\") pod \"perf-node-gather-daemonset-z4v2h\" (UID: \"5d57895d-f637-452a-888e-7eeb1e05cdd0\") " pod="openshift-must-gather-z2kfm/perf-node-gather-daemonset-z4v2h" Jul 29 23:04:18.515978 ip-10-0-138-112 kubenswrapper[2562]: I0729 23:04:18.515857 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/5d57895d-f637-452a-888e-7eeb1e05cdd0-lib-modules\") pod \"perf-node-gather-daemonset-z4v2h\" (UID: \"5d57895d-f637-452a-888e-7eeb1e05cdd0\") " pod="openshift-must-gather-z2kfm/perf-node-gather-daemonset-z4v2h" Jul 29 23:04:18.616503 ip-10-0-138-112 kubenswrapper[2562]: I0729 23:04:18.616390 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/5d57895d-f637-452a-888e-7eeb1e05cdd0-lib-modules\") pod \"perf-node-gather-daemonset-z4v2h\" (UID: \"5d57895d-f637-452a-888e-7eeb1e05cdd0\") " pod="openshift-must-gather-z2kfm/perf-node-gather-daemonset-z4v2h" Jul 29 23:04:18.616503 ip-10-0-138-112 kubenswrapper[2562]: I0729 23:04:18.616480 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/5d57895d-f637-452a-888e-7eeb1e05cdd0-proc\") pod \"perf-node-gather-daemonset-z4v2h\" (UID: \"5d57895d-f637-452a-888e-7eeb1e05cdd0\") " pod="openshift-must-gather-z2kfm/perf-node-gather-daemonset-z4v2h" Jul 29 23:04:18.616503 ip-10-0-138-112 kubenswrapper[2562]: I0729 23:04:18.616499 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-clcll\" (UniqueName: \"kubernetes.io/projected/5d57895d-f637-452a-888e-7eeb1e05cdd0-kube-api-access-clcll\") pod \"perf-node-gather-daemonset-z4v2h\" (UID: \"5d57895d-f637-452a-888e-7eeb1e05cdd0\") " pod="openshift-must-gather-z2kfm/perf-node-gather-daemonset-z4v2h" Jul 29 23:04:18.616701 ip-10-0-138-112 kubenswrapper[2562]: I0729 23:04:18.616519 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/5d57895d-f637-452a-888e-7eeb1e05cdd0-podres\") pod \"perf-node-gather-daemonset-z4v2h\" (UID: \"5d57895d-f637-452a-888e-7eeb1e05cdd0\") " pod="openshift-must-gather-z2kfm/perf-node-gather-daemonset-z4v2h" Jul 29 23:04:18.616701 ip-10-0-138-112 kubenswrapper[2562]: I0729 23:04:18.616542 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5d57895d-f637-452a-888e-7eeb1e05cdd0-sys\") pod \"perf-node-gather-daemonset-z4v2h\" (UID: \"5d57895d-f637-452a-888e-7eeb1e05cdd0\") " pod="openshift-must-gather-z2kfm/perf-node-gather-daemonset-z4v2h" Jul 29 23:04:18.616701 ip-10-0-138-112 kubenswrapper[2562]: I0729 23:04:18.616581 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/5d57895d-f637-452a-888e-7eeb1e05cdd0-lib-modules\") pod \"perf-node-gather-daemonset-z4v2h\" (UID: \"5d57895d-f637-452a-888e-7eeb1e05cdd0\") " pod="openshift-must-gather-z2kfm/perf-node-gather-daemonset-z4v2h" Jul 29 23:04:18.616701 ip-10-0-138-112 kubenswrapper[2562]: I0729 23:04:18.616628 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5d57895d-f637-452a-888e-7eeb1e05cdd0-sys\") pod \"perf-node-gather-daemonset-z4v2h\" (UID: \"5d57895d-f637-452a-888e-7eeb1e05cdd0\") " pod="openshift-must-gather-z2kfm/perf-node-gather-daemonset-z4v2h" Jul 29 23:04:18.616701 ip-10-0-138-112 kubenswrapper[2562]: I0729 23:04:18.616631 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/5d57895d-f637-452a-888e-7eeb1e05cdd0-proc\") pod \"perf-node-gather-daemonset-z4v2h\" (UID: \"5d57895d-f637-452a-888e-7eeb1e05cdd0\") " pod="openshift-must-gather-z2kfm/perf-node-gather-daemonset-z4v2h" Jul 29 23:04:18.616701 ip-10-0-138-112 kubenswrapper[2562]: I0729 23:04:18.616639 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/5d57895d-f637-452a-888e-7eeb1e05cdd0-podres\") pod \"perf-node-gather-daemonset-z4v2h\" (UID: \"5d57895d-f637-452a-888e-7eeb1e05cdd0\") " pod="openshift-must-gather-z2kfm/perf-node-gather-daemonset-z4v2h" Jul 29 23:04:18.624808 ip-10-0-138-112 kubenswrapper[2562]: I0729 23:04:18.624786 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-clcll\" (UniqueName: \"kubernetes.io/projected/5d57895d-f637-452a-888e-7eeb1e05cdd0-kube-api-access-clcll\") pod \"perf-node-gather-daemonset-z4v2h\" (UID: \"5d57895d-f637-452a-888e-7eeb1e05cdd0\") " pod="openshift-must-gather-z2kfm/perf-node-gather-daemonset-z4v2h" Jul 29 23:04:18.669870 ip-10-0-138-112 kubenswrapper[2562]: I0729 23:04:18.669835 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-z2kfm/perf-node-gather-daemonset-z4v2h" Jul 29 23:04:18.795042 ip-10-0-138-112 kubenswrapper[2562]: I0729 23:04:18.795009 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-z2kfm/perf-node-gather-daemonset-z4v2h"] Jul 29 23:04:18.798188 ip-10-0-138-112 kubenswrapper[2562]: W0729 23:04:18.798161 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod5d57895d_f637_452a_888e_7eeb1e05cdd0.slice/crio-6053b9f9e808a85810a0b177cb16d717f66b072a2de2f9769bbbd91f6fd93f53 WatchSource:0}: Error finding container 6053b9f9e808a85810a0b177cb16d717f66b072a2de2f9769bbbd91f6fd93f53: Status 404 returned error can't find the container with id 6053b9f9e808a85810a0b177cb16d717f66b072a2de2f9769bbbd91f6fd93f53 Jul 29 23:04:18.932962 ip-10-0-138-112 kubenswrapper[2562]: I0729 23:04:18.932932 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-z2kfm/perf-node-gather-daemonset-z4v2h" event={"ID":"5d57895d-f637-452a-888e-7eeb1e05cdd0","Type":"ContainerStarted","Data":"f20ba2f83a1d27260f4bf952bacc34e12de1b2aa8085ede8dadbc8e856f9366c"} Jul 29 23:04:18.933106 ip-10-0-138-112 kubenswrapper[2562]: I0729 23:04:18.933042 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-z2kfm/perf-node-gather-daemonset-z4v2h" Jul 29 23:04:18.933106 ip-10-0-138-112 kubenswrapper[2562]: I0729 23:04:18.933069 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-z2kfm/perf-node-gather-daemonset-z4v2h" event={"ID":"5d57895d-f637-452a-888e-7eeb1e05cdd0","Type":"ContainerStarted","Data":"6053b9f9e808a85810a0b177cb16d717f66b072a2de2f9769bbbd91f6fd93f53"} Jul 29 23:04:18.953221 ip-10-0-138-112 kubenswrapper[2562]: I0729 23:04:18.953165 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-z2kfm/perf-node-gather-daemonset-z4v2h" podStartSLOduration=0.953150463 podStartE2EDuration="953.150463ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 23:04:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 23:04:18.950540151 +0000 UTC m=+1073.862365839" watchObservedRunningTime="2026-07-29 23:04:18.953150463 +0000 UTC m=+1073.864976178" Jul 29 23:04:19.183157 ip-10-0-138-112 kubenswrapper[2562]: I0729 23:04:19.183047 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-ncs4k_cb3010ad-c703-4901-979e-88a3e1b07ec3/dns/0.log" Jul 29 23:04:19.205365 ip-10-0-138-112 kubenswrapper[2562]: I0729 23:04:19.205324 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-ncs4k_cb3010ad-c703-4901-979e-88a3e1b07ec3/kube-rbac-proxy/0.log" Jul 29 23:04:19.368574 ip-10-0-138-112 kubenswrapper[2562]: I0729 23:04:19.368548 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-k8fl9_51d8cffe-16b4-4bab-9193-953163e45417/dns-node-resolver/0.log" Jul 29 23:04:19.849136 ip-10-0-138-112 kubenswrapper[2562]: I0729 23:04:19.849093 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-7f7448c9db-mqm8x_e40c1be2-77bb-4d00-971f-a99e666826eb/registry/0.log" Jul 29 23:04:19.922781 ip-10-0-138-112 kubenswrapper[2562]: I0729 23:04:19.922752 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-vwtj2_f6d8c777-cb9a-4505-8e30-ffaa01e53239/node-ca/0.log" Jul 29 23:04:20.570858 ip-10-0-138-112 kubenswrapper[2562]: I0729 23:04:20.570826 2562 scope.go:117] "RemoveContainer" containerID="2312686273fc0156575f3b8d3483858b96661665b16c04c93364738d690cdeb9" Jul 29 23:04:20.571309 ip-10-0-138-112 kubenswrapper[2562]: E0729 23:04:20.571008 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 5m0s restarting failed container=manager pod=llmisvc-controller-manager-654c557856-k2ld6_kserve(43ca558c-d85d-426c-9560-c71ccdefcae1)\"" pod="kserve/llmisvc-controller-manager-654c557856-k2ld6" podUID="43ca558c-d85d-426c-9560-c71ccdefcae1" Jul 29 23:04:20.990037 ip-10-0-138-112 kubenswrapper[2562]: I0729 23:04:20.989925 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-c5tdl_9b18dc65-d191-4eef-b23e-14561cbe95f1/serve-healthcheck-canary/0.log" Jul 29 23:04:21.474171 ip-10-0-138-112 kubenswrapper[2562]: I0729 23:04:21.474124 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-rhx9q_ad207e3b-5129-47d5-9151-d39346f17cdd/kube-rbac-proxy/0.log" Jul 29 23:04:21.496285 ip-10-0-138-112 kubenswrapper[2562]: I0729 23:04:21.496254 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-rhx9q_ad207e3b-5129-47d5-9151-d39346f17cdd/exporter/0.log" Jul 29 23:04:21.517067 ip-10-0-138-112 kubenswrapper[2562]: I0729 23:04:21.517032 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-rhx9q_ad207e3b-5129-47d5-9151-d39346f17cdd/extractor/0.log" Jul 29 23:04:23.803683 ip-10-0-138-112 kubenswrapper[2562]: I0729 23:04:23.803646 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve_kserve-controller-manager-844cf74686-242x8_23059c0f-1b26-41dc-9031-e96ddce46577/manager/0.log" Jul 29 23:04:23.830455 ip-10-0-138-112 kubenswrapper[2562]: I0729 23:04:23.830409 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve_llmisvc-controller-manager-654c557856-k2ld6_43ca558c-d85d-426c-9560-c71ccdefcae1/manager/6.log" Jul 29 23:04:23.830455 ip-10-0-138-112 kubenswrapper[2562]: I0729 23:04:23.830419 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve_llmisvc-controller-manager-654c557856-k2ld6_43ca558c-d85d-426c-9560-c71ccdefcae1/manager/6.log" Jul 29 23:04:23.850780 ip-10-0-138-112 kubenswrapper[2562]: I0729 23:04:23.850754 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve_seaweedfs-64568bcd49-6wt5l_a02e53c9-9d46-4f90-a406-0fbb66307fa8/seaweedfs/0.log" Jul 29 23:04:24.946134 ip-10-0-138-112 kubenswrapper[2562]: I0729 23:04:24.946107 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-z2kfm/perf-node-gather-daemonset-z4v2h" Jul 29 23:04:29.167367 ip-10-0-138-112 kubenswrapper[2562]: I0729 23:04:29.167335 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-2mwmf_6a64a07e-1c4a-4990-90b9-80623be37f60/kube-multus/0.log" Jul 29 23:04:29.382937 ip-10-0-138-112 kubenswrapper[2562]: I0729 23:04:29.382907 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-h6flc_d40a4cdf-f141-46d3-8159-986f680b2dec/kube-multus-additional-cni-plugins/0.log" Jul 29 23:04:29.411966 ip-10-0-138-112 kubenswrapper[2562]: I0729 23:04:29.411936 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-h6flc_d40a4cdf-f141-46d3-8159-986f680b2dec/egress-router-binary-copy/0.log" Jul 29 23:04:29.436863 ip-10-0-138-112 kubenswrapper[2562]: I0729 23:04:29.436776 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-h6flc_d40a4cdf-f141-46d3-8159-986f680b2dec/cni-plugins/0.log" Jul 29 23:04:29.457674 ip-10-0-138-112 kubenswrapper[2562]: I0729 23:04:29.457647 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-h6flc_d40a4cdf-f141-46d3-8159-986f680b2dec/bond-cni-plugin/0.log" Jul 29 23:04:29.478147 ip-10-0-138-112 kubenswrapper[2562]: I0729 23:04:29.478120 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-h6flc_d40a4cdf-f141-46d3-8159-986f680b2dec/routeoverride-cni/0.log" Jul 29 23:04:29.498608 ip-10-0-138-112 kubenswrapper[2562]: I0729 23:04:29.498576 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-h6flc_d40a4cdf-f141-46d3-8159-986f680b2dec/whereabouts-cni-bincopy/0.log" Jul 29 23:04:29.522096 ip-10-0-138-112 kubenswrapper[2562]: I0729 23:04:29.522066 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-h6flc_d40a4cdf-f141-46d3-8159-986f680b2dec/whereabouts-cni/0.log" Jul 29 23:04:29.739138 ip-10-0-138-112 kubenswrapper[2562]: I0729 23:04:29.739058 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-fxb52_f0a6baed-8bea-4992-8852-0b84808a78d1/network-metrics-daemon/0.log" Jul 29 23:04:29.758267 ip-10-0-138-112 kubenswrapper[2562]: I0729 23:04:29.758244 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-fxb52_f0a6baed-8bea-4992-8852-0b84808a78d1/kube-rbac-proxy/0.log" Jul 29 23:04:30.826765 ip-10-0-138-112 kubenswrapper[2562]: I0729 23:04:30.826727 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9g28r_6050ad9f-817a-48e4-b939-12efa11a6a73/ovn-controller/0.log" Jul 29 23:04:30.846083 ip-10-0-138-112 kubenswrapper[2562]: I0729 23:04:30.846046 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9g28r_6050ad9f-817a-48e4-b939-12efa11a6a73/ovn-acl-logging/0.log" Jul 29 23:04:30.872795 ip-10-0-138-112 kubenswrapper[2562]: I0729 23:04:30.872767 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9g28r_6050ad9f-817a-48e4-b939-12efa11a6a73/kube-rbac-proxy-node/0.log" Jul 29 23:04:30.894779 ip-10-0-138-112 kubenswrapper[2562]: I0729 23:04:30.894750 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9g28r_6050ad9f-817a-48e4-b939-12efa11a6a73/kube-rbac-proxy-ovn-metrics/0.log" Jul 29 23:04:30.914721 ip-10-0-138-112 kubenswrapper[2562]: I0729 23:04:30.914694 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9g28r_6050ad9f-817a-48e4-b939-12efa11a6a73/northd/0.log" Jul 29 23:04:30.945015 ip-10-0-138-112 kubenswrapper[2562]: I0729 23:04:30.944984 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9g28r_6050ad9f-817a-48e4-b939-12efa11a6a73/nbdb/0.log" Jul 29 23:04:30.972762 ip-10-0-138-112 kubenswrapper[2562]: I0729 23:04:30.972733 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9g28r_6050ad9f-817a-48e4-b939-12efa11a6a73/sbdb/0.log" Jul 29 23:04:31.010243 ip-10-0-138-112 kubenswrapper[2562]: I0729 23:04:31.010210 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9g28r_6050ad9f-817a-48e4-b939-12efa11a6a73/ovnkube-controller/0.log" Jul 29 23:04:32.273563 ip-10-0-138-112 kubenswrapper[2562]: I0729 23:04:32.273533 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-diagnostics_network-check-target-z92kj_2771bdbc-d054-464b-a01f-82344308451f/network-check-target-container/0.log" Jul 29 23:04:33.197244 ip-10-0-138-112 kubenswrapper[2562]: I0729 23:04:33.197214 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-operator_iptables-alerter-frd9g_5ce0a47c-e715-472b-a0d9-761a5a6777dd/iptables-alerter/0.log" Jul 29 23:04:33.811328 ip-10-0-138-112 kubenswrapper[2562]: I0729 23:04:33.811299 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-node-tuning-operator_tuned-84tng_4c31fa1a-fda4-43d6-84ef-e43f0bafdcb2/tuned/0.log" Jul 29 23:04:34.570134 ip-10-0-138-112 kubenswrapper[2562]: I0729 23:04:34.570097 2562 scope.go:117] "RemoveContainer" containerID="2312686273fc0156575f3b8d3483858b96661665b16c04c93364738d690cdeb9" Jul 29 23:04:34.570389 ip-10-0-138-112 kubenswrapper[2562]: I0729 23:04:34.570376 2562 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider