Jul 27 09:24:31.923093 ip-10-0-128-103 systemd[1]: Starting Kubernetes Kubelet... Jul 27 09:24:32.300927 ip-10-0-128-103 kubenswrapper[2565]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 27 09:24:32.300927 ip-10-0-128-103 kubenswrapper[2565]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 27 09:24:32.300927 ip-10-0-128-103 kubenswrapper[2565]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 27 09:24:32.300927 ip-10-0-128-103 kubenswrapper[2565]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 27 09:24:32.300927 ip-10-0-128-103 kubenswrapper[2565]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 27 09:24:32.301604 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:32.300925 2565 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 27 09:24:32.306614 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.306597 2565 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 27 09:24:32.306614 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.306613 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 27 09:24:32.306683 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.306619 2565 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 27 09:24:32.306683 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.306624 2565 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 27 09:24:32.306683 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.306627 2565 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 27 09:24:32.306683 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.306630 2565 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 27 09:24:32.306683 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.306633 2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 27 09:24:32.306683 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.306637 2565 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 27 09:24:32.306683 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.306639 2565 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 27 09:24:32.306683 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.306642 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 27 09:24:32.306683 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.306644 2565 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 27 09:24:32.306683 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.306647 2565 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 27 09:24:32.306683 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.306650 2565 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 27 09:24:32.306683 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.306652 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 27 09:24:32.306683 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.306655 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 27 09:24:32.306683 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.306658 2565 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 27 09:24:32.306683 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.306668 2565 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 27 09:24:32.306683 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.306671 2565 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 27 09:24:32.306683 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.306673 2565 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 27 09:24:32.306683 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.306676 2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 27 09:24:32.307131 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.306679 2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 27 09:24:32.307131 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.306682 2565 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 27 09:24:32.307131 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.306685 2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 27 09:24:32.307131 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.306687 2565 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 27 09:24:32.307131 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.306690 2565 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 27 09:24:32.307131 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.306693 2565 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 27 09:24:32.307131 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.306696 2565 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 27 09:24:32.307131 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.306698 2565 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 27 09:24:32.307131 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.306701 2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 27 09:24:32.307131 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.306703 2565 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 27 09:24:32.307131 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.306706 2565 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 27 09:24:32.307131 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.306708 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 27 09:24:32.307131 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.306711 2565 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 27 09:24:32.307131 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.306713 2565 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 27 09:24:32.307131 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.306716 2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 27 09:24:32.307131 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.306718 2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 27 09:24:32.307131 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.306720 2565 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 27 09:24:32.307131 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.306723 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 27 09:24:32.307131 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.306726 2565 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 27 09:24:32.307131 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.306728 2565 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 27 09:24:32.307609 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.306731 2565 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 27 09:24:32.307609 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.306733 2565 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 27 09:24:32.307609 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.306736 2565 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 27 09:24:32.307609 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.306738 2565 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 27 09:24:32.307609 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.306741 2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 27 09:24:32.307609 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.306745 2565 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 27 09:24:32.307609 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.306748 2565 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 27 09:24:32.307609 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.306751 2565 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 27 09:24:32.307609 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.306753 2565 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 27 09:24:32.307609 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.306756 2565 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 27 09:24:32.307609 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.306758 2565 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 27 09:24:32.307609 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.306761 2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 27 09:24:32.307609 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.306783 2565 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 27 09:24:32.307609 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.306787 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 27 09:24:32.307609 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.306792 2565 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 27 09:24:32.307609 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.306796 2565 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 27 09:24:32.307609 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.306799 2565 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 27 09:24:32.307609 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.306801 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 27 09:24:32.307609 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.306804 2565 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 27 09:24:32.307609 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.306806 2565 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 27 09:24:32.308110 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.306809 2565 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 27 09:24:32.308110 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.306812 2565 feature_gate.go:328] unrecognized feature gate: Example Jul 27 09:24:32.308110 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.306814 2565 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 27 09:24:32.308110 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.306817 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 27 09:24:32.308110 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.306819 2565 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 27 09:24:32.308110 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.306822 2565 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 27 09:24:32.308110 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.306824 2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 27 09:24:32.308110 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.306827 2565 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 27 09:24:32.308110 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.306829 2565 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 27 09:24:32.308110 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.306832 2565 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 27 09:24:32.308110 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.306834 2565 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 27 09:24:32.308110 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.306838 2565 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 27 09:24:32.308110 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.306840 2565 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 27 09:24:32.308110 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.306843 2565 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 27 09:24:32.308110 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.306845 2565 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 27 09:24:32.308110 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.306848 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 27 09:24:32.308110 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.306850 2565 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 27 09:24:32.308110 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.306853 2565 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 27 09:24:32.308110 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.306856 2565 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 27 09:24:32.308110 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.306858 2565 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 27 09:24:32.308588 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.306860 2565 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 27 09:24:32.308588 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.306863 2565 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 27 09:24:32.308588 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.306865 2565 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 27 09:24:32.308588 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.306868 2565 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 27 09:24:32.308588 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.306873 2565 feature_gate.go:328] unrecognized feature gate: Example2 Jul 27 09:24:32.308588 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.306876 2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 27 09:24:32.308588 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.306879 2565 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 27 09:24:32.308588 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.306881 2565 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 27 09:24:32.308588 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.306884 2565 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 27 09:24:32.308588 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.306887 2565 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 27 09:24:32.308588 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.306889 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 27 09:24:32.308588 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.306892 2565 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 27 09:24:32.308588 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.306894 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 27 09:24:32.308588 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.306897 2565 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 27 09:24:32.308588 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.306900 2565 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 27 09:24:32.308588 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.306902 2565 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 27 09:24:32.308588 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.306906 2565 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 27 09:24:32.308588 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.306909 2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 27 09:24:32.308588 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.306912 2565 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 27 09:24:32.308588 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.307629 2565 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 27 09:24:32.309125 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.307636 2565 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 27 09:24:32.309125 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.307639 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 27 09:24:32.309125 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.307642 2565 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 27 09:24:32.309125 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.307644 2565 feature_gate.go:328] unrecognized feature gate: Example2 Jul 27 09:24:32.309125 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.307648 2565 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 27 09:24:32.309125 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.307651 2565 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 27 09:24:32.309125 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.307654 2565 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 27 09:24:32.309125 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.307656 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 27 09:24:32.309125 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.307659 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 27 09:24:32.309125 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.307662 2565 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 27 09:24:32.309125 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.307664 2565 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 27 09:24:32.309125 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.307667 2565 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 27 09:24:32.309125 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.307669 2565 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 27 09:24:32.309125 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.307672 2565 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 27 09:24:32.309125 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.307674 2565 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 27 09:24:32.309125 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.307678 2565 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 27 09:24:32.309125 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.307687 2565 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 27 09:24:32.309125 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.307692 2565 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 27 09:24:32.309125 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.307696 2565 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 27 09:24:32.309125 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.307720 2565 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 27 09:24:32.309632 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.307742 2565 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 27 09:24:32.309632 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.307746 2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 27 09:24:32.309632 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.307751 2565 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 27 09:24:32.309632 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.307755 2565 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 27 09:24:32.309632 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.307758 2565 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 27 09:24:32.309632 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.307761 2565 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 27 09:24:32.309632 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.307778 2565 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 27 09:24:32.309632 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.307782 2565 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 27 09:24:32.309632 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.307786 2565 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 27 09:24:32.309632 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.307789 2565 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 27 09:24:32.309632 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.307792 2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 27 09:24:32.309632 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.307795 2565 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 27 09:24:32.309632 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.307798 2565 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 27 09:24:32.309632 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.307800 2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 27 09:24:32.309632 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.307803 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 27 09:24:32.309632 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.307806 2565 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 27 09:24:32.309632 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.307809 2565 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 27 09:24:32.309632 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.307812 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 27 09:24:32.309632 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.307816 2565 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 27 09:24:32.310109 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.307821 2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 27 09:24:32.310109 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.307824 2565 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 27 09:24:32.310109 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.307827 2565 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 27 09:24:32.310109 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.307832 2565 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 27 09:24:32.310109 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.307836 2565 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 27 09:24:32.310109 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.307839 2565 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 27 09:24:32.310109 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.307841 2565 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 27 09:24:32.310109 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.307844 2565 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 27 09:24:32.310109 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.307847 2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 27 09:24:32.310109 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.307850 2565 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 27 09:24:32.310109 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.307853 2565 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 27 09:24:32.310109 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.307855 2565 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 27 09:24:32.310109 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.307858 2565 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 27 09:24:32.310109 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.307863 2565 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 27 09:24:32.310109 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.307867 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 27 09:24:32.310109 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.307870 2565 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 27 09:24:32.310109 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.307873 2565 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 27 09:24:32.310109 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.307876 2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 27 09:24:32.310109 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.307879 2565 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 27 09:24:32.310109 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.307881 2565 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 27 09:24:32.310601 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.307884 2565 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 27 09:24:32.310601 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.307886 2565 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 27 09:24:32.310601 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.307889 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 27 09:24:32.310601 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.307892 2565 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 27 09:24:32.310601 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.307894 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 27 09:24:32.310601 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.307897 2565 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 27 09:24:32.310601 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.307899 2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 27 09:24:32.310601 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.307902 2565 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 27 09:24:32.310601 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.307904 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 27 09:24:32.310601 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.307907 2565 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 27 09:24:32.310601 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.307910 2565 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 27 09:24:32.310601 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.307912 2565 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 27 09:24:32.310601 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.307915 2565 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 27 09:24:32.310601 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.307917 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 27 09:24:32.310601 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.307920 2565 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 27 09:24:32.310601 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.307922 2565 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 27 09:24:32.310601 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.307925 2565 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 27 09:24:32.310601 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.307927 2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 27 09:24:32.310601 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.307930 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 27 09:24:32.310601 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.307932 2565 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 27 09:24:32.311104 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.307935 2565 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 27 09:24:32.311104 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.307939 2565 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 27 09:24:32.311104 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.307943 2565 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 27 09:24:32.311104 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.307947 2565 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 27 09:24:32.311104 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.307950 2565 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 27 09:24:32.311104 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.307954 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 27 09:24:32.311104 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.307957 2565 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 27 09:24:32.311104 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.307960 2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 27 09:24:32.311104 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.307963 2565 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 27 09:24:32.311104 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.307966 2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 27 09:24:32.311104 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.307968 2565 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 27 09:24:32.311104 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.307971 2565 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 27 09:24:32.311104 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.307974 2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 27 09:24:32.311104 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.307976 2565 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 27 09:24:32.311104 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.307979 2565 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 27 09:24:32.311104 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.307981 2565 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 27 09:24:32.311104 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.307984 2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 27 09:24:32.311104 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.307986 2565 feature_gate.go:328] unrecognized feature gate: Example Jul 27 09:24:32.311104 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.307989 2565 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 27 09:24:32.311577 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.308387 2565 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 27 09:24:32.311577 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.308393 2565 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 27 09:24:32.311577 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.308396 2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 27 09:24:32.311577 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.308399 2565 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 27 09:24:32.311577 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.308402 2565 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 27 09:24:32.311577 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.308404 2565 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 27 09:24:32.311577 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.308407 2565 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 27 09:24:32.311577 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.308410 2565 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 27 09:24:32.311577 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.308413 2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 27 09:24:32.311577 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.308415 2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 27 09:24:32.311577 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.308418 2565 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 27 09:24:32.311577 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.308421 2565 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 27 09:24:32.311577 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.308424 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 27 09:24:32.311577 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.308426 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 27 09:24:32.311577 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.308431 2565 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 27 09:24:32.311577 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.308434 2565 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 27 09:24:32.311577 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.308437 2565 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 27 09:24:32.311577 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.308440 2565 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 27 09:24:32.311577 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.308443 2565 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 27 09:24:32.312046 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.308446 2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 27 09:24:32.312046 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.308449 2565 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 27 09:24:32.312046 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.308452 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 27 09:24:32.312046 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.308456 2565 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 27 09:24:32.312046 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.308460 2565 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 27 09:24:32.312046 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.308463 2565 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 27 09:24:32.312046 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.308466 2565 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 27 09:24:32.312046 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.308469 2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 27 09:24:32.312046 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.308471 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 27 09:24:32.312046 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.308474 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 27 09:24:32.312046 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.308476 2565 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 27 09:24:32.312046 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.308479 2565 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 27 09:24:32.312046 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.308481 2565 feature_gate.go:328] unrecognized feature gate: Example2 Jul 27 09:24:32.312046 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.308484 2565 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 27 09:24:32.312046 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.308487 2565 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 27 09:24:32.312046 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.308489 2565 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 27 09:24:32.312046 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.308492 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 27 09:24:32.312046 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.308495 2565 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 27 09:24:32.312046 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.308497 2565 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 27 09:24:32.312512 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.308500 2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 27 09:24:32.312512 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.308503 2565 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 27 09:24:32.312512 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.308505 2565 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 27 09:24:32.312512 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.308508 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 27 09:24:32.312512 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.308511 2565 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 27 09:24:32.312512 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.308513 2565 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 27 09:24:32.312512 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.308516 2565 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 27 09:24:32.312512 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.308519 2565 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 27 09:24:32.312512 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.308521 2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 27 09:24:32.312512 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.308524 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 27 09:24:32.312512 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.308527 2565 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 27 09:24:32.312512 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.308530 2565 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 27 09:24:32.312512 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.308532 2565 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 27 09:24:32.312512 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.308535 2565 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 27 09:24:32.312512 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.308537 2565 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 27 09:24:32.312512 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.308540 2565 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 27 09:24:32.312512 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.308543 2565 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 27 09:24:32.312512 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.308546 2565 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 27 09:24:32.312512 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.308548 2565 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 27 09:24:32.312512 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.308551 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 27 09:24:32.313011 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.308553 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 27 09:24:32.313011 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.308557 2565 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 27 09:24:32.313011 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.308559 2565 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 27 09:24:32.313011 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.308562 2565 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 27 09:24:32.313011 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.308564 2565 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 27 09:24:32.313011 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.308567 2565 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 27 09:24:32.313011 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.308569 2565 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 27 09:24:32.313011 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.308572 2565 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 27 09:24:32.313011 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.308574 2565 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 27 09:24:32.313011 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.308577 2565 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 27 09:24:32.313011 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.308579 2565 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 27 09:24:32.313011 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.308582 2565 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 27 09:24:32.313011 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.308584 2565 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 27 09:24:32.313011 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.308587 2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 27 09:24:32.313011 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.308589 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 27 09:24:32.313011 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.308592 2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 27 09:24:32.313011 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.308597 2565 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 27 09:24:32.313011 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.308600 2565 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 27 09:24:32.313011 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.308602 2565 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 27 09:24:32.313011 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.308606 2565 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 27 09:24:32.313501 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.308609 2565 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 27 09:24:32.313501 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.308611 2565 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 27 09:24:32.313501 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.308614 2565 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 27 09:24:32.313501 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.308617 2565 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 27 09:24:32.313501 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.308619 2565 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 27 09:24:32.313501 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.308622 2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 27 09:24:32.313501 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.308624 2565 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 27 09:24:32.313501 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.308627 2565 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 27 09:24:32.313501 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.308630 2565 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 27 09:24:32.313501 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.308633 2565 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 27 09:24:32.313501 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.308635 2565 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 27 09:24:32.313501 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.308638 2565 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 27 09:24:32.313501 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.308640 2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 27 09:24:32.313501 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.308643 2565 feature_gate.go:328] unrecognized feature gate: Example Jul 27 09:24:32.313501 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.308645 2565 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 27 09:24:32.313501 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.308648 2565 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 27 09:24:32.313501 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.308650 2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 27 09:24:32.313501 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.308653 2565 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 27 09:24:32.313501 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.308655 2565 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 27 09:24:32.313501 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.308658 2565 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 27 09:24:32.314040 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.308660 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 27 09:24:32.314040 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:32.308736 2565 flags.go:64] FLAG: --address="0.0.0.0" Jul 27 09:24:32.314040 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:32.308743 2565 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 27 09:24:32.314040 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:32.308750 2565 flags.go:64] FLAG: --anonymous-auth="true" Jul 27 09:24:32.314040 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:32.308755 2565 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 27 09:24:32.314040 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:32.308759 2565 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 27 09:24:32.314040 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:32.308775 2565 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 27 09:24:32.314040 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:32.308781 2565 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 27 09:24:32.314040 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:32.308786 2565 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 27 09:24:32.314040 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:32.308790 2565 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 27 09:24:32.314040 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:32.308793 2565 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 27 09:24:32.314040 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:32.308796 2565 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 27 09:24:32.314040 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:32.308799 2565 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 27 09:24:32.314040 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:32.308803 2565 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 27 09:24:32.314040 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:32.308806 2565 flags.go:64] FLAG: --cgroup-root="" Jul 27 09:24:32.314040 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:32.308809 2565 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 27 09:24:32.314040 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:32.308812 2565 flags.go:64] FLAG: --client-ca-file="" Jul 27 09:24:32.314040 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:32.308815 2565 flags.go:64] FLAG: --cloud-provider="external" Jul 27 09:24:32.314040 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:32.308818 2565 flags.go:64] FLAG: --cluster-dns="[]" Jul 27 09:24:32.314040 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:32.308822 2565 flags.go:64] FLAG: --cluster-domain="" Jul 27 09:24:32.314040 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:32.308825 2565 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 27 09:24:32.314040 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:32.308828 2565 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 27 09:24:32.314040 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:32.308831 2565 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 27 09:24:32.314040 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:32.308835 2565 flags.go:64] FLAG: --container-log-max-files="5" Jul 27 09:24:32.314603 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:32.308838 2565 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 27 09:24:32.314603 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:32.308841 2565 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 27 09:24:32.314603 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:32.308844 2565 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 27 09:24:32.314603 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:32.308848 2565 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 27 09:24:32.314603 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:32.308851 2565 flags.go:64] FLAG: --contention-profiling="false" Jul 27 09:24:32.314603 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:32.308854 2565 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 27 09:24:32.314603 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:32.308857 2565 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 27 09:24:32.314603 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:32.308860 2565 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 27 09:24:32.314603 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:32.308862 2565 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 27 09:24:32.314603 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:32.308867 2565 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 27 09:24:32.314603 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:32.308870 2565 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 27 09:24:32.314603 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:32.308872 2565 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 27 09:24:32.314603 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:32.308875 2565 flags.go:64] FLAG: --enable-load-reader="false" Jul 27 09:24:32.314603 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:32.308878 2565 flags.go:64] FLAG: --enable-server="true" Jul 27 09:24:32.314603 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:32.308881 2565 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 27 09:24:32.314603 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:32.308886 2565 flags.go:64] FLAG: --event-burst="100" Jul 27 09:24:32.314603 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:32.308891 2565 flags.go:64] FLAG: --event-qps="50" Jul 27 09:24:32.314603 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:32.308894 2565 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 27 09:24:32.314603 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:32.308897 2565 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 27 09:24:32.314603 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:32.308900 2565 flags.go:64] FLAG: --eviction-hard="" Jul 27 09:24:32.314603 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:32.308903 2565 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 27 09:24:32.314603 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:32.308908 2565 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 27 09:24:32.314603 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:32.308911 2565 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 27 09:24:32.314603 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:32.308914 2565 flags.go:64] FLAG: --eviction-soft="" Jul 27 09:24:32.314603 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:32.308917 2565 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 27 09:24:32.315223 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:32.308920 2565 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 27 09:24:32.315223 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:32.308923 2565 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 27 09:24:32.315223 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:32.308926 2565 flags.go:64] FLAG: --experimental-mounter-path="" Jul 27 09:24:32.315223 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:32.308929 2565 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 27 09:24:32.315223 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:32.308932 2565 flags.go:64] FLAG: --fail-swap-on="true" Jul 27 09:24:32.315223 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:32.308936 2565 flags.go:64] FLAG: --feature-gates="" Jul 27 09:24:32.315223 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:32.308940 2565 flags.go:64] FLAG: --file-check-frequency="20s" Jul 27 09:24:32.315223 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:32.308943 2565 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 27 09:24:32.315223 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:32.308946 2565 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 27 09:24:32.315223 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:32.308949 2565 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 27 09:24:32.315223 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:32.308952 2565 flags.go:64] FLAG: --healthz-port="10248" Jul 27 09:24:32.315223 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:32.308955 2565 flags.go:64] FLAG: --help="false" Jul 27 09:24:32.315223 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:32.308958 2565 flags.go:64] FLAG: --hostname-override="ip-10-0-128-103.ec2.internal" Jul 27 09:24:32.315223 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:32.308961 2565 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 27 09:24:32.315223 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:32.308964 2565 flags.go:64] FLAG: --http-check-frequency="20s" Jul 27 09:24:32.315223 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:32.308967 2565 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 27 09:24:32.315223 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:32.308970 2565 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 27 09:24:32.315223 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:32.308974 2565 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 27 09:24:32.315223 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:32.308977 2565 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 27 09:24:32.315223 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:32.308979 2565 flags.go:64] FLAG: --image-service-endpoint="" Jul 27 09:24:32.315223 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:32.308982 2565 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 27 09:24:32.315223 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:32.308985 2565 flags.go:64] FLAG: --kube-api-burst="100" Jul 27 09:24:32.315223 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:32.308988 2565 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 27 09:24:32.315864 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:32.308992 2565 flags.go:64] FLAG: --kube-api-qps="50" Jul 27 09:24:32.315864 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:32.308995 2565 flags.go:64] FLAG: --kube-reserved="" Jul 27 09:24:32.315864 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:32.308998 2565 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 27 09:24:32.315864 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:32.309001 2565 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 27 09:24:32.315864 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:32.309004 2565 flags.go:64] FLAG: --kubelet-cgroups="" Jul 27 09:24:32.315864 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:32.309009 2565 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 27 09:24:32.315864 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:32.309012 2565 flags.go:64] FLAG: --lock-file="" Jul 27 09:24:32.315864 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:32.309014 2565 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 27 09:24:32.315864 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:32.309017 2565 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 27 09:24:32.315864 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:32.309020 2565 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 27 09:24:32.315864 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:32.309026 2565 flags.go:64] FLAG: --log-json-split-stream="false" Jul 27 09:24:32.315864 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:32.309029 2565 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 27 09:24:32.315864 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:32.309032 2565 flags.go:64] FLAG: --log-text-split-stream="false" Jul 27 09:24:32.315864 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:32.309034 2565 flags.go:64] FLAG: --logging-format="text" Jul 27 09:24:32.315864 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:32.309037 2565 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 27 09:24:32.315864 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:32.309041 2565 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 27 09:24:32.315864 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:32.309044 2565 flags.go:64] FLAG: --manifest-url="" Jul 27 09:24:32.315864 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:32.309047 2565 flags.go:64] FLAG: --manifest-url-header="" Jul 27 09:24:32.315864 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:32.309052 2565 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 27 09:24:32.315864 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:32.309054 2565 flags.go:64] FLAG: --max-open-files="1000000" Jul 27 09:24:32.315864 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:32.309058 2565 flags.go:64] FLAG: --max-pods="110" Jul 27 09:24:32.315864 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:32.309061 2565 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 27 09:24:32.315864 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:32.309064 2565 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 27 09:24:32.315864 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:32.309067 2565 flags.go:64] FLAG: --memory-manager-policy="None" Jul 27 09:24:32.315864 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:32.309070 2565 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 27 09:24:32.316473 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:32.309073 2565 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 27 09:24:32.316473 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:32.309076 2565 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 27 09:24:32.316473 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:32.309079 2565 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 27 09:24:32.316473 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:32.309086 2565 flags.go:64] FLAG: --node-status-max-images="50" Jul 27 09:24:32.316473 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:32.309089 2565 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 27 09:24:32.316473 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:32.309092 2565 flags.go:64] FLAG: --oom-score-adj="-999" Jul 27 09:24:32.316473 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:32.309095 2565 flags.go:64] FLAG: --pod-cidr="" Jul 27 09:24:32.316473 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:32.309099 2565 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7505336a406394b8c1c77d14d890a9f628aa4cfff172312179cb9af8c28c30bb" Jul 27 09:24:32.316473 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:32.309104 2565 flags.go:64] FLAG: --pod-manifest-path="" Jul 27 09:24:32.316473 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:32.309107 2565 flags.go:64] FLAG: --pod-max-pids="-1" Jul 27 09:24:32.316473 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:32.309111 2565 flags.go:64] FLAG: --pods-per-core="0" Jul 27 09:24:32.316473 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:32.309114 2565 flags.go:64] FLAG: --port="10250" Jul 27 09:24:32.316473 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:32.309117 2565 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 27 09:24:32.316473 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:32.309123 2565 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0ae59e138865c3890" Jul 27 09:24:32.316473 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:32.309126 2565 flags.go:64] FLAG: --qos-reserved="" Jul 27 09:24:32.316473 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:32.309129 2565 flags.go:64] FLAG: --read-only-port="10255" Jul 27 09:24:32.316473 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:32.309132 2565 flags.go:64] FLAG: --register-node="true" Jul 27 09:24:32.316473 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:32.309135 2565 flags.go:64] FLAG: --register-with-taints="" Jul 27 09:24:32.316473 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:32.309139 2565 flags.go:64] FLAG: --registry-burst="10" Jul 27 09:24:32.316473 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:32.309157 2565 flags.go:64] FLAG: --registry-qps="5" Jul 27 09:24:32.316473 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:32.309161 2565 flags.go:64] FLAG: --reserved-cpus="" Jul 27 09:24:32.316473 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:32.309164 2565 flags.go:64] FLAG: --reserved-memory="" Jul 27 09:24:32.316473 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:32.309168 2565 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 27 09:24:32.316473 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:32.309171 2565 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 27 09:24:32.316473 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:32.309175 2565 flags.go:64] FLAG: --rotate-certificates="false" Jul 27 09:24:32.317131 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:32.309178 2565 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 27 09:24:32.317131 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:32.309180 2565 flags.go:64] FLAG: --runonce="false" Jul 27 09:24:32.317131 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:32.309183 2565 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 27 09:24:32.317131 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:32.309186 2565 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 27 09:24:32.317131 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:32.309189 2565 flags.go:64] FLAG: --seccomp-default="false" Jul 27 09:24:32.317131 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:32.309192 2565 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 27 09:24:32.317131 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:32.309195 2565 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 27 09:24:32.317131 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:32.309198 2565 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 27 09:24:32.317131 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:32.309201 2565 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 27 09:24:32.317131 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:32.309204 2565 flags.go:64] FLAG: --storage-driver-password="root" Jul 27 09:24:32.317131 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:32.309207 2565 flags.go:64] FLAG: --storage-driver-secure="false" Jul 27 09:24:32.317131 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:32.309210 2565 flags.go:64] FLAG: --storage-driver-table="stats" Jul 27 09:24:32.317131 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:32.309212 2565 flags.go:64] FLAG: --storage-driver-user="root" Jul 27 09:24:32.317131 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:32.309215 2565 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 27 09:24:32.317131 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:32.309220 2565 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 27 09:24:32.317131 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:32.309223 2565 flags.go:64] FLAG: --system-cgroups="" Jul 27 09:24:32.317131 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:32.309226 2565 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 27 09:24:32.317131 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:32.309231 2565 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 27 09:24:32.317131 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:32.309234 2565 flags.go:64] FLAG: --tls-cert-file="" Jul 27 09:24:32.317131 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:32.309237 2565 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 27 09:24:32.317131 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:32.309243 2565 flags.go:64] FLAG: --tls-min-version="" Jul 27 09:24:32.317131 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:32.309246 2565 flags.go:64] FLAG: --tls-private-key-file="" Jul 27 09:24:32.317131 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:32.309249 2565 flags.go:64] FLAG: --topology-manager-policy="none" Jul 27 09:24:32.317131 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:32.309251 2565 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 27 09:24:32.317131 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:32.309254 2565 flags.go:64] FLAG: --topology-manager-scope="container" Jul 27 09:24:32.317739 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:32.309257 2565 flags.go:64] FLAG: --v="2" Jul 27 09:24:32.317739 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:32.309261 2565 flags.go:64] FLAG: --version="false" Jul 27 09:24:32.317739 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:32.309265 2565 flags.go:64] FLAG: --vmodule="" Jul 27 09:24:32.317739 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:32.309268 2565 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 27 09:24:32.317739 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:32.309271 2565 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 27 09:24:32.317739 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.309384 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 27 09:24:32.317739 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.309388 2565 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 27 09:24:32.317739 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.309391 2565 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 27 09:24:32.317739 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.309394 2565 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 27 09:24:32.317739 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.309398 2565 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 27 09:24:32.317739 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.309401 2565 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 27 09:24:32.317739 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.309404 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 27 09:24:32.317739 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.309407 2565 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 27 09:24:32.317739 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.309409 2565 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 27 09:24:32.317739 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.309412 2565 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 27 09:24:32.317739 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.309415 2565 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 27 09:24:32.317739 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.309417 2565 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 27 09:24:32.317739 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.309420 2565 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 27 09:24:32.317739 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.309422 2565 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 27 09:24:32.317739 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.309425 2565 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 27 09:24:32.317739 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.309427 2565 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 27 09:24:32.318440 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.309432 2565 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 27 09:24:32.318440 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.309435 2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 27 09:24:32.318440 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.309437 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 27 09:24:32.318440 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.309440 2565 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 27 09:24:32.318440 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.309443 2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 27 09:24:32.318440 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.309445 2565 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 27 09:24:32.318440 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.309449 2565 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 27 09:24:32.318440 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.309451 2565 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 27 09:24:32.318440 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.309454 2565 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 27 09:24:32.318440 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.309456 2565 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 27 09:24:32.318440 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.309459 2565 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 27 09:24:32.318440 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.309461 2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 27 09:24:32.318440 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.309464 2565 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 27 09:24:32.318440 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.309467 2565 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 27 09:24:32.318440 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.309469 2565 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 27 09:24:32.318440 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.309472 2565 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 27 09:24:32.318440 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.309474 2565 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 27 09:24:32.318440 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.309477 2565 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 27 09:24:32.318440 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.309480 2565 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 27 09:24:32.319147 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.309482 2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 27 09:24:32.319147 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.309485 2565 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 27 09:24:32.319147 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.309488 2565 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 27 09:24:32.319147 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.309490 2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 27 09:24:32.319147 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.309493 2565 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 27 09:24:32.319147 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.309495 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 27 09:24:32.319147 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.309498 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 27 09:24:32.319147 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.309500 2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 27 09:24:32.319147 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.309503 2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 27 09:24:32.319147 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.309507 2565 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 27 09:24:32.319147 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.309511 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 27 09:24:32.319147 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.309514 2565 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 27 09:24:32.319147 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.309517 2565 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 27 09:24:32.319147 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.309521 2565 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 27 09:24:32.319147 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.309525 2565 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 27 09:24:32.319147 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.309528 2565 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 27 09:24:32.319147 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.309530 2565 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 27 09:24:32.319147 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.309534 2565 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 27 09:24:32.319147 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.309538 2565 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 27 09:24:32.319900 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.309542 2565 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 27 09:24:32.319900 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.309545 2565 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 27 09:24:32.319900 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.309548 2565 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 27 09:24:32.319900 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.309551 2565 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 27 09:24:32.319900 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.309553 2565 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 27 09:24:32.319900 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.309556 2565 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 27 09:24:32.319900 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.309558 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 27 09:24:32.319900 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.309561 2565 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 27 09:24:32.319900 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.309564 2565 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 27 09:24:32.319900 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.309566 2565 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 27 09:24:32.319900 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.309569 2565 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 27 09:24:32.319900 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.309572 2565 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 27 09:24:32.319900 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.309582 2565 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 27 09:24:32.319900 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.309586 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 27 09:24:32.319900 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.309589 2565 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 27 09:24:32.319900 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.309591 2565 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 27 09:24:32.319900 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.309594 2565 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 27 09:24:32.319900 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.309597 2565 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 27 09:24:32.319900 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.309600 2565 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 27 09:24:32.319900 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.309602 2565 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 27 09:24:32.320401 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.309605 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 27 09:24:32.320401 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.309607 2565 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 27 09:24:32.320401 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.309610 2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 27 09:24:32.320401 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.309613 2565 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 27 09:24:32.320401 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.309615 2565 feature_gate.go:328] unrecognized feature gate: Example2 Jul 27 09:24:32.320401 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.309618 2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 27 09:24:32.320401 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.309622 2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 27 09:24:32.320401 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.309624 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 27 09:24:32.320401 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.309627 2565 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 27 09:24:32.320401 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.309629 2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 27 09:24:32.320401 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.309632 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 27 09:24:32.320401 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.309635 2565 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 27 09:24:32.320401 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.309638 2565 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 27 09:24:32.320401 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.309641 2565 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 27 09:24:32.320401 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.309644 2565 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 27 09:24:32.320401 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.309661 2565 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 27 09:24:32.320401 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.309664 2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 27 09:24:32.320401 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.309667 2565 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 27 09:24:32.320401 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.309670 2565 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 27 09:24:32.320401 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.309673 2565 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 27 09:24:32.320944 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.309676 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 27 09:24:32.320944 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.309679 2565 feature_gate.go:328] unrecognized feature gate: Example Jul 27 09:24:32.320944 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.309682 2565 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 27 09:24:32.320944 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.309685 2565 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 27 09:24:32.320944 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.309687 2565 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 27 09:24:32.320944 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:32.310323 2565 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 27 09:24:32.320944 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:32.316721 2565 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Jul 27 09:24:32.320944 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:32.316741 2565 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 27 09:24:32.320944 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:32.317438 2565 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 27 09:24:32.320944 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:32.317446 2565 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 27 09:24:32.320944 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.317505 2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 27 09:24:32.320944 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.317511 2565 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 27 09:24:32.320944 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.317515 2565 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 27 09:24:32.320944 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.317519 2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 27 09:24:32.320944 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.317522 2565 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 27 09:24:32.321323 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.317525 2565 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 27 09:24:32.321323 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.317528 2565 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 27 09:24:32.321323 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.317531 2565 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 27 09:24:32.321323 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.317534 2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 27 09:24:32.321323 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.317536 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 27 09:24:32.321323 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.317539 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 27 09:24:32.321323 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.317542 2565 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 27 09:24:32.321323 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.317544 2565 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 27 09:24:32.321323 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.317547 2565 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 27 09:24:32.321323 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.317550 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 27 09:24:32.321323 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.317552 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 27 09:24:32.321323 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.317555 2565 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 27 09:24:32.321323 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.317557 2565 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 27 09:24:32.321323 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.317560 2565 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 27 09:24:32.321323 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.317563 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 27 09:24:32.321323 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.317565 2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 27 09:24:32.321323 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.317568 2565 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 27 09:24:32.321323 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.317570 2565 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 27 09:24:32.321323 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.317580 2565 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 27 09:24:32.321323 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.317584 2565 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 27 09:24:32.321821 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.317586 2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 27 09:24:32.321821 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.317589 2565 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 27 09:24:32.321821 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.317591 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 27 09:24:32.321821 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.317594 2565 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 27 09:24:32.321821 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.317599 2565 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 27 09:24:32.321821 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.317602 2565 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 27 09:24:32.321821 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.317605 2565 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 27 09:24:32.321821 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.317608 2565 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 27 09:24:32.321821 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.317611 2565 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 27 09:24:32.321821 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.317613 2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 27 09:24:32.321821 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.317616 2565 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 27 09:24:32.321821 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.317618 2565 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 27 09:24:32.321821 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.317621 2565 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 27 09:24:32.321821 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.317623 2565 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 27 09:24:32.321821 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.317626 2565 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 27 09:24:32.321821 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.317628 2565 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 27 09:24:32.321821 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.317631 2565 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 27 09:24:32.321821 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.317633 2565 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 27 09:24:32.321821 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.317636 2565 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 27 09:24:32.321821 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.317638 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 27 09:24:32.322314 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.317642 2565 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 27 09:24:32.322314 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.317644 2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 27 09:24:32.322314 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.317647 2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 27 09:24:32.322314 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.317650 2565 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 27 09:24:32.322314 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.317653 2565 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 27 09:24:32.322314 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.317655 2565 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 27 09:24:32.322314 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.317658 2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 27 09:24:32.322314 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.317660 2565 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 27 09:24:32.322314 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.317662 2565 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 27 09:24:32.322314 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.317667 2565 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 27 09:24:32.322314 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.317670 2565 feature_gate.go:328] unrecognized feature gate: Example2 Jul 27 09:24:32.322314 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.317673 2565 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 27 09:24:32.322314 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.317676 2565 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 27 09:24:32.322314 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.317678 2565 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 27 09:24:32.322314 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.317681 2565 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 27 09:24:32.322314 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.317685 2565 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 27 09:24:32.322314 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.317689 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 27 09:24:32.322314 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.317693 2565 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 27 09:24:32.322314 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.317696 2565 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 27 09:24:32.322902 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.317698 2565 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 27 09:24:32.322902 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.317701 2565 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 27 09:24:32.322902 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.317704 2565 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 27 09:24:32.322902 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.317706 2565 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 27 09:24:32.322902 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.317709 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 27 09:24:32.322902 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.317711 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 27 09:24:32.322902 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.317714 2565 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 27 09:24:32.322902 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.317716 2565 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 27 09:24:32.322902 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.317720 2565 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 27 09:24:32.322902 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.317723 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 27 09:24:32.322902 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.317726 2565 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 27 09:24:32.322902 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.317729 2565 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 27 09:24:32.322902 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.317731 2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 27 09:24:32.322902 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.317734 2565 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 27 09:24:32.322902 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.317736 2565 feature_gate.go:328] unrecognized feature gate: Example Jul 27 09:24:32.322902 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.317739 2565 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 27 09:24:32.322902 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.317741 2565 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 27 09:24:32.322902 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.317744 2565 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 27 09:24:32.322902 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.317747 2565 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 27 09:24:32.322902 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.317750 2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 27 09:24:32.323385 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.317752 2565 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 27 09:24:32.323385 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.317755 2565 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 27 09:24:32.323385 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.317758 2565 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 27 09:24:32.323385 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.317761 2565 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 27 09:24:32.323385 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.317777 2565 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 27 09:24:32.323385 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.317781 2565 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 27 09:24:32.323385 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.317784 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 27 09:24:32.323385 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.317787 2565 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 27 09:24:32.323385 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.317789 2565 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 27 09:24:32.323385 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.317793 2565 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 27 09:24:32.323385 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.317795 2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 27 09:24:32.323385 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.317799 2565 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 27 09:24:32.323385 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.317802 2565 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 27 09:24:32.323385 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.317805 2565 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 27 09:24:32.323385 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.317807 2565 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 27 09:24:32.323826 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:32.317813 2565 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 27 09:24:32.323826 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.318083 2565 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 27 09:24:32.323826 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.318112 2565 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 27 09:24:32.323826 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.318117 2565 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 27 09:24:32.323826 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.318122 2565 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 27 09:24:32.323826 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.318126 2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 27 09:24:32.323826 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.318141 2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 27 09:24:32.323826 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.318146 2565 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 27 09:24:32.323826 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.318152 2565 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 27 09:24:32.323826 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.318159 2565 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 27 09:24:32.323826 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.318165 2565 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 27 09:24:32.323826 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.318170 2565 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 27 09:24:32.323826 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.319203 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 27 09:24:32.323826 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.319217 2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 27 09:24:32.323826 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.319223 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 27 09:24:32.324204 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.319228 2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 27 09:24:32.324204 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.319236 2565 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 27 09:24:32.324204 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.319676 2565 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 27 09:24:32.324204 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.319685 2565 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 27 09:24:32.324204 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.319689 2565 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 27 09:24:32.324204 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.319693 2565 feature_gate.go:328] unrecognized feature gate: Example Jul 27 09:24:32.324204 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.319697 2565 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 27 09:24:32.324204 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.319701 2565 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 27 09:24:32.324204 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.319704 2565 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 27 09:24:32.324204 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.319707 2565 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 27 09:24:32.324204 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.319710 2565 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 27 09:24:32.324204 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.319713 2565 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 27 09:24:32.324204 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.319717 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 27 09:24:32.324204 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.319720 2565 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 27 09:24:32.324204 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.319727 2565 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 27 09:24:32.324204 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.319730 2565 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 27 09:24:32.324204 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.319733 2565 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 27 09:24:32.324204 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.319736 2565 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 27 09:24:32.324204 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.319739 2565 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 27 09:24:32.324668 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.319742 2565 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 27 09:24:32.324668 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.319745 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 27 09:24:32.324668 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.319748 2565 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 27 09:24:32.324668 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.319751 2565 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 27 09:24:32.324668 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.319754 2565 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 27 09:24:32.324668 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.319757 2565 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 27 09:24:32.324668 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.319775 2565 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 27 09:24:32.324668 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.319782 2565 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 27 09:24:32.324668 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.319789 2565 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 27 09:24:32.324668 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.319792 2565 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 27 09:24:32.324668 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.319796 2565 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 27 09:24:32.324668 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.319799 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 27 09:24:32.324668 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.319911 2565 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 27 09:24:32.324668 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.319917 2565 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 27 09:24:32.324668 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.319920 2565 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 27 09:24:32.324668 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.319923 2565 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 27 09:24:32.324668 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.319926 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 27 09:24:32.324668 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.319928 2565 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 27 09:24:32.324668 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.319931 2565 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 27 09:24:32.324668 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.319934 2565 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 27 09:24:32.325191 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.319937 2565 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 27 09:24:32.325191 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.319939 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 27 09:24:32.325191 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.319942 2565 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 27 09:24:32.325191 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.319944 2565 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 27 09:24:32.325191 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.319947 2565 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 27 09:24:32.325191 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.319949 2565 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 27 09:24:32.325191 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.319952 2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 27 09:24:32.325191 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.319955 2565 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 27 09:24:32.325191 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.319958 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 27 09:24:32.325191 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.319960 2565 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 27 09:24:32.325191 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.319963 2565 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 27 09:24:32.325191 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.319966 2565 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 27 09:24:32.325191 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.319969 2565 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 27 09:24:32.325191 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.319972 2565 feature_gate.go:328] unrecognized feature gate: Example2 Jul 27 09:24:32.325191 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.319978 2565 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 27 09:24:32.325191 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.319981 2565 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 27 09:24:32.325191 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.319983 2565 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 27 09:24:32.325191 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.319986 2565 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 27 09:24:32.325191 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.319988 2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 27 09:24:32.325678 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.319991 2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 27 09:24:32.325678 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.319994 2565 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 27 09:24:32.325678 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.319996 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 27 09:24:32.325678 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.319999 2565 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 27 09:24:32.325678 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.320002 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 27 09:24:32.325678 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.320005 2565 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 27 09:24:32.325678 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.320007 2565 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 27 09:24:32.325678 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.320010 2565 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 27 09:24:32.325678 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.320013 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 27 09:24:32.325678 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.320016 2565 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 27 09:24:32.325678 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.320018 2565 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 27 09:24:32.325678 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.320021 2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 27 09:24:32.325678 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.320024 2565 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 27 09:24:32.325678 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.320026 2565 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 27 09:24:32.325678 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.320029 2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 27 09:24:32.325678 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.320032 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 27 09:24:32.325678 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.320035 2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 27 09:24:32.325678 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.320037 2565 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 27 09:24:32.325678 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.320040 2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 27 09:24:32.325678 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.320043 2565 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 27 09:24:32.326295 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.320045 2565 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 27 09:24:32.326295 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.320048 2565 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 27 09:24:32.326295 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.320050 2565 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 27 09:24:32.326295 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.320054 2565 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 27 09:24:32.326295 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.320057 2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 27 09:24:32.326295 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.320060 2565 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 27 09:24:32.326295 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:32.320063 2565 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 27 09:24:32.326295 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:32.320070 2565 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 27 09:24:32.326295 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:32.320222 2565 server.go:962] "Client rotation is on, will bootstrap in background" Jul 27 09:24:32.326295 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:32.322899 2565 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 27 09:24:32.326295 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:32.324058 2565 server.go:1019] "Starting client certificate rotation" Jul 27 09:24:32.326295 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:32.324153 2565 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 27 09:24:32.326295 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:32.324194 2565 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 27 09:24:32.326630 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:32.326518 2565 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 27 09:24:32.327797 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:32.327685 2565 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 27 09:24:32.338323 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:32.338304 2565 log.go:25] "Validated CRI v1 runtime API" Jul 27 09:24:32.341377 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:32.341362 2565 log.go:25] "Validated CRI v1 image API" Jul 27 09:24:32.342142 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:32.342123 2565 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 27 09:24:32.344435 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:32.344408 2565 fs.go:135] Filesystem UUIDs: map[690e5169-d9c8-494c-982a-c2b2d7d5c938:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 9e90398e-d48c-4ffe-865b-5366294bc2b6:/dev/nvme0n1p4] Jul 27 09:24:32.344522 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:32.344434 2565 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 27 09:24:32.349422 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:32.349275 2565 manager.go:216] Machine: {Timestamp:2026-07-27 09:24:32.348082319 +0000 UTC m=+0.311868287 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3117165 MemoryCapacity:33164484608 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec20c7573f78308d767a2986bf1ca266 SystemUUID:ec20c757-3f78-308d-767a-2986bf1ca266 BootID:e56cae04-d4b4-409a-a8b5-bc0900296cba Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582242304 Type:vfs Inodes:4048399 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582242304 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:ad:2d:83:d3:c9 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:ad:2d:83:d3:c9 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:ce:9f:12:55:bd:1a Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164484608 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 27 09:24:32.349422 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:32.349409 2565 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 27 09:24:32.349597 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:32.349520 2565 manager.go:232] Version: {KernelVersion:5.14.0-570.129.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260717-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 27 09:24:32.352997 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:32.352968 2565 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 27 09:24:32.353138 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:32.352998 2565 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-128-103.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 27 09:24:32.353530 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:32.353509 2565 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 27 09:24:32.353916 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:32.353904 2565 topology_manager.go:138] "Creating topology manager with none policy" Jul 27 09:24:32.353961 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:32.353918 2565 container_manager_linux.go:309] "Creating device plugin manager" Jul 27 09:24:32.353961 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:32.353926 2565 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 27 09:24:32.354683 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:32.354672 2565 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 27 09:24:32.355377 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:32.355367 2565 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 27 09:24:32.357050 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:32.357039 2565 state_mem.go:36] "Initialized new in-memory state store" Jul 27 09:24:32.357174 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:32.357165 2565 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 27 09:24:32.357210 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:32.357195 2565 kubelet.go:486] "Attempting to sync node with API server" Jul 27 09:24:32.357210 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:32.357204 2565 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 27 09:24:32.357263 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:32.357217 2565 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 27 09:24:32.357263 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:32.357227 2565 kubelet.go:398] "Adding apiserver pod source" Jul 27 09:24:32.357263 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:32.357237 2565 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 27 09:24:32.358195 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:32.358172 2565 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 27 09:24:32.358277 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:32.358219 2565 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 27 09:24:32.360583 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:32.360564 2565 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1" Jul 27 09:24:32.362607 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:32.362590 2565 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 27 09:24:32.362667 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:32.362624 2565 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 27 09:24:32.364180 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:32.364167 2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 27 09:24:32.364180 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:32.364183 2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 27 09:24:32.364280 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:32.364190 2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 27 09:24:32.364280 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:32.364196 2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 27 09:24:32.364280 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:32.364205 2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 27 09:24:32.364280 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:32.364211 2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 27 09:24:32.364280 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:32.364219 2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 27 09:24:32.364280 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:32.364225 2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 27 09:24:32.364280 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:32.364232 2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 27 09:24:32.364280 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:32.364237 2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 27 09:24:32.364280 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:32.364246 2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 27 09:24:32.364280 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:32.364251 2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 27 09:24:32.364859 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:32.364849 2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 27 09:24:32.364859 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:32.364858 2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 27 09:24:32.368037 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:32.368022 2565 server.go:1268] "Started kubelet" Jul 27 09:24:32.368144 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:32.368093 2565 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 27 09:24:32.368220 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:32.368172 2565 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 27 09:24:32.368271 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:32.368244 2565 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 27 09:24:32.368912 ip-10-0-128-103 kubenswrapper[2565]: E0727 09:24:32.368886 2565 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 27 09:24:32.368992 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:32.368957 2565 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-128-103.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 27 09:24:32.369049 ip-10-0-128-103 systemd[1]: Started Kubernetes Kubelet. Jul 27 09:24:32.369221 ip-10-0-128-103 kubenswrapper[2565]: E0727 09:24:32.369054 2565 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-128-103.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 27 09:24:32.369828 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:32.369812 2565 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 27 09:24:32.370164 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:32.370149 2565 server.go:310] "Adding debug handlers to kubelet server" Jul 27 09:24:32.373286 ip-10-0-128-103 kubenswrapper[2565]: E0727 09:24:32.372418 2565 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-128-103.ec2.internal.18c61b41f35148ab default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-128-103.ec2.internal,UID:ip-10-0-128-103.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-128-103.ec2.internal,},FirstTimestamp:2026-07-27 09:24:32.367995051 +0000 UTC m=+0.331781000,LastTimestamp:2026-07-27 09:24:32.367995051 +0000 UTC m=+0.331781000,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-128-103.ec2.internal,}" Jul 27 09:24:32.374718 ip-10-0-128-103 kubenswrapper[2565]: E0727 09:24:32.374699 2565 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 27 09:24:32.377076 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:32.377062 2565 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 27 09:24:32.377143 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:32.377086 2565 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 27 09:24:32.377618 ip-10-0-128-103 kubenswrapper[2565]: E0727 09:24:32.377229 2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-103.ec2.internal\" not found" Jul 27 09:24:32.377618 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:32.377432 2565 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-rgfpf" Jul 27 09:24:32.377782 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:32.377727 2565 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 27 09:24:32.377871 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:32.377852 2565 factory.go:55] Registering systemd factory Jul 27 09:24:32.377916 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:32.377872 2565 factory.go:223] Registration of the systemd container factory successfully Jul 27 09:24:32.377987 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:32.377974 2565 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 27 09:24:32.378021 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:32.377990 2565 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 27 09:24:32.378076 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:32.378050 2565 reconstruct.go:97] "Volume reconstruction finished" Jul 27 09:24:32.378076 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:32.378058 2565 reconciler.go:29] "Reconciler: start to sync state" Jul 27 09:24:32.378166 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:32.378092 2565 factory.go:153] Registering CRI-O factory Jul 27 09:24:32.378166 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:32.378107 2565 factory.go:223] Registration of the crio container factory successfully Jul 27 09:24:32.378166 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:32.378156 2565 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 27 09:24:32.378303 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:32.378183 2565 factory.go:103] Registering Raw factory Jul 27 09:24:32.378303 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:32.378199 2565 manager.go:1196] Started watching for new ooms in manager Jul 27 09:24:32.378544 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:32.378529 2565 manager.go:349] Starting recovery of all containers Jul 27 09:24:32.379451 ip-10-0-128-103 kubenswrapper[2565]: E0727 09:24:32.379428 2565 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 27 09:24:32.379658 ip-10-0-128-103 kubenswrapper[2565]: E0727 09:24:32.379622 2565 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-128-103.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 27 09:24:32.386843 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:32.386811 2565 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-rgfpf" Jul 27 09:24:32.390009 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:32.389992 2565 manager.go:354] Recovery completed Jul 27 09:24:32.394479 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:32.394465 2565 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 27 09:24:32.396390 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:32.396372 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-103.ec2.internal" event="NodeHasSufficientMemory" Jul 27 09:24:32.396476 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:32.396407 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-103.ec2.internal" event="NodeHasNoDiskPressure" Jul 27 09:24:32.396476 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:32.396422 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-103.ec2.internal" event="NodeHasSufficientPID" Jul 27 09:24:32.397064 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:32.397048 2565 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 27 09:24:32.397064 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:32.397063 2565 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 27 09:24:32.397201 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:32.397081 2565 state_mem.go:36] "Initialized new in-memory state store" Jul 27 09:24:32.397201 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:32.397080 2565 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 27 09:24:32.397826 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:32.397804 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-103.ec2.internal" event="NodeHasSufficientMemory" Jul 27 09:24:32.397920 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:32.397840 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-103.ec2.internal" event="NodeHasNoDiskPressure" Jul 27 09:24:32.397920 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:32.397855 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-103.ec2.internal" event="NodeHasSufficientPID" Jul 27 09:24:32.398698 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:32.398687 2565 policy_none.go:49] "None policy: Start" Jul 27 09:24:32.398747 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:32.398703 2565 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 27 09:24:32.398747 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:32.398714 2565 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 27 09:24:32.399572 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:32.399560 2565 policy_none.go:47] "Start" Jul 27 09:24:32.424646 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:32.424446 2565 manager.go:337] "Starting Device Plugin manager" Jul 27 09:24:32.424756 ip-10-0-128-103 kubenswrapper[2565]: E0727 09:24:32.424686 2565 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 27 09:24:32.424756 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:32.424700 2565 server.go:87] "Starting device plugin registration server" Jul 27 09:24:32.424969 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:32.424958 2565 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 27 09:24:32.425002 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:32.424972 2565 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 27 09:24:32.425075 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:32.425059 2565 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 27 09:24:32.425182 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:32.425165 2565 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 27 09:24:32.425182 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:32.425181 2565 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 27 09:24:32.425721 ip-10-0-128-103 kubenswrapper[2565]: E0727 09:24:32.425688 2565 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 27 09:24:32.425822 ip-10-0-128-103 kubenswrapper[2565]: E0727 09:24:32.425760 2565 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-128-103.ec2.internal\" not found" Jul 27 09:24:32.445139 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:32.445116 2565 container_manager_linux.go:676] "Failed to ensure state" containerName="/system.slice" err="failed to move PID 2598 into the system container \"/system.slice\": " Jul 27 09:24:32.525272 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:32.525224 2565 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 27 09:24:32.526328 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:32.526311 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-103.ec2.internal" event="NodeHasSufficientMemory" Jul 27 09:24:32.526402 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:32.526346 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-103.ec2.internal" event="NodeHasNoDiskPressure" Jul 27 09:24:32.526402 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:32.526362 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-103.ec2.internal" event="NodeHasSufficientPID" Jul 27 09:24:32.526402 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:32.526396 2565 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-128-103.ec2.internal" Jul 27 09:24:32.536964 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:32.536942 2565 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-128-103.ec2.internal" Jul 27 09:24:32.537102 ip-10-0-128-103 kubenswrapper[2565]: E0727 09:24:32.536972 2565 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-128-103.ec2.internal\": node \"ip-10-0-128-103.ec2.internal\" not found" Jul 27 09:24:32.555498 ip-10-0-128-103 kubenswrapper[2565]: E0727 09:24:32.555429 2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-103.ec2.internal\" not found" Jul 27 09:24:32.555840 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:32.555818 2565 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 27 09:24:32.557124 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:32.557098 2565 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 27 09:24:32.557229 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:32.557139 2565 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 27 09:24:32.557229 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:32.557162 2565 kubelet.go:2448] "Starting kubelet main sync loop" Jul 27 09:24:32.557229 ip-10-0-128-103 kubenswrapper[2565]: E0727 09:24:32.557194 2565 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 27 09:24:32.560135 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:32.560116 2565 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 27 09:24:32.655840 ip-10-0-128-103 kubenswrapper[2565]: E0727 09:24:32.655804 2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-103.ec2.internal\" not found" Jul 27 09:24:32.657963 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:32.657928 2565 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-103.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-128-103.ec2.internal"] Jul 27 09:24:32.658106 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:32.658028 2565 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 27 09:24:32.659249 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:32.659233 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-103.ec2.internal" event="NodeHasSufficientMemory" Jul 27 09:24:32.659347 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:32.659267 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-103.ec2.internal" event="NodeHasNoDiskPressure" Jul 27 09:24:32.659347 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:32.659298 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-103.ec2.internal" event="NodeHasSufficientPID" Jul 27 09:24:32.660625 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:32.660610 2565 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 27 09:24:32.660800 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:32.660787 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-103.ec2.internal" Jul 27 09:24:32.660891 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:32.660823 2565 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 27 09:24:32.661364 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:32.661349 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-103.ec2.internal" event="NodeHasSufficientMemory" Jul 27 09:24:32.661433 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:32.661377 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-103.ec2.internal" event="NodeHasNoDiskPressure" Jul 27 09:24:32.661433 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:32.661387 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-103.ec2.internal" event="NodeHasSufficientPID" Jul 27 09:24:32.661505 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:32.661463 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-103.ec2.internal" event="NodeHasSufficientMemory" Jul 27 09:24:32.661505 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:32.661488 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-103.ec2.internal" event="NodeHasNoDiskPressure" Jul 27 09:24:32.661505 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:32.661499 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-103.ec2.internal" event="NodeHasSufficientPID" Jul 27 09:24:32.662778 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:32.662746 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-103.ec2.internal" Jul 27 09:24:32.662842 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:32.662789 2565 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 27 09:24:32.663545 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:32.663530 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-103.ec2.internal" event="NodeHasSufficientMemory" Jul 27 09:24:32.663618 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:32.663562 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-103.ec2.internal" event="NodeHasNoDiskPressure" Jul 27 09:24:32.663618 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:32.663572 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-103.ec2.internal" event="NodeHasSufficientPID" Jul 27 09:24:32.679316 ip-10-0-128-103 kubenswrapper[2565]: E0727 09:24:32.679285 2565 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-128-103.ec2.internal\" not found" node="ip-10-0-128-103.ec2.internal" Jul 27 09:24:32.679997 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:32.679964 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/f9f6fe57433891594e1efa7d6875336f-config\") pod \"kube-apiserver-proxy-ip-10-0-128-103.ec2.internal\" (UID: \"f9f6fe57433891594e1efa7d6875336f\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-103.ec2.internal" Jul 27 09:24:32.680100 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:32.680032 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/519b669acb80623490c3cff7ef66ed98-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-103.ec2.internal\" (UID: \"519b669acb80623490c3cff7ef66ed98\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-103.ec2.internal" Jul 27 09:24:32.680100 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:32.680068 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/519b669acb80623490c3cff7ef66ed98-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-103.ec2.internal\" (UID: \"519b669acb80623490c3cff7ef66ed98\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-103.ec2.internal" Jul 27 09:24:32.684658 ip-10-0-128-103 kubenswrapper[2565]: E0727 09:24:32.684639 2565 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-128-103.ec2.internal\" not found" node="ip-10-0-128-103.ec2.internal" Jul 27 09:24:32.756057 ip-10-0-128-103 kubenswrapper[2565]: E0727 09:24:32.756027 2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-103.ec2.internal\" not found" Jul 27 09:24:32.780822 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:32.780798 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/519b669acb80623490c3cff7ef66ed98-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-103.ec2.internal\" (UID: \"519b669acb80623490c3cff7ef66ed98\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-103.ec2.internal" Jul 27 09:24:32.780884 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:32.780834 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/f9f6fe57433891594e1efa7d6875336f-config\") pod \"kube-apiserver-proxy-ip-10-0-128-103.ec2.internal\" (UID: \"f9f6fe57433891594e1efa7d6875336f\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-103.ec2.internal" Jul 27 09:24:32.780884 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:32.780853 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/519b669acb80623490c3cff7ef66ed98-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-103.ec2.internal\" (UID: \"519b669acb80623490c3cff7ef66ed98\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-103.ec2.internal" Jul 27 09:24:32.780950 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:32.780901 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/f9f6fe57433891594e1efa7d6875336f-config\") pod \"kube-apiserver-proxy-ip-10-0-128-103.ec2.internal\" (UID: \"f9f6fe57433891594e1efa7d6875336f\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-103.ec2.internal" Jul 27 09:24:32.780950 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:32.780905 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/519b669acb80623490c3cff7ef66ed98-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-103.ec2.internal\" (UID: \"519b669acb80623490c3cff7ef66ed98\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-103.ec2.internal" Jul 27 09:24:32.781012 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:32.780949 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/519b669acb80623490c3cff7ef66ed98-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-103.ec2.internal\" (UID: \"519b669acb80623490c3cff7ef66ed98\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-103.ec2.internal" Jul 27 09:24:32.857137 ip-10-0-128-103 kubenswrapper[2565]: E0727 09:24:32.857073 2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-103.ec2.internal\" not found" Jul 27 09:24:32.957836 ip-10-0-128-103 kubenswrapper[2565]: E0727 09:24:32.957804 2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-103.ec2.internal\" not found" Jul 27 09:24:32.982144 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:32.982116 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-103.ec2.internal" Jul 27 09:24:32.987881 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:32.987857 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-103.ec2.internal" Jul 27 09:24:33.058871 ip-10-0-128-103 kubenswrapper[2565]: E0727 09:24:33.058834 2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-103.ec2.internal\" not found" Jul 27 09:24:33.159420 ip-10-0-128-103 kubenswrapper[2565]: E0727 09:24:33.159384 2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-103.ec2.internal\" not found" Jul 27 09:24:33.217783 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:33.217752 2565 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 27 09:24:33.262795 ip-10-0-128-103 kubenswrapper[2565]: E0727 09:24:33.262756 2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-103.ec2.internal\" not found" Jul 27 09:24:33.324355 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:33.324311 2565 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 27 09:24:33.324894 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:33.324527 2565 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 27 09:24:33.324894 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:33.324524 2565 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 27 09:24:33.363937 ip-10-0-128-103 kubenswrapper[2565]: E0727 09:24:33.363902 2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-103.ec2.internal\" not found" Jul 27 09:24:33.392256 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:33.392217 2565 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-26 09:19:33 +0000 UTC" deadline="2028-03-09 15:58:11.121734829 +0000 UTC" Jul 27 09:24:33.392256 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:33.392246 2565 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14190h33m37.729492386s" Jul 27 09:24:33.464434 ip-10-0-128-103 kubenswrapper[2565]: E0727 09:24:33.464341 2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-103.ec2.internal\" not found" Jul 27 09:24:33.493612 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:33.493585 2565 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 27 09:24:33.530889 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:33.530856 2565 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 27 09:24:33.573012 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:33.572975 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod519b669acb80623490c3cff7ef66ed98.slice/crio-d4e3de8edbf38384fc22c5b147ca1ead30d7fe7a7a778b9d76c3ee3cb17c3ca2 WatchSource:0}: Error finding container d4e3de8edbf38384fc22c5b147ca1ead30d7fe7a7a778b9d76c3ee3cb17c3ca2: Status 404 returned error can't find the container with id d4e3de8edbf38384fc22c5b147ca1ead30d7fe7a7a778b9d76c3ee3cb17c3ca2 Jul 27 09:24:33.573211 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:33.573191 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf9f6fe57433891594e1efa7d6875336f.slice/crio-a78e391be9f2e4e5713bffa26ae33ece50f7aa3c6d1fd2c16816e0cbd4325541 WatchSource:0}: Error finding container a78e391be9f2e4e5713bffa26ae33ece50f7aa3c6d1fd2c16816e0cbd4325541: Status 404 returned error can't find the container with id a78e391be9f2e4e5713bffa26ae33ece50f7aa3c6d1fd2c16816e0cbd4325541 Jul 27 09:24:33.577546 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:33.577521 2565 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-103.ec2.internal" Jul 27 09:24:33.577652 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:33.577591 2565 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 27 09:24:33.590965 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:33.590945 2565 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 27 09:24:33.592034 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:33.592019 2565 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-103.ec2.internal" Jul 27 09:24:33.602354 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:33.602325 2565 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 27 09:24:34.359542 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:34.359505 2565 apiserver.go:52] "Watching apiserver" Jul 27 09:24:34.370268 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:34.370236 2565 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 27 09:24:34.370745 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:34.370713 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-multus/multus-jmgz6","openshift-multus/network-metrics-daemon-trcl2","openshift-ovn-kubernetes/ovnkube-node-mqhsn","kube-system/konnectivity-agent-848w4","kube-system/kube-apiserver-proxy-ip-10-0-128-103.ec2.internal","openshift-multus/multus-additional-cni-plugins-54xcx","openshift-network-diagnostics/network-check-target-5zqgm","openshift-network-operator/iptables-alerter-rcftn","kube-system/global-pull-secret-syncer-rqz85","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kb5qg","openshift-cluster-node-tuning-operator/tuned-bk5w8","openshift-image-registry/node-ca-bmklx","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-103.ec2.internal"] Jul 27 09:24:34.372767 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:34.372707 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5zqgm" Jul 27 09:24:34.372913 ip-10-0-128-103 kubenswrapper[2565]: E0727 09:24:34.372799 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5zqgm" podUID="91816deb-5261-44b7-857a-04cf036feffa" Jul 27 09:24:34.374314 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:34.374188 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-trcl2" Jul 27 09:24:34.374314 ip-10-0-128-103 kubenswrapper[2565]: E0727 09:24:34.374275 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-trcl2" podUID="0d6ba35e-ec8a-4ec9-8520-0d32a5586a1d" Jul 27 09:24:34.376343 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:34.375587 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-mqhsn" Jul 27 09:24:34.386996 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:34.386008 2565 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 27 09:24:34.386996 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:34.386315 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 27 09:24:34.386996 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:34.386580 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 27 09:24:34.386996 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:34.386702 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-zr2x6\"" Jul 27 09:24:34.386996 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:34.386581 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 27 09:24:34.386996 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:34.386815 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 27 09:24:34.386996 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:34.386870 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 27 09:24:34.386996 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:34.386964 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-848w4" Jul 27 09:24:34.387474 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:34.387116 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 27 09:24:34.388566 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:34.388424 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-54xcx" Jul 27 09:24:34.389597 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:34.389568 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5c031f05-bf4a-4ff7-81ad-381d31b163ff-var-lib-openvswitch\") pod \"ovnkube-node-mqhsn\" (UID: \"5c031f05-bf4a-4ff7-81ad-381d31b163ff\") " pod="openshift-ovn-kubernetes/ovnkube-node-mqhsn" Jul 27 09:24:34.390091 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:34.390050 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2ckrw\" (UniqueName: \"kubernetes.io/projected/91816deb-5261-44b7-857a-04cf036feffa-kube-api-access-2ckrw\") pod \"network-check-target-5zqgm\" (UID: \"91816deb-5261-44b7-857a-04cf036feffa\") " pod="openshift-network-diagnostics/network-check-target-5zqgm" Jul 27 09:24:34.390190 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:34.390094 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4pnnj\" (UniqueName: \"kubernetes.io/projected/0d6ba35e-ec8a-4ec9-8520-0d32a5586a1d-kube-api-access-4pnnj\") pod \"network-metrics-daemon-trcl2\" (UID: \"0d6ba35e-ec8a-4ec9-8520-0d32a5586a1d\") " pod="openshift-multus/network-metrics-daemon-trcl2" Jul 27 09:24:34.390190 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:34.390131 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/5c031f05-bf4a-4ff7-81ad-381d31b163ff-host-kubelet\") pod \"ovnkube-node-mqhsn\" (UID: \"5c031f05-bf4a-4ff7-81ad-381d31b163ff\") " pod="openshift-ovn-kubernetes/ovnkube-node-mqhsn" Jul 27 09:24:34.390190 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:34.390157 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/5c031f05-bf4a-4ff7-81ad-381d31b163ff-host-slash\") pod \"ovnkube-node-mqhsn\" (UID: \"5c031f05-bf4a-4ff7-81ad-381d31b163ff\") " pod="openshift-ovn-kubernetes/ovnkube-node-mqhsn" Jul 27 09:24:34.390190 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:34.390181 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/5c031f05-bf4a-4ff7-81ad-381d31b163ff-log-socket\") pod \"ovnkube-node-mqhsn\" (UID: \"5c031f05-bf4a-4ff7-81ad-381d31b163ff\") " pod="openshift-ovn-kubernetes/ovnkube-node-mqhsn" Jul 27 09:24:34.390400 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:34.390205 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/5c031f05-bf4a-4ff7-81ad-381d31b163ff-env-overrides\") pod \"ovnkube-node-mqhsn\" (UID: \"5c031f05-bf4a-4ff7-81ad-381d31b163ff\") " pod="openshift-ovn-kubernetes/ovnkube-node-mqhsn" Jul 27 09:24:34.390400 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:34.390230 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/5c031f05-bf4a-4ff7-81ad-381d31b163ff-ovnkube-script-lib\") pod \"ovnkube-node-mqhsn\" (UID: \"5c031f05-bf4a-4ff7-81ad-381d31b163ff\") " pod="openshift-ovn-kubernetes/ovnkube-node-mqhsn" Jul 27 09:24:34.390400 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:34.390258 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/5c031f05-bf4a-4ff7-81ad-381d31b163ff-run-systemd\") pod \"ovnkube-node-mqhsn\" (UID: \"5c031f05-bf4a-4ff7-81ad-381d31b163ff\") " pod="openshift-ovn-kubernetes/ovnkube-node-mqhsn" Jul 27 09:24:34.390400 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:34.390287 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5c031f05-bf4a-4ff7-81ad-381d31b163ff-etc-openvswitch\") pod \"ovnkube-node-mqhsn\" (UID: \"5c031f05-bf4a-4ff7-81ad-381d31b163ff\") " pod="openshift-ovn-kubernetes/ovnkube-node-mqhsn" Jul 27 09:24:34.390400 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:34.390311 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/5c031f05-bf4a-4ff7-81ad-381d31b163ff-host-cni-netd\") pod \"ovnkube-node-mqhsn\" (UID: \"5c031f05-bf4a-4ff7-81ad-381d31b163ff\") " pod="openshift-ovn-kubernetes/ovnkube-node-mqhsn" Jul 27 09:24:34.390400 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:34.390336 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/5c031f05-bf4a-4ff7-81ad-381d31b163ff-ovnkube-config\") pod \"ovnkube-node-mqhsn\" (UID: \"5c031f05-bf4a-4ff7-81ad-381d31b163ff\") " pod="openshift-ovn-kubernetes/ovnkube-node-mqhsn" Jul 27 09:24:34.390400 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:34.390360 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b8hnw\" (UniqueName: \"kubernetes.io/projected/5c031f05-bf4a-4ff7-81ad-381d31b163ff-kube-api-access-b8hnw\") pod \"ovnkube-node-mqhsn\" (UID: \"5c031f05-bf4a-4ff7-81ad-381d31b163ff\") " pod="openshift-ovn-kubernetes/ovnkube-node-mqhsn" Jul 27 09:24:34.390742 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:34.390443 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/2c043d82-d300-41ba-b138-60a03d65ea11-konnectivity-ca\") pod \"konnectivity-agent-848w4\" (UID: \"2c043d82-d300-41ba-b138-60a03d65ea11\") " pod="kube-system/konnectivity-agent-848w4" Jul 27 09:24:34.390742 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:34.390481 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0d6ba35e-ec8a-4ec9-8520-0d32a5586a1d-metrics-certs\") pod \"network-metrics-daemon-trcl2\" (UID: \"0d6ba35e-ec8a-4ec9-8520-0d32a5586a1d\") " pod="openshift-multus/network-metrics-daemon-trcl2" Jul 27 09:24:34.390742 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:34.390510 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/5c031f05-bf4a-4ff7-81ad-381d31b163ff-systemd-units\") pod \"ovnkube-node-mqhsn\" (UID: \"5c031f05-bf4a-4ff7-81ad-381d31b163ff\") " pod="openshift-ovn-kubernetes/ovnkube-node-mqhsn" Jul 27 09:24:34.390742 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:34.390562 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/5c031f05-bf4a-4ff7-81ad-381d31b163ff-run-ovn\") pod \"ovnkube-node-mqhsn\" (UID: \"5c031f05-bf4a-4ff7-81ad-381d31b163ff\") " pod="openshift-ovn-kubernetes/ovnkube-node-mqhsn" Jul 27 09:24:34.390742 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:34.390588 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/5c031f05-bf4a-4ff7-81ad-381d31b163ff-node-log\") pod \"ovnkube-node-mqhsn\" (UID: \"5c031f05-bf4a-4ff7-81ad-381d31b163ff\") " pod="openshift-ovn-kubernetes/ovnkube-node-mqhsn" Jul 27 09:24:34.390742 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:34.390613 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/5c031f05-bf4a-4ff7-81ad-381d31b163ff-host-cni-bin\") pod \"ovnkube-node-mqhsn\" (UID: \"5c031f05-bf4a-4ff7-81ad-381d31b163ff\") " pod="openshift-ovn-kubernetes/ovnkube-node-mqhsn" Jul 27 09:24:34.390742 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:34.390637 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/2c043d82-d300-41ba-b138-60a03d65ea11-agent-certs\") pod \"konnectivity-agent-848w4\" (UID: \"2c043d82-d300-41ba-b138-60a03d65ea11\") " pod="kube-system/konnectivity-agent-848w4" Jul 27 09:24:34.390742 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:34.390664 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/5c031f05-bf4a-4ff7-81ad-381d31b163ff-host-run-netns\") pod \"ovnkube-node-mqhsn\" (UID: \"5c031f05-bf4a-4ff7-81ad-381d31b163ff\") " pod="openshift-ovn-kubernetes/ovnkube-node-mqhsn" Jul 27 09:24:34.390742 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:34.390688 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5c031f05-bf4a-4ff7-81ad-381d31b163ff-run-openvswitch\") pod \"ovnkube-node-mqhsn\" (UID: \"5c031f05-bf4a-4ff7-81ad-381d31b163ff\") " pod="openshift-ovn-kubernetes/ovnkube-node-mqhsn" Jul 27 09:24:34.390742 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:34.390722 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5c031f05-bf4a-4ff7-81ad-381d31b163ff-host-run-ovn-kubernetes\") pod \"ovnkube-node-mqhsn\" (UID: \"5c031f05-bf4a-4ff7-81ad-381d31b163ff\") " pod="openshift-ovn-kubernetes/ovnkube-node-mqhsn" Jul 27 09:24:34.391253 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:34.390749 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5c031f05-bf4a-4ff7-81ad-381d31b163ff-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-mqhsn\" (UID: \"5c031f05-bf4a-4ff7-81ad-381d31b163ff\") " pod="openshift-ovn-kubernetes/ovnkube-node-mqhsn" Jul 27 09:24:34.391253 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:34.390869 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/5c031f05-bf4a-4ff7-81ad-381d31b163ff-ovn-node-metrics-cert\") pod \"ovnkube-node-mqhsn\" (UID: \"5c031f05-bf4a-4ff7-81ad-381d31b163ff\") " pod="openshift-ovn-kubernetes/ovnkube-node-mqhsn" Jul 27 09:24:34.391253 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:34.390895 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 27 09:24:34.391253 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:34.391080 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 27 09:24:34.391253 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:34.391116 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-8s8dw\"" Jul 27 09:24:34.392083 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:34.392061 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 27 09:24:34.392313 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:34.392280 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-h928n\"" Jul 27 09:24:34.392524 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:34.392511 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 27 09:24:34.392718 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:34.392703 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 27 09:24:34.393384 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:34.393363 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 27 09:24:34.393634 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:34.393616 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 27 09:24:34.394291 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:34.393964 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-jmgz6" Jul 27 09:24:34.394291 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:34.394053 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-rcftn" Jul 27 09:24:34.394866 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:34.394823 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rqz85" Jul 27 09:24:34.394958 ip-10-0-128-103 kubenswrapper[2565]: E0727 09:24:34.394898 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rqz85" podUID="204c97b6-2efe-4db7-bc9f-a5653fed77c7" Jul 27 09:24:34.396433 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:34.396412 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 27 09:24:34.396663 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:34.396648 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kb5qg" Jul 27 09:24:34.396994 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:34.396796 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 27 09:24:34.397626 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:34.397384 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 27 09:24:34.397626 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:34.397454 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-rrvnr\"" Jul 27 09:24:34.397968 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:34.397947 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 27 09:24:34.398147 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:34.398127 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-n5xtz\"" Jul 27 09:24:34.398455 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:34.398435 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-bk5w8" Jul 27 09:24:34.399508 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:34.399453 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 27 09:24:34.399640 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:34.399574 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 27 09:24:34.399942 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:34.399921 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 27 09:24:34.400080 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:34.400062 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-r9vdx\"" Jul 27 09:24:34.400424 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:34.400394 2565 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 27 09:24:34.401280 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:34.401259 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-w52tm\"" Jul 27 09:24:34.402104 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:34.402081 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-bmklx" Jul 27 09:24:34.402391 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:34.402371 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 27 09:24:34.402631 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:34.402612 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 27 09:24:34.404477 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:34.404422 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 27 09:24:34.404883 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:34.404859 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 27 09:24:34.405030 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:34.405014 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-2699m\"" Jul 27 09:24:34.405030 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:34.405026 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 27 09:24:34.426459 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:34.426434 2565 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-9xtp6" Jul 27 09:24:34.437627 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:34.437606 2565 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-9xtp6" Jul 27 09:24:34.479003 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:34.478974 2565 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 27 09:24:34.491347 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:34.491315 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mp4mq\" (UniqueName: \"kubernetes.io/projected/29406f41-dc4d-4f8a-9627-9589659ae0d4-kube-api-access-mp4mq\") pod \"multus-jmgz6\" (UID: \"29406f41-dc4d-4f8a-9627-9589659ae0d4\") " pod="openshift-multus/multus-jmgz6" Jul 27 09:24:34.491510 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:34.491358 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/09e4c928-4b42-484e-808c-dfaa1b17977b-etc-selinux\") pod \"aws-ebs-csi-driver-node-kb5qg\" (UID: \"09e4c928-4b42-484e-808c-dfaa1b17977b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kb5qg" Jul 27 09:24:34.491510 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:34.491385 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/9cc406f3-cdff-4cde-ac7b-ff31abb20e0d-etc-sysctl-d\") pod \"tuned-bk5w8\" (UID: \"9cc406f3-cdff-4cde-ac7b-ff31abb20e0d\") " pod="openshift-cluster-node-tuning-operator/tuned-bk5w8" Jul 27 09:24:34.491510 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:34.491406 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/204c97b6-2efe-4db7-bc9f-a5653fed77c7-original-pull-secret\") pod \"global-pull-secret-syncer-rqz85\" (UID: \"204c97b6-2efe-4db7-bc9f-a5653fed77c7\") " pod="kube-system/global-pull-secret-syncer-rqz85" Jul 27 09:24:34.491510 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:34.491463 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4pnnj\" (UniqueName: \"kubernetes.io/projected/0d6ba35e-ec8a-4ec9-8520-0d32a5586a1d-kube-api-access-4pnnj\") pod \"network-metrics-daemon-trcl2\" (UID: \"0d6ba35e-ec8a-4ec9-8520-0d32a5586a1d\") " pod="openshift-multus/network-metrics-daemon-trcl2" Jul 27 09:24:34.491510 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:34.491507 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/5c031f05-bf4a-4ff7-81ad-381d31b163ff-host-slash\") pod \"ovnkube-node-mqhsn\" (UID: \"5c031f05-bf4a-4ff7-81ad-381d31b163ff\") " pod="openshift-ovn-kubernetes/ovnkube-node-mqhsn" Jul 27 09:24:34.491853 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:34.491540 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/29406f41-dc4d-4f8a-9627-9589659ae0d4-multus-cni-dir\") pod \"multus-jmgz6\" (UID: \"29406f41-dc4d-4f8a-9627-9589659ae0d4\") " pod="openshift-multus/multus-jmgz6" Jul 27 09:24:34.491853 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:34.491565 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/29406f41-dc4d-4f8a-9627-9589659ae0d4-cni-binary-copy\") pod \"multus-jmgz6\" (UID: \"29406f41-dc4d-4f8a-9627-9589659ae0d4\") " pod="openshift-multus/multus-jmgz6" Jul 27 09:24:34.491853 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:34.491588 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/29406f41-dc4d-4f8a-9627-9589659ae0d4-hostroot\") pod \"multus-jmgz6\" (UID: \"29406f41-dc4d-4f8a-9627-9589659ae0d4\") " pod="openshift-multus/multus-jmgz6" Jul 27 09:24:34.491853 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:34.491614 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/5c031f05-bf4a-4ff7-81ad-381d31b163ff-ovnkube-config\") pod \"ovnkube-node-mqhsn\" (UID: \"5c031f05-bf4a-4ff7-81ad-381d31b163ff\") " pod="openshift-ovn-kubernetes/ovnkube-node-mqhsn" Jul 27 09:24:34.491853 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:34.491625 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/5c031f05-bf4a-4ff7-81ad-381d31b163ff-host-slash\") pod \"ovnkube-node-mqhsn\" (UID: \"5c031f05-bf4a-4ff7-81ad-381d31b163ff\") " pod="openshift-ovn-kubernetes/ovnkube-node-mqhsn" Jul 27 09:24:34.491853 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:34.491638 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/204c97b6-2efe-4db7-bc9f-a5653fed77c7-dbus\") pod \"global-pull-secret-syncer-rqz85\" (UID: \"204c97b6-2efe-4db7-bc9f-a5653fed77c7\") " pod="kube-system/global-pull-secret-syncer-rqz85" Jul 27 09:24:34.491853 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:34.491668 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/9cc406f3-cdff-4cde-ac7b-ff31abb20e0d-etc-systemd\") pod \"tuned-bk5w8\" (UID: \"9cc406f3-cdff-4cde-ac7b-ff31abb20e0d\") " pod="openshift-cluster-node-tuning-operator/tuned-bk5w8" Jul 27 09:24:34.491853 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:34.491695 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9cc406f3-cdff-4cde-ac7b-ff31abb20e0d-sys\") pod \"tuned-bk5w8\" (UID: \"9cc406f3-cdff-4cde-ac7b-ff31abb20e0d\") " pod="openshift-cluster-node-tuning-operator/tuned-bk5w8" Jul 27 09:24:34.491853 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:34.491733 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/9cc406f3-cdff-4cde-ac7b-ff31abb20e0d-host\") pod \"tuned-bk5w8\" (UID: \"9cc406f3-cdff-4cde-ac7b-ff31abb20e0d\") " pod="openshift-cluster-node-tuning-operator/tuned-bk5w8" Jul 27 09:24:34.491853 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:34.491783 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w2t6q\" (UniqueName: \"kubernetes.io/projected/9cc406f3-cdff-4cde-ac7b-ff31abb20e0d-kube-api-access-w2t6q\") pod \"tuned-bk5w8\" (UID: \"9cc406f3-cdff-4cde-ac7b-ff31abb20e0d\") " pod="openshift-cluster-node-tuning-operator/tuned-bk5w8" Jul 27 09:24:34.491853 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:34.491815 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/5c031f05-bf4a-4ff7-81ad-381d31b163ff-run-ovn\") pod \"ovnkube-node-mqhsn\" (UID: \"5c031f05-bf4a-4ff7-81ad-381d31b163ff\") " pod="openshift-ovn-kubernetes/ovnkube-node-mqhsn" Jul 27 09:24:34.491853 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:34.491840 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/2c043d82-d300-41ba-b138-60a03d65ea11-agent-certs\") pod \"konnectivity-agent-848w4\" (UID: \"2c043d82-d300-41ba-b138-60a03d65ea11\") " pod="kube-system/konnectivity-agent-848w4" Jul 27 09:24:34.492457 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:34.491862 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/5c031f05-bf4a-4ff7-81ad-381d31b163ff-run-ovn\") pod \"ovnkube-node-mqhsn\" (UID: \"5c031f05-bf4a-4ff7-81ad-381d31b163ff\") " pod="openshift-ovn-kubernetes/ovnkube-node-mqhsn" Jul 27 09:24:34.492457 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:34.491866 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/9cc406f3-cdff-4cde-ac7b-ff31abb20e0d-etc-sysctl-conf\") pod \"tuned-bk5w8\" (UID: \"9cc406f3-cdff-4cde-ac7b-ff31abb20e0d\") " pod="openshift-cluster-node-tuning-operator/tuned-bk5w8" Jul 27 09:24:34.492457 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:34.491912 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9cc406f3-cdff-4cde-ac7b-ff31abb20e0d-tmp\") pod \"tuned-bk5w8\" (UID: \"9cc406f3-cdff-4cde-ac7b-ff31abb20e0d\") " pod="openshift-cluster-node-tuning-operator/tuned-bk5w8" Jul 27 09:24:34.492457 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:34.491942 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/5c031f05-bf4a-4ff7-81ad-381d31b163ff-host-run-netns\") pod \"ovnkube-node-mqhsn\" (UID: \"5c031f05-bf4a-4ff7-81ad-381d31b163ff\") " pod="openshift-ovn-kubernetes/ovnkube-node-mqhsn" Jul 27 09:24:34.492457 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:34.491974 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/5c031f05-bf4a-4ff7-81ad-381d31b163ff-ovn-node-metrics-cert\") pod \"ovnkube-node-mqhsn\" (UID: \"5c031f05-bf4a-4ff7-81ad-381d31b163ff\") " pod="openshift-ovn-kubernetes/ovnkube-node-mqhsn" Jul 27 09:24:34.492457 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:34.492060 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/29406f41-dc4d-4f8a-9627-9589659ae0d4-etc-kubernetes\") pod \"multus-jmgz6\" (UID: \"29406f41-dc4d-4f8a-9627-9589659ae0d4\") " pod="openshift-multus/multus-jmgz6" Jul 27 09:24:34.492457 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:34.492065 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/5c031f05-bf4a-4ff7-81ad-381d31b163ff-host-run-netns\") pod \"ovnkube-node-mqhsn\" (UID: \"5c031f05-bf4a-4ff7-81ad-381d31b163ff\") " pod="openshift-ovn-kubernetes/ovnkube-node-mqhsn" Jul 27 09:24:34.492457 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:34.492092 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mblhr\" (UniqueName: \"kubernetes.io/projected/d0206685-d6b8-43ea-a691-ca3c1acd237a-kube-api-access-mblhr\") pod \"multus-additional-cni-plugins-54xcx\" (UID: \"d0206685-d6b8-43ea-a691-ca3c1acd237a\") " pod="openshift-multus/multus-additional-cni-plugins-54xcx" Jul 27 09:24:34.492457 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:34.492118 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9cc406f3-cdff-4cde-ac7b-ff31abb20e0d-etc-kubernetes\") pod \"tuned-bk5w8\" (UID: \"9cc406f3-cdff-4cde-ac7b-ff31abb20e0d\") " pod="openshift-cluster-node-tuning-operator/tuned-bk5w8" Jul 27 09:24:34.492457 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:34.492136 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/9cc406f3-cdff-4cde-ac7b-ff31abb20e0d-lib-modules\") pod \"tuned-bk5w8\" (UID: \"9cc406f3-cdff-4cde-ac7b-ff31abb20e0d\") " pod="openshift-cluster-node-tuning-operator/tuned-bk5w8" Jul 27 09:24:34.492457 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:34.492152 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/996c29bd-3969-4268-a795-4f4d21881d48-host\") pod \"node-ca-bmklx\" (UID: \"996c29bd-3969-4268-a795-4f4d21881d48\") " pod="openshift-image-registry/node-ca-bmklx" Jul 27 09:24:34.492457 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:34.492176 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5c031f05-bf4a-4ff7-81ad-381d31b163ff-var-lib-openvswitch\") pod \"ovnkube-node-mqhsn\" (UID: \"5c031f05-bf4a-4ff7-81ad-381d31b163ff\") " pod="openshift-ovn-kubernetes/ovnkube-node-mqhsn" Jul 27 09:24:34.492457 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:34.492233 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/29406f41-dc4d-4f8a-9627-9589659ae0d4-host-var-lib-cni-bin\") pod \"multus-jmgz6\" (UID: \"29406f41-dc4d-4f8a-9627-9589659ae0d4\") " pod="openshift-multus/multus-jmgz6" Jul 27 09:24:34.492457 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:34.492224 2565 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 27 09:24:34.492457 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:34.492273 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5c031f05-bf4a-4ff7-81ad-381d31b163ff-var-lib-openvswitch\") pod \"ovnkube-node-mqhsn\" (UID: \"5c031f05-bf4a-4ff7-81ad-381d31b163ff\") " pod="openshift-ovn-kubernetes/ovnkube-node-mqhsn" Jul 27 09:24:34.492457 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:34.492285 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/29406f41-dc4d-4f8a-9627-9589659ae0d4-multus-conf-dir\") pod \"multus-jmgz6\" (UID: \"29406f41-dc4d-4f8a-9627-9589659ae0d4\") " pod="openshift-multus/multus-jmgz6" Jul 27 09:24:34.492457 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:34.492304 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/5c031f05-bf4a-4ff7-81ad-381d31b163ff-ovnkube-config\") pod \"ovnkube-node-mqhsn\" (UID: \"5c031f05-bf4a-4ff7-81ad-381d31b163ff\") " pod="openshift-ovn-kubernetes/ovnkube-node-mqhsn" Jul 27 09:24:34.493321 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:34.492316 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/29406f41-dc4d-4f8a-9627-9589659ae0d4-multus-daemon-config\") pod \"multus-jmgz6\" (UID: \"29406f41-dc4d-4f8a-9627-9589659ae0d4\") " pod="openshift-multus/multus-jmgz6" Jul 27 09:24:34.493321 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:34.492346 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/09e4c928-4b42-484e-808c-dfaa1b17977b-kubelet-dir\") pod \"aws-ebs-csi-driver-node-kb5qg\" (UID: \"09e4c928-4b42-484e-808c-dfaa1b17977b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kb5qg" Jul 27 09:24:34.493321 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:34.492375 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5c031f05-bf4a-4ff7-81ad-381d31b163ff-etc-openvswitch\") pod \"ovnkube-node-mqhsn\" (UID: \"5c031f05-bf4a-4ff7-81ad-381d31b163ff\") " pod="openshift-ovn-kubernetes/ovnkube-node-mqhsn" Jul 27 09:24:34.493321 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:34.492401 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/5c031f05-bf4a-4ff7-81ad-381d31b163ff-host-cni-netd\") pod \"ovnkube-node-mqhsn\" (UID: \"5c031f05-bf4a-4ff7-81ad-381d31b163ff\") " pod="openshift-ovn-kubernetes/ovnkube-node-mqhsn" Jul 27 09:24:34.493321 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:34.492437 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/6fd3f632-9d80-4e6e-8d4f-f5314c0f174a-iptables-alerter-script\") pod \"iptables-alerter-rcftn\" (UID: \"6fd3f632-9d80-4e6e-8d4f-f5314c0f174a\") " pod="openshift-network-operator/iptables-alerter-rcftn" Jul 27 09:24:34.493321 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:34.492462 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/5c031f05-bf4a-4ff7-81ad-381d31b163ff-systemd-units\") pod \"ovnkube-node-mqhsn\" (UID: \"5c031f05-bf4a-4ff7-81ad-381d31b163ff\") " pod="openshift-ovn-kubernetes/ovnkube-node-mqhsn" Jul 27 09:24:34.493321 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:34.492478 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/5c031f05-bf4a-4ff7-81ad-381d31b163ff-host-cni-netd\") pod \"ovnkube-node-mqhsn\" (UID: \"5c031f05-bf4a-4ff7-81ad-381d31b163ff\") " pod="openshift-ovn-kubernetes/ovnkube-node-mqhsn" Jul 27 09:24:34.493321 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:34.492487 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/5c031f05-bf4a-4ff7-81ad-381d31b163ff-node-log\") pod \"ovnkube-node-mqhsn\" (UID: \"5c031f05-bf4a-4ff7-81ad-381d31b163ff\") " pod="openshift-ovn-kubernetes/ovnkube-node-mqhsn" Jul 27 09:24:34.493321 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:34.492490 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5c031f05-bf4a-4ff7-81ad-381d31b163ff-etc-openvswitch\") pod \"ovnkube-node-mqhsn\" (UID: \"5c031f05-bf4a-4ff7-81ad-381d31b163ff\") " pod="openshift-ovn-kubernetes/ovnkube-node-mqhsn" Jul 27 09:24:34.493321 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:34.492513 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/29406f41-dc4d-4f8a-9627-9589659ae0d4-cnibin\") pod \"multus-jmgz6\" (UID: \"29406f41-dc4d-4f8a-9627-9589659ae0d4\") " pod="openshift-multus/multus-jmgz6" Jul 27 09:24:34.493321 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:34.492535 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/5c031f05-bf4a-4ff7-81ad-381d31b163ff-systemd-units\") pod \"ovnkube-node-mqhsn\" (UID: \"5c031f05-bf4a-4ff7-81ad-381d31b163ff\") " pod="openshift-ovn-kubernetes/ovnkube-node-mqhsn" Jul 27 09:24:34.493321 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:34.492538 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/29406f41-dc4d-4f8a-9627-9589659ae0d4-host-run-k8s-cni-cncf-io\") pod \"multus-jmgz6\" (UID: \"29406f41-dc4d-4f8a-9627-9589659ae0d4\") " pod="openshift-multus/multus-jmgz6" Jul 27 09:24:34.493321 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:34.492551 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/5c031f05-bf4a-4ff7-81ad-381d31b163ff-node-log\") pod \"ovnkube-node-mqhsn\" (UID: \"5c031f05-bf4a-4ff7-81ad-381d31b163ff\") " pod="openshift-ovn-kubernetes/ovnkube-node-mqhsn" Jul 27 09:24:34.493321 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:34.492565 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q2z5h\" (UniqueName: \"kubernetes.io/projected/09e4c928-4b42-484e-808c-dfaa1b17977b-kube-api-access-q2z5h\") pod \"aws-ebs-csi-driver-node-kb5qg\" (UID: \"09e4c928-4b42-484e-808c-dfaa1b17977b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kb5qg" Jul 27 09:24:34.493321 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:34.492590 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/9cc406f3-cdff-4cde-ac7b-ff31abb20e0d-etc-modprobe-d\") pod \"tuned-bk5w8\" (UID: \"9cc406f3-cdff-4cde-ac7b-ff31abb20e0d\") " pod="openshift-cluster-node-tuning-operator/tuned-bk5w8" Jul 27 09:24:34.493321 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:34.492616 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/29406f41-dc4d-4f8a-9627-9589659ae0d4-host-var-lib-cni-multus\") pod \"multus-jmgz6\" (UID: \"29406f41-dc4d-4f8a-9627-9589659ae0d4\") " pod="openshift-multus/multus-jmgz6" Jul 27 09:24:34.493321 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:34.492641 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/29406f41-dc4d-4f8a-9627-9589659ae0d4-host-var-lib-kubelet\") pod \"multus-jmgz6\" (UID: \"29406f41-dc4d-4f8a-9627-9589659ae0d4\") " pod="openshift-multus/multus-jmgz6" Jul 27 09:24:34.494220 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:34.492668 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/29406f41-dc4d-4f8a-9627-9589659ae0d4-host-run-multus-certs\") pod \"multus-jmgz6\" (UID: \"29406f41-dc4d-4f8a-9627-9589659ae0d4\") " pod="openshift-multus/multus-jmgz6" Jul 27 09:24:34.494220 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:34.492692 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/9cc406f3-cdff-4cde-ac7b-ff31abb20e0d-etc-sysconfig\") pod \"tuned-bk5w8\" (UID: \"9cc406f3-cdff-4cde-ac7b-ff31abb20e0d\") " pod="openshift-cluster-node-tuning-operator/tuned-bk5w8" Jul 27 09:24:34.494220 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:34.492717 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/9cc406f3-cdff-4cde-ac7b-ff31abb20e0d-etc-tuned\") pod \"tuned-bk5w8\" (UID: \"9cc406f3-cdff-4cde-ac7b-ff31abb20e0d\") " pod="openshift-cluster-node-tuning-operator/tuned-bk5w8" Jul 27 09:24:34.494220 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:34.492746 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2ckrw\" (UniqueName: \"kubernetes.io/projected/91816deb-5261-44b7-857a-04cf036feffa-kube-api-access-2ckrw\") pod \"network-check-target-5zqgm\" (UID: \"91816deb-5261-44b7-857a-04cf036feffa\") " pod="openshift-network-diagnostics/network-check-target-5zqgm" Jul 27 09:24:34.494220 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:34.492798 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/5c031f05-bf4a-4ff7-81ad-381d31b163ff-log-socket\") pod \"ovnkube-node-mqhsn\" (UID: \"5c031f05-bf4a-4ff7-81ad-381d31b163ff\") " pod="openshift-ovn-kubernetes/ovnkube-node-mqhsn" Jul 27 09:24:34.494220 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:34.492831 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b8hnw\" (UniqueName: \"kubernetes.io/projected/5c031f05-bf4a-4ff7-81ad-381d31b163ff-kube-api-access-b8hnw\") pod \"ovnkube-node-mqhsn\" (UID: \"5c031f05-bf4a-4ff7-81ad-381d31b163ff\") " pod="openshift-ovn-kubernetes/ovnkube-node-mqhsn" Jul 27 09:24:34.494220 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:34.492857 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/2c043d82-d300-41ba-b138-60a03d65ea11-konnectivity-ca\") pod \"konnectivity-agent-848w4\" (UID: \"2c043d82-d300-41ba-b138-60a03d65ea11\") " pod="kube-system/konnectivity-agent-848w4" Jul 27 09:24:34.494220 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:34.493046 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/5c031f05-bf4a-4ff7-81ad-381d31b163ff-log-socket\") pod \"ovnkube-node-mqhsn\" (UID: \"5c031f05-bf4a-4ff7-81ad-381d31b163ff\") " pod="openshift-ovn-kubernetes/ovnkube-node-mqhsn" Jul 27 09:24:34.494220 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:34.493092 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/29406f41-dc4d-4f8a-9627-9589659ae0d4-os-release\") pod \"multus-jmgz6\" (UID: \"29406f41-dc4d-4f8a-9627-9589659ae0d4\") " pod="openshift-multus/multus-jmgz6" Jul 27 09:24:34.494220 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:34.493122 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/29406f41-dc4d-4f8a-9627-9589659ae0d4-host-run-netns\") pod \"multus-jmgz6\" (UID: \"29406f41-dc4d-4f8a-9627-9589659ae0d4\") " pod="openshift-multus/multus-jmgz6" Jul 27 09:24:34.494220 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:34.493150 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sqdxv\" (UniqueName: \"kubernetes.io/projected/6fd3f632-9d80-4e6e-8d4f-f5314c0f174a-kube-api-access-sqdxv\") pod \"iptables-alerter-rcftn\" (UID: \"6fd3f632-9d80-4e6e-8d4f-f5314c0f174a\") " pod="openshift-network-operator/iptables-alerter-rcftn" Jul 27 09:24:34.494220 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:34.493188 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d0206685-d6b8-43ea-a691-ca3c1acd237a-system-cni-dir\") pod \"multus-additional-cni-plugins-54xcx\" (UID: \"d0206685-d6b8-43ea-a691-ca3c1acd237a\") " pod="openshift-multus/multus-additional-cni-plugins-54xcx" Jul 27 09:24:34.494220 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:34.493213 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/09e4c928-4b42-484e-808c-dfaa1b17977b-registration-dir\") pod \"aws-ebs-csi-driver-node-kb5qg\" (UID: \"09e4c928-4b42-484e-808c-dfaa1b17977b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kb5qg" Jul 27 09:24:34.494220 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:34.493245 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/5c031f05-bf4a-4ff7-81ad-381d31b163ff-run-systemd\") pod \"ovnkube-node-mqhsn\" (UID: \"5c031f05-bf4a-4ff7-81ad-381d31b163ff\") " pod="openshift-ovn-kubernetes/ovnkube-node-mqhsn" Jul 27 09:24:34.494220 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:34.493270 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/29406f41-dc4d-4f8a-9627-9589659ae0d4-multus-socket-dir-parent\") pod \"multus-jmgz6\" (UID: \"29406f41-dc4d-4f8a-9627-9589659ae0d4\") " pod="openshift-multus/multus-jmgz6" Jul 27 09:24:34.494220 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:34.493295 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/d0206685-d6b8-43ea-a691-ca3c1acd237a-cnibin\") pod \"multus-additional-cni-plugins-54xcx\" (UID: \"d0206685-d6b8-43ea-a691-ca3c1acd237a\") " pod="openshift-multus/multus-additional-cni-plugins-54xcx" Jul 27 09:24:34.494220 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:34.493305 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/5c031f05-bf4a-4ff7-81ad-381d31b163ff-run-systemd\") pod \"ovnkube-node-mqhsn\" (UID: \"5c031f05-bf4a-4ff7-81ad-381d31b163ff\") " pod="openshift-ovn-kubernetes/ovnkube-node-mqhsn" Jul 27 09:24:34.495105 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:34.493321 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/d0206685-d6b8-43ea-a691-ca3c1acd237a-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-54xcx\" (UID: \"d0206685-d6b8-43ea-a691-ca3c1acd237a\") " pod="openshift-multus/multus-additional-cni-plugins-54xcx" Jul 27 09:24:34.495105 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:34.493381 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/09e4c928-4b42-484e-808c-dfaa1b17977b-device-dir\") pod \"aws-ebs-csi-driver-node-kb5qg\" (UID: \"09e4c928-4b42-484e-808c-dfaa1b17977b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kb5qg" Jul 27 09:24:34.495105 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:34.493411 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0d6ba35e-ec8a-4ec9-8520-0d32a5586a1d-metrics-certs\") pod \"network-metrics-daemon-trcl2\" (UID: \"0d6ba35e-ec8a-4ec9-8520-0d32a5586a1d\") " pod="openshift-multus/network-metrics-daemon-trcl2" Jul 27 09:24:34.495105 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:34.493437 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/29406f41-dc4d-4f8a-9627-9589659ae0d4-system-cni-dir\") pod \"multus-jmgz6\" (UID: \"29406f41-dc4d-4f8a-9627-9589659ae0d4\") " pod="openshift-multus/multus-jmgz6" Jul 27 09:24:34.495105 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:34.493464 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/09e4c928-4b42-484e-808c-dfaa1b17977b-socket-dir\") pod \"aws-ebs-csi-driver-node-kb5qg\" (UID: \"09e4c928-4b42-484e-808c-dfaa1b17977b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kb5qg" Jul 27 09:24:34.495105 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:34.493489 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/d0206685-d6b8-43ea-a691-ca3c1acd237a-os-release\") pod \"multus-additional-cni-plugins-54xcx\" (UID: \"d0206685-d6b8-43ea-a691-ca3c1acd237a\") " pod="openshift-multus/multus-additional-cni-plugins-54xcx" Jul 27 09:24:34.495105 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:34.493514 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9cc406f3-cdff-4cde-ac7b-ff31abb20e0d-var-lib-kubelet\") pod \"tuned-bk5w8\" (UID: \"9cc406f3-cdff-4cde-ac7b-ff31abb20e0d\") " pod="openshift-cluster-node-tuning-operator/tuned-bk5w8" Jul 27 09:24:34.495105 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:34.493541 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5c031f05-bf4a-4ff7-81ad-381d31b163ff-run-openvswitch\") pod \"ovnkube-node-mqhsn\" (UID: \"5c031f05-bf4a-4ff7-81ad-381d31b163ff\") " pod="openshift-ovn-kubernetes/ovnkube-node-mqhsn" Jul 27 09:24:34.495105 ip-10-0-128-103 kubenswrapper[2565]: E0727 09:24:34.493572 2565 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 27 09:24:34.495105 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:34.493617 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5c031f05-bf4a-4ff7-81ad-381d31b163ff-host-run-ovn-kubernetes\") pod \"ovnkube-node-mqhsn\" (UID: \"5c031f05-bf4a-4ff7-81ad-381d31b163ff\") " pod="openshift-ovn-kubernetes/ovnkube-node-mqhsn" Jul 27 09:24:34.495105 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:34.493625 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/2c043d82-d300-41ba-b138-60a03d65ea11-konnectivity-ca\") pod \"konnectivity-agent-848w4\" (UID: \"2c043d82-d300-41ba-b138-60a03d65ea11\") " pod="kube-system/konnectivity-agent-848w4" Jul 27 09:24:34.495105 ip-10-0-128-103 kubenswrapper[2565]: E0727 09:24:34.493647 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0d6ba35e-ec8a-4ec9-8520-0d32a5586a1d-metrics-certs podName:0d6ba35e-ec8a-4ec9-8520-0d32a5586a1d nodeName:}" failed. No retries permitted until 2026-07-27 09:24:34.993625466 +0000 UTC m=+2.957411405 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0d6ba35e-ec8a-4ec9-8520-0d32a5586a1d-metrics-certs") pod "network-metrics-daemon-trcl2" (UID: "0d6ba35e-ec8a-4ec9-8520-0d32a5586a1d") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 27 09:24:34.495105 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:34.493572 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5c031f05-bf4a-4ff7-81ad-381d31b163ff-host-run-ovn-kubernetes\") pod \"ovnkube-node-mqhsn\" (UID: \"5c031f05-bf4a-4ff7-81ad-381d31b163ff\") " pod="openshift-ovn-kubernetes/ovnkube-node-mqhsn" Jul 27 09:24:34.495105 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:34.493695 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5c031f05-bf4a-4ff7-81ad-381d31b163ff-run-openvswitch\") pod \"ovnkube-node-mqhsn\" (UID: \"5c031f05-bf4a-4ff7-81ad-381d31b163ff\") " pod="openshift-ovn-kubernetes/ovnkube-node-mqhsn" Jul 27 09:24:34.495105 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:34.493733 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5c031f05-bf4a-4ff7-81ad-381d31b163ff-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-mqhsn\" (UID: \"5c031f05-bf4a-4ff7-81ad-381d31b163ff\") " pod="openshift-ovn-kubernetes/ovnkube-node-mqhsn" Jul 27 09:24:34.495105 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:34.493767 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/9cc406f3-cdff-4cde-ac7b-ff31abb20e0d-run\") pod \"tuned-bk5w8\" (UID: \"9cc406f3-cdff-4cde-ac7b-ff31abb20e0d\") " pod="openshift-cluster-node-tuning-operator/tuned-bk5w8" Jul 27 09:24:34.495930 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:34.493792 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/5c031f05-bf4a-4ff7-81ad-381d31b163ff-host-kubelet\") pod \"ovnkube-node-mqhsn\" (UID: \"5c031f05-bf4a-4ff7-81ad-381d31b163ff\") " pod="openshift-ovn-kubernetes/ovnkube-node-mqhsn" Jul 27 09:24:34.495930 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:34.493817 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/5c031f05-bf4a-4ff7-81ad-381d31b163ff-env-overrides\") pod \"ovnkube-node-mqhsn\" (UID: \"5c031f05-bf4a-4ff7-81ad-381d31b163ff\") " pod="openshift-ovn-kubernetes/ovnkube-node-mqhsn" Jul 27 09:24:34.495930 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:34.493859 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5c031f05-bf4a-4ff7-81ad-381d31b163ff-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-mqhsn\" (UID: \"5c031f05-bf4a-4ff7-81ad-381d31b163ff\") " pod="openshift-ovn-kubernetes/ovnkube-node-mqhsn" Jul 27 09:24:34.495930 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:34.493863 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/5c031f05-bf4a-4ff7-81ad-381d31b163ff-ovnkube-script-lib\") pod \"ovnkube-node-mqhsn\" (UID: \"5c031f05-bf4a-4ff7-81ad-381d31b163ff\") " pod="openshift-ovn-kubernetes/ovnkube-node-mqhsn" Jul 27 09:24:34.495930 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:34.493913 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/5c031f05-bf4a-4ff7-81ad-381d31b163ff-host-kubelet\") pod \"ovnkube-node-mqhsn\" (UID: \"5c031f05-bf4a-4ff7-81ad-381d31b163ff\") " pod="openshift-ovn-kubernetes/ovnkube-node-mqhsn" Jul 27 09:24:34.495930 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:34.494005 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/d0206685-d6b8-43ea-a691-ca3c1acd237a-tuning-conf-dir\") pod \"multus-additional-cni-plugins-54xcx\" (UID: \"d0206685-d6b8-43ea-a691-ca3c1acd237a\") " pod="openshift-multus/multus-additional-cni-plugins-54xcx" Jul 27 09:24:34.495930 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:34.494028 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/d0206685-d6b8-43ea-a691-ca3c1acd237a-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-54xcx\" (UID: \"d0206685-d6b8-43ea-a691-ca3c1acd237a\") " pod="openshift-multus/multus-additional-cni-plugins-54xcx" Jul 27 09:24:34.495930 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:34.494077 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/996c29bd-3969-4268-a795-4f4d21881d48-serviceca\") pod \"node-ca-bmklx\" (UID: \"996c29bd-3969-4268-a795-4f4d21881d48\") " pod="openshift-image-registry/node-ca-bmklx" Jul 27 09:24:34.495930 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:34.494095 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mcwvb\" (UniqueName: \"kubernetes.io/projected/996c29bd-3969-4268-a795-4f4d21881d48-kube-api-access-mcwvb\") pod \"node-ca-bmklx\" (UID: \"996c29bd-3969-4268-a795-4f4d21881d48\") " pod="openshift-image-registry/node-ca-bmklx" Jul 27 09:24:34.495930 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:34.494114 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/6fd3f632-9d80-4e6e-8d4f-f5314c0f174a-host-slash\") pod \"iptables-alerter-rcftn\" (UID: \"6fd3f632-9d80-4e6e-8d4f-f5314c0f174a\") " pod="openshift-network-operator/iptables-alerter-rcftn" Jul 27 09:24:34.495930 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:34.494135 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/204c97b6-2efe-4db7-bc9f-a5653fed77c7-kubelet-config\") pod \"global-pull-secret-syncer-rqz85\" (UID: \"204c97b6-2efe-4db7-bc9f-a5653fed77c7\") " pod="kube-system/global-pull-secret-syncer-rqz85" Jul 27 09:24:34.495930 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:34.494152 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/09e4c928-4b42-484e-808c-dfaa1b17977b-sys-fs\") pod \"aws-ebs-csi-driver-node-kb5qg\" (UID: \"09e4c928-4b42-484e-808c-dfaa1b17977b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kb5qg" Jul 27 09:24:34.495930 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:34.494168 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/5c031f05-bf4a-4ff7-81ad-381d31b163ff-host-cni-bin\") pod \"ovnkube-node-mqhsn\" (UID: \"5c031f05-bf4a-4ff7-81ad-381d31b163ff\") " pod="openshift-ovn-kubernetes/ovnkube-node-mqhsn" Jul 27 09:24:34.495930 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:34.494185 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/d0206685-d6b8-43ea-a691-ca3c1acd237a-cni-binary-copy\") pod \"multus-additional-cni-plugins-54xcx\" (UID: \"d0206685-d6b8-43ea-a691-ca3c1acd237a\") " pod="openshift-multus/multus-additional-cni-plugins-54xcx" Jul 27 09:24:34.495930 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:34.494275 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/5c031f05-bf4a-4ff7-81ad-381d31b163ff-host-cni-bin\") pod \"ovnkube-node-mqhsn\" (UID: \"5c031f05-bf4a-4ff7-81ad-381d31b163ff\") " pod="openshift-ovn-kubernetes/ovnkube-node-mqhsn" Jul 27 09:24:34.495930 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:34.494276 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/5c031f05-bf4a-4ff7-81ad-381d31b163ff-env-overrides\") pod \"ovnkube-node-mqhsn\" (UID: \"5c031f05-bf4a-4ff7-81ad-381d31b163ff\") " pod="openshift-ovn-kubernetes/ovnkube-node-mqhsn" Jul 27 09:24:34.495930 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:34.494510 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/5c031f05-bf4a-4ff7-81ad-381d31b163ff-ovnkube-script-lib\") pod \"ovnkube-node-mqhsn\" (UID: \"5c031f05-bf4a-4ff7-81ad-381d31b163ff\") " pod="openshift-ovn-kubernetes/ovnkube-node-mqhsn" Jul 27 09:24:34.498741 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:34.498668 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/5c031f05-bf4a-4ff7-81ad-381d31b163ff-ovn-node-metrics-cert\") pod \"ovnkube-node-mqhsn\" (UID: \"5c031f05-bf4a-4ff7-81ad-381d31b163ff\") " pod="openshift-ovn-kubernetes/ovnkube-node-mqhsn" Jul 27 09:24:34.499050 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:34.499005 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/2c043d82-d300-41ba-b138-60a03d65ea11-agent-certs\") pod \"konnectivity-agent-848w4\" (UID: \"2c043d82-d300-41ba-b138-60a03d65ea11\") " pod="kube-system/konnectivity-agent-848w4" Jul 27 09:24:34.503063 ip-10-0-128-103 kubenswrapper[2565]: E0727 09:24:34.503039 2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 27 09:24:34.503063 ip-10-0-128-103 kubenswrapper[2565]: E0727 09:24:34.503066 2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 27 09:24:34.503245 ip-10-0-128-103 kubenswrapper[2565]: E0727 09:24:34.503080 2565 projected.go:196] Error preparing data for projected volume kube-api-access-2ckrw for pod openshift-network-diagnostics/network-check-target-5zqgm: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 27 09:24:34.503245 ip-10-0-128-103 kubenswrapper[2565]: E0727 09:24:34.503159 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/91816deb-5261-44b7-857a-04cf036feffa-kube-api-access-2ckrw podName:91816deb-5261-44b7-857a-04cf036feffa nodeName:}" failed. No retries permitted until 2026-07-27 09:24:35.003137494 +0000 UTC m=+2.966923476 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-2ckrw" (UniqueName: "kubernetes.io/projected/91816deb-5261-44b7-857a-04cf036feffa-kube-api-access-2ckrw") pod "network-check-target-5zqgm" (UID: "91816deb-5261-44b7-857a-04cf036feffa") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 27 09:24:34.505953 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:34.505921 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4pnnj\" (UniqueName: \"kubernetes.io/projected/0d6ba35e-ec8a-4ec9-8520-0d32a5586a1d-kube-api-access-4pnnj\") pod \"network-metrics-daemon-trcl2\" (UID: \"0d6ba35e-ec8a-4ec9-8520-0d32a5586a1d\") " pod="openshift-multus/network-metrics-daemon-trcl2" Jul 27 09:24:34.506039 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:34.506006 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b8hnw\" (UniqueName: \"kubernetes.io/projected/5c031f05-bf4a-4ff7-81ad-381d31b163ff-kube-api-access-b8hnw\") pod \"ovnkube-node-mqhsn\" (UID: \"5c031f05-bf4a-4ff7-81ad-381d31b163ff\") " pod="openshift-ovn-kubernetes/ovnkube-node-mqhsn" Jul 27 09:24:34.521130 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:34.521096 2565 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 27 09:24:34.563171 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:34.563105 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-103.ec2.internal" event={"ID":"f9f6fe57433891594e1efa7d6875336f","Type":"ContainerStarted","Data":"a78e391be9f2e4e5713bffa26ae33ece50f7aa3c6d1fd2c16816e0cbd4325541"} Jul 27 09:24:34.564542 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:34.564509 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-103.ec2.internal" event={"ID":"519b669acb80623490c3cff7ef66ed98","Type":"ContainerStarted","Data":"d4e3de8edbf38384fc22c5b147ca1ead30d7fe7a7a778b9d76c3ee3cb17c3ca2"} Jul 27 09:24:34.594901 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:34.594823 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/9cc406f3-cdff-4cde-ac7b-ff31abb20e0d-run\") pod \"tuned-bk5w8\" (UID: \"9cc406f3-cdff-4cde-ac7b-ff31abb20e0d\") " pod="openshift-cluster-node-tuning-operator/tuned-bk5w8" Jul 27 09:24:34.594901 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:34.594877 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/d0206685-d6b8-43ea-a691-ca3c1acd237a-tuning-conf-dir\") pod \"multus-additional-cni-plugins-54xcx\" (UID: \"d0206685-d6b8-43ea-a691-ca3c1acd237a\") " pod="openshift-multus/multus-additional-cni-plugins-54xcx" Jul 27 09:24:34.594901 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:34.594904 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/d0206685-d6b8-43ea-a691-ca3c1acd237a-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-54xcx\" (UID: \"d0206685-d6b8-43ea-a691-ca3c1acd237a\") " pod="openshift-multus/multus-additional-cni-plugins-54xcx" Jul 27 09:24:34.595196 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:34.594932 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/996c29bd-3969-4268-a795-4f4d21881d48-serviceca\") pod \"node-ca-bmklx\" (UID: \"996c29bd-3969-4268-a795-4f4d21881d48\") " pod="openshift-image-registry/node-ca-bmklx" Jul 27 09:24:34.595196 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:34.594974 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mcwvb\" (UniqueName: \"kubernetes.io/projected/996c29bd-3969-4268-a795-4f4d21881d48-kube-api-access-mcwvb\") pod \"node-ca-bmklx\" (UID: \"996c29bd-3969-4268-a795-4f4d21881d48\") " pod="openshift-image-registry/node-ca-bmklx" Jul 27 09:24:34.595196 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:34.594998 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/6fd3f632-9d80-4e6e-8d4f-f5314c0f174a-host-slash\") pod \"iptables-alerter-rcftn\" (UID: \"6fd3f632-9d80-4e6e-8d4f-f5314c0f174a\") " pod="openshift-network-operator/iptables-alerter-rcftn" Jul 27 09:24:34.595196 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:34.595021 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/204c97b6-2efe-4db7-bc9f-a5653fed77c7-kubelet-config\") pod \"global-pull-secret-syncer-rqz85\" (UID: \"204c97b6-2efe-4db7-bc9f-a5653fed77c7\") " pod="kube-system/global-pull-secret-syncer-rqz85" Jul 27 09:24:34.595196 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:34.595046 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/09e4c928-4b42-484e-808c-dfaa1b17977b-sys-fs\") pod \"aws-ebs-csi-driver-node-kb5qg\" (UID: \"09e4c928-4b42-484e-808c-dfaa1b17977b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kb5qg" Jul 27 09:24:34.595196 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:34.595071 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/d0206685-d6b8-43ea-a691-ca3c1acd237a-cni-binary-copy\") pod \"multus-additional-cni-plugins-54xcx\" (UID: \"d0206685-d6b8-43ea-a691-ca3c1acd237a\") " pod="openshift-multus/multus-additional-cni-plugins-54xcx" Jul 27 09:24:34.595196 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:34.595095 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mp4mq\" (UniqueName: \"kubernetes.io/projected/29406f41-dc4d-4f8a-9627-9589659ae0d4-kube-api-access-mp4mq\") pod \"multus-jmgz6\" (UID: \"29406f41-dc4d-4f8a-9627-9589659ae0d4\") " pod="openshift-multus/multus-jmgz6" Jul 27 09:24:34.595196 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:34.595077 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/9cc406f3-cdff-4cde-ac7b-ff31abb20e0d-run\") pod \"tuned-bk5w8\" (UID: \"9cc406f3-cdff-4cde-ac7b-ff31abb20e0d\") " pod="openshift-cluster-node-tuning-operator/tuned-bk5w8" Jul 27 09:24:34.595196 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:34.595137 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/d0206685-d6b8-43ea-a691-ca3c1acd237a-tuning-conf-dir\") pod \"multus-additional-cni-plugins-54xcx\" (UID: \"d0206685-d6b8-43ea-a691-ca3c1acd237a\") " pod="openshift-multus/multus-additional-cni-plugins-54xcx" Jul 27 09:24:34.595196 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:34.595153 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/09e4c928-4b42-484e-808c-dfaa1b17977b-etc-selinux\") pod \"aws-ebs-csi-driver-node-kb5qg\" (UID: \"09e4c928-4b42-484e-808c-dfaa1b17977b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kb5qg" Jul 27 09:24:34.595196 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:34.595186 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/9cc406f3-cdff-4cde-ac7b-ff31abb20e0d-etc-sysctl-d\") pod \"tuned-bk5w8\" (UID: \"9cc406f3-cdff-4cde-ac7b-ff31abb20e0d\") " pod="openshift-cluster-node-tuning-operator/tuned-bk5w8" Jul 27 09:24:34.595728 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:34.595217 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/204c97b6-2efe-4db7-bc9f-a5653fed77c7-original-pull-secret\") pod \"global-pull-secret-syncer-rqz85\" (UID: \"204c97b6-2efe-4db7-bc9f-a5653fed77c7\") " pod="kube-system/global-pull-secret-syncer-rqz85" Jul 27 09:24:34.595728 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:34.595247 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/29406f41-dc4d-4f8a-9627-9589659ae0d4-multus-cni-dir\") pod \"multus-jmgz6\" (UID: \"29406f41-dc4d-4f8a-9627-9589659ae0d4\") " pod="openshift-multus/multus-jmgz6" Jul 27 09:24:34.595728 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:34.595272 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/29406f41-dc4d-4f8a-9627-9589659ae0d4-cni-binary-copy\") pod \"multus-jmgz6\" (UID: \"29406f41-dc4d-4f8a-9627-9589659ae0d4\") " pod="openshift-multus/multus-jmgz6" Jul 27 09:24:34.595728 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:34.595296 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/29406f41-dc4d-4f8a-9627-9589659ae0d4-hostroot\") pod \"multus-jmgz6\" (UID: \"29406f41-dc4d-4f8a-9627-9589659ae0d4\") " pod="openshift-multus/multus-jmgz6" Jul 27 09:24:34.595728 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:34.595323 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/204c97b6-2efe-4db7-bc9f-a5653fed77c7-dbus\") pod \"global-pull-secret-syncer-rqz85\" (UID: \"204c97b6-2efe-4db7-bc9f-a5653fed77c7\") " pod="kube-system/global-pull-secret-syncer-rqz85" Jul 27 09:24:34.595728 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:34.595352 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/9cc406f3-cdff-4cde-ac7b-ff31abb20e0d-etc-systemd\") pod \"tuned-bk5w8\" (UID: \"9cc406f3-cdff-4cde-ac7b-ff31abb20e0d\") " pod="openshift-cluster-node-tuning-operator/tuned-bk5w8" Jul 27 09:24:34.595728 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:34.595376 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9cc406f3-cdff-4cde-ac7b-ff31abb20e0d-sys\") pod \"tuned-bk5w8\" (UID: \"9cc406f3-cdff-4cde-ac7b-ff31abb20e0d\") " pod="openshift-cluster-node-tuning-operator/tuned-bk5w8" Jul 27 09:24:34.595728 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:34.595400 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/9cc406f3-cdff-4cde-ac7b-ff31abb20e0d-host\") pod \"tuned-bk5w8\" (UID: \"9cc406f3-cdff-4cde-ac7b-ff31abb20e0d\") " pod="openshift-cluster-node-tuning-operator/tuned-bk5w8" Jul 27 09:24:34.595728 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:34.595428 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w2t6q\" (UniqueName: \"kubernetes.io/projected/9cc406f3-cdff-4cde-ac7b-ff31abb20e0d-kube-api-access-w2t6q\") pod \"tuned-bk5w8\" (UID: \"9cc406f3-cdff-4cde-ac7b-ff31abb20e0d\") " pod="openshift-cluster-node-tuning-operator/tuned-bk5w8" Jul 27 09:24:34.595728 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:34.595456 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/9cc406f3-cdff-4cde-ac7b-ff31abb20e0d-etc-sysctl-conf\") pod \"tuned-bk5w8\" (UID: \"9cc406f3-cdff-4cde-ac7b-ff31abb20e0d\") " pod="openshift-cluster-node-tuning-operator/tuned-bk5w8" Jul 27 09:24:34.595728 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:34.595479 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9cc406f3-cdff-4cde-ac7b-ff31abb20e0d-tmp\") pod \"tuned-bk5w8\" (UID: \"9cc406f3-cdff-4cde-ac7b-ff31abb20e0d\") " pod="openshift-cluster-node-tuning-operator/tuned-bk5w8" Jul 27 09:24:34.595728 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:34.595481 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/204c97b6-2efe-4db7-bc9f-a5653fed77c7-kubelet-config\") pod \"global-pull-secret-syncer-rqz85\" (UID: \"204c97b6-2efe-4db7-bc9f-a5653fed77c7\") " pod="kube-system/global-pull-secret-syncer-rqz85" Jul 27 09:24:34.595728 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:34.595424 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/6fd3f632-9d80-4e6e-8d4f-f5314c0f174a-host-slash\") pod \"iptables-alerter-rcftn\" (UID: \"6fd3f632-9d80-4e6e-8d4f-f5314c0f174a\") " pod="openshift-network-operator/iptables-alerter-rcftn" Jul 27 09:24:34.595728 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:34.595518 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/09e4c928-4b42-484e-808c-dfaa1b17977b-sys-fs\") pod \"aws-ebs-csi-driver-node-kb5qg\" (UID: \"09e4c928-4b42-484e-808c-dfaa1b17977b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kb5qg" Jul 27 09:24:34.595728 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:34.595539 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/29406f41-dc4d-4f8a-9627-9589659ae0d4-hostroot\") pod \"multus-jmgz6\" (UID: \"29406f41-dc4d-4f8a-9627-9589659ae0d4\") " pod="openshift-multus/multus-jmgz6" Jul 27 09:24:34.595728 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:34.595558 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9cc406f3-cdff-4cde-ac7b-ff31abb20e0d-sys\") pod \"tuned-bk5w8\" (UID: \"9cc406f3-cdff-4cde-ac7b-ff31abb20e0d\") " pod="openshift-cluster-node-tuning-operator/tuned-bk5w8" Jul 27 09:24:34.595728 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:34.595558 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/9cc406f3-cdff-4cde-ac7b-ff31abb20e0d-etc-systemd\") pod \"tuned-bk5w8\" (UID: \"9cc406f3-cdff-4cde-ac7b-ff31abb20e0d\") " pod="openshift-cluster-node-tuning-operator/tuned-bk5w8" Jul 27 09:24:34.595728 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:34.595569 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/996c29bd-3969-4268-a795-4f4d21881d48-serviceca\") pod \"node-ca-bmklx\" (UID: \"996c29bd-3969-4268-a795-4f4d21881d48\") " pod="openshift-image-registry/node-ca-bmklx" Jul 27 09:24:34.596627 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:34.595592 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/9cc406f3-cdff-4cde-ac7b-ff31abb20e0d-host\") pod \"tuned-bk5w8\" (UID: \"9cc406f3-cdff-4cde-ac7b-ff31abb20e0d\") " pod="openshift-cluster-node-tuning-operator/tuned-bk5w8" Jul 27 09:24:34.596627 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:34.595615 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/29406f41-dc4d-4f8a-9627-9589659ae0d4-etc-kubernetes\") pod \"multus-jmgz6\" (UID: \"29406f41-dc4d-4f8a-9627-9589659ae0d4\") " pod="openshift-multus/multus-jmgz6" Jul 27 09:24:34.596627 ip-10-0-128-103 kubenswrapper[2565]: E0727 09:24:34.595634 2565 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 27 09:24:34.596627 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:34.595644 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mblhr\" (UniqueName: \"kubernetes.io/projected/d0206685-d6b8-43ea-a691-ca3c1acd237a-kube-api-access-mblhr\") pod \"multus-additional-cni-plugins-54xcx\" (UID: \"d0206685-d6b8-43ea-a691-ca3c1acd237a\") " pod="openshift-multus/multus-additional-cni-plugins-54xcx" Jul 27 09:24:34.596627 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:34.595675 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9cc406f3-cdff-4cde-ac7b-ff31abb20e0d-etc-kubernetes\") pod \"tuned-bk5w8\" (UID: \"9cc406f3-cdff-4cde-ac7b-ff31abb20e0d\") " pod="openshift-cluster-node-tuning-operator/tuned-bk5w8" Jul 27 09:24:34.596627 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:34.595690 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/09e4c928-4b42-484e-808c-dfaa1b17977b-etc-selinux\") pod \"aws-ebs-csi-driver-node-kb5qg\" (UID: \"09e4c928-4b42-484e-808c-dfaa1b17977b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kb5qg" Jul 27 09:24:34.596627 ip-10-0-128-103 kubenswrapper[2565]: E0727 09:24:34.595703 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/204c97b6-2efe-4db7-bc9f-a5653fed77c7-original-pull-secret podName:204c97b6-2efe-4db7-bc9f-a5653fed77c7 nodeName:}" failed. No retries permitted until 2026-07-27 09:24:35.095682386 +0000 UTC m=+3.059468337 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/204c97b6-2efe-4db7-bc9f-a5653fed77c7-original-pull-secret") pod "global-pull-secret-syncer-rqz85" (UID: "204c97b6-2efe-4db7-bc9f-a5653fed77c7") : object "kube-system"/"original-pull-secret" not registered Jul 27 09:24:34.596627 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:34.595718 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9cc406f3-cdff-4cde-ac7b-ff31abb20e0d-etc-kubernetes\") pod \"tuned-bk5w8\" (UID: \"9cc406f3-cdff-4cde-ac7b-ff31abb20e0d\") " pod="openshift-cluster-node-tuning-operator/tuned-bk5w8" Jul 27 09:24:34.596627 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:34.595758 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/29406f41-dc4d-4f8a-9627-9589659ae0d4-etc-kubernetes\") pod \"multus-jmgz6\" (UID: \"29406f41-dc4d-4f8a-9627-9589659ae0d4\") " pod="openshift-multus/multus-jmgz6" Jul 27 09:24:34.596627 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:34.595819 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/9cc406f3-cdff-4cde-ac7b-ff31abb20e0d-etc-sysctl-d\") pod \"tuned-bk5w8\" (UID: \"9cc406f3-cdff-4cde-ac7b-ff31abb20e0d\") " pod="openshift-cluster-node-tuning-operator/tuned-bk5w8" Jul 27 09:24:34.596627 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:34.595646 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/9cc406f3-cdff-4cde-ac7b-ff31abb20e0d-etc-sysctl-conf\") pod \"tuned-bk5w8\" (UID: \"9cc406f3-cdff-4cde-ac7b-ff31abb20e0d\") " pod="openshift-cluster-node-tuning-operator/tuned-bk5w8" Jul 27 09:24:34.596627 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:34.595717 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/204c97b6-2efe-4db7-bc9f-a5653fed77c7-dbus\") pod \"global-pull-secret-syncer-rqz85\" (UID: \"204c97b6-2efe-4db7-bc9f-a5653fed77c7\") " pod="kube-system/global-pull-secret-syncer-rqz85" Jul 27 09:24:34.596627 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:34.595936 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/9cc406f3-cdff-4cde-ac7b-ff31abb20e0d-lib-modules\") pod \"tuned-bk5w8\" (UID: \"9cc406f3-cdff-4cde-ac7b-ff31abb20e0d\") " pod="openshift-cluster-node-tuning-operator/tuned-bk5w8" Jul 27 09:24:34.596627 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:34.595968 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/996c29bd-3969-4268-a795-4f4d21881d48-host\") pod \"node-ca-bmklx\" (UID: \"996c29bd-3969-4268-a795-4f4d21881d48\") " pod="openshift-image-registry/node-ca-bmklx" Jul 27 09:24:34.596627 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:34.595996 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/29406f41-dc4d-4f8a-9627-9589659ae0d4-host-var-lib-cni-bin\") pod \"multus-jmgz6\" (UID: \"29406f41-dc4d-4f8a-9627-9589659ae0d4\") " pod="openshift-multus/multus-jmgz6" Jul 27 09:24:34.596627 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:34.596007 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/29406f41-dc4d-4f8a-9627-9589659ae0d4-multus-cni-dir\") pod \"multus-jmgz6\" (UID: \"29406f41-dc4d-4f8a-9627-9589659ae0d4\") " pod="openshift-multus/multus-jmgz6" Jul 27 09:24:34.596627 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:34.596023 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/29406f41-dc4d-4f8a-9627-9589659ae0d4-multus-conf-dir\") pod \"multus-jmgz6\" (UID: \"29406f41-dc4d-4f8a-9627-9589659ae0d4\") " pod="openshift-multus/multus-jmgz6" Jul 27 09:24:34.597485 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:34.596040 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/d0206685-d6b8-43ea-a691-ca3c1acd237a-cni-binary-copy\") pod \"multus-additional-cni-plugins-54xcx\" (UID: \"d0206685-d6b8-43ea-a691-ca3c1acd237a\") " pod="openshift-multus/multus-additional-cni-plugins-54xcx" Jul 27 09:24:34.597485 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:34.596051 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/29406f41-dc4d-4f8a-9627-9589659ae0d4-multus-daemon-config\") pod \"multus-jmgz6\" (UID: \"29406f41-dc4d-4f8a-9627-9589659ae0d4\") " pod="openshift-multus/multus-jmgz6" Jul 27 09:24:34.597485 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:34.596081 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/09e4c928-4b42-484e-808c-dfaa1b17977b-kubelet-dir\") pod \"aws-ebs-csi-driver-node-kb5qg\" (UID: \"09e4c928-4b42-484e-808c-dfaa1b17977b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kb5qg" Jul 27 09:24:34.597485 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:34.596085 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/29406f41-dc4d-4f8a-9627-9589659ae0d4-multus-conf-dir\") pod \"multus-jmgz6\" (UID: \"29406f41-dc4d-4f8a-9627-9589659ae0d4\") " pod="openshift-multus/multus-jmgz6" Jul 27 09:24:34.597485 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:34.596054 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/29406f41-dc4d-4f8a-9627-9589659ae0d4-host-var-lib-cni-bin\") pod \"multus-jmgz6\" (UID: \"29406f41-dc4d-4f8a-9627-9589659ae0d4\") " pod="openshift-multus/multus-jmgz6" Jul 27 09:24:34.597485 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:34.596101 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/9cc406f3-cdff-4cde-ac7b-ff31abb20e0d-lib-modules\") pod \"tuned-bk5w8\" (UID: \"9cc406f3-cdff-4cde-ac7b-ff31abb20e0d\") " pod="openshift-cluster-node-tuning-operator/tuned-bk5w8" Jul 27 09:24:34.597485 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:34.596113 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/6fd3f632-9d80-4e6e-8d4f-f5314c0f174a-iptables-alerter-script\") pod \"iptables-alerter-rcftn\" (UID: \"6fd3f632-9d80-4e6e-8d4f-f5314c0f174a\") " pod="openshift-network-operator/iptables-alerter-rcftn" Jul 27 09:24:34.597485 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:34.596121 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/29406f41-dc4d-4f8a-9627-9589659ae0d4-cni-binary-copy\") pod \"multus-jmgz6\" (UID: \"29406f41-dc4d-4f8a-9627-9589659ae0d4\") " pod="openshift-multus/multus-jmgz6" Jul 27 09:24:34.597485 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:34.596054 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/996c29bd-3969-4268-a795-4f4d21881d48-host\") pod \"node-ca-bmklx\" (UID: \"996c29bd-3969-4268-a795-4f4d21881d48\") " pod="openshift-image-registry/node-ca-bmklx" Jul 27 09:24:34.597485 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:34.596142 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/09e4c928-4b42-484e-808c-dfaa1b17977b-kubelet-dir\") pod \"aws-ebs-csi-driver-node-kb5qg\" (UID: \"09e4c928-4b42-484e-808c-dfaa1b17977b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kb5qg" Jul 27 09:24:34.597485 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:34.596160 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/29406f41-dc4d-4f8a-9627-9589659ae0d4-cnibin\") pod \"multus-jmgz6\" (UID: \"29406f41-dc4d-4f8a-9627-9589659ae0d4\") " pod="openshift-multus/multus-jmgz6" Jul 27 09:24:34.597485 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:34.596458 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/29406f41-dc4d-4f8a-9627-9589659ae0d4-host-run-k8s-cni-cncf-io\") pod \"multus-jmgz6\" (UID: \"29406f41-dc4d-4f8a-9627-9589659ae0d4\") " pod="openshift-multus/multus-jmgz6" Jul 27 09:24:34.597485 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:34.596489 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q2z5h\" (UniqueName: \"kubernetes.io/projected/09e4c928-4b42-484e-808c-dfaa1b17977b-kube-api-access-q2z5h\") pod \"aws-ebs-csi-driver-node-kb5qg\" (UID: \"09e4c928-4b42-484e-808c-dfaa1b17977b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kb5qg" Jul 27 09:24:34.597485 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:34.596526 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/9cc406f3-cdff-4cde-ac7b-ff31abb20e0d-etc-modprobe-d\") pod \"tuned-bk5w8\" (UID: \"9cc406f3-cdff-4cde-ac7b-ff31abb20e0d\") " pod="openshift-cluster-node-tuning-operator/tuned-bk5w8" Jul 27 09:24:34.597485 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:34.596532 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/29406f41-dc4d-4f8a-9627-9589659ae0d4-host-run-k8s-cni-cncf-io\") pod \"multus-jmgz6\" (UID: \"29406f41-dc4d-4f8a-9627-9589659ae0d4\") " pod="openshift-multus/multus-jmgz6" Jul 27 09:24:34.597485 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:34.596541 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/29406f41-dc4d-4f8a-9627-9589659ae0d4-multus-daemon-config\") pod \"multus-jmgz6\" (UID: \"29406f41-dc4d-4f8a-9627-9589659ae0d4\") " pod="openshift-multus/multus-jmgz6" Jul 27 09:24:34.597485 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:34.596228 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/29406f41-dc4d-4f8a-9627-9589659ae0d4-cnibin\") pod \"multus-jmgz6\" (UID: \"29406f41-dc4d-4f8a-9627-9589659ae0d4\") " pod="openshift-multus/multus-jmgz6" Jul 27 09:24:34.597485 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:34.596383 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/d0206685-d6b8-43ea-a691-ca3c1acd237a-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-54xcx\" (UID: \"d0206685-d6b8-43ea-a691-ca3c1acd237a\") " pod="openshift-multus/multus-additional-cni-plugins-54xcx" Jul 27 09:24:34.598299 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:34.596606 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/29406f41-dc4d-4f8a-9627-9589659ae0d4-host-var-lib-cni-multus\") pod \"multus-jmgz6\" (UID: \"29406f41-dc4d-4f8a-9627-9589659ae0d4\") " pod="openshift-multus/multus-jmgz6" Jul 27 09:24:34.598299 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:34.596553 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/29406f41-dc4d-4f8a-9627-9589659ae0d4-host-var-lib-cni-multus\") pod \"multus-jmgz6\" (UID: \"29406f41-dc4d-4f8a-9627-9589659ae0d4\") " pod="openshift-multus/multus-jmgz6" Jul 27 09:24:34.598299 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:34.596641 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/29406f41-dc4d-4f8a-9627-9589659ae0d4-host-var-lib-kubelet\") pod \"multus-jmgz6\" (UID: \"29406f41-dc4d-4f8a-9627-9589659ae0d4\") " pod="openshift-multus/multus-jmgz6" Jul 27 09:24:34.598299 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:34.596665 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/29406f41-dc4d-4f8a-9627-9589659ae0d4-host-run-multus-certs\") pod \"multus-jmgz6\" (UID: \"29406f41-dc4d-4f8a-9627-9589659ae0d4\") " pod="openshift-multus/multus-jmgz6" Jul 27 09:24:34.598299 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:34.596679 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/9cc406f3-cdff-4cde-ac7b-ff31abb20e0d-etc-modprobe-d\") pod \"tuned-bk5w8\" (UID: \"9cc406f3-cdff-4cde-ac7b-ff31abb20e0d\") " pod="openshift-cluster-node-tuning-operator/tuned-bk5w8" Jul 27 09:24:34.598299 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:34.596691 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/9cc406f3-cdff-4cde-ac7b-ff31abb20e0d-etc-sysconfig\") pod \"tuned-bk5w8\" (UID: \"9cc406f3-cdff-4cde-ac7b-ff31abb20e0d\") " pod="openshift-cluster-node-tuning-operator/tuned-bk5w8" Jul 27 09:24:34.598299 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:34.596717 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/9cc406f3-cdff-4cde-ac7b-ff31abb20e0d-etc-tuned\") pod \"tuned-bk5w8\" (UID: \"9cc406f3-cdff-4cde-ac7b-ff31abb20e0d\") " pod="openshift-cluster-node-tuning-operator/tuned-bk5w8" Jul 27 09:24:34.598299 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:34.596727 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/29406f41-dc4d-4f8a-9627-9589659ae0d4-host-run-multus-certs\") pod \"multus-jmgz6\" (UID: \"29406f41-dc4d-4f8a-9627-9589659ae0d4\") " pod="openshift-multus/multus-jmgz6" Jul 27 09:24:34.598299 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:34.596741 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/29406f41-dc4d-4f8a-9627-9589659ae0d4-host-var-lib-kubelet\") pod \"multus-jmgz6\" (UID: \"29406f41-dc4d-4f8a-9627-9589659ae0d4\") " pod="openshift-multus/multus-jmgz6" Jul 27 09:24:34.598299 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:34.596664 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/6fd3f632-9d80-4e6e-8d4f-f5314c0f174a-iptables-alerter-script\") pod \"iptables-alerter-rcftn\" (UID: \"6fd3f632-9d80-4e6e-8d4f-f5314c0f174a\") " pod="openshift-network-operator/iptables-alerter-rcftn" Jul 27 09:24:34.598299 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:34.596795 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/9cc406f3-cdff-4cde-ac7b-ff31abb20e0d-etc-sysconfig\") pod \"tuned-bk5w8\" (UID: \"9cc406f3-cdff-4cde-ac7b-ff31abb20e0d\") " pod="openshift-cluster-node-tuning-operator/tuned-bk5w8" Jul 27 09:24:34.598299 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:34.596815 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/29406f41-dc4d-4f8a-9627-9589659ae0d4-os-release\") pod \"multus-jmgz6\" (UID: \"29406f41-dc4d-4f8a-9627-9589659ae0d4\") " pod="openshift-multus/multus-jmgz6" Jul 27 09:24:34.598299 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:34.596847 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/29406f41-dc4d-4f8a-9627-9589659ae0d4-host-run-netns\") pod \"multus-jmgz6\" (UID: \"29406f41-dc4d-4f8a-9627-9589659ae0d4\") " pod="openshift-multus/multus-jmgz6" Jul 27 09:24:34.598299 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:34.596886 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sqdxv\" (UniqueName: \"kubernetes.io/projected/6fd3f632-9d80-4e6e-8d4f-f5314c0f174a-kube-api-access-sqdxv\") pod \"iptables-alerter-rcftn\" (UID: \"6fd3f632-9d80-4e6e-8d4f-f5314c0f174a\") " pod="openshift-network-operator/iptables-alerter-rcftn" Jul 27 09:24:34.598299 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:34.596913 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d0206685-d6b8-43ea-a691-ca3c1acd237a-system-cni-dir\") pod \"multus-additional-cni-plugins-54xcx\" (UID: \"d0206685-d6b8-43ea-a691-ca3c1acd237a\") " pod="openshift-multus/multus-additional-cni-plugins-54xcx" Jul 27 09:24:34.598299 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:34.596932 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/29406f41-dc4d-4f8a-9627-9589659ae0d4-os-release\") pod \"multus-jmgz6\" (UID: \"29406f41-dc4d-4f8a-9627-9589659ae0d4\") " pod="openshift-multus/multus-jmgz6" Jul 27 09:24:34.598299 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:34.596941 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/09e4c928-4b42-484e-808c-dfaa1b17977b-registration-dir\") pod \"aws-ebs-csi-driver-node-kb5qg\" (UID: \"09e4c928-4b42-484e-808c-dfaa1b17977b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kb5qg" Jul 27 09:24:34.598299 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:34.596972 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/29406f41-dc4d-4f8a-9627-9589659ae0d4-multus-socket-dir-parent\") pod \"multus-jmgz6\" (UID: \"29406f41-dc4d-4f8a-9627-9589659ae0d4\") " pod="openshift-multus/multus-jmgz6" Jul 27 09:24:34.599209 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:34.596954 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/29406f41-dc4d-4f8a-9627-9589659ae0d4-host-run-netns\") pod \"multus-jmgz6\" (UID: \"29406f41-dc4d-4f8a-9627-9589659ae0d4\") " pod="openshift-multus/multus-jmgz6" Jul 27 09:24:34.599209 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:34.596997 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/d0206685-d6b8-43ea-a691-ca3c1acd237a-cnibin\") pod \"multus-additional-cni-plugins-54xcx\" (UID: \"d0206685-d6b8-43ea-a691-ca3c1acd237a\") " pod="openshift-multus/multus-additional-cni-plugins-54xcx" Jul 27 09:24:34.599209 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:34.597025 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/d0206685-d6b8-43ea-a691-ca3c1acd237a-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-54xcx\" (UID: \"d0206685-d6b8-43ea-a691-ca3c1acd237a\") " pod="openshift-multus/multus-additional-cni-plugins-54xcx" Jul 27 09:24:34.599209 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:34.597047 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/09e4c928-4b42-484e-808c-dfaa1b17977b-registration-dir\") pod \"aws-ebs-csi-driver-node-kb5qg\" (UID: \"09e4c928-4b42-484e-808c-dfaa1b17977b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kb5qg" Jul 27 09:24:34.599209 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:34.597093 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/29406f41-dc4d-4f8a-9627-9589659ae0d4-multus-socket-dir-parent\") pod \"multus-jmgz6\" (UID: \"29406f41-dc4d-4f8a-9627-9589659ae0d4\") " pod="openshift-multus/multus-jmgz6" Jul 27 09:24:34.599209 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:34.597095 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d0206685-d6b8-43ea-a691-ca3c1acd237a-system-cni-dir\") pod \"multus-additional-cni-plugins-54xcx\" (UID: \"d0206685-d6b8-43ea-a691-ca3c1acd237a\") " pod="openshift-multus/multus-additional-cni-plugins-54xcx" Jul 27 09:24:34.599209 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:34.597130 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/d0206685-d6b8-43ea-a691-ca3c1acd237a-cnibin\") pod \"multus-additional-cni-plugins-54xcx\" (UID: \"d0206685-d6b8-43ea-a691-ca3c1acd237a\") " pod="openshift-multus/multus-additional-cni-plugins-54xcx" Jul 27 09:24:34.599209 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:34.597137 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/09e4c928-4b42-484e-808c-dfaa1b17977b-device-dir\") pod \"aws-ebs-csi-driver-node-kb5qg\" (UID: \"09e4c928-4b42-484e-808c-dfaa1b17977b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kb5qg" Jul 27 09:24:34.599209 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:34.597185 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/09e4c928-4b42-484e-808c-dfaa1b17977b-device-dir\") pod \"aws-ebs-csi-driver-node-kb5qg\" (UID: \"09e4c928-4b42-484e-808c-dfaa1b17977b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kb5qg" Jul 27 09:24:34.599209 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:34.597188 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/29406f41-dc4d-4f8a-9627-9589659ae0d4-system-cni-dir\") pod \"multus-jmgz6\" (UID: \"29406f41-dc4d-4f8a-9627-9589659ae0d4\") " pod="openshift-multus/multus-jmgz6" Jul 27 09:24:34.599209 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:34.597220 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/09e4c928-4b42-484e-808c-dfaa1b17977b-socket-dir\") pod \"aws-ebs-csi-driver-node-kb5qg\" (UID: \"09e4c928-4b42-484e-808c-dfaa1b17977b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kb5qg" Jul 27 09:24:34.599209 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:34.597248 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/d0206685-d6b8-43ea-a691-ca3c1acd237a-os-release\") pod \"multus-additional-cni-plugins-54xcx\" (UID: \"d0206685-d6b8-43ea-a691-ca3c1acd237a\") " pod="openshift-multus/multus-additional-cni-plugins-54xcx" Jul 27 09:24:34.599209 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:34.597272 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9cc406f3-cdff-4cde-ac7b-ff31abb20e0d-var-lib-kubelet\") pod \"tuned-bk5w8\" (UID: \"9cc406f3-cdff-4cde-ac7b-ff31abb20e0d\") " pod="openshift-cluster-node-tuning-operator/tuned-bk5w8" Jul 27 09:24:34.599209 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:34.597271 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/29406f41-dc4d-4f8a-9627-9589659ae0d4-system-cni-dir\") pod \"multus-jmgz6\" (UID: \"29406f41-dc4d-4f8a-9627-9589659ae0d4\") " pod="openshift-multus/multus-jmgz6" Jul 27 09:24:34.599209 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:34.597326 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/d0206685-d6b8-43ea-a691-ca3c1acd237a-os-release\") pod \"multus-additional-cni-plugins-54xcx\" (UID: \"d0206685-d6b8-43ea-a691-ca3c1acd237a\") " pod="openshift-multus/multus-additional-cni-plugins-54xcx" Jul 27 09:24:34.599209 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:34.597365 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9cc406f3-cdff-4cde-ac7b-ff31abb20e0d-var-lib-kubelet\") pod \"tuned-bk5w8\" (UID: \"9cc406f3-cdff-4cde-ac7b-ff31abb20e0d\") " pod="openshift-cluster-node-tuning-operator/tuned-bk5w8" Jul 27 09:24:34.599209 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:34.597377 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/09e4c928-4b42-484e-808c-dfaa1b17977b-socket-dir\") pod \"aws-ebs-csi-driver-node-kb5qg\" (UID: \"09e4c928-4b42-484e-808c-dfaa1b17977b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kb5qg" Jul 27 09:24:34.600072 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:34.597517 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/d0206685-d6b8-43ea-a691-ca3c1acd237a-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-54xcx\" (UID: \"d0206685-d6b8-43ea-a691-ca3c1acd237a\") " pod="openshift-multus/multus-additional-cni-plugins-54xcx" Jul 27 09:24:34.604509 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:34.604482 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9cc406f3-cdff-4cde-ac7b-ff31abb20e0d-tmp\") pod \"tuned-bk5w8\" (UID: \"9cc406f3-cdff-4cde-ac7b-ff31abb20e0d\") " pod="openshift-cluster-node-tuning-operator/tuned-bk5w8" Jul 27 09:24:34.604509 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:34.604506 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/9cc406f3-cdff-4cde-ac7b-ff31abb20e0d-etc-tuned\") pod \"tuned-bk5w8\" (UID: \"9cc406f3-cdff-4cde-ac7b-ff31abb20e0d\") " pod="openshift-cluster-node-tuning-operator/tuned-bk5w8" Jul 27 09:24:34.612004 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:34.611940 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mcwvb\" (UniqueName: \"kubernetes.io/projected/996c29bd-3969-4268-a795-4f4d21881d48-kube-api-access-mcwvb\") pod \"node-ca-bmklx\" (UID: \"996c29bd-3969-4268-a795-4f4d21881d48\") " pod="openshift-image-registry/node-ca-bmklx" Jul 27 09:24:34.612612 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:34.612581 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mp4mq\" (UniqueName: \"kubernetes.io/projected/29406f41-dc4d-4f8a-9627-9589659ae0d4-kube-api-access-mp4mq\") pod \"multus-jmgz6\" (UID: \"29406f41-dc4d-4f8a-9627-9589659ae0d4\") " pod="openshift-multus/multus-jmgz6" Jul 27 09:24:34.614757 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:34.614735 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sqdxv\" (UniqueName: \"kubernetes.io/projected/6fd3f632-9d80-4e6e-8d4f-f5314c0f174a-kube-api-access-sqdxv\") pod \"iptables-alerter-rcftn\" (UID: \"6fd3f632-9d80-4e6e-8d4f-f5314c0f174a\") " pod="openshift-network-operator/iptables-alerter-rcftn" Jul 27 09:24:34.615422 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:34.615362 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q2z5h\" (UniqueName: \"kubernetes.io/projected/09e4c928-4b42-484e-808c-dfaa1b17977b-kube-api-access-q2z5h\") pod \"aws-ebs-csi-driver-node-kb5qg\" (UID: \"09e4c928-4b42-484e-808c-dfaa1b17977b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kb5qg" Jul 27 09:24:34.616246 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:34.616188 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w2t6q\" (UniqueName: \"kubernetes.io/projected/9cc406f3-cdff-4cde-ac7b-ff31abb20e0d-kube-api-access-w2t6q\") pod \"tuned-bk5w8\" (UID: \"9cc406f3-cdff-4cde-ac7b-ff31abb20e0d\") " pod="openshift-cluster-node-tuning-operator/tuned-bk5w8" Jul 27 09:24:34.616711 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:34.616690 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mblhr\" (UniqueName: \"kubernetes.io/projected/d0206685-d6b8-43ea-a691-ca3c1acd237a-kube-api-access-mblhr\") pod \"multus-additional-cni-plugins-54xcx\" (UID: \"d0206685-d6b8-43ea-a691-ca3c1acd237a\") " pod="openshift-multus/multus-additional-cni-plugins-54xcx" Jul 27 09:24:34.702443 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:34.702401 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-mqhsn" Jul 27 09:24:34.711488 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:34.711455 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-848w4" Jul 27 09:24:34.721300 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:34.721270 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-54xcx" Jul 27 09:24:34.730157 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:34.730133 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-jmgz6" Jul 27 09:24:34.736895 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:34.736872 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-rcftn" Jul 27 09:24:34.744815 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:34.744791 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kb5qg" Jul 27 09:24:34.752562 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:34.752543 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-bk5w8" Jul 27 09:24:34.759254 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:34.759229 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-bmklx" Jul 27 09:24:35.000280 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:35.000196 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0d6ba35e-ec8a-4ec9-8520-0d32a5586a1d-metrics-certs\") pod \"network-metrics-daemon-trcl2\" (UID: \"0d6ba35e-ec8a-4ec9-8520-0d32a5586a1d\") " pod="openshift-multus/network-metrics-daemon-trcl2" Jul 27 09:24:35.000444 ip-10-0-128-103 kubenswrapper[2565]: E0727 09:24:35.000353 2565 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 27 09:24:35.000444 ip-10-0-128-103 kubenswrapper[2565]: E0727 09:24:35.000434 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0d6ba35e-ec8a-4ec9-8520-0d32a5586a1d-metrics-certs podName:0d6ba35e-ec8a-4ec9-8520-0d32a5586a1d nodeName:}" failed. No retries permitted until 2026-07-27 09:24:36.000415223 +0000 UTC m=+3.964201181 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0d6ba35e-ec8a-4ec9-8520-0d32a5586a1d-metrics-certs") pod "network-metrics-daemon-trcl2" (UID: "0d6ba35e-ec8a-4ec9-8520-0d32a5586a1d") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 27 09:24:35.088564 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:35.088524 2565 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 27 09:24:35.101328 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:35.101266 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/204c97b6-2efe-4db7-bc9f-a5653fed77c7-original-pull-secret\") pod \"global-pull-secret-syncer-rqz85\" (UID: \"204c97b6-2efe-4db7-bc9f-a5653fed77c7\") " pod="kube-system/global-pull-secret-syncer-rqz85" Jul 27 09:24:35.101328 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:35.101330 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2ckrw\" (UniqueName: \"kubernetes.io/projected/91816deb-5261-44b7-857a-04cf036feffa-kube-api-access-2ckrw\") pod \"network-check-target-5zqgm\" (UID: \"91816deb-5261-44b7-857a-04cf036feffa\") " pod="openshift-network-diagnostics/network-check-target-5zqgm" Jul 27 09:24:35.101552 ip-10-0-128-103 kubenswrapper[2565]: E0727 09:24:35.101426 2565 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 27 09:24:35.101552 ip-10-0-128-103 kubenswrapper[2565]: E0727 09:24:35.101480 2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 27 09:24:35.101552 ip-10-0-128-103 kubenswrapper[2565]: E0727 09:24:35.101496 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/204c97b6-2efe-4db7-bc9f-a5653fed77c7-original-pull-secret podName:204c97b6-2efe-4db7-bc9f-a5653fed77c7 nodeName:}" failed. No retries permitted until 2026-07-27 09:24:36.101475355 +0000 UTC m=+4.065261308 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/204c97b6-2efe-4db7-bc9f-a5653fed77c7-original-pull-secret") pod "global-pull-secret-syncer-rqz85" (UID: "204c97b6-2efe-4db7-bc9f-a5653fed77c7") : object "kube-system"/"original-pull-secret" not registered Jul 27 09:24:35.101552 ip-10-0-128-103 kubenswrapper[2565]: E0727 09:24:35.101504 2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 27 09:24:35.101552 ip-10-0-128-103 kubenswrapper[2565]: E0727 09:24:35.101516 2565 projected.go:196] Error preparing data for projected volume kube-api-access-2ckrw for pod openshift-network-diagnostics/network-check-target-5zqgm: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 27 09:24:35.101783 ip-10-0-128-103 kubenswrapper[2565]: E0727 09:24:35.101565 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/91816deb-5261-44b7-857a-04cf036feffa-kube-api-access-2ckrw podName:91816deb-5261-44b7-857a-04cf036feffa nodeName:}" failed. No retries permitted until 2026-07-27 09:24:36.101547637 +0000 UTC m=+4.065333578 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-2ckrw" (UniqueName: "kubernetes.io/projected/91816deb-5261-44b7-857a-04cf036feffa-kube-api-access-2ckrw") pod "network-check-target-5zqgm" (UID: "91816deb-5261-44b7-857a-04cf036feffa") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 27 09:24:35.440151 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:35.440040 2565 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-26 09:19:35 +0000 UTC" deadline="2028-02-10 04:39:14.52331408 +0000 UTC" Jul 27 09:24:35.440151 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:35.440111 2565 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13507h14m39.083206072s" Jul 27 09:24:35.494380 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:35.494344 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod29406f41_dc4d_4f8a_9627_9589659ae0d4.slice/crio-4d08bab7f6ef12e55b578d0bb61dec879f9a2b2bc62645e495a2afe281eea00d WatchSource:0}: Error finding container 4d08bab7f6ef12e55b578d0bb61dec879f9a2b2bc62645e495a2afe281eea00d: Status 404 returned error can't find the container with id 4d08bab7f6ef12e55b578d0bb61dec879f9a2b2bc62645e495a2afe281eea00d Jul 27 09:24:35.500379 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:35.500328 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2c043d82_d300_41ba_b138_60a03d65ea11.slice/crio-86f1ed65886d8ee6b72094304c35b382b95228d2373405debed4a92b9c2f6ed8 WatchSource:0}: Error finding container 86f1ed65886d8ee6b72094304c35b382b95228d2373405debed4a92b9c2f6ed8: Status 404 returned error can't find the container with id 86f1ed65886d8ee6b72094304c35b382b95228d2373405debed4a92b9c2f6ed8 Jul 27 09:24:35.502104 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:35.501963 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9cc406f3_cdff_4cde_ac7b_ff31abb20e0d.slice/crio-7fb315989e9083c244712170baf4578d37171cb01846c7f1e44565df11b59f98 WatchSource:0}: Error finding container 7fb315989e9083c244712170baf4578d37171cb01846c7f1e44565df11b59f98: Status 404 returned error can't find the container with id 7fb315989e9083c244712170baf4578d37171cb01846c7f1e44565df11b59f98 Jul 27 09:24:35.502625 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:35.502599 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod09e4c928_4b42_484e_808c_dfaa1b17977b.slice/crio-8188b5d555da21d5395ef3006b1d45141d4dbe9b11c1c63d0896cd8ad07d84d4 WatchSource:0}: Error finding container 8188b5d555da21d5395ef3006b1d45141d4dbe9b11c1c63d0896cd8ad07d84d4: Status 404 returned error can't find the container with id 8188b5d555da21d5395ef3006b1d45141d4dbe9b11c1c63d0896cd8ad07d84d4 Jul 27 09:24:35.503850 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:35.503766 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd0206685_d6b8_43ea_a691_ca3c1acd237a.slice/crio-8b4d87609cf706ececf5efd5de04a0be45e8240931f4fa3dd99ac41996212e16 WatchSource:0}: Error finding container 8b4d87609cf706ececf5efd5de04a0be45e8240931f4fa3dd99ac41996212e16: Status 404 returned error can't find the container with id 8b4d87609cf706ececf5efd5de04a0be45e8240931f4fa3dd99ac41996212e16 Jul 27 09:24:35.504298 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:35.504262 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod996c29bd_3969_4268_a795_4f4d21881d48.slice/crio-cf0abdf001561b0e6c6793d6b8ab2f1b6a071f16fdf24ee67b22d810dd68c43f WatchSource:0}: Error finding container cf0abdf001561b0e6c6793d6b8ab2f1b6a071f16fdf24ee67b22d810dd68c43f: Status 404 returned error can't find the container with id cf0abdf001561b0e6c6793d6b8ab2f1b6a071f16fdf24ee67b22d810dd68c43f Jul 27 09:24:35.505408 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:35.505372 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6fd3f632_9d80_4e6e_8d4f_f5314c0f174a.slice/crio-d7705cc528c70163ac23744c13615ebb3d4f690bbfdc0ce9cd4e0edb1f3c1451 WatchSource:0}: Error finding container d7705cc528c70163ac23744c13615ebb3d4f690bbfdc0ce9cd4e0edb1f3c1451: Status 404 returned error can't find the container with id d7705cc528c70163ac23744c13615ebb3d4f690bbfdc0ce9cd4e0edb1f3c1451 Jul 27 09:24:35.507001 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:35.506865 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5c031f05_bf4a_4ff7_81ad_381d31b163ff.slice/crio-6faf4b2ab2b860245d50e61cb9e3c82a00ce32d3525aa69f17cd88936c800626 WatchSource:0}: Error finding container 6faf4b2ab2b860245d50e61cb9e3c82a00ce32d3525aa69f17cd88936c800626: Status 404 returned error can't find the container with id 6faf4b2ab2b860245d50e61cb9e3c82a00ce32d3525aa69f17cd88936c800626 Jul 27 09:24:35.557446 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:35.557419 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-trcl2" Jul 27 09:24:35.557597 ip-10-0-128-103 kubenswrapper[2565]: E0727 09:24:35.557563 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-trcl2" podUID="0d6ba35e-ec8a-4ec9-8520-0d32a5586a1d" Jul 27 09:24:35.567549 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:35.567503 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-mqhsn" event={"ID":"5c031f05-bf4a-4ff7-81ad-381d31b163ff","Type":"ContainerStarted","Data":"6faf4b2ab2b860245d50e61cb9e3c82a00ce32d3525aa69f17cd88936c800626"} Jul 27 09:24:35.568739 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:35.568716 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-rcftn" event={"ID":"6fd3f632-9d80-4e6e-8d4f-f5314c0f174a","Type":"ContainerStarted","Data":"d7705cc528c70163ac23744c13615ebb3d4f690bbfdc0ce9cd4e0edb1f3c1451"} Jul 27 09:24:35.570100 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:35.570071 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-bmklx" event={"ID":"996c29bd-3969-4268-a795-4f4d21881d48","Type":"ContainerStarted","Data":"cf0abdf001561b0e6c6793d6b8ab2f1b6a071f16fdf24ee67b22d810dd68c43f"} Jul 27 09:24:35.571494 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:35.571317 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-54xcx" event={"ID":"d0206685-d6b8-43ea-a691-ca3c1acd237a","Type":"ContainerStarted","Data":"8b4d87609cf706ececf5efd5de04a0be45e8240931f4fa3dd99ac41996212e16"} Jul 27 09:24:35.572561 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:35.572531 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kb5qg" event={"ID":"09e4c928-4b42-484e-808c-dfaa1b17977b","Type":"ContainerStarted","Data":"8188b5d555da21d5395ef3006b1d45141d4dbe9b11c1c63d0896cd8ad07d84d4"} Jul 27 09:24:35.573927 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:35.573900 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-bk5w8" event={"ID":"9cc406f3-cdff-4cde-ac7b-ff31abb20e0d","Type":"ContainerStarted","Data":"7fb315989e9083c244712170baf4578d37171cb01846c7f1e44565df11b59f98"} Jul 27 09:24:35.575120 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:35.575097 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-848w4" event={"ID":"2c043d82-d300-41ba-b138-60a03d65ea11","Type":"ContainerStarted","Data":"86f1ed65886d8ee6b72094304c35b382b95228d2373405debed4a92b9c2f6ed8"} Jul 27 09:24:35.576316 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:35.576296 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-jmgz6" event={"ID":"29406f41-dc4d-4f8a-9627-9589659ae0d4","Type":"ContainerStarted","Data":"4d08bab7f6ef12e55b578d0bb61dec879f9a2b2bc62645e495a2afe281eea00d"} Jul 27 09:24:36.009216 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:36.008543 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0d6ba35e-ec8a-4ec9-8520-0d32a5586a1d-metrics-certs\") pod \"network-metrics-daemon-trcl2\" (UID: \"0d6ba35e-ec8a-4ec9-8520-0d32a5586a1d\") " pod="openshift-multus/network-metrics-daemon-trcl2" Jul 27 09:24:36.009216 ip-10-0-128-103 kubenswrapper[2565]: E0727 09:24:36.008698 2565 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 27 09:24:36.009216 ip-10-0-128-103 kubenswrapper[2565]: E0727 09:24:36.008772 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0d6ba35e-ec8a-4ec9-8520-0d32a5586a1d-metrics-certs podName:0d6ba35e-ec8a-4ec9-8520-0d32a5586a1d nodeName:}" failed. No retries permitted until 2026-07-27 09:24:38.00875082 +0000 UTC m=+5.972536762 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0d6ba35e-ec8a-4ec9-8520-0d32a5586a1d-metrics-certs") pod "network-metrics-daemon-trcl2" (UID: "0d6ba35e-ec8a-4ec9-8520-0d32a5586a1d") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 27 09:24:36.109402 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:36.109336 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2ckrw\" (UniqueName: \"kubernetes.io/projected/91816deb-5261-44b7-857a-04cf036feffa-kube-api-access-2ckrw\") pod \"network-check-target-5zqgm\" (UID: \"91816deb-5261-44b7-857a-04cf036feffa\") " pod="openshift-network-diagnostics/network-check-target-5zqgm" Jul 27 09:24:36.109596 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:36.109439 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/204c97b6-2efe-4db7-bc9f-a5653fed77c7-original-pull-secret\") pod \"global-pull-secret-syncer-rqz85\" (UID: \"204c97b6-2efe-4db7-bc9f-a5653fed77c7\") " pod="kube-system/global-pull-secret-syncer-rqz85" Jul 27 09:24:36.109659 ip-10-0-128-103 kubenswrapper[2565]: E0727 09:24:36.109602 2565 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 27 09:24:36.109724 ip-10-0-128-103 kubenswrapper[2565]: E0727 09:24:36.109678 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/204c97b6-2efe-4db7-bc9f-a5653fed77c7-original-pull-secret podName:204c97b6-2efe-4db7-bc9f-a5653fed77c7 nodeName:}" failed. No retries permitted until 2026-07-27 09:24:38.109654947 +0000 UTC m=+6.073440904 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/204c97b6-2efe-4db7-bc9f-a5653fed77c7-original-pull-secret") pod "global-pull-secret-syncer-rqz85" (UID: "204c97b6-2efe-4db7-bc9f-a5653fed77c7") : object "kube-system"/"original-pull-secret" not registered Jul 27 09:24:36.110144 ip-10-0-128-103 kubenswrapper[2565]: E0727 09:24:36.110119 2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 27 09:24:36.110255 ip-10-0-128-103 kubenswrapper[2565]: E0727 09:24:36.110150 2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 27 09:24:36.110255 ip-10-0-128-103 kubenswrapper[2565]: E0727 09:24:36.110167 2565 projected.go:196] Error preparing data for projected volume kube-api-access-2ckrw for pod openshift-network-diagnostics/network-check-target-5zqgm: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 27 09:24:36.110255 ip-10-0-128-103 kubenswrapper[2565]: E0727 09:24:36.110218 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/91816deb-5261-44b7-857a-04cf036feffa-kube-api-access-2ckrw podName:91816deb-5261-44b7-857a-04cf036feffa nodeName:}" failed. No retries permitted until 2026-07-27 09:24:38.110201982 +0000 UTC m=+6.073987923 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-2ckrw" (UniqueName: "kubernetes.io/projected/91816deb-5261-44b7-857a-04cf036feffa-kube-api-access-2ckrw") pod "network-check-target-5zqgm" (UID: "91816deb-5261-44b7-857a-04cf036feffa") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 27 09:24:36.441259 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:36.441165 2565 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-26 09:19:35 +0000 UTC" deadline="2027-12-31 00:38:56.322687158 +0000 UTC" Jul 27 09:24:36.441259 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:36.441209 2565 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12519h14m19.881484708s" Jul 27 09:24:36.559059 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:36.559026 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rqz85" Jul 27 09:24:36.559247 ip-10-0-128-103 kubenswrapper[2565]: E0727 09:24:36.559162 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rqz85" podUID="204c97b6-2efe-4db7-bc9f-a5653fed77c7" Jul 27 09:24:36.561537 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:36.561509 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5zqgm" Jul 27 09:24:36.561692 ip-10-0-128-103 kubenswrapper[2565]: E0727 09:24:36.561657 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5zqgm" podUID="91816deb-5261-44b7-857a-04cf036feffa" Jul 27 09:24:36.596579 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:36.596535 2565 generic.go:356] "Generic (PLEG): container finished" podID="519b669acb80623490c3cff7ef66ed98" containerID="87b622a9cec365c51716385087c050a7b2a301500b6d83fc897605b3cbcaced1" exitCode=0 Jul 27 09:24:36.596790 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:36.596657 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-103.ec2.internal" event={"ID":"519b669acb80623490c3cff7ef66ed98","Type":"ContainerDied","Data":"87b622a9cec365c51716385087c050a7b2a301500b6d83fc897605b3cbcaced1"} Jul 27 09:24:36.623344 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:36.623242 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-103.ec2.internal" event={"ID":"f9f6fe57433891594e1efa7d6875336f","Type":"ContainerStarted","Data":"546846f5871528f370bfee42dd59c5170c5ea8b8744e97866e206ba5065d6392"} Jul 27 09:24:37.275142 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:37.274204 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-103.ec2.internal" podStartSLOduration=3.274179963 podStartE2EDuration="3.274179963s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-27 09:24:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 09:24:36.644984516 +0000 UTC m=+4.608770479" watchObservedRunningTime="2026-07-27 09:24:37.274179963 +0000 UTC m=+5.237965926" Jul 27 09:24:37.275142 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:37.274593 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/node-resolver-r2wsn"] Jul 27 09:24:37.278472 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:37.278445 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-r2wsn" Jul 27 09:24:37.281832 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:37.281656 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 27 09:24:37.281832 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:37.281683 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-c7qpr\"" Jul 27 09:24:37.282044 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:37.281866 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 27 09:24:37.320950 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:37.320911 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n4vnx\" (UniqueName: \"kubernetes.io/projected/199af0d6-b2fc-4951-941f-948e403d3865-kube-api-access-n4vnx\") pod \"node-resolver-r2wsn\" (UID: \"199af0d6-b2fc-4951-941f-948e403d3865\") " pod="openshift-dns/node-resolver-r2wsn" Jul 27 09:24:37.321149 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:37.320992 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/199af0d6-b2fc-4951-941f-948e403d3865-hosts-file\") pod \"node-resolver-r2wsn\" (UID: \"199af0d6-b2fc-4951-941f-948e403d3865\") " pod="openshift-dns/node-resolver-r2wsn" Jul 27 09:24:37.321149 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:37.321040 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/199af0d6-b2fc-4951-941f-948e403d3865-tmp-dir\") pod \"node-resolver-r2wsn\" (UID: \"199af0d6-b2fc-4951-941f-948e403d3865\") " pod="openshift-dns/node-resolver-r2wsn" Jul 27 09:24:37.421775 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:37.421399 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n4vnx\" (UniqueName: \"kubernetes.io/projected/199af0d6-b2fc-4951-941f-948e403d3865-kube-api-access-n4vnx\") pod \"node-resolver-r2wsn\" (UID: \"199af0d6-b2fc-4951-941f-948e403d3865\") " pod="openshift-dns/node-resolver-r2wsn" Jul 27 09:24:37.421775 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:37.421470 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/199af0d6-b2fc-4951-941f-948e403d3865-hosts-file\") pod \"node-resolver-r2wsn\" (UID: \"199af0d6-b2fc-4951-941f-948e403d3865\") " pod="openshift-dns/node-resolver-r2wsn" Jul 27 09:24:37.421775 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:37.421516 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/199af0d6-b2fc-4951-941f-948e403d3865-tmp-dir\") pod \"node-resolver-r2wsn\" (UID: \"199af0d6-b2fc-4951-941f-948e403d3865\") " pod="openshift-dns/node-resolver-r2wsn" Jul 27 09:24:37.422091 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:37.421886 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/199af0d6-b2fc-4951-941f-948e403d3865-hosts-file\") pod \"node-resolver-r2wsn\" (UID: \"199af0d6-b2fc-4951-941f-948e403d3865\") " pod="openshift-dns/node-resolver-r2wsn" Jul 27 09:24:37.423277 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:37.423137 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/199af0d6-b2fc-4951-941f-948e403d3865-tmp-dir\") pod \"node-resolver-r2wsn\" (UID: \"199af0d6-b2fc-4951-941f-948e403d3865\") " pod="openshift-dns/node-resolver-r2wsn" Jul 27 09:24:37.459289 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:37.459255 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n4vnx\" (UniqueName: \"kubernetes.io/projected/199af0d6-b2fc-4951-941f-948e403d3865-kube-api-access-n4vnx\") pod \"node-resolver-r2wsn\" (UID: \"199af0d6-b2fc-4951-941f-948e403d3865\") " pod="openshift-dns/node-resolver-r2wsn" Jul 27 09:24:37.558670 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:37.558079 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-trcl2" Jul 27 09:24:37.558670 ip-10-0-128-103 kubenswrapper[2565]: E0727 09:24:37.558245 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-trcl2" podUID="0d6ba35e-ec8a-4ec9-8520-0d32a5586a1d" Jul 27 09:24:37.593619 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:37.593352 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-r2wsn" Jul 27 09:24:37.646401 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:37.645985 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-103.ec2.internal" event={"ID":"519b669acb80623490c3cff7ef66ed98","Type":"ContainerStarted","Data":"926846b941c38a438594976c11e1a9c1d19ec4f69e972f1c70411678312f2484"} Jul 27 09:24:38.026087 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:38.025999 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0d6ba35e-ec8a-4ec9-8520-0d32a5586a1d-metrics-certs\") pod \"network-metrics-daemon-trcl2\" (UID: \"0d6ba35e-ec8a-4ec9-8520-0d32a5586a1d\") " pod="openshift-multus/network-metrics-daemon-trcl2" Jul 27 09:24:38.026266 ip-10-0-128-103 kubenswrapper[2565]: E0727 09:24:38.026136 2565 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 27 09:24:38.026266 ip-10-0-128-103 kubenswrapper[2565]: E0727 09:24:38.026199 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0d6ba35e-ec8a-4ec9-8520-0d32a5586a1d-metrics-certs podName:0d6ba35e-ec8a-4ec9-8520-0d32a5586a1d nodeName:}" failed. No retries permitted until 2026-07-27 09:24:42.026180103 +0000 UTC m=+9.989966041 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0d6ba35e-ec8a-4ec9-8520-0d32a5586a1d-metrics-certs") pod "network-metrics-daemon-trcl2" (UID: "0d6ba35e-ec8a-4ec9-8520-0d32a5586a1d") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 27 09:24:38.126321 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:38.126282 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/204c97b6-2efe-4db7-bc9f-a5653fed77c7-original-pull-secret\") pod \"global-pull-secret-syncer-rqz85\" (UID: \"204c97b6-2efe-4db7-bc9f-a5653fed77c7\") " pod="kube-system/global-pull-secret-syncer-rqz85" Jul 27 09:24:38.126521 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:38.126368 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2ckrw\" (UniqueName: \"kubernetes.io/projected/91816deb-5261-44b7-857a-04cf036feffa-kube-api-access-2ckrw\") pod \"network-check-target-5zqgm\" (UID: \"91816deb-5261-44b7-857a-04cf036feffa\") " pod="openshift-network-diagnostics/network-check-target-5zqgm" Jul 27 09:24:38.126749 ip-10-0-128-103 kubenswrapper[2565]: E0727 09:24:38.126724 2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 27 09:24:38.126864 ip-10-0-128-103 kubenswrapper[2565]: E0727 09:24:38.126759 2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 27 09:24:38.126864 ip-10-0-128-103 kubenswrapper[2565]: E0727 09:24:38.126774 2565 projected.go:196] Error preparing data for projected volume kube-api-access-2ckrw for pod openshift-network-diagnostics/network-check-target-5zqgm: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 27 09:24:38.126864 ip-10-0-128-103 kubenswrapper[2565]: E0727 09:24:38.126835 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/91816deb-5261-44b7-857a-04cf036feffa-kube-api-access-2ckrw podName:91816deb-5261-44b7-857a-04cf036feffa nodeName:}" failed. No retries permitted until 2026-07-27 09:24:42.12681546 +0000 UTC m=+10.090601412 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-2ckrw" (UniqueName: "kubernetes.io/projected/91816deb-5261-44b7-857a-04cf036feffa-kube-api-access-2ckrw") pod "network-check-target-5zqgm" (UID: "91816deb-5261-44b7-857a-04cf036feffa") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 27 09:24:38.127052 ip-10-0-128-103 kubenswrapper[2565]: E0727 09:24:38.126938 2565 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 27 09:24:38.127052 ip-10-0-128-103 kubenswrapper[2565]: E0727 09:24:38.126980 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/204c97b6-2efe-4db7-bc9f-a5653fed77c7-original-pull-secret podName:204c97b6-2efe-4db7-bc9f-a5653fed77c7 nodeName:}" failed. No retries permitted until 2026-07-27 09:24:42.126967102 +0000 UTC m=+10.090753042 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/204c97b6-2efe-4db7-bc9f-a5653fed77c7-original-pull-secret") pod "global-pull-secret-syncer-rqz85" (UID: "204c97b6-2efe-4db7-bc9f-a5653fed77c7") : object "kube-system"/"original-pull-secret" not registered Jul 27 09:24:38.558684 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:38.558648 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5zqgm" Jul 27 09:24:38.559268 ip-10-0-128-103 kubenswrapper[2565]: E0727 09:24:38.558790 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5zqgm" podUID="91816deb-5261-44b7-857a-04cf036feffa" Jul 27 09:24:38.559268 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:38.558653 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rqz85" Jul 27 09:24:38.559494 ip-10-0-128-103 kubenswrapper[2565]: E0727 09:24:38.559292 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rqz85" podUID="204c97b6-2efe-4db7-bc9f-a5653fed77c7" Jul 27 09:24:39.558298 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:39.558269 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-trcl2" Jul 27 09:24:39.558511 ip-10-0-128-103 kubenswrapper[2565]: E0727 09:24:39.558406 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-trcl2" podUID="0d6ba35e-ec8a-4ec9-8520-0d32a5586a1d" Jul 27 09:24:40.558084 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:40.558048 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rqz85" Jul 27 09:24:40.558576 ip-10-0-128-103 kubenswrapper[2565]: E0727 09:24:40.558193 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rqz85" podUID="204c97b6-2efe-4db7-bc9f-a5653fed77c7" Jul 27 09:24:40.558576 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:40.558566 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5zqgm" Jul 27 09:24:40.558733 ip-10-0-128-103 kubenswrapper[2565]: E0727 09:24:40.558662 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5zqgm" podUID="91816deb-5261-44b7-857a-04cf036feffa" Jul 27 09:24:41.557754 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:41.557627 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-trcl2" Jul 27 09:24:41.558019 ip-10-0-128-103 kubenswrapper[2565]: E0727 09:24:41.557821 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-trcl2" podUID="0d6ba35e-ec8a-4ec9-8520-0d32a5586a1d" Jul 27 09:24:42.059703 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:42.059655 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0d6ba35e-ec8a-4ec9-8520-0d32a5586a1d-metrics-certs\") pod \"network-metrics-daemon-trcl2\" (UID: \"0d6ba35e-ec8a-4ec9-8520-0d32a5586a1d\") " pod="openshift-multus/network-metrics-daemon-trcl2" Jul 27 09:24:42.060215 ip-10-0-128-103 kubenswrapper[2565]: E0727 09:24:42.059879 2565 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 27 09:24:42.060215 ip-10-0-128-103 kubenswrapper[2565]: E0727 09:24:42.059972 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0d6ba35e-ec8a-4ec9-8520-0d32a5586a1d-metrics-certs podName:0d6ba35e-ec8a-4ec9-8520-0d32a5586a1d nodeName:}" failed. No retries permitted until 2026-07-27 09:24:50.059946735 +0000 UTC m=+18.023732696 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0d6ba35e-ec8a-4ec9-8520-0d32a5586a1d-metrics-certs") pod "network-metrics-daemon-trcl2" (UID: "0d6ba35e-ec8a-4ec9-8520-0d32a5586a1d") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 27 09:24:42.160933 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:42.160893 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2ckrw\" (UniqueName: \"kubernetes.io/projected/91816deb-5261-44b7-857a-04cf036feffa-kube-api-access-2ckrw\") pod \"network-check-target-5zqgm\" (UID: \"91816deb-5261-44b7-857a-04cf036feffa\") " pod="openshift-network-diagnostics/network-check-target-5zqgm" Jul 27 09:24:42.161128 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:42.160979 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/204c97b6-2efe-4db7-bc9f-a5653fed77c7-original-pull-secret\") pod \"global-pull-secret-syncer-rqz85\" (UID: \"204c97b6-2efe-4db7-bc9f-a5653fed77c7\") " pod="kube-system/global-pull-secret-syncer-rqz85" Jul 27 09:24:42.161128 ip-10-0-128-103 kubenswrapper[2565]: E0727 09:24:42.161091 2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 27 09:24:42.161128 ip-10-0-128-103 kubenswrapper[2565]: E0727 09:24:42.161122 2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 27 09:24:42.161279 ip-10-0-128-103 kubenswrapper[2565]: E0727 09:24:42.161165 2565 projected.go:196] Error preparing data for projected volume kube-api-access-2ckrw for pod openshift-network-diagnostics/network-check-target-5zqgm: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 27 09:24:42.161279 ip-10-0-128-103 kubenswrapper[2565]: E0727 09:24:42.161096 2565 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 27 09:24:42.161357 ip-10-0-128-103 kubenswrapper[2565]: E0727 09:24:42.161234 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/91816deb-5261-44b7-857a-04cf036feffa-kube-api-access-2ckrw podName:91816deb-5261-44b7-857a-04cf036feffa nodeName:}" failed. No retries permitted until 2026-07-27 09:24:50.161213781 +0000 UTC m=+18.124999720 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-2ckrw" (UniqueName: "kubernetes.io/projected/91816deb-5261-44b7-857a-04cf036feffa-kube-api-access-2ckrw") pod "network-check-target-5zqgm" (UID: "91816deb-5261-44b7-857a-04cf036feffa") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 27 09:24:42.161357 ip-10-0-128-103 kubenswrapper[2565]: E0727 09:24:42.161323 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/204c97b6-2efe-4db7-bc9f-a5653fed77c7-original-pull-secret podName:204c97b6-2efe-4db7-bc9f-a5653fed77c7 nodeName:}" failed. No retries permitted until 2026-07-27 09:24:50.161301835 +0000 UTC m=+18.125087794 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/204c97b6-2efe-4db7-bc9f-a5653fed77c7-original-pull-secret") pod "global-pull-secret-syncer-rqz85" (UID: "204c97b6-2efe-4db7-bc9f-a5653fed77c7") : object "kube-system"/"original-pull-secret" not registered Jul 27 09:24:42.557625 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:42.557529 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rqz85" Jul 27 09:24:42.557824 ip-10-0-128-103 kubenswrapper[2565]: E0727 09:24:42.557673 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rqz85" podUID="204c97b6-2efe-4db7-bc9f-a5653fed77c7" Jul 27 09:24:42.559018 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:42.558987 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5zqgm" Jul 27 09:24:42.559235 ip-10-0-128-103 kubenswrapper[2565]: E0727 09:24:42.559080 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5zqgm" podUID="91816deb-5261-44b7-857a-04cf036feffa" Jul 27 09:24:43.558196 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:43.558161 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-trcl2" Jul 27 09:24:43.558679 ip-10-0-128-103 kubenswrapper[2565]: E0727 09:24:43.558302 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-trcl2" podUID="0d6ba35e-ec8a-4ec9-8520-0d32a5586a1d" Jul 27 09:24:44.557688 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:44.557658 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5zqgm" Jul 27 09:24:44.557902 ip-10-0-128-103 kubenswrapper[2565]: E0727 09:24:44.557788 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5zqgm" podUID="91816deb-5261-44b7-857a-04cf036feffa" Jul 27 09:24:44.558327 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:44.558306 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rqz85" Jul 27 09:24:44.558680 ip-10-0-128-103 kubenswrapper[2565]: E0727 09:24:44.558374 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rqz85" podUID="204c97b6-2efe-4db7-bc9f-a5653fed77c7" Jul 27 09:24:45.557712 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:45.557669 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-trcl2" Jul 27 09:24:45.557915 ip-10-0-128-103 kubenswrapper[2565]: E0727 09:24:45.557833 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-trcl2" podUID="0d6ba35e-ec8a-4ec9-8520-0d32a5586a1d" Jul 27 09:24:46.558445 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:46.558353 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rqz85" Jul 27 09:24:46.558445 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:46.558400 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5zqgm" Jul 27 09:24:46.558984 ip-10-0-128-103 kubenswrapper[2565]: E0727 09:24:46.558490 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rqz85" podUID="204c97b6-2efe-4db7-bc9f-a5653fed77c7" Jul 27 09:24:46.558984 ip-10-0-128-103 kubenswrapper[2565]: E0727 09:24:46.558600 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5zqgm" podUID="91816deb-5261-44b7-857a-04cf036feffa" Jul 27 09:24:47.557873 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:47.557838 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-trcl2" Jul 27 09:24:47.558072 ip-10-0-128-103 kubenswrapper[2565]: E0727 09:24:47.557974 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-trcl2" podUID="0d6ba35e-ec8a-4ec9-8520-0d32a5586a1d" Jul 27 09:24:48.557906 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:48.557864 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5zqgm" Jul 27 09:24:48.558342 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:48.557871 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rqz85" Jul 27 09:24:48.558342 ip-10-0-128-103 kubenswrapper[2565]: E0727 09:24:48.558018 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5zqgm" podUID="91816deb-5261-44b7-857a-04cf036feffa" Jul 27 09:24:48.558342 ip-10-0-128-103 kubenswrapper[2565]: E0727 09:24:48.558089 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rqz85" podUID="204c97b6-2efe-4db7-bc9f-a5653fed77c7" Jul 27 09:24:49.558295 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:49.558260 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-trcl2" Jul 27 09:24:49.558806 ip-10-0-128-103 kubenswrapper[2565]: E0727 09:24:49.558420 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-trcl2" podUID="0d6ba35e-ec8a-4ec9-8520-0d32a5586a1d" Jul 27 09:24:50.119679 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:50.119635 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0d6ba35e-ec8a-4ec9-8520-0d32a5586a1d-metrics-certs\") pod \"network-metrics-daemon-trcl2\" (UID: \"0d6ba35e-ec8a-4ec9-8520-0d32a5586a1d\") " pod="openshift-multus/network-metrics-daemon-trcl2" Jul 27 09:24:50.119870 ip-10-0-128-103 kubenswrapper[2565]: E0727 09:24:50.119805 2565 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 27 09:24:50.119930 ip-10-0-128-103 kubenswrapper[2565]: E0727 09:24:50.119888 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0d6ba35e-ec8a-4ec9-8520-0d32a5586a1d-metrics-certs podName:0d6ba35e-ec8a-4ec9-8520-0d32a5586a1d nodeName:}" failed. No retries permitted until 2026-07-27 09:25:06.119867219 +0000 UTC m=+34.083653178 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0d6ba35e-ec8a-4ec9-8520-0d32a5586a1d-metrics-certs") pod "network-metrics-daemon-trcl2" (UID: "0d6ba35e-ec8a-4ec9-8520-0d32a5586a1d") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 27 09:24:50.220218 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:50.220129 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/204c97b6-2efe-4db7-bc9f-a5653fed77c7-original-pull-secret\") pod \"global-pull-secret-syncer-rqz85\" (UID: \"204c97b6-2efe-4db7-bc9f-a5653fed77c7\") " pod="kube-system/global-pull-secret-syncer-rqz85" Jul 27 09:24:50.220218 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:50.220205 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2ckrw\" (UniqueName: \"kubernetes.io/projected/91816deb-5261-44b7-857a-04cf036feffa-kube-api-access-2ckrw\") pod \"network-check-target-5zqgm\" (UID: \"91816deb-5261-44b7-857a-04cf036feffa\") " pod="openshift-network-diagnostics/network-check-target-5zqgm" Jul 27 09:24:50.220436 ip-10-0-128-103 kubenswrapper[2565]: E0727 09:24:50.220269 2565 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 27 09:24:50.220436 ip-10-0-128-103 kubenswrapper[2565]: E0727 09:24:50.220348 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/204c97b6-2efe-4db7-bc9f-a5653fed77c7-original-pull-secret podName:204c97b6-2efe-4db7-bc9f-a5653fed77c7 nodeName:}" failed. No retries permitted until 2026-07-27 09:25:06.220328718 +0000 UTC m=+34.184114666 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/204c97b6-2efe-4db7-bc9f-a5653fed77c7-original-pull-secret") pod "global-pull-secret-syncer-rqz85" (UID: "204c97b6-2efe-4db7-bc9f-a5653fed77c7") : object "kube-system"/"original-pull-secret" not registered Jul 27 09:24:50.220436 ip-10-0-128-103 kubenswrapper[2565]: E0727 09:24:50.220351 2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 27 09:24:50.220436 ip-10-0-128-103 kubenswrapper[2565]: E0727 09:24:50.220369 2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 27 09:24:50.220436 ip-10-0-128-103 kubenswrapper[2565]: E0727 09:24:50.220382 2565 projected.go:196] Error preparing data for projected volume kube-api-access-2ckrw for pod openshift-network-diagnostics/network-check-target-5zqgm: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 27 09:24:50.220673 ip-10-0-128-103 kubenswrapper[2565]: E0727 09:24:50.220457 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/91816deb-5261-44b7-857a-04cf036feffa-kube-api-access-2ckrw podName:91816deb-5261-44b7-857a-04cf036feffa nodeName:}" failed. No retries permitted until 2026-07-27 09:25:06.220441602 +0000 UTC m=+34.184227538 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-2ckrw" (UniqueName: "kubernetes.io/projected/91816deb-5261-44b7-857a-04cf036feffa-kube-api-access-2ckrw") pod "network-check-target-5zqgm" (UID: "91816deb-5261-44b7-857a-04cf036feffa") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 27 09:24:50.558225 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:50.558136 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5zqgm" Jul 27 09:24:50.558225 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:50.558185 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rqz85" Jul 27 09:24:50.558674 ip-10-0-128-103 kubenswrapper[2565]: E0727 09:24:50.558275 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5zqgm" podUID="91816deb-5261-44b7-857a-04cf036feffa" Jul 27 09:24:50.558674 ip-10-0-128-103 kubenswrapper[2565]: E0727 09:24:50.558354 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rqz85" podUID="204c97b6-2efe-4db7-bc9f-a5653fed77c7" Jul 27 09:24:51.557671 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:51.557629 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-trcl2" Jul 27 09:24:51.557858 ip-10-0-128-103 kubenswrapper[2565]: E0727 09:24:51.557779 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-trcl2" podUID="0d6ba35e-ec8a-4ec9-8520-0d32a5586a1d" Jul 27 09:24:52.559012 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:52.558977 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rqz85" Jul 27 09:24:52.559427 ip-10-0-128-103 kubenswrapper[2565]: E0727 09:24:52.559084 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rqz85" podUID="204c97b6-2efe-4db7-bc9f-a5653fed77c7" Jul 27 09:24:52.559427 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:52.559131 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5zqgm" Jul 27 09:24:52.559427 ip-10-0-128-103 kubenswrapper[2565]: E0727 09:24:52.559243 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5zqgm" podUID="91816deb-5261-44b7-857a-04cf036feffa" Jul 27 09:24:53.557564 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:53.557530 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-trcl2" Jul 27 09:24:53.557786 ip-10-0-128-103 kubenswrapper[2565]: E0727 09:24:53.557678 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-trcl2" podUID="0d6ba35e-ec8a-4ec9-8520-0d32a5586a1d" Jul 27 09:24:54.016377 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:24:54.016303 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod199af0d6_b2fc_4951_941f_948e403d3865.slice/crio-42f0ef366a204d186e2ff2c73bcea970f2b8022f1ab2f36196a9db9c227e0522 WatchSource:0}: Error finding container 42f0ef366a204d186e2ff2c73bcea970f2b8022f1ab2f36196a9db9c227e0522: Status 404 returned error can't find the container with id 42f0ef366a204d186e2ff2c73bcea970f2b8022f1ab2f36196a9db9c227e0522 Jul 27 09:24:54.557869 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:54.557674 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rqz85" Jul 27 09:24:54.557998 ip-10-0-128-103 kubenswrapper[2565]: E0727 09:24:54.557974 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rqz85" podUID="204c97b6-2efe-4db7-bc9f-a5653fed77c7" Jul 27 09:24:54.558467 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:54.558443 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5zqgm" Jul 27 09:24:54.558570 ip-10-0-128-103 kubenswrapper[2565]: E0727 09:24:54.558548 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5zqgm" podUID="91816deb-5261-44b7-857a-04cf036feffa" Jul 27 09:24:54.685499 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:54.685460 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-bmklx" event={"ID":"996c29bd-3969-4268-a795-4f4d21881d48","Type":"ContainerStarted","Data":"1ef26b6cad82d4b3f058589663126e44c8615be263894849c7d4c847f3bfa944"} Jul 27 09:24:54.687116 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:54.687083 2565 generic.go:356] "Generic (PLEG): container finished" podID="d0206685-d6b8-43ea-a691-ca3c1acd237a" containerID="38dd3f44045b3daf4553d65bdc4eb7a3a5ede425a1e293136a40eb5a896177ab" exitCode=0 Jul 27 09:24:54.687241 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:54.687169 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-54xcx" event={"ID":"d0206685-d6b8-43ea-a691-ca3c1acd237a","Type":"ContainerDied","Data":"38dd3f44045b3daf4553d65bdc4eb7a3a5ede425a1e293136a40eb5a896177ab"} Jul 27 09:24:54.688895 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:54.688868 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kb5qg" event={"ID":"09e4c928-4b42-484e-808c-dfaa1b17977b","Type":"ContainerStarted","Data":"c1f536ef8af8735704ed37e4223852fbe45f8546b3df3291cd4eee5c63e150dc"} Jul 27 09:24:54.690588 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:54.690567 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-bk5w8" event={"ID":"9cc406f3-cdff-4cde-ac7b-ff31abb20e0d","Type":"ContainerStarted","Data":"d04ad36f7a144c88863836a96dddbc462ab9d55402b2342dd5da416b3f85987c"} Jul 27 09:24:54.691922 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:54.691899 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-848w4" event={"ID":"2c043d82-d300-41ba-b138-60a03d65ea11","Type":"ContainerStarted","Data":"cd5cb2eaa599fcd3df713a1e3e208c8ed20211f9ff203b1fcaf8abda7c61c53b"} Jul 27 09:24:54.693384 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:54.693352 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-jmgz6" event={"ID":"29406f41-dc4d-4f8a-9627-9589659ae0d4","Type":"ContainerStarted","Data":"fdedafe9109bf71383694a302618204f83b194adb56f4f9c5c7a0175e63dcdd0"} Jul 27 09:24:54.694753 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:54.694731 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-r2wsn" event={"ID":"199af0d6-b2fc-4951-941f-948e403d3865","Type":"ContainerStarted","Data":"a9ad4800328d6aa670b9eade6fd01b393f2aca4a47c2677a19b09ca5e9644037"} Jul 27 09:24:54.694881 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:54.694787 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-r2wsn" event={"ID":"199af0d6-b2fc-4951-941f-948e403d3865","Type":"ContainerStarted","Data":"42f0ef366a204d186e2ff2c73bcea970f2b8022f1ab2f36196a9db9c227e0522"} Jul 27 09:24:54.703350 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:54.703293 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-bmklx" podStartSLOduration=2.840436988 podStartE2EDuration="21.703280404s" totalImagesPullingTime="18.862843416s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-27 09:24:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 09:24:54.703147623 +0000 UTC m=+22.666933580" watchObservedRunningTime="2026-07-27 09:24:54.703280404 +0000 UTC m=+22.667066360" Jul 27 09:24:54.703511 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:54.703371 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-103.ec2.internal" podStartSLOduration=20.703367567 podStartE2EDuration="20.703367567s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-27 09:24:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 09:24:37.663546281 +0000 UTC m=+5.627332243" watchObservedRunningTime="2026-07-27 09:24:54.703367567 +0000 UTC m=+22.667153524" Jul 27 09:24:54.719228 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:54.719175 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-848w4" podStartSLOduration=7.320047075 podStartE2EDuration="21.719159302s" totalImagesPullingTime="14.399112227s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 09:24:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 09:24:54.719094677 +0000 UTC m=+22.682880630" watchObservedRunningTime="2026-07-27 09:24:54.719159302 +0000 UTC m=+22.682945259" Jul 27 09:24:54.739747 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:54.739694 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-jmgz6" podStartSLOduration=2.833111846 podStartE2EDuration="21.739679823s" totalImagesPullingTime="18.906567977s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-27 09:24:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 09:24:54.7396323 +0000 UTC m=+22.703418257" watchObservedRunningTime="2026-07-27 09:24:54.739679823 +0000 UTC m=+22.703465780" Jul 27 09:24:54.760834 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:54.760761 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-bk5w8" podStartSLOduration=2.880956409 podStartE2EDuration="21.760747933s" totalImagesPullingTime="18.879791524s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-27 09:24:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 09:24:54.760387665 +0000 UTC m=+22.724173623" watchObservedRunningTime="2026-07-27 09:24:54.760747933 +0000 UTC m=+22.724533889" Jul 27 09:24:54.819072 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:54.819027 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-r2wsn" podStartSLOduration=17.819009626 podStartE2EDuration="17.819009626s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-27 09:24:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 09:24:54.818454873 +0000 UTC m=+22.782240841" watchObservedRunningTime="2026-07-27 09:24:54.819009626 +0000 UTC m=+22.782795583" Jul 27 09:24:55.415151 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:55.414999 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-848w4" Jul 27 09:24:55.416103 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:55.415913 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-848w4" Jul 27 09:24:55.557670 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:55.557632 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-trcl2" Jul 27 09:24:55.557894 ip-10-0-128-103 kubenswrapper[2565]: E0727 09:24:55.557760 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-trcl2" podUID="0d6ba35e-ec8a-4ec9-8520-0d32a5586a1d" Jul 27 09:24:55.700480 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:55.700463 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-mqhsn_5c031f05-bf4a-4ff7-81ad-381d31b163ff/ovn-acl-logging/0.log" Jul 27 09:24:55.700854 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:55.700830 2565 generic.go:356] "Generic (PLEG): container finished" podID="5c031f05-bf4a-4ff7-81ad-381d31b163ff" containerID="a5cfec00367737744d31d47cde602ed9207eea39af92281083d5324a46ed3b5d" exitCode=1 Jul 27 09:24:55.701147 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:55.700960 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-mqhsn" event={"ID":"5c031f05-bf4a-4ff7-81ad-381d31b163ff","Type":"ContainerStarted","Data":"f8ad2f9eac78c03645c52214c1d8abb800d6e9186481d6519d22527bcca11bd6"} Jul 27 09:24:55.701147 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:55.701029 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-mqhsn" event={"ID":"5c031f05-bf4a-4ff7-81ad-381d31b163ff","Type":"ContainerStarted","Data":"9e988fcf7dbec90bec70fba4c90c3af82e6100b45a71c34fa90a75386147682b"} Jul 27 09:24:55.701147 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:55.701063 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-mqhsn" event={"ID":"5c031f05-bf4a-4ff7-81ad-381d31b163ff","Type":"ContainerStarted","Data":"a10a3d05c756263f6eb32386060aded4a48cf011ef55c03455eb0e6effea0ec3"} Jul 27 09:24:55.701147 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:55.701093 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-mqhsn" event={"ID":"5c031f05-bf4a-4ff7-81ad-381d31b163ff","Type":"ContainerStarted","Data":"87d974086b777b61ba877702c29bb1f50388734680b2750d937a0b95f7c1020b"} Jul 27 09:24:55.701147 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:55.701134 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-mqhsn" event={"ID":"5c031f05-bf4a-4ff7-81ad-381d31b163ff","Type":"ContainerDied","Data":"a5cfec00367737744d31d47cde602ed9207eea39af92281083d5324a46ed3b5d"} Jul 27 09:24:55.701432 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:55.701185 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-mqhsn" event={"ID":"5c031f05-bf4a-4ff7-81ad-381d31b163ff","Type":"ContainerStarted","Data":"ab1f179c2b362a6fe98b2976d6057adff4b87ca5307916712e77cb1469bcbef7"} Jul 27 09:24:55.701930 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:55.701731 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-848w4" Jul 27 09:24:55.702249 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:55.702226 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-848w4" Jul 27 09:24:55.752119 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:55.752087 2565 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 27 09:24:56.437313 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:56.437150 2565 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-27T09:24:55.752109944Z","UUID":"5127b21a-dc4e-4d5d-bd05-c20a08943bec","Handler":null,"Name":"","Endpoint":""} Jul 27 09:24:56.439068 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:56.439032 2565 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 27 09:24:56.439213 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:56.439079 2565 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 27 09:24:56.557910 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:56.557884 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rqz85" Jul 27 09:24:56.558096 ip-10-0-128-103 kubenswrapper[2565]: E0727 09:24:56.558026 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rqz85" podUID="204c97b6-2efe-4db7-bc9f-a5653fed77c7" Jul 27 09:24:56.558096 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:56.558073 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5zqgm" Jul 27 09:24:56.559821 ip-10-0-128-103 kubenswrapper[2565]: E0727 09:24:56.559494 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5zqgm" podUID="91816deb-5261-44b7-857a-04cf036feffa" Jul 27 09:24:56.704923 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:56.704785 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-rcftn" event={"ID":"6fd3f632-9d80-4e6e-8d4f-f5314c0f174a","Type":"ContainerStarted","Data":"e10bf8cc781ab26f73497856bb4cd251c055bb067d9d17caf82496458fe4e6a0"} Jul 27 09:24:56.706898 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:56.706860 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kb5qg" event={"ID":"09e4c928-4b42-484e-808c-dfaa1b17977b","Type":"ContainerStarted","Data":"508e2a99a1687163c4cbf5c04e095deba0ed328a6787dd61a275da9e5ca900eb"} Jul 27 09:24:56.739601 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:56.739547 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-rcftn" podStartSLOduration=4.865151086 podStartE2EDuration="23.739525265s" totalImagesPullingTime="18.874374179s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-27 09:24:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 09:24:56.739040818 +0000 UTC m=+24.702826799" watchObservedRunningTime="2026-07-27 09:24:56.739525265 +0000 UTC m=+24.703311224" Jul 27 09:24:57.558275 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:57.558046 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-trcl2" Jul 27 09:24:57.558731 ip-10-0-128-103 kubenswrapper[2565]: E0727 09:24:57.558386 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-trcl2" podUID="0d6ba35e-ec8a-4ec9-8520-0d32a5586a1d" Jul 27 09:24:57.712197 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:57.712114 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kb5qg" event={"ID":"09e4c928-4b42-484e-808c-dfaa1b17977b","Type":"ContainerStarted","Data":"8eacf4f798eb1397f58233c71572df47f602cbf7b1b327b51bfdee23ee814fb4"} Jul 27 09:24:58.557717 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:58.557677 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5zqgm" Jul 27 09:24:58.557912 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:58.557677 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rqz85" Jul 27 09:24:58.557912 ip-10-0-128-103 kubenswrapper[2565]: E0727 09:24:58.557842 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5zqgm" podUID="91816deb-5261-44b7-857a-04cf036feffa" Jul 27 09:24:58.558034 ip-10-0-128-103 kubenswrapper[2565]: E0727 09:24:58.557904 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rqz85" podUID="204c97b6-2efe-4db7-bc9f-a5653fed77c7" Jul 27 09:24:59.143804 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:59.143692 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kb5qg" podStartSLOduration=5.047277298 podStartE2EDuration="26.143673789s" totalImagesPullingTime="21.096396491s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-27 09:24:33 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 09:24:58.877329906 +0000 UTC m=+26.841115862" watchObservedRunningTime="2026-07-27 09:24:59.143673789 +0000 UTC m=+27.107459746" Jul 27 09:24:59.558173 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:59.558098 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-trcl2" Jul 27 09:24:59.558367 ip-10-0-128-103 kubenswrapper[2565]: E0727 09:24:59.558211 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-trcl2" podUID="0d6ba35e-ec8a-4ec9-8520-0d32a5586a1d" Jul 27 09:24:59.718075 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:59.718042 2565 generic.go:356] "Generic (PLEG): container finished" podID="d0206685-d6b8-43ea-a691-ca3c1acd237a" containerID="a7e52469c2aa25c282c8c87c7619fbb95dbea141b6d3ea04bc68bc918dc6dd0d" exitCode=0 Jul 27 09:24:59.718240 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:59.718078 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-54xcx" event={"ID":"d0206685-d6b8-43ea-a691-ca3c1acd237a","Type":"ContainerDied","Data":"a7e52469c2aa25c282c8c87c7619fbb95dbea141b6d3ea04bc68bc918dc6dd0d"} Jul 27 09:24:59.725961 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:59.725941 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-mqhsn_5c031f05-bf4a-4ff7-81ad-381d31b163ff/ovn-acl-logging/0.log" Jul 27 09:24:59.726297 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:24:59.726278 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-mqhsn" event={"ID":"5c031f05-bf4a-4ff7-81ad-381d31b163ff","Type":"ContainerStarted","Data":"152b3cfb10fad65734a2c63f33dd24955920c211a25f3b1be527437a40d9afe4"} Jul 27 09:25:00.557720 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:25:00.557685 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5zqgm" Jul 27 09:25:00.558188 ip-10-0-128-103 kubenswrapper[2565]: E0727 09:25:00.557820 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5zqgm" podUID="91816deb-5261-44b7-857a-04cf036feffa" Jul 27 09:25:00.558188 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:25:00.557871 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rqz85" Jul 27 09:25:00.558188 ip-10-0-128-103 kubenswrapper[2565]: E0727 09:25:00.557958 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rqz85" podUID="204c97b6-2efe-4db7-bc9f-a5653fed77c7" Jul 27 09:25:01.558349 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:25:01.558123 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-trcl2" Jul 27 09:25:01.558842 ip-10-0-128-103 kubenswrapper[2565]: E0727 09:25:01.558365 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-trcl2" podUID="0d6ba35e-ec8a-4ec9-8520-0d32a5586a1d" Jul 27 09:25:01.732878 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:25:01.732846 2565 generic.go:356] "Generic (PLEG): container finished" podID="d0206685-d6b8-43ea-a691-ca3c1acd237a" containerID="adc3e9df573db0fd80b0c3b3759d56f0dcdb5919c7df28d57d8beb351aabe87c" exitCode=0 Jul 27 09:25:01.733029 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:25:01.732933 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-54xcx" event={"ID":"d0206685-d6b8-43ea-a691-ca3c1acd237a","Type":"ContainerDied","Data":"adc3e9df573db0fd80b0c3b3759d56f0dcdb5919c7df28d57d8beb351aabe87c"} Jul 27 09:25:01.736385 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:25:01.736367 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-mqhsn_5c031f05-bf4a-4ff7-81ad-381d31b163ff/ovn-acl-logging/0.log" Jul 27 09:25:01.736746 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:25:01.736724 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-mqhsn" event={"ID":"5c031f05-bf4a-4ff7-81ad-381d31b163ff","Type":"ContainerStarted","Data":"4a747c17dee951e06ce9e8a75ed6e0ebdcf1fb61ad0e826a29ae9d43a2d68098"} Jul 27 09:25:01.736995 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:25:01.736977 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-mqhsn" Jul 27 09:25:01.737101 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:25:01.737036 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-mqhsn" Jul 27 09:25:01.737140 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:25:01.737118 2565 scope.go:117] "RemoveContainer" containerID="a5cfec00367737744d31d47cde602ed9207eea39af92281083d5324a46ed3b5d" Jul 27 09:25:01.753523 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:25:01.753496 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-mqhsn" Jul 27 09:25:02.559139 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:25:02.559109 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rqz85" Jul 27 09:25:02.559632 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:25:02.559187 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5zqgm" Jul 27 09:25:02.559632 ip-10-0-128-103 kubenswrapper[2565]: E0727 09:25:02.559208 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rqz85" podUID="204c97b6-2efe-4db7-bc9f-a5653fed77c7" Jul 27 09:25:02.559632 ip-10-0-128-103 kubenswrapper[2565]: E0727 09:25:02.559275 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5zqgm" podUID="91816deb-5261-44b7-857a-04cf036feffa" Jul 27 09:25:02.742407 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:25:02.742383 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-mqhsn_5c031f05-bf4a-4ff7-81ad-381d31b163ff/ovn-acl-logging/0.log" Jul 27 09:25:02.742735 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:25:02.742707 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-mqhsn" event={"ID":"5c031f05-bf4a-4ff7-81ad-381d31b163ff","Type":"ContainerStarted","Data":"85192d234527a8bc1194b8fcc6ca99395e5d6de4544738df58709fc4d7c6cb7c"} Jul 27 09:25:02.743027 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:25:02.743000 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-mqhsn" Jul 27 09:25:02.760969 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:25:02.760940 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-mqhsn" Jul 27 09:25:02.772640 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:25:02.772581 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-mqhsn" podStartSLOduration=10.634063806 podStartE2EDuration="29.772566278s" totalImagesPullingTime="19.138502472s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-27 09:24:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 09:25:02.772131536 +0000 UTC m=+30.735917499" watchObservedRunningTime="2026-07-27 09:25:02.772566278 +0000 UTC m=+30.736352263" Jul 27 09:25:03.118075 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:25:03.117850 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-5zqgm"] Jul 27 09:25:03.118225 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:25:03.118177 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5zqgm" Jul 27 09:25:03.118313 ip-10-0-128-103 kubenswrapper[2565]: E0727 09:25:03.118288 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5zqgm" podUID="91816deb-5261-44b7-857a-04cf036feffa" Jul 27 09:25:03.120815 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:25:03.120511 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-rqz85"] Jul 27 09:25:03.120815 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:25:03.120626 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rqz85" Jul 27 09:25:03.120815 ip-10-0-128-103 kubenswrapper[2565]: E0727 09:25:03.120745 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rqz85" podUID="204c97b6-2efe-4db7-bc9f-a5653fed77c7" Jul 27 09:25:03.121370 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:25:03.121153 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-trcl2"] Jul 27 09:25:03.121370 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:25:03.121253 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-trcl2" Jul 27 09:25:03.121370 ip-10-0-128-103 kubenswrapper[2565]: E0727 09:25:03.121331 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-trcl2" podUID="0d6ba35e-ec8a-4ec9-8520-0d32a5586a1d" Jul 27 09:25:03.747606 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:25:03.747567 2565 generic.go:356] "Generic (PLEG): container finished" podID="d0206685-d6b8-43ea-a691-ca3c1acd237a" containerID="d27c3f2c7fa559772f25ba86e774f032cb422765fff7edb94a5e7ca719290074" exitCode=0 Jul 27 09:25:03.748284 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:25:03.747651 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-54xcx" event={"ID":"d0206685-d6b8-43ea-a691-ca3c1acd237a","Type":"ContainerDied","Data":"d27c3f2c7fa559772f25ba86e774f032cb422765fff7edb94a5e7ca719290074"} Jul 27 09:25:04.557722 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:25:04.557688 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rqz85" Jul 27 09:25:04.557901 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:25:04.557689 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-trcl2" Jul 27 09:25:04.557901 ip-10-0-128-103 kubenswrapper[2565]: E0727 09:25:04.557830 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rqz85" podUID="204c97b6-2efe-4db7-bc9f-a5653fed77c7" Jul 27 09:25:04.557980 ip-10-0-128-103 kubenswrapper[2565]: E0727 09:25:04.557949 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-trcl2" podUID="0d6ba35e-ec8a-4ec9-8520-0d32a5586a1d" Jul 27 09:25:04.558044 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:25:04.558015 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5zqgm" Jul 27 09:25:04.558126 ip-10-0-128-103 kubenswrapper[2565]: E0727 09:25:04.558106 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5zqgm" podUID="91816deb-5261-44b7-857a-04cf036feffa" Jul 27 09:25:06.145348 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:25:06.145314 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0d6ba35e-ec8a-4ec9-8520-0d32a5586a1d-metrics-certs\") pod \"network-metrics-daemon-trcl2\" (UID: \"0d6ba35e-ec8a-4ec9-8520-0d32a5586a1d\") " pod="openshift-multus/network-metrics-daemon-trcl2" Jul 27 09:25:06.146071 ip-10-0-128-103 kubenswrapper[2565]: E0727 09:25:06.145506 2565 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 27 09:25:06.146071 ip-10-0-128-103 kubenswrapper[2565]: E0727 09:25:06.145591 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0d6ba35e-ec8a-4ec9-8520-0d32a5586a1d-metrics-certs podName:0d6ba35e-ec8a-4ec9-8520-0d32a5586a1d nodeName:}" failed. No retries permitted until 2026-07-27 09:25:38.145569575 +0000 UTC m=+66.109355529 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0d6ba35e-ec8a-4ec9-8520-0d32a5586a1d-metrics-certs") pod "network-metrics-daemon-trcl2" (UID: "0d6ba35e-ec8a-4ec9-8520-0d32a5586a1d") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 27 09:25:06.245741 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:25:06.245707 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/204c97b6-2efe-4db7-bc9f-a5653fed77c7-original-pull-secret\") pod \"global-pull-secret-syncer-rqz85\" (UID: \"204c97b6-2efe-4db7-bc9f-a5653fed77c7\") " pod="kube-system/global-pull-secret-syncer-rqz85" Jul 27 09:25:06.245930 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:25:06.245757 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2ckrw\" (UniqueName: \"kubernetes.io/projected/91816deb-5261-44b7-857a-04cf036feffa-kube-api-access-2ckrw\") pod \"network-check-target-5zqgm\" (UID: \"91816deb-5261-44b7-857a-04cf036feffa\") " pod="openshift-network-diagnostics/network-check-target-5zqgm" Jul 27 09:25:06.245930 ip-10-0-128-103 kubenswrapper[2565]: E0727 09:25:06.245875 2565 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 27 09:25:06.246007 ip-10-0-128-103 kubenswrapper[2565]: E0727 09:25:06.245945 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/204c97b6-2efe-4db7-bc9f-a5653fed77c7-original-pull-secret podName:204c97b6-2efe-4db7-bc9f-a5653fed77c7 nodeName:}" failed. No retries permitted until 2026-07-27 09:25:38.245929072 +0000 UTC m=+66.209715037 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/204c97b6-2efe-4db7-bc9f-a5653fed77c7-original-pull-secret") pod "global-pull-secret-syncer-rqz85" (UID: "204c97b6-2efe-4db7-bc9f-a5653fed77c7") : object "kube-system"/"original-pull-secret" not registered Jul 27 09:25:06.246007 ip-10-0-128-103 kubenswrapper[2565]: E0727 09:25:06.245880 2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 27 09:25:06.246007 ip-10-0-128-103 kubenswrapper[2565]: E0727 09:25:06.245995 2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 27 09:25:06.246007 ip-10-0-128-103 kubenswrapper[2565]: E0727 09:25:06.246007 2565 projected.go:196] Error preparing data for projected volume kube-api-access-2ckrw for pod openshift-network-diagnostics/network-check-target-5zqgm: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 27 09:25:06.246130 ip-10-0-128-103 kubenswrapper[2565]: E0727 09:25:06.246049 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/91816deb-5261-44b7-857a-04cf036feffa-kube-api-access-2ckrw podName:91816deb-5261-44b7-857a-04cf036feffa nodeName:}" failed. No retries permitted until 2026-07-27 09:25:38.246037334 +0000 UTC m=+66.209823272 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-2ckrw" (UniqueName: "kubernetes.io/projected/91816deb-5261-44b7-857a-04cf036feffa-kube-api-access-2ckrw") pod "network-check-target-5zqgm" (UID: "91816deb-5261-44b7-857a-04cf036feffa") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 27 09:25:06.558266 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:25:06.558176 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rqz85" Jul 27 09:25:06.558266 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:25:06.558210 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-trcl2" Jul 27 09:25:06.558266 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:25:06.558180 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5zqgm" Jul 27 09:25:06.558536 ip-10-0-128-103 kubenswrapper[2565]: E0727 09:25:06.558303 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-rqz85" podUID="204c97b6-2efe-4db7-bc9f-a5653fed77c7" Jul 27 09:25:06.558536 ip-10-0-128-103 kubenswrapper[2565]: E0727 09:25:06.558392 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5zqgm" podUID="91816deb-5261-44b7-857a-04cf036feffa" Jul 27 09:25:06.558536 ip-10-0-128-103 kubenswrapper[2565]: E0727 09:25:06.558493 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-trcl2" podUID="0d6ba35e-ec8a-4ec9-8520-0d32a5586a1d" Jul 27 09:25:07.423636 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:25:07.423601 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-103.ec2.internal" event="NodeReady" Jul 27 09:25:07.424098 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:25:07.423792 2565 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 27 09:25:07.465729 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:25:07.465694 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-7lzb6"] Jul 27 09:25:07.470754 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:25:07.470728 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-7lzb6" Jul 27 09:25:07.473122 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:25:07.473088 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-q52r7"] Jul 27 09:25:07.473269 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:25:07.473213 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 27 09:25:07.473269 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:25:07.473226 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-v4cq5\"" Jul 27 09:25:07.473269 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:25:07.473234 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 27 09:25:07.476513 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:25:07.476454 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-q52r7" Jul 27 09:25:07.477943 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:25:07.477883 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-7lzb6"] Jul 27 09:25:07.478849 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:25:07.478808 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 27 09:25:07.478980 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:25:07.478961 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-zzcwm\"" Jul 27 09:25:07.479068 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:25:07.479039 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 27 09:25:07.479170 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:25:07.479151 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 27 09:25:07.488663 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:25:07.488635 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-q52r7"] Jul 27 09:25:07.557539 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:25:07.557499 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/ef01a458-240c-4477-9ef9-d5b21026df4f-config-volume\") pod \"dns-default-7lzb6\" (UID: \"ef01a458-240c-4477-9ef9-d5b21026df4f\") " pod="openshift-dns/dns-default-7lzb6" Jul 27 09:25:07.557701 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:25:07.557551 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ef01a458-240c-4477-9ef9-d5b21026df4f-tmp-dir\") pod \"dns-default-7lzb6\" (UID: \"ef01a458-240c-4477-9ef9-d5b21026df4f\") " pod="openshift-dns/dns-default-7lzb6" Jul 27 09:25:07.557701 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:25:07.557649 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jgbsg\" (UniqueName: \"kubernetes.io/projected/ef01a458-240c-4477-9ef9-d5b21026df4f-kube-api-access-jgbsg\") pod \"dns-default-7lzb6\" (UID: \"ef01a458-240c-4477-9ef9-d5b21026df4f\") " pod="openshift-dns/dns-default-7lzb6" Jul 27 09:25:07.557701 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:25:07.557692 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9b739ad1-657a-40f1-a932-0a94f3960ca4-cert\") pod \"ingress-canary-q52r7\" (UID: \"9b739ad1-657a-40f1-a932-0a94f3960ca4\") " pod="openshift-ingress-canary/ingress-canary-q52r7" Jul 27 09:25:07.557882 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:25:07.557711 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ef01a458-240c-4477-9ef9-d5b21026df4f-metrics-tls\") pod \"dns-default-7lzb6\" (UID: \"ef01a458-240c-4477-9ef9-d5b21026df4f\") " pod="openshift-dns/dns-default-7lzb6" Jul 27 09:25:07.557882 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:25:07.557740 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zk8q8\" (UniqueName: \"kubernetes.io/projected/9b739ad1-657a-40f1-a932-0a94f3960ca4-kube-api-access-zk8q8\") pod \"ingress-canary-q52r7\" (UID: \"9b739ad1-657a-40f1-a932-0a94f3960ca4\") " pod="openshift-ingress-canary/ingress-canary-q52r7" Jul 27 09:25:07.658428 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:25:07.658390 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jgbsg\" (UniqueName: \"kubernetes.io/projected/ef01a458-240c-4477-9ef9-d5b21026df4f-kube-api-access-jgbsg\") pod \"dns-default-7lzb6\" (UID: \"ef01a458-240c-4477-9ef9-d5b21026df4f\") " pod="openshift-dns/dns-default-7lzb6" Jul 27 09:25:07.658611 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:25:07.658439 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9b739ad1-657a-40f1-a932-0a94f3960ca4-cert\") pod \"ingress-canary-q52r7\" (UID: \"9b739ad1-657a-40f1-a932-0a94f3960ca4\") " pod="openshift-ingress-canary/ingress-canary-q52r7" Jul 27 09:25:07.658611 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:25:07.658466 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ef01a458-240c-4477-9ef9-d5b21026df4f-metrics-tls\") pod \"dns-default-7lzb6\" (UID: \"ef01a458-240c-4477-9ef9-d5b21026df4f\") " pod="openshift-dns/dns-default-7lzb6" Jul 27 09:25:07.658611 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:25:07.658498 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zk8q8\" (UniqueName: \"kubernetes.io/projected/9b739ad1-657a-40f1-a932-0a94f3960ca4-kube-api-access-zk8q8\") pod \"ingress-canary-q52r7\" (UID: \"9b739ad1-657a-40f1-a932-0a94f3960ca4\") " pod="openshift-ingress-canary/ingress-canary-q52r7" Jul 27 09:25:07.658611 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:25:07.658580 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/ef01a458-240c-4477-9ef9-d5b21026df4f-config-volume\") pod \"dns-default-7lzb6\" (UID: \"ef01a458-240c-4477-9ef9-d5b21026df4f\") " pod="openshift-dns/dns-default-7lzb6" Jul 27 09:25:07.658850 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:25:07.658614 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ef01a458-240c-4477-9ef9-d5b21026df4f-tmp-dir\") pod \"dns-default-7lzb6\" (UID: \"ef01a458-240c-4477-9ef9-d5b21026df4f\") " pod="openshift-dns/dns-default-7lzb6" Jul 27 09:25:07.658850 ip-10-0-128-103 kubenswrapper[2565]: E0727 09:25:07.658686 2565 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 27 09:25:07.658850 ip-10-0-128-103 kubenswrapper[2565]: E0727 09:25:07.658759 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ef01a458-240c-4477-9ef9-d5b21026df4f-metrics-tls podName:ef01a458-240c-4477-9ef9-d5b21026df4f nodeName:}" failed. No retries permitted until 2026-07-27 09:25:08.15873793 +0000 UTC m=+36.122523880 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/ef01a458-240c-4477-9ef9-d5b21026df4f-metrics-tls") pod "dns-default-7lzb6" (UID: "ef01a458-240c-4477-9ef9-d5b21026df4f") : secret "dns-default-metrics-tls" not found Jul 27 09:25:07.658976 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:25:07.658957 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ef01a458-240c-4477-9ef9-d5b21026df4f-tmp-dir\") pod \"dns-default-7lzb6\" (UID: \"ef01a458-240c-4477-9ef9-d5b21026df4f\") " pod="openshift-dns/dns-default-7lzb6" Jul 27 09:25:07.659460 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:25:07.659437 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/ef01a458-240c-4477-9ef9-d5b21026df4f-config-volume\") pod \"dns-default-7lzb6\" (UID: \"ef01a458-240c-4477-9ef9-d5b21026df4f\") " pod="openshift-dns/dns-default-7lzb6" Jul 27 09:25:07.663558 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:25:07.663535 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9b739ad1-657a-40f1-a932-0a94f3960ca4-cert\") pod \"ingress-canary-q52r7\" (UID: \"9b739ad1-657a-40f1-a932-0a94f3960ca4\") " pod="openshift-ingress-canary/ingress-canary-q52r7" Jul 27 09:25:07.669351 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:25:07.669328 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jgbsg\" (UniqueName: \"kubernetes.io/projected/ef01a458-240c-4477-9ef9-d5b21026df4f-kube-api-access-jgbsg\") pod \"dns-default-7lzb6\" (UID: \"ef01a458-240c-4477-9ef9-d5b21026df4f\") " pod="openshift-dns/dns-default-7lzb6" Jul 27 09:25:07.669461 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:25:07.669422 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zk8q8\" (UniqueName: \"kubernetes.io/projected/9b739ad1-657a-40f1-a932-0a94f3960ca4-kube-api-access-zk8q8\") pod \"ingress-canary-q52r7\" (UID: \"9b739ad1-657a-40f1-a932-0a94f3960ca4\") " pod="openshift-ingress-canary/ingress-canary-q52r7" Jul 27 09:25:07.791208 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:25:07.791125 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-q52r7" Jul 27 09:25:08.161977 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:25:08.161726 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ef01a458-240c-4477-9ef9-d5b21026df4f-metrics-tls\") pod \"dns-default-7lzb6\" (UID: \"ef01a458-240c-4477-9ef9-d5b21026df4f\") " pod="openshift-dns/dns-default-7lzb6" Jul 27 09:25:08.162115 ip-10-0-128-103 kubenswrapper[2565]: E0727 09:25:08.161883 2565 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 27 09:25:08.162162 ip-10-0-128-103 kubenswrapper[2565]: E0727 09:25:08.162132 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ef01a458-240c-4477-9ef9-d5b21026df4f-metrics-tls podName:ef01a458-240c-4477-9ef9-d5b21026df4f nodeName:}" failed. No retries permitted until 2026-07-27 09:25:09.162108297 +0000 UTC m=+37.125894247 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/ef01a458-240c-4477-9ef9-d5b21026df4f-metrics-tls") pod "dns-default-7lzb6" (UID: "ef01a458-240c-4477-9ef9-d5b21026df4f") : secret "dns-default-metrics-tls" not found Jul 27 09:25:08.162502 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:25:08.162485 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-q52r7"] Jul 27 09:25:08.166387 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:25:08.166365 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9b739ad1_657a_40f1_a932_0a94f3960ca4.slice/crio-5eb6bceaf259caf7443a402e18da7a595738853d97a178f1e80671efb40b86c9 WatchSource:0}: Error finding container 5eb6bceaf259caf7443a402e18da7a595738853d97a178f1e80671efb40b86c9: Status 404 returned error can't find the container with id 5eb6bceaf259caf7443a402e18da7a595738853d97a178f1e80671efb40b86c9 Jul 27 09:25:08.558414 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:25:08.558377 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rqz85" Jul 27 09:25:08.558932 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:25:08.558377 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5zqgm" Jul 27 09:25:08.558932 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:25:08.558384 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-trcl2" Jul 27 09:25:08.561247 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:25:08.561162 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 27 09:25:08.561247 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:25:08.561170 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 27 09:25:08.561247 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:25:08.561223 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 27 09:25:08.562077 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:25:08.562059 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-fpm4f\"" Jul 27 09:25:08.562175 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:25:08.562102 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 27 09:25:08.562175 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:25:08.562119 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-bnzn6\"" Jul 27 09:25:08.764140 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:25:08.764104 2565 generic.go:356] "Generic (PLEG): container finished" podID="d0206685-d6b8-43ea-a691-ca3c1acd237a" containerID="ad9f19e79e39047fa35d007d559b31ad173985f0dcb800b366e05a06ebd1f289" exitCode=0 Jul 27 09:25:08.764293 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:25:08.764228 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-54xcx" event={"ID":"d0206685-d6b8-43ea-a691-ca3c1acd237a","Type":"ContainerDied","Data":"ad9f19e79e39047fa35d007d559b31ad173985f0dcb800b366e05a06ebd1f289"} Jul 27 09:25:08.765587 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:25:08.765552 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-q52r7" event={"ID":"9b739ad1-657a-40f1-a932-0a94f3960ca4","Type":"ContainerStarted","Data":"5eb6bceaf259caf7443a402e18da7a595738853d97a178f1e80671efb40b86c9"} Jul 27 09:25:09.170473 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:25:09.170434 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ef01a458-240c-4477-9ef9-d5b21026df4f-metrics-tls\") pod \"dns-default-7lzb6\" (UID: \"ef01a458-240c-4477-9ef9-d5b21026df4f\") " pod="openshift-dns/dns-default-7lzb6" Jul 27 09:25:09.170668 ip-10-0-128-103 kubenswrapper[2565]: E0727 09:25:09.170599 2565 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 27 09:25:09.170668 ip-10-0-128-103 kubenswrapper[2565]: E0727 09:25:09.170667 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ef01a458-240c-4477-9ef9-d5b21026df4f-metrics-tls podName:ef01a458-240c-4477-9ef9-d5b21026df4f nodeName:}" failed. No retries permitted until 2026-07-27 09:25:11.170650492 +0000 UTC m=+39.134436427 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/ef01a458-240c-4477-9ef9-d5b21026df4f-metrics-tls") pod "dns-default-7lzb6" (UID: "ef01a458-240c-4477-9ef9-d5b21026df4f") : secret "dns-default-metrics-tls" not found Jul 27 09:25:09.772384 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:25:09.772342 2565 generic.go:356] "Generic (PLEG): container finished" podID="d0206685-d6b8-43ea-a691-ca3c1acd237a" containerID="b00390d7c77bca3052a65f184ecdb3f304434d91ebe47677613a92961e346e36" exitCode=0 Jul 27 09:25:09.772817 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:25:09.772385 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-54xcx" event={"ID":"d0206685-d6b8-43ea-a691-ca3c1acd237a","Type":"ContainerDied","Data":"b00390d7c77bca3052a65f184ecdb3f304434d91ebe47677613a92961e346e36"} Jul 27 09:25:09.886299 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:25:09.886270 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-848w4_2c043d82-d300-41ba-b138-60a03d65ea11/konnectivity-agent/0.log" limit=1024 Jul 27 09:25:10.056423 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:25:10.056339 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-848w4_2c043d82-d300-41ba-b138-60a03d65ea11/konnectivity-agent/0.log" limit=1024 Jul 27 09:25:10.777671 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:25:10.777430 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-q52r7" event={"ID":"9b739ad1-657a-40f1-a932-0a94f3960ca4","Type":"ContainerStarted","Data":"f5b83533039b0bd0e7346bdbf3acaf9eb5cd686e5565d84d5541009cd8fb3f9f"} Jul 27 09:25:10.780826 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:25:10.780801 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-54xcx" event={"ID":"d0206685-d6b8-43ea-a691-ca3c1acd237a","Type":"ContainerStarted","Data":"922c1ddae5c95414a5e668179e460b5144725f081f89a1a0b5a4040ff9dc09f4"} Jul 27 09:25:10.797345 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:25:10.797298 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-q52r7" podStartSLOduration=1.3591087499999999 podStartE2EDuration="3.797281659s" totalImagesPullingTime="2.438172909s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 09:25:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 09:25:10.796417318 +0000 UTC m=+38.760203287" watchObservedRunningTime="2026-07-27 09:25:10.797281659 +0000 UTC m=+38.761067615" Jul 27 09:25:10.820467 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:25:10.820408 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-54xcx" podStartSLOduration=8.029603357 podStartE2EDuration="37.820392107s" totalImagesPullingTime="29.79078875s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-27 09:24:33 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 09:25:10.819116176 +0000 UTC m=+38.782902133" watchObservedRunningTime="2026-07-27 09:25:10.820392107 +0000 UTC m=+38.784178064" Jul 27 09:25:10.850188 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:25:10.850155 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-848w4_2c043d82-d300-41ba-b138-60a03d65ea11/konnectivity-agent/0.log" limit=1024 Jul 27 09:25:10.941460 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:25:10.941362 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-848w4_2c043d82-d300-41ba-b138-60a03d65ea11/konnectivity-agent/0.log" limit=1024 Jul 27 09:25:11.003084 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:25:11.003051 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-848w4_2c043d82-d300-41ba-b138-60a03d65ea11/konnectivity-agent/0.log" limit=1024 Jul 27 09:25:11.188288 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:25:11.188257 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ef01a458-240c-4477-9ef9-d5b21026df4f-metrics-tls\") pod \"dns-default-7lzb6\" (UID: \"ef01a458-240c-4477-9ef9-d5b21026df4f\") " pod="openshift-dns/dns-default-7lzb6" Jul 27 09:25:11.188419 ip-10-0-128-103 kubenswrapper[2565]: E0727 09:25:11.188402 2565 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 27 09:25:11.188467 ip-10-0-128-103 kubenswrapper[2565]: E0727 09:25:11.188462 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ef01a458-240c-4477-9ef9-d5b21026df4f-metrics-tls podName:ef01a458-240c-4477-9ef9-d5b21026df4f nodeName:}" failed. No retries permitted until 2026-07-27 09:25:15.18844955 +0000 UTC m=+43.152235485 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/ef01a458-240c-4477-9ef9-d5b21026df4f-metrics-tls") pod "dns-default-7lzb6" (UID: "ef01a458-240c-4477-9ef9-d5b21026df4f") : secret "dns-default-metrics-tls" not found Jul 27 09:25:12.641424 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:25:12.641391 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-848w4_2c043d82-d300-41ba-b138-60a03d65ea11/konnectivity-agent/0.log" limit=1024 Jul 27 09:25:12.700494 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:25:12.700462 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-848w4_2c043d82-d300-41ba-b138-60a03d65ea11/konnectivity-agent/0.log" limit=1024 Jul 27 09:25:12.934064 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:25:12.933988 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-848w4_2c043d82-d300-41ba-b138-60a03d65ea11/konnectivity-agent/0.log" limit=1024 Jul 27 09:25:12.993156 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:25:12.993121 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-848w4_2c043d82-d300-41ba-b138-60a03d65ea11/konnectivity-agent/0.log" limit=1024 Jul 27 09:25:15.215432 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:25:15.215376 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ef01a458-240c-4477-9ef9-d5b21026df4f-metrics-tls\") pod \"dns-default-7lzb6\" (UID: \"ef01a458-240c-4477-9ef9-d5b21026df4f\") " pod="openshift-dns/dns-default-7lzb6" Jul 27 09:25:15.215934 ip-10-0-128-103 kubenswrapper[2565]: E0727 09:25:15.215534 2565 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 27 09:25:15.215934 ip-10-0-128-103 kubenswrapper[2565]: E0727 09:25:15.215597 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ef01a458-240c-4477-9ef9-d5b21026df4f-metrics-tls podName:ef01a458-240c-4477-9ef9-d5b21026df4f nodeName:}" failed. No retries permitted until 2026-07-27 09:25:23.21557952 +0000 UTC m=+51.179365470 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/ef01a458-240c-4477-9ef9-d5b21026df4f-metrics-tls") pod "dns-default-7lzb6" (UID: "ef01a458-240c-4477-9ef9-d5b21026df4f") : secret "dns-default-metrics-tls" not found Jul 27 09:25:17.912101 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:25:17.912068 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-848w4_2c043d82-d300-41ba-b138-60a03d65ea11/konnectivity-agent/0.log" limit=1024 Jul 27 09:25:18.002286 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:25:18.002252 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-848w4_2c043d82-d300-41ba-b138-60a03d65ea11/konnectivity-agent/0.log" limit=1024 Jul 27 09:25:19.076342 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:25:19.076305 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-848w4_2c043d82-d300-41ba-b138-60a03d65ea11/konnectivity-agent/0.log" limit=1024 Jul 27 09:25:19.152849 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:25:19.152814 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-848w4_2c043d82-d300-41ba-b138-60a03d65ea11/konnectivity-agent/0.log" limit=1024 Jul 27 09:25:19.946236 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:25:19.946199 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-848w4_2c043d82-d300-41ba-b138-60a03d65ea11/konnectivity-agent/0.log" limit=1024 Jul 27 09:25:20.022123 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:25:20.022091 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-848w4_2c043d82-d300-41ba-b138-60a03d65ea11/konnectivity-agent/0.log" limit=1024 Jul 27 09:25:23.277903 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:25:23.277866 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ef01a458-240c-4477-9ef9-d5b21026df4f-metrics-tls\") pod \"dns-default-7lzb6\" (UID: \"ef01a458-240c-4477-9ef9-d5b21026df4f\") " pod="openshift-dns/dns-default-7lzb6" Jul 27 09:25:23.278371 ip-10-0-128-103 kubenswrapper[2565]: E0727 09:25:23.278016 2565 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 27 09:25:23.278371 ip-10-0-128-103 kubenswrapper[2565]: E0727 09:25:23.278079 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ef01a458-240c-4477-9ef9-d5b21026df4f-metrics-tls podName:ef01a458-240c-4477-9ef9-d5b21026df4f nodeName:}" failed. No retries permitted until 2026-07-27 09:25:39.278063748 +0000 UTC m=+67.241849696 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/ef01a458-240c-4477-9ef9-d5b21026df4f-metrics-tls") pod "dns-default-7lzb6" (UID: "ef01a458-240c-4477-9ef9-d5b21026df4f") : secret "dns-default-metrics-tls" not found Jul 27 09:25:33.835513 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:25:33.835472 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-848w4_2c043d82-d300-41ba-b138-60a03d65ea11/konnectivity-agent/0.log" limit=1024 Jul 27 09:25:33.993581 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:25:33.993549 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-848w4_2c043d82-d300-41ba-b138-60a03d65ea11/konnectivity-agent/0.log" limit=1024 Jul 27 09:25:34.048112 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:25:34.048068 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-848w4_2c043d82-d300-41ba-b138-60a03d65ea11/konnectivity-agent/0.log" limit=1024 Jul 27 09:25:34.760861 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:25:34.760832 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-mqhsn" Jul 27 09:25:34.932324 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:25:34.932287 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-848w4_2c043d82-d300-41ba-b138-60a03d65ea11/konnectivity-agent/0.log" limit=1024 Jul 27 09:25:35.006760 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:25:35.006711 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-848w4_2c043d82-d300-41ba-b138-60a03d65ea11/konnectivity-agent/0.log" limit=1024 Jul 27 09:25:35.078597 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:25:35.078520 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-848w4_2c043d82-d300-41ba-b138-60a03d65ea11/konnectivity-agent/0.log" limit=1024 Jul 27 09:25:38.176285 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:25:38.176246 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0d6ba35e-ec8a-4ec9-8520-0d32a5586a1d-metrics-certs\") pod \"network-metrics-daemon-trcl2\" (UID: \"0d6ba35e-ec8a-4ec9-8520-0d32a5586a1d\") " pod="openshift-multus/network-metrics-daemon-trcl2" Jul 27 09:25:38.178954 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:25:38.178934 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 27 09:25:38.186851 ip-10-0-128-103 kubenswrapper[2565]: E0727 09:25:38.186833 2565 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jul 27 09:25:38.186905 ip-10-0-128-103 kubenswrapper[2565]: E0727 09:25:38.186888 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0d6ba35e-ec8a-4ec9-8520-0d32a5586a1d-metrics-certs podName:0d6ba35e-ec8a-4ec9-8520-0d32a5586a1d nodeName:}" failed. No retries permitted until 2026-07-27 09:26:42.186872487 +0000 UTC m=+130.150658422 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0d6ba35e-ec8a-4ec9-8520-0d32a5586a1d-metrics-certs") pod "network-metrics-daemon-trcl2" (UID: "0d6ba35e-ec8a-4ec9-8520-0d32a5586a1d") : secret "metrics-daemon-secret" not found Jul 27 09:25:38.276725 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:25:38.276691 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/204c97b6-2efe-4db7-bc9f-a5653fed77c7-original-pull-secret\") pod \"global-pull-secret-syncer-rqz85\" (UID: \"204c97b6-2efe-4db7-bc9f-a5653fed77c7\") " pod="kube-system/global-pull-secret-syncer-rqz85" Jul 27 09:25:38.276852 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:25:38.276751 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2ckrw\" (UniqueName: \"kubernetes.io/projected/91816deb-5261-44b7-857a-04cf036feffa-kube-api-access-2ckrw\") pod \"network-check-target-5zqgm\" (UID: \"91816deb-5261-44b7-857a-04cf036feffa\") " pod="openshift-network-diagnostics/network-check-target-5zqgm" Jul 27 09:25:38.279502 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:25:38.279479 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 27 09:25:38.279554 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:25:38.279478 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 27 09:25:38.289243 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:25:38.289211 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 27 09:25:38.291161 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:25:38.291142 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/204c97b6-2efe-4db7-bc9f-a5653fed77c7-original-pull-secret\") pod \"global-pull-secret-syncer-rqz85\" (UID: \"204c97b6-2efe-4db7-bc9f-a5653fed77c7\") " pod="kube-system/global-pull-secret-syncer-rqz85" Jul 27 09:25:38.300414 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:25:38.300388 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2ckrw\" (UniqueName: \"kubernetes.io/projected/91816deb-5261-44b7-857a-04cf036feffa-kube-api-access-2ckrw\") pod \"network-check-target-5zqgm\" (UID: \"91816deb-5261-44b7-857a-04cf036feffa\") " pod="openshift-network-diagnostics/network-check-target-5zqgm" Jul 27 09:25:38.568212 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:25:38.568135 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-rqz85" Jul 27 09:25:38.576860 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:25:38.576839 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-bnzn6\"" Jul 27 09:25:38.585008 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:25:38.584990 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5zqgm" Jul 27 09:25:38.734163 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:25:38.734132 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-rqz85"] Jul 27 09:25:38.755757 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:25:38.755729 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-5zqgm"] Jul 27 09:25:38.759537 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:25:38.759509 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod91816deb_5261_44b7_857a_04cf036feffa.slice/crio-00d1d2ee2485945c6b58c91eb24d72ced30a609fbc9870cf833a9b516ea1440b WatchSource:0}: Error finding container 00d1d2ee2485945c6b58c91eb24d72ced30a609fbc9870cf833a9b516ea1440b: Status 404 returned error can't find the container with id 00d1d2ee2485945c6b58c91eb24d72ced30a609fbc9870cf833a9b516ea1440b Jul 27 09:25:38.844416 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:25:38.844333 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-5zqgm" event={"ID":"91816deb-5261-44b7-857a-04cf036feffa","Type":"ContainerStarted","Data":"00d1d2ee2485945c6b58c91eb24d72ced30a609fbc9870cf833a9b516ea1440b"} Jul 27 09:25:38.845414 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:25:38.845390 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-rqz85" event={"ID":"204c97b6-2efe-4db7-bc9f-a5653fed77c7","Type":"ContainerStarted","Data":"6e5ef35874b5812a427af204f9c4a615561b5f89a752e3ed75f9fb1861684e83"} Jul 27 09:25:39.283198 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:25:39.283164 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ef01a458-240c-4477-9ef9-d5b21026df4f-metrics-tls\") pod \"dns-default-7lzb6\" (UID: \"ef01a458-240c-4477-9ef9-d5b21026df4f\") " pod="openshift-dns/dns-default-7lzb6" Jul 27 09:25:39.283660 ip-10-0-128-103 kubenswrapper[2565]: E0727 09:25:39.283298 2565 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 27 09:25:39.283660 ip-10-0-128-103 kubenswrapper[2565]: E0727 09:25:39.283375 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ef01a458-240c-4477-9ef9-d5b21026df4f-metrics-tls podName:ef01a458-240c-4477-9ef9-d5b21026df4f nodeName:}" failed. No retries permitted until 2026-07-27 09:26:11.283352796 +0000 UTC m=+99.247138754 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/ef01a458-240c-4477-9ef9-d5b21026df4f-metrics-tls") pod "dns-default-7lzb6" (UID: "ef01a458-240c-4477-9ef9-d5b21026df4f") : secret "dns-default-metrics-tls" not found Jul 27 09:25:42.060318 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:25:42.060273 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-848w4_2c043d82-d300-41ba-b138-60a03d65ea11/konnectivity-agent/0.log" limit=1024 Jul 27 09:25:42.165051 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:25:42.165003 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-848w4_2c043d82-d300-41ba-b138-60a03d65ea11/konnectivity-agent/0.log" limit=1024 Jul 27 09:25:42.234067 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:25:42.234028 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-848w4_2c043d82-d300-41ba-b138-60a03d65ea11/konnectivity-agent/0.log" limit=1024 Jul 27 09:25:43.860513 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:25:43.860421 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-5zqgm" event={"ID":"91816deb-5261-44b7-857a-04cf036feffa","Type":"ContainerStarted","Data":"5d81105454318a4088a7b853d4bdcd46e73c58c2013b106a60bb68d9aa8a6812"} Jul 27 09:25:43.860932 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:25:43.860535 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-5zqgm" Jul 27 09:25:43.862019 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:25:43.861996 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-rqz85" event={"ID":"204c97b6-2efe-4db7-bc9f-a5653fed77c7","Type":"ContainerStarted","Data":"931fabec643cd8ffef65030ff8ec1fdafa60611183a766e4f99d81fc7b5c26c8"} Jul 27 09:25:43.876951 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:25:43.876909 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-5zqgm" podStartSLOduration=66.230145225 podStartE2EDuration="1m10.87689547s" totalImagesPullingTime="4.646750245s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 09:24:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 09:25:43.876284685 +0000 UTC m=+71.840070654" watchObservedRunningTime="2026-07-27 09:25:43.87689547 +0000 UTC m=+71.840681428" Jul 27 09:25:43.895863 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:25:43.895809 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-rqz85" podStartSLOduration=66.226414742 podStartE2EDuration="1m10.895791733s" totalImagesPullingTime="4.669376991s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-27 09:24:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 09:25:43.895598305 +0000 UTC m=+71.859384262" watchObservedRunningTime="2026-07-27 09:25:43.895791733 +0000 UTC m=+71.859577687" Jul 27 09:26:11.314783 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:26:11.314725 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ef01a458-240c-4477-9ef9-d5b21026df4f-metrics-tls\") pod \"dns-default-7lzb6\" (UID: \"ef01a458-240c-4477-9ef9-d5b21026df4f\") " pod="openshift-dns/dns-default-7lzb6" Jul 27 09:26:11.315272 ip-10-0-128-103 kubenswrapper[2565]: E0727 09:26:11.314887 2565 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 27 09:26:11.315272 ip-10-0-128-103 kubenswrapper[2565]: E0727 09:26:11.314965 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ef01a458-240c-4477-9ef9-d5b21026df4f-metrics-tls podName:ef01a458-240c-4477-9ef9-d5b21026df4f nodeName:}" failed. No retries permitted until 2026-07-27 09:27:15.314946623 +0000 UTC m=+163.278732579 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/ef01a458-240c-4477-9ef9-d5b21026df4f-metrics-tls") pod "dns-default-7lzb6" (UID: "ef01a458-240c-4477-9ef9-d5b21026df4f") : secret "dns-default-metrics-tls" not found Jul 27 09:26:14.867205 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:26:14.867173 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-5zqgm" Jul 27 09:26:33.717388 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:26:33.717350 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-848w4_2c043d82-d300-41ba-b138-60a03d65ea11/konnectivity-agent/0.log" limit=1024 Jul 27 09:26:34.932224 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:26:34.932183 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-848w4_2c043d82-d300-41ba-b138-60a03d65ea11/konnectivity-agent/0.log" limit=1024 Jul 27 09:26:42.221348 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:26:42.221302 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0d6ba35e-ec8a-4ec9-8520-0d32a5586a1d-metrics-certs\") pod \"network-metrics-daemon-trcl2\" (UID: \"0d6ba35e-ec8a-4ec9-8520-0d32a5586a1d\") " pod="openshift-multus/network-metrics-daemon-trcl2" Jul 27 09:26:42.221889 ip-10-0-128-103 kubenswrapper[2565]: E0727 09:26:42.221482 2565 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jul 27 09:26:42.221889 ip-10-0-128-103 kubenswrapper[2565]: E0727 09:26:42.221594 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0d6ba35e-ec8a-4ec9-8520-0d32a5586a1d-metrics-certs podName:0d6ba35e-ec8a-4ec9-8520-0d32a5586a1d nodeName:}" failed. No retries permitted until 2026-07-27 09:28:44.221568964 +0000 UTC m=+252.185354917 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0d6ba35e-ec8a-4ec9-8520-0d32a5586a1d-metrics-certs") pod "network-metrics-daemon-trcl2" (UID: "0d6ba35e-ec8a-4ec9-8520-0d32a5586a1d") : secret "metrics-daemon-secret" not found Jul 27 09:26:56.789861 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:26:56.789828 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d6c75f84f-pjm9w"] Jul 27 09:26:56.791384 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:26:56.791366 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-pjm9w" Jul 27 09:26:56.792922 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:26:56.792897 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-6748489656-hsqzj"] Jul 27 09:26:56.794104 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:26:56.794084 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\"" Jul 27 09:26:56.794288 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:26:56.794275 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\"" Jul 27 09:26:56.794543 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:26:56.794527 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-hsqzj" Jul 27 09:26:56.795047 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:26:56.795028 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\"" Jul 27 09:26:56.795112 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:26:56.795045 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-rv2ln\"" Jul 27 09:26:56.795161 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:26:56.795055 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\"" Jul 27 09:26:56.796560 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:26:56.796542 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\"" Jul 27 09:26:56.796725 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:26:56.796710 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\"" Jul 27 09:26:56.796945 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:26:56.796864 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-9nkvz\"" Jul 27 09:26:56.796945 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:26:56.796898 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\"" Jul 27 09:26:56.803448 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:26:56.803429 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d6c75f84f-pjm9w"] Jul 27 09:26:56.806634 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:26:56.806613 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-6748489656-hsqzj"] Jul 27 09:26:56.892366 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:26:56.892332 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-548b489cb8-lgr4b"] Jul 27 09:26:56.894196 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:26:56.894178 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-lgr4b" Jul 27 09:26:56.897147 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:26:56.897128 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 27 09:26:56.897434 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:26:56.897418 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\"" Jul 27 09:26:56.899660 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:26:56.899640 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 27 09:26:56.899660 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:26:56.899653 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-d8zqd\"" Jul 27 09:26:56.899790 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:26:56.899777 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\"" Jul 27 09:26:56.910958 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:26:56.910927 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-548b489cb8-lgr4b"] Jul 27 09:26:56.924952 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:26:56.924925 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/fd3e4e31-3c65-472d-bd64-e3bd969db33e-config\") pod \"service-ca-operator-5d6c75f84f-pjm9w\" (UID: \"fd3e4e31-3c65-472d-bd64-e3bd969db33e\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-pjm9w" Jul 27 09:26:56.925078 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:26:56.924979 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/f3c02965-8034-4cc6-9cd6-eeff280ebb58-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-hsqzj\" (UID: \"f3c02965-8034-4cc6-9cd6-eeff280ebb58\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-hsqzj" Jul 27 09:26:56.925078 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:26:56.925038 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h2bs2\" (UniqueName: \"kubernetes.io/projected/fd3e4e31-3c65-472d-bd64-e3bd969db33e-kube-api-access-h2bs2\") pod \"service-ca-operator-5d6c75f84f-pjm9w\" (UID: \"fd3e4e31-3c65-472d-bd64-e3bd969db33e\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-pjm9w" Jul 27 09:26:56.925148 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:26:56.925099 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/fd3e4e31-3c65-472d-bd64-e3bd969db33e-serving-cert\") pod \"service-ca-operator-5d6c75f84f-pjm9w\" (UID: \"fd3e4e31-3c65-472d-bd64-e3bd969db33e\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-pjm9w" Jul 27 09:26:56.925148 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:26:56.925116 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hb4nn\" (UniqueName: \"kubernetes.io/projected/f3c02965-8034-4cc6-9cd6-eeff280ebb58-kube-api-access-hb4nn\") pod \"cluster-samples-operator-6748489656-hsqzj\" (UID: \"f3c02965-8034-4cc6-9cd6-eeff280ebb58\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-hsqzj" Jul 27 09:26:56.990716 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:26:56.990683 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-584dcb6b54-w7grd"] Jul 27 09:26:56.992399 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:26:56.992380 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-584dcb6b54-w7grd" Jul 27 09:26:56.994755 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:26:56.994730 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\"" Jul 27 09:26:56.994755 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:26:56.994746 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-crgwg\"" Jul 27 09:26:56.994992 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:26:56.994746 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\"" Jul 27 09:26:56.994992 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:26:56.994817 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\"" Jul 27 09:26:56.994992 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:26:56.994990 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\"" Jul 27 09:26:56.995163 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:26:56.994996 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-b5jrj"] Jul 27 09:26:56.996601 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:26:56.996583 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-b5jrj" Jul 27 09:26:56.999646 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:26:56.999628 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\"" Jul 27 09:26:56.999875 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:26:56.999806 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\"" Jul 27 09:26:57.000359 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:26:57.000341 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-2xtxv\"" Jul 27 09:26:57.000690 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:26:57.000677 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\"" Jul 27 09:26:57.001119 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:26:57.001101 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\"" Jul 27 09:26:57.003222 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:26:57.003201 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\"" Jul 27 09:26:57.009742 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:26:57.009724 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-584dcb6b54-w7grd"] Jul 27 09:26:57.010552 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:26:57.010532 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-b5jrj"] Jul 27 09:26:57.025702 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:26:57.025677 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/fb21c72b-5cea-49fa-9b28-d1362f405ab0-telemetry-config\") pod \"cluster-monitoring-operator-548b489cb8-lgr4b\" (UID: \"fb21c72b-5cea-49fa-9b28-d1362f405ab0\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-lgr4b" Jul 27 09:26:57.025847 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:26:57.025714 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/f3c02965-8034-4cc6-9cd6-eeff280ebb58-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-hsqzj\" (UID: \"f3c02965-8034-4cc6-9cd6-eeff280ebb58\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-hsqzj" Jul 27 09:26:57.025847 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:26:57.025737 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h2bs2\" (UniqueName: \"kubernetes.io/projected/fd3e4e31-3c65-472d-bd64-e3bd969db33e-kube-api-access-h2bs2\") pod \"service-ca-operator-5d6c75f84f-pjm9w\" (UID: \"fd3e4e31-3c65-472d-bd64-e3bd969db33e\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-pjm9w" Jul 27 09:26:57.025847 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:26:57.025781 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/fb21c72b-5cea-49fa-9b28-d1362f405ab0-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-lgr4b\" (UID: \"fb21c72b-5cea-49fa-9b28-d1362f405ab0\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-lgr4b" Jul 27 09:26:57.025847 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:26:57.025827 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/fd3e4e31-3c65-472d-bd64-e3bd969db33e-serving-cert\") pod \"service-ca-operator-5d6c75f84f-pjm9w\" (UID: \"fd3e4e31-3c65-472d-bd64-e3bd969db33e\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-pjm9w" Jul 27 09:26:57.026049 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:26:57.025853 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hb4nn\" (UniqueName: \"kubernetes.io/projected/f3c02965-8034-4cc6-9cd6-eeff280ebb58-kube-api-access-hb4nn\") pod \"cluster-samples-operator-6748489656-hsqzj\" (UID: \"f3c02965-8034-4cc6-9cd6-eeff280ebb58\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-hsqzj" Jul 27 09:26:57.026049 ip-10-0-128-103 kubenswrapper[2565]: E0727 09:26:57.025893 2565 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 27 09:26:57.026049 ip-10-0-128-103 kubenswrapper[2565]: E0727 09:26:57.025956 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f3c02965-8034-4cc6-9cd6-eeff280ebb58-samples-operator-tls podName:f3c02965-8034-4cc6-9cd6-eeff280ebb58 nodeName:}" failed. No retries permitted until 2026-07-27 09:26:57.525936451 +0000 UTC m=+145.489722401 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/f3c02965-8034-4cc6-9cd6-eeff280ebb58-samples-operator-tls") pod "cluster-samples-operator-6748489656-hsqzj" (UID: "f3c02965-8034-4cc6-9cd6-eeff280ebb58") : secret "samples-operator-tls" not found Jul 27 09:26:57.026049 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:26:57.026004 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gnbz4\" (UniqueName: \"kubernetes.io/projected/fb21c72b-5cea-49fa-9b28-d1362f405ab0-kube-api-access-gnbz4\") pod \"cluster-monitoring-operator-548b489cb8-lgr4b\" (UID: \"fb21c72b-5cea-49fa-9b28-d1362f405ab0\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-lgr4b" Jul 27 09:26:57.026211 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:26:57.026063 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/fd3e4e31-3c65-472d-bd64-e3bd969db33e-config\") pod \"service-ca-operator-5d6c75f84f-pjm9w\" (UID: \"fd3e4e31-3c65-472d-bd64-e3bd969db33e\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-pjm9w" Jul 27 09:26:57.026450 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:26:57.026434 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/fd3e4e31-3c65-472d-bd64-e3bd969db33e-config\") pod \"service-ca-operator-5d6c75f84f-pjm9w\" (UID: \"fd3e4e31-3c65-472d-bd64-e3bd969db33e\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-pjm9w" Jul 27 09:26:57.028004 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:26:57.027987 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/fd3e4e31-3c65-472d-bd64-e3bd969db33e-serving-cert\") pod \"service-ca-operator-5d6c75f84f-pjm9w\" (UID: \"fd3e4e31-3c65-472d-bd64-e3bd969db33e\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-pjm9w" Jul 27 09:26:57.039227 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:26:57.039198 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h2bs2\" (UniqueName: \"kubernetes.io/projected/fd3e4e31-3c65-472d-bd64-e3bd969db33e-kube-api-access-h2bs2\") pod \"service-ca-operator-5d6c75f84f-pjm9w\" (UID: \"fd3e4e31-3c65-472d-bd64-e3bd969db33e\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-pjm9w" Jul 27 09:26:57.039361 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:26:57.039343 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hb4nn\" (UniqueName: \"kubernetes.io/projected/f3c02965-8034-4cc6-9cd6-eeff280ebb58-kube-api-access-hb4nn\") pod \"cluster-samples-operator-6748489656-hsqzj\" (UID: \"f3c02965-8034-4cc6-9cd6-eeff280ebb58\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-hsqzj" Jul 27 09:26:57.101433 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:26:57.101344 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-pjm9w" Jul 27 09:26:57.127202 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:26:57.127168 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/fb21c72b-5cea-49fa-9b28-d1362f405ab0-telemetry-config\") pod \"cluster-monitoring-operator-548b489cb8-lgr4b\" (UID: \"fb21c72b-5cea-49fa-9b28-d1362f405ab0\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-lgr4b" Jul 27 09:26:57.127368 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:26:57.127214 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/0badf8bd-7f64-46ef-872c-39f59dd11d0d-config\") pod \"console-operator-584dcb6b54-w7grd\" (UID: \"0badf8bd-7f64-46ef-872c-39f59dd11d0d\") " pod="openshift-console-operator/console-operator-584dcb6b54-w7grd" Jul 27 09:26:57.127368 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:26:57.127251 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/0badf8bd-7f64-46ef-872c-39f59dd11d0d-serving-cert\") pod \"console-operator-584dcb6b54-w7grd\" (UID: \"0badf8bd-7f64-46ef-872c-39f59dd11d0d\") " pod="openshift-console-operator/console-operator-584dcb6b54-w7grd" Jul 27 09:26:57.127368 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:26:57.127290 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/85bf3791-57a8-4f6a-aba7-85b8a44e170c-serving-cert\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-b5jrj\" (UID: \"85bf3791-57a8-4f6a-aba7-85b8a44e170c\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-b5jrj" Jul 27 09:26:57.127501 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:26:57.127373 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-br4rc\" (UniqueName: \"kubernetes.io/projected/0badf8bd-7f64-46ef-872c-39f59dd11d0d-kube-api-access-br4rc\") pod \"console-operator-584dcb6b54-w7grd\" (UID: \"0badf8bd-7f64-46ef-872c-39f59dd11d0d\") " pod="openshift-console-operator/console-operator-584dcb6b54-w7grd" Jul 27 09:26:57.127501 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:26:57.127449 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/fb21c72b-5cea-49fa-9b28-d1362f405ab0-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-lgr4b\" (UID: \"fb21c72b-5cea-49fa-9b28-d1362f405ab0\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-lgr4b" Jul 27 09:26:57.127589 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:26:57.127546 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b8k74\" (UniqueName: \"kubernetes.io/projected/85bf3791-57a8-4f6a-aba7-85b8a44e170c-kube-api-access-b8k74\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-b5jrj\" (UID: \"85bf3791-57a8-4f6a-aba7-85b8a44e170c\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-b5jrj" Jul 27 09:26:57.127637 ip-10-0-128-103 kubenswrapper[2565]: E0727 09:26:57.127582 2565 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 27 09:26:57.127637 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:26:57.127590 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/85bf3791-57a8-4f6a-aba7-85b8a44e170c-config\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-b5jrj\" (UID: \"85bf3791-57a8-4f6a-aba7-85b8a44e170c\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-b5jrj" Jul 27 09:26:57.127711 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:26:57.127634 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/0badf8bd-7f64-46ef-872c-39f59dd11d0d-trusted-ca\") pod \"console-operator-584dcb6b54-w7grd\" (UID: \"0badf8bd-7f64-46ef-872c-39f59dd11d0d\") " pod="openshift-console-operator/console-operator-584dcb6b54-w7grd" Jul 27 09:26:57.127711 ip-10-0-128-103 kubenswrapper[2565]: E0727 09:26:57.127668 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fb21c72b-5cea-49fa-9b28-d1362f405ab0-cluster-monitoring-operator-tls podName:fb21c72b-5cea-49fa-9b28-d1362f405ab0 nodeName:}" failed. No retries permitted until 2026-07-27 09:26:57.627646294 +0000 UTC m=+145.591432244 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/fb21c72b-5cea-49fa-9b28-d1362f405ab0-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-lgr4b" (UID: "fb21c72b-5cea-49fa-9b28-d1362f405ab0") : secret "cluster-monitoring-operator-tls" not found Jul 27 09:26:57.127846 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:26:57.127749 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gnbz4\" (UniqueName: \"kubernetes.io/projected/fb21c72b-5cea-49fa-9b28-d1362f405ab0-kube-api-access-gnbz4\") pod \"cluster-monitoring-operator-548b489cb8-lgr4b\" (UID: \"fb21c72b-5cea-49fa-9b28-d1362f405ab0\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-lgr4b" Jul 27 09:26:57.127900 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:26:57.127868 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/fb21c72b-5cea-49fa-9b28-d1362f405ab0-telemetry-config\") pod \"cluster-monitoring-operator-548b489cb8-lgr4b\" (UID: \"fb21c72b-5cea-49fa-9b28-d1362f405ab0\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-lgr4b" Jul 27 09:26:57.138628 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:26:57.138595 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gnbz4\" (UniqueName: \"kubernetes.io/projected/fb21c72b-5cea-49fa-9b28-d1362f405ab0-kube-api-access-gnbz4\") pod \"cluster-monitoring-operator-548b489cb8-lgr4b\" (UID: \"fb21c72b-5cea-49fa-9b28-d1362f405ab0\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-lgr4b" Jul 27 09:26:57.162731 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:26:57.162696 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-848w4_2c043d82-d300-41ba-b138-60a03d65ea11/konnectivity-agent/0.log" limit=1024 Jul 27 09:26:57.220081 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:26:57.220048 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d6c75f84f-pjm9w"] Jul 27 09:26:57.223017 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:26:57.222984 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfd3e4e31_3c65_472d_bd64_e3bd969db33e.slice/crio-2cb079def2595eb0f02da4d192927d9b7767a4c07ab921350b4fa5e5a8b6643e WatchSource:0}: Error finding container 2cb079def2595eb0f02da4d192927d9b7767a4c07ab921350b4fa5e5a8b6643e: Status 404 returned error can't find the container with id 2cb079def2595eb0f02da4d192927d9b7767a4c07ab921350b4fa5e5a8b6643e Jul 27 09:26:57.228848 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:26:57.228826 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/85bf3791-57a8-4f6a-aba7-85b8a44e170c-serving-cert\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-b5jrj\" (UID: \"85bf3791-57a8-4f6a-aba7-85b8a44e170c\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-b5jrj" Jul 27 09:26:57.228957 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:26:57.228857 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-br4rc\" (UniqueName: \"kubernetes.io/projected/0badf8bd-7f64-46ef-872c-39f59dd11d0d-kube-api-access-br4rc\") pod \"console-operator-584dcb6b54-w7grd\" (UID: \"0badf8bd-7f64-46ef-872c-39f59dd11d0d\") " pod="openshift-console-operator/console-operator-584dcb6b54-w7grd" Jul 27 09:26:57.228957 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:26:57.228892 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b8k74\" (UniqueName: \"kubernetes.io/projected/85bf3791-57a8-4f6a-aba7-85b8a44e170c-kube-api-access-b8k74\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-b5jrj\" (UID: \"85bf3791-57a8-4f6a-aba7-85b8a44e170c\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-b5jrj" Jul 27 09:26:57.228957 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:26:57.228935 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/85bf3791-57a8-4f6a-aba7-85b8a44e170c-config\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-b5jrj\" (UID: \"85bf3791-57a8-4f6a-aba7-85b8a44e170c\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-b5jrj" Jul 27 09:26:57.229105 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:26:57.228960 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/0badf8bd-7f64-46ef-872c-39f59dd11d0d-trusted-ca\") pod \"console-operator-584dcb6b54-w7grd\" (UID: \"0badf8bd-7f64-46ef-872c-39f59dd11d0d\") " pod="openshift-console-operator/console-operator-584dcb6b54-w7grd" Jul 27 09:26:57.229105 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:26:57.229022 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/0badf8bd-7f64-46ef-872c-39f59dd11d0d-config\") pod \"console-operator-584dcb6b54-w7grd\" (UID: \"0badf8bd-7f64-46ef-872c-39f59dd11d0d\") " pod="openshift-console-operator/console-operator-584dcb6b54-w7grd" Jul 27 09:26:57.229105 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:26:57.229051 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/0badf8bd-7f64-46ef-872c-39f59dd11d0d-serving-cert\") pod \"console-operator-584dcb6b54-w7grd\" (UID: \"0badf8bd-7f64-46ef-872c-39f59dd11d0d\") " pod="openshift-console-operator/console-operator-584dcb6b54-w7grd" Jul 27 09:26:57.229550 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:26:57.229525 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/85bf3791-57a8-4f6a-aba7-85b8a44e170c-config\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-b5jrj\" (UID: \"85bf3791-57a8-4f6a-aba7-85b8a44e170c\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-b5jrj" Jul 27 09:26:57.229895 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:26:57.229866 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/0badf8bd-7f64-46ef-872c-39f59dd11d0d-config\") pod \"console-operator-584dcb6b54-w7grd\" (UID: \"0badf8bd-7f64-46ef-872c-39f59dd11d0d\") " pod="openshift-console-operator/console-operator-584dcb6b54-w7grd" Jul 27 09:26:57.230059 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:26:57.230041 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/0badf8bd-7f64-46ef-872c-39f59dd11d0d-trusted-ca\") pod \"console-operator-584dcb6b54-w7grd\" (UID: \"0badf8bd-7f64-46ef-872c-39f59dd11d0d\") " pod="openshift-console-operator/console-operator-584dcb6b54-w7grd" Jul 27 09:26:57.231199 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:26:57.231175 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/85bf3791-57a8-4f6a-aba7-85b8a44e170c-serving-cert\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-b5jrj\" (UID: \"85bf3791-57a8-4f6a-aba7-85b8a44e170c\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-b5jrj" Jul 27 09:26:57.231388 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:26:57.231368 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/0badf8bd-7f64-46ef-872c-39f59dd11d0d-serving-cert\") pod \"console-operator-584dcb6b54-w7grd\" (UID: \"0badf8bd-7f64-46ef-872c-39f59dd11d0d\") " pod="openshift-console-operator/console-operator-584dcb6b54-w7grd" Jul 27 09:26:57.231455 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:26:57.231402 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-848w4_2c043d82-d300-41ba-b138-60a03d65ea11/konnectivity-agent/0.log" limit=1024 Jul 27 09:26:57.236364 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:26:57.236344 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b8k74\" (UniqueName: \"kubernetes.io/projected/85bf3791-57a8-4f6a-aba7-85b8a44e170c-kube-api-access-b8k74\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-b5jrj\" (UID: \"85bf3791-57a8-4f6a-aba7-85b8a44e170c\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-b5jrj" Jul 27 09:26:57.236808 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:26:57.236793 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-br4rc\" (UniqueName: \"kubernetes.io/projected/0badf8bd-7f64-46ef-872c-39f59dd11d0d-kube-api-access-br4rc\") pod \"console-operator-584dcb6b54-w7grd\" (UID: \"0badf8bd-7f64-46ef-872c-39f59dd11d0d\") " pod="openshift-console-operator/console-operator-584dcb6b54-w7grd" Jul 27 09:26:57.304106 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:26:57.304066 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-584dcb6b54-w7grd" Jul 27 09:26:57.312806 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:26:57.312780 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-b5jrj" Jul 27 09:26:57.444617 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:26:57.444580 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-584dcb6b54-w7grd"] Jul 27 09:26:57.447627 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:26:57.447594 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0badf8bd_7f64_46ef_872c_39f59dd11d0d.slice/crio-26717e32788bb099c7da94497a56047ab1b9ee12155d87aa094606ca64fd9a80 WatchSource:0}: Error finding container 26717e32788bb099c7da94497a56047ab1b9ee12155d87aa094606ca64fd9a80: Status 404 returned error can't find the container with id 26717e32788bb099c7da94497a56047ab1b9ee12155d87aa094606ca64fd9a80 Jul 27 09:26:57.453322 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:26:57.453298 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-b5jrj"] Jul 27 09:26:57.456539 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:26:57.456515 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod85bf3791_57a8_4f6a_aba7_85b8a44e170c.slice/crio-866bdb744d1a9c2bf60ee96c29b4b81fc8fdd8dcdd797700ce96c0a26abef39f WatchSource:0}: Error finding container 866bdb744d1a9c2bf60ee96c29b4b81fc8fdd8dcdd797700ce96c0a26abef39f: Status 404 returned error can't find the container with id 866bdb744d1a9c2bf60ee96c29b4b81fc8fdd8dcdd797700ce96c0a26abef39f Jul 27 09:26:57.530893 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:26:57.530859 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/f3c02965-8034-4cc6-9cd6-eeff280ebb58-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-hsqzj\" (UID: \"f3c02965-8034-4cc6-9cd6-eeff280ebb58\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-hsqzj" Jul 27 09:26:57.531027 ip-10-0-128-103 kubenswrapper[2565]: E0727 09:26:57.530960 2565 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 27 09:26:57.531027 ip-10-0-128-103 kubenswrapper[2565]: E0727 09:26:57.531015 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f3c02965-8034-4cc6-9cd6-eeff280ebb58-samples-operator-tls podName:f3c02965-8034-4cc6-9cd6-eeff280ebb58 nodeName:}" failed. No retries permitted until 2026-07-27 09:26:58.531002162 +0000 UTC m=+146.494788098 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/f3c02965-8034-4cc6-9cd6-eeff280ebb58-samples-operator-tls") pod "cluster-samples-operator-6748489656-hsqzj" (UID: "f3c02965-8034-4cc6-9cd6-eeff280ebb58") : secret "samples-operator-tls" not found Jul 27 09:26:57.632102 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:26:57.632067 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/fb21c72b-5cea-49fa-9b28-d1362f405ab0-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-lgr4b\" (UID: \"fb21c72b-5cea-49fa-9b28-d1362f405ab0\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-lgr4b" Jul 27 09:26:57.632292 ip-10-0-128-103 kubenswrapper[2565]: E0727 09:26:57.632263 2565 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 27 09:26:57.632358 ip-10-0-128-103 kubenswrapper[2565]: E0727 09:26:57.632344 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fb21c72b-5cea-49fa-9b28-d1362f405ab0-cluster-monitoring-operator-tls podName:fb21c72b-5cea-49fa-9b28-d1362f405ab0 nodeName:}" failed. No retries permitted until 2026-07-27 09:26:58.632326281 +0000 UTC m=+146.596112217 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/fb21c72b-5cea-49fa-9b28-d1362f405ab0-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-lgr4b" (UID: "fb21c72b-5cea-49fa-9b28-d1362f405ab0") : secret "cluster-monitoring-operator-tls" not found Jul 27 09:26:58.040724 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:26:58.040636 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-584dcb6b54-w7grd" event={"ID":"0badf8bd-7f64-46ef-872c-39f59dd11d0d","Type":"ContainerStarted","Data":"26717e32788bb099c7da94497a56047ab1b9ee12155d87aa094606ca64fd9a80"} Jul 27 09:26:58.042102 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:26:58.042075 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-b5jrj" event={"ID":"85bf3791-57a8-4f6a-aba7-85b8a44e170c","Type":"ContainerStarted","Data":"866bdb744d1a9c2bf60ee96c29b4b81fc8fdd8dcdd797700ce96c0a26abef39f"} Jul 27 09:26:58.043978 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:26:58.043920 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-pjm9w" event={"ID":"fd3e4e31-3c65-472d-bd64-e3bd969db33e","Type":"ContainerStarted","Data":"2cb079def2595eb0f02da4d192927d9b7767a4c07ab921350b4fa5e5a8b6643e"} Jul 27 09:26:58.541339 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:26:58.541301 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/f3c02965-8034-4cc6-9cd6-eeff280ebb58-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-hsqzj\" (UID: \"f3c02965-8034-4cc6-9cd6-eeff280ebb58\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-hsqzj" Jul 27 09:26:58.541525 ip-10-0-128-103 kubenswrapper[2565]: E0727 09:26:58.541438 2565 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 27 09:26:58.541525 ip-10-0-128-103 kubenswrapper[2565]: E0727 09:26:58.541495 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f3c02965-8034-4cc6-9cd6-eeff280ebb58-samples-operator-tls podName:f3c02965-8034-4cc6-9cd6-eeff280ebb58 nodeName:}" failed. No retries permitted until 2026-07-27 09:27:00.541481639 +0000 UTC m=+148.505267573 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/f3c02965-8034-4cc6-9cd6-eeff280ebb58-samples-operator-tls") pod "cluster-samples-operator-6748489656-hsqzj" (UID: "f3c02965-8034-4cc6-9cd6-eeff280ebb58") : secret "samples-operator-tls" not found Jul 27 09:26:58.641964 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:26:58.641928 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/fb21c72b-5cea-49fa-9b28-d1362f405ab0-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-lgr4b\" (UID: \"fb21c72b-5cea-49fa-9b28-d1362f405ab0\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-lgr4b" Jul 27 09:26:58.642157 ip-10-0-128-103 kubenswrapper[2565]: E0727 09:26:58.642119 2565 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 27 09:26:58.642231 ip-10-0-128-103 kubenswrapper[2565]: E0727 09:26:58.642204 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fb21c72b-5cea-49fa-9b28-d1362f405ab0-cluster-monitoring-operator-tls podName:fb21c72b-5cea-49fa-9b28-d1362f405ab0 nodeName:}" failed. No retries permitted until 2026-07-27 09:27:00.642181987 +0000 UTC m=+148.605967946 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/fb21c72b-5cea-49fa-9b28-d1362f405ab0-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-lgr4b" (UID: "fb21c72b-5cea-49fa-9b28-d1362f405ab0") : secret "cluster-monitoring-operator-tls" not found Jul 27 09:26:59.234299 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:26:59.234257 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-848w4_2c043d82-d300-41ba-b138-60a03d65ea11/konnectivity-agent/0.log" limit=1024 Jul 27 09:26:59.334306 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:26:59.334236 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-848w4_2c043d82-d300-41ba-b138-60a03d65ea11/konnectivity-agent/0.log" limit=1024 Jul 27 09:27:00.052164 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:00.051826 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-584dcb6b54-w7grd" event={"ID":"0badf8bd-7f64-46ef-872c-39f59dd11d0d","Type":"ContainerStarted","Data":"d00d87f7ebeeb23df2a15af5af26cf55e5d1bee94ef83113c1e1e429e1cae440"} Jul 27 09:27:00.052164 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:00.052087 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-584dcb6b54-w7grd" Jul 27 09:27:00.053403 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:00.053370 2565 patch_prober.go:28] interesting pod/console-operator-584dcb6b54-w7grd container/console-operator namespace/openshift-console-operator: Readiness probe status=failure output="Get \"https://10.133.0.11:8443/readyz\": dial tcp 10.133.0.11:8443: connect: connection refused" start-of-body= Jul 27 09:27:00.053530 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:00.053425 2565 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-console-operator/console-operator-584dcb6b54-w7grd" podUID="0badf8bd-7f64-46ef-872c-39f59dd11d0d" containerName="console-operator" probeResult="failure" output="Get \"https://10.133.0.11:8443/readyz\": dial tcp 10.133.0.11:8443: connect: connection refused" Jul 27 09:27:00.054161 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:00.054101 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-b5jrj" event={"ID":"85bf3791-57a8-4f6a-aba7-85b8a44e170c","Type":"ContainerStarted","Data":"f1bd60cf2ef9c6ea383212107d2a8dda47942a509e44f370b68a866d3755e0e7"} Jul 27 09:27:00.056248 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:00.056225 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-pjm9w" event={"ID":"fd3e4e31-3c65-472d-bd64-e3bd969db33e","Type":"ContainerStarted","Data":"4dc76c2e2bb6a749bcb3cbcbd9b6df5bbf2230ab735fe7ab99917f180320dd2b"} Jul 27 09:27:00.070268 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:00.070161 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-584dcb6b54-w7grd" podStartSLOduration=1.579007014 podStartE2EDuration="4.070139349s" totalImagesPullingTime="2.491132335s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 09:26:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 09:27:00.069000242 +0000 UTC m=+148.032786225" watchObservedRunningTime="2026-07-27 09:27:00.070139349 +0000 UTC m=+148.033925308" Jul 27 09:27:00.088863 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:00.088793 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-pjm9w" podStartSLOduration=1.378314861 podStartE2EDuration="4.088753351s" totalImagesPullingTime="2.71043849s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 09:26:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 09:27:00.08796447 +0000 UTC m=+148.051750430" watchObservedRunningTime="2026-07-27 09:27:00.088753351 +0000 UTC m=+148.052539309" Jul 27 09:27:00.107437 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:00.107312 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-b5jrj" podStartSLOduration=1.625648446 podStartE2EDuration="4.107291726s" totalImagesPullingTime="2.48164328s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 09:26:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 09:27:00.106808434 +0000 UTC m=+148.070594411" watchObservedRunningTime="2026-07-27 09:27:00.107291726 +0000 UTC m=+148.071077684" Jul 27 09:27:00.558947 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:00.558910 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/f3c02965-8034-4cc6-9cd6-eeff280ebb58-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-hsqzj\" (UID: \"f3c02965-8034-4cc6-9cd6-eeff280ebb58\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-hsqzj" Jul 27 09:27:00.559331 ip-10-0-128-103 kubenswrapper[2565]: E0727 09:27:00.559041 2565 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 27 09:27:00.559331 ip-10-0-128-103 kubenswrapper[2565]: E0727 09:27:00.559104 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f3c02965-8034-4cc6-9cd6-eeff280ebb58-samples-operator-tls podName:f3c02965-8034-4cc6-9cd6-eeff280ebb58 nodeName:}" failed. No retries permitted until 2026-07-27 09:27:04.559086485 +0000 UTC m=+152.522872438 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/f3c02965-8034-4cc6-9cd6-eeff280ebb58-samples-operator-tls") pod "cluster-samples-operator-6748489656-hsqzj" (UID: "f3c02965-8034-4cc6-9cd6-eeff280ebb58") : secret "samples-operator-tls" not found Jul 27 09:27:00.646714 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:00.646650 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-xfvbm"] Jul 27 09:27:00.648760 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:00.648740 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-xfvbm" Jul 27 09:27:00.651048 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:00.651016 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 27 09:27:00.651048 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:00.651020 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 27 09:27:00.651352 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:00.651333 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 27 09:27:00.651985 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:00.651967 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-5ldlr\"" Jul 27 09:27:00.651985 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:00.651976 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 27 09:27:00.659636 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:00.659614 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/fb21c72b-5cea-49fa-9b28-d1362f405ab0-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-lgr4b\" (UID: \"fb21c72b-5cea-49fa-9b28-d1362f405ab0\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-lgr4b" Jul 27 09:27:00.659732 ip-10-0-128-103 kubenswrapper[2565]: E0727 09:27:00.659719 2565 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 27 09:27:00.659801 ip-10-0-128-103 kubenswrapper[2565]: E0727 09:27:00.659786 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fb21c72b-5cea-49fa-9b28-d1362f405ab0-cluster-monitoring-operator-tls podName:fb21c72b-5cea-49fa-9b28-d1362f405ab0 nodeName:}" failed. No retries permitted until 2026-07-27 09:27:04.659758801 +0000 UTC m=+152.623544736 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/fb21c72b-5cea-49fa-9b28-d1362f405ab0-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-lgr4b" (UID: "fb21c72b-5cea-49fa-9b28-d1362f405ab0") : secret "cluster-monitoring-operator-tls" not found Jul 27 09:27:00.662631 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:00.662609 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-xfvbm"] Jul 27 09:27:00.678644 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:00.678598 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-848w4_2c043d82-d300-41ba-b138-60a03d65ea11/konnectivity-agent/0.log" limit=1024 Jul 27 09:27:00.760735 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:00.760701 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a2f0dc08-339d-4aad-bae9-e13a8f594c70-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-xfvbm\" (UID: \"a2f0dc08-339d-4aad-bae9-e13a8f594c70\") " pod="openshift-insights/insights-runtime-extractor-xfvbm" Jul 27 09:27:00.760930 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:00.760758 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/a2f0dc08-339d-4aad-bae9-e13a8f594c70-crio-socket\") pod \"insights-runtime-extractor-xfvbm\" (UID: \"a2f0dc08-339d-4aad-bae9-e13a8f594c70\") " pod="openshift-insights/insights-runtime-extractor-xfvbm" Jul 27 09:27:00.760930 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:00.760861 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/a2f0dc08-339d-4aad-bae9-e13a8f594c70-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-xfvbm\" (UID: \"a2f0dc08-339d-4aad-bae9-e13a8f594c70\") " pod="openshift-insights/insights-runtime-extractor-xfvbm" Jul 27 09:27:00.761045 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:00.760955 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/a2f0dc08-339d-4aad-bae9-e13a8f594c70-data-volume\") pod \"insights-runtime-extractor-xfvbm\" (UID: \"a2f0dc08-339d-4aad-bae9-e13a8f594c70\") " pod="openshift-insights/insights-runtime-extractor-xfvbm" Jul 27 09:27:00.761045 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:00.761021 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cdhsc\" (UniqueName: \"kubernetes.io/projected/a2f0dc08-339d-4aad-bae9-e13a8f594c70-kube-api-access-cdhsc\") pod \"insights-runtime-extractor-xfvbm\" (UID: \"a2f0dc08-339d-4aad-bae9-e13a8f594c70\") " pod="openshift-insights/insights-runtime-extractor-xfvbm" Jul 27 09:27:00.862329 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:00.862214 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cdhsc\" (UniqueName: \"kubernetes.io/projected/a2f0dc08-339d-4aad-bae9-e13a8f594c70-kube-api-access-cdhsc\") pod \"insights-runtime-extractor-xfvbm\" (UID: \"a2f0dc08-339d-4aad-bae9-e13a8f594c70\") " pod="openshift-insights/insights-runtime-extractor-xfvbm" Jul 27 09:27:00.862329 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:00.862274 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a2f0dc08-339d-4aad-bae9-e13a8f594c70-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-xfvbm\" (UID: \"a2f0dc08-339d-4aad-bae9-e13a8f594c70\") " pod="openshift-insights/insights-runtime-extractor-xfvbm" Jul 27 09:27:00.862329 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:00.862325 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/a2f0dc08-339d-4aad-bae9-e13a8f594c70-crio-socket\") pod \"insights-runtime-extractor-xfvbm\" (UID: \"a2f0dc08-339d-4aad-bae9-e13a8f594c70\") " pod="openshift-insights/insights-runtime-extractor-xfvbm" Jul 27 09:27:00.862614 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:00.862362 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/a2f0dc08-339d-4aad-bae9-e13a8f594c70-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-xfvbm\" (UID: \"a2f0dc08-339d-4aad-bae9-e13a8f594c70\") " pod="openshift-insights/insights-runtime-extractor-xfvbm" Jul 27 09:27:00.862614 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:00.862428 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/a2f0dc08-339d-4aad-bae9-e13a8f594c70-data-volume\") pod \"insights-runtime-extractor-xfvbm\" (UID: \"a2f0dc08-339d-4aad-bae9-e13a8f594c70\") " pod="openshift-insights/insights-runtime-extractor-xfvbm" Jul 27 09:27:00.862614 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:00.862435 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/a2f0dc08-339d-4aad-bae9-e13a8f594c70-crio-socket\") pod \"insights-runtime-extractor-xfvbm\" (UID: \"a2f0dc08-339d-4aad-bae9-e13a8f594c70\") " pod="openshift-insights/insights-runtime-extractor-xfvbm" Jul 27 09:27:00.862614 ip-10-0-128-103 kubenswrapper[2565]: E0727 09:27:00.862448 2565 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 27 09:27:00.862614 ip-10-0-128-103 kubenswrapper[2565]: E0727 09:27:00.862502 2565 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 27 09:27:00.862614 ip-10-0-128-103 kubenswrapper[2565]: E0727 09:27:00.862516 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a2f0dc08-339d-4aad-bae9-e13a8f594c70-insights-runtime-extractor-tls podName:a2f0dc08-339d-4aad-bae9-e13a8f594c70 nodeName:}" failed. No retries permitted until 2026-07-27 09:27:01.362500791 +0000 UTC m=+149.326286726 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/a2f0dc08-339d-4aad-bae9-e13a8f594c70-insights-runtime-extractor-tls") pod "insights-runtime-extractor-xfvbm" (UID: "a2f0dc08-339d-4aad-bae9-e13a8f594c70") : secret "insights-runtime-extractor-tls" not found Jul 27 09:27:00.862614 ip-10-0-128-103 kubenswrapper[2565]: E0727 09:27:00.862563 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/a2f0dc08-339d-4aad-bae9-e13a8f594c70-kube-rbac-proxy-cm podName:a2f0dc08-339d-4aad-bae9-e13a8f594c70 nodeName:}" failed. No retries permitted until 2026-07-27 09:27:01.362547554 +0000 UTC m=+149.326333494 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/a2f0dc08-339d-4aad-bae9-e13a8f594c70-kube-rbac-proxy-cm") pod "insights-runtime-extractor-xfvbm" (UID: "a2f0dc08-339d-4aad-bae9-e13a8f594c70") : configmap "kube-rbac-proxy" not found Jul 27 09:27:00.862971 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:00.862761 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/a2f0dc08-339d-4aad-bae9-e13a8f594c70-data-volume\") pod \"insights-runtime-extractor-xfvbm\" (UID: \"a2f0dc08-339d-4aad-bae9-e13a8f594c70\") " pod="openshift-insights/insights-runtime-extractor-xfvbm" Jul 27 09:27:00.871537 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:00.871505 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cdhsc\" (UniqueName: \"kubernetes.io/projected/a2f0dc08-339d-4aad-bae9-e13a8f594c70-kube-api-access-cdhsc\") pod \"insights-runtime-extractor-xfvbm\" (UID: \"a2f0dc08-339d-4aad-bae9-e13a8f594c70\") " pod="openshift-insights/insights-runtime-extractor-xfvbm" Jul 27 09:27:01.061649 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:01.061617 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-w7grd_0badf8bd-7f64-46ef-872c-39f59dd11d0d/console-operator/0.log" Jul 27 09:27:01.061825 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:01.061665 2565 generic.go:356] "Generic (PLEG): container finished" podID="0badf8bd-7f64-46ef-872c-39f59dd11d0d" containerID="d00d87f7ebeeb23df2a15af5af26cf55e5d1bee94ef83113c1e1e429e1cae440" exitCode=255 Jul 27 09:27:01.061825 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:01.061751 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-584dcb6b54-w7grd" event={"ID":"0badf8bd-7f64-46ef-872c-39f59dd11d0d","Type":"ContainerDied","Data":"d00d87f7ebeeb23df2a15af5af26cf55e5d1bee94ef83113c1e1e429e1cae440"} Jul 27 09:27:01.062061 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:01.062045 2565 scope.go:117] "RemoveContainer" containerID="d00d87f7ebeeb23df2a15af5af26cf55e5d1bee94ef83113c1e1e429e1cae440" Jul 27 09:27:01.366447 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:01.366414 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a2f0dc08-339d-4aad-bae9-e13a8f594c70-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-xfvbm\" (UID: \"a2f0dc08-339d-4aad-bae9-e13a8f594c70\") " pod="openshift-insights/insights-runtime-extractor-xfvbm" Jul 27 09:27:01.366629 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:01.366477 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/a2f0dc08-339d-4aad-bae9-e13a8f594c70-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-xfvbm\" (UID: \"a2f0dc08-339d-4aad-bae9-e13a8f594c70\") " pod="openshift-insights/insights-runtime-extractor-xfvbm" Jul 27 09:27:01.366629 ip-10-0-128-103 kubenswrapper[2565]: E0727 09:27:01.366582 2565 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 27 09:27:01.366629 ip-10-0-128-103 kubenswrapper[2565]: E0727 09:27:01.366594 2565 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 27 09:27:01.366804 ip-10-0-128-103 kubenswrapper[2565]: E0727 09:27:01.366639 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/a2f0dc08-339d-4aad-bae9-e13a8f594c70-kube-rbac-proxy-cm podName:a2f0dc08-339d-4aad-bae9-e13a8f594c70 nodeName:}" failed. No retries permitted until 2026-07-27 09:27:02.366622865 +0000 UTC m=+150.330408822 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/a2f0dc08-339d-4aad-bae9-e13a8f594c70-kube-rbac-proxy-cm") pod "insights-runtime-extractor-xfvbm" (UID: "a2f0dc08-339d-4aad-bae9-e13a8f594c70") : configmap "kube-rbac-proxy" not found Jul 27 09:27:01.366804 ip-10-0-128-103 kubenswrapper[2565]: E0727 09:27:01.366674 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a2f0dc08-339d-4aad-bae9-e13a8f594c70-insights-runtime-extractor-tls podName:a2f0dc08-339d-4aad-bae9-e13a8f594c70 nodeName:}" failed. No retries permitted until 2026-07-27 09:27:02.366654983 +0000 UTC m=+150.330440922 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/a2f0dc08-339d-4aad-bae9-e13a8f594c70-insights-runtime-extractor-tls") pod "insights-runtime-extractor-xfvbm" (UID: "a2f0dc08-339d-4aad-bae9-e13a8f594c70") : secret "insights-runtime-extractor-tls" not found Jul 27 09:27:01.756371 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:01.756287 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-848w4_2c043d82-d300-41ba-b138-60a03d65ea11/konnectivity-agent/0.log" limit=1024 Jul 27 09:27:01.849136 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:01.849099 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-848w4_2c043d82-d300-41ba-b138-60a03d65ea11/konnectivity-agent/0.log" limit=1024 Jul 27 09:27:02.067228 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:02.067155 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-w7grd_0badf8bd-7f64-46ef-872c-39f59dd11d0d/console-operator/1.log" Jul 27 09:27:02.067590 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:02.067576 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-w7grd_0badf8bd-7f64-46ef-872c-39f59dd11d0d/console-operator/0.log" Jul 27 09:27:02.067662 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:02.067608 2565 generic.go:356] "Generic (PLEG): container finished" podID="0badf8bd-7f64-46ef-872c-39f59dd11d0d" containerID="533b092dffb6a6b30e4f5626b68feede4e0431c4b9eb6e71433b9bf7df2f853e" exitCode=255 Jul 27 09:27:02.067662 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:02.067640 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-584dcb6b54-w7grd" event={"ID":"0badf8bd-7f64-46ef-872c-39f59dd11d0d","Type":"ContainerDied","Data":"533b092dffb6a6b30e4f5626b68feede4e0431c4b9eb6e71433b9bf7df2f853e"} Jul 27 09:27:02.067788 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:02.067699 2565 scope.go:117] "RemoveContainer" containerID="d00d87f7ebeeb23df2a15af5af26cf55e5d1bee94ef83113c1e1e429e1cae440" Jul 27 09:27:02.067992 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:02.067976 2565 scope.go:117] "RemoveContainer" containerID="533b092dffb6a6b30e4f5626b68feede4e0431c4b9eb6e71433b9bf7df2f853e" Jul 27 09:27:02.068171 ip-10-0-128-103 kubenswrapper[2565]: E0727 09:27:02.068152 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-584dcb6b54-w7grd_openshift-console-operator(0badf8bd-7f64-46ef-872c-39f59dd11d0d)\"" pod="openshift-console-operator/console-operator-584dcb6b54-w7grd" podUID="0badf8bd-7f64-46ef-872c-39f59dd11d0d" Jul 27 09:27:02.237257 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:02.237210 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-848w4_2c043d82-d300-41ba-b138-60a03d65ea11/konnectivity-agent/0.log" limit=1024 Jul 27 09:27:02.298077 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:02.298047 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-848w4_2c043d82-d300-41ba-b138-60a03d65ea11/konnectivity-agent/0.log" limit=1024 Jul 27 09:27:02.375421 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:02.375390 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a2f0dc08-339d-4aad-bae9-e13a8f594c70-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-xfvbm\" (UID: \"a2f0dc08-339d-4aad-bae9-e13a8f594c70\") " pod="openshift-insights/insights-runtime-extractor-xfvbm" Jul 27 09:27:02.375574 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:02.375443 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/a2f0dc08-339d-4aad-bae9-e13a8f594c70-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-xfvbm\" (UID: \"a2f0dc08-339d-4aad-bae9-e13a8f594c70\") " pod="openshift-insights/insights-runtime-extractor-xfvbm" Jul 27 09:27:02.375574 ip-10-0-128-103 kubenswrapper[2565]: E0727 09:27:02.375522 2565 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 27 09:27:02.375574 ip-10-0-128-103 kubenswrapper[2565]: E0727 09:27:02.375554 2565 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 27 09:27:02.375574 ip-10-0-128-103 kubenswrapper[2565]: E0727 09:27:02.375576 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/a2f0dc08-339d-4aad-bae9-e13a8f594c70-kube-rbac-proxy-cm podName:a2f0dc08-339d-4aad-bae9-e13a8f594c70 nodeName:}" failed. No retries permitted until 2026-07-27 09:27:04.3755624 +0000 UTC m=+152.339348334 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/a2f0dc08-339d-4aad-bae9-e13a8f594c70-kube-rbac-proxy-cm") pod "insights-runtime-extractor-xfvbm" (UID: "a2f0dc08-339d-4aad-bae9-e13a8f594c70") : configmap "kube-rbac-proxy" not found Jul 27 09:27:02.375713 ip-10-0-128-103 kubenswrapper[2565]: E0727 09:27:02.375607 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a2f0dc08-339d-4aad-bae9-e13a8f594c70-insights-runtime-extractor-tls podName:a2f0dc08-339d-4aad-bae9-e13a8f594c70 nodeName:}" failed. No retries permitted until 2026-07-27 09:27:04.375593588 +0000 UTC m=+152.339379523 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/a2f0dc08-339d-4aad-bae9-e13a8f594c70-insights-runtime-extractor-tls") pod "insights-runtime-extractor-xfvbm" (UID: "a2f0dc08-339d-4aad-bae9-e13a8f594c70") : secret "insights-runtime-extractor-tls" not found Jul 27 09:27:02.500703 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:02.500652 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-848w4_2c043d82-d300-41ba-b138-60a03d65ea11/konnectivity-agent/0.log" limit=1024 Jul 27 09:27:02.609003 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:02.608967 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-848w4_2c043d82-d300-41ba-b138-60a03d65ea11/konnectivity-agent/0.log" limit=1024 Jul 27 09:27:02.816194 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:02.816110 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-848w4_2c043d82-d300-41ba-b138-60a03d65ea11/konnectivity-agent/0.log" limit=1024 Jul 27 09:27:02.824371 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:02.824346 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-r2wsn_199af0d6-b2fc-4951-941f-948e403d3865/dns-node-resolver/0.log" Jul 27 09:27:02.917526 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:02.917492 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-848w4_2c043d82-d300-41ba-b138-60a03d65ea11/konnectivity-agent/0.log" limit=1024 Jul 27 09:27:03.072147 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:03.072053 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-w7grd_0badf8bd-7f64-46ef-872c-39f59dd11d0d/console-operator/1.log" Jul 27 09:27:03.072418 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:03.072403 2565 scope.go:117] "RemoveContainer" containerID="533b092dffb6a6b30e4f5626b68feede4e0431c4b9eb6e71433b9bf7df2f853e" Jul 27 09:27:03.072560 ip-10-0-128-103 kubenswrapper[2565]: E0727 09:27:03.072542 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-584dcb6b54-w7grd_openshift-console-operator(0badf8bd-7f64-46ef-872c-39f59dd11d0d)\"" pod="openshift-console-operator/console-operator-584dcb6b54-w7grd" podUID="0badf8bd-7f64-46ef-872c-39f59dd11d0d" Jul 27 09:27:03.224144 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:03.224111 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-bmklx_996c29bd-3969-4268-a795-4f4d21881d48/node-ca/0.log" Jul 27 09:27:04.086921 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:04.086889 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-544bcd5f5d-5jm4h"] Jul 27 09:27:04.091170 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:04.091148 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-544bcd5f5d-5jm4h" Jul 27 09:27:04.093314 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:04.093291 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 27 09:27:04.093460 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:04.093412 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-rntjf\"" Jul 27 09:27:04.093514 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:04.093491 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 27 09:27:04.093693 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:04.093672 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 27 09:27:04.098811 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:04.098792 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 27 09:27:04.103686 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:04.103663 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-544bcd5f5d-5jm4h"] Jul 27 09:27:04.170421 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:04.170387 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-848w4_2c043d82-d300-41ba-b138-60a03d65ea11/konnectivity-agent/0.log" limit=1024 Jul 27 09:27:04.190740 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:04.190703 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/1221dd0e-e994-44c0-8082-780934175e8e-registry-certificates\") pod \"image-registry-544bcd5f5d-5jm4h\" (UID: \"1221dd0e-e994-44c0-8082-780934175e8e\") " pod="openshift-image-registry/image-registry-544bcd5f5d-5jm4h" Jul 27 09:27:04.190950 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:04.190803 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/1221dd0e-e994-44c0-8082-780934175e8e-image-registry-private-configuration\") pod \"image-registry-544bcd5f5d-5jm4h\" (UID: \"1221dd0e-e994-44c0-8082-780934175e8e\") " pod="openshift-image-registry/image-registry-544bcd5f5d-5jm4h" Jul 27 09:27:04.190950 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:04.190860 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1221dd0e-e994-44c0-8082-780934175e8e-registry-tls\") pod \"image-registry-544bcd5f5d-5jm4h\" (UID: \"1221dd0e-e994-44c0-8082-780934175e8e\") " pod="openshift-image-registry/image-registry-544bcd5f5d-5jm4h" Jul 27 09:27:04.190950 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:04.190925 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j7mcj\" (UniqueName: \"kubernetes.io/projected/1221dd0e-e994-44c0-8082-780934175e8e-kube-api-access-j7mcj\") pod \"image-registry-544bcd5f5d-5jm4h\" (UID: \"1221dd0e-e994-44c0-8082-780934175e8e\") " pod="openshift-image-registry/image-registry-544bcd5f5d-5jm4h" Jul 27 09:27:04.191104 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:04.190958 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/1221dd0e-e994-44c0-8082-780934175e8e-bound-sa-token\") pod \"image-registry-544bcd5f5d-5jm4h\" (UID: \"1221dd0e-e994-44c0-8082-780934175e8e\") " pod="openshift-image-registry/image-registry-544bcd5f5d-5jm4h" Jul 27 09:27:04.191104 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:04.191002 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/1221dd0e-e994-44c0-8082-780934175e8e-installation-pull-secrets\") pod \"image-registry-544bcd5f5d-5jm4h\" (UID: \"1221dd0e-e994-44c0-8082-780934175e8e\") " pod="openshift-image-registry/image-registry-544bcd5f5d-5jm4h" Jul 27 09:27:04.191104 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:04.191064 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/1221dd0e-e994-44c0-8082-780934175e8e-trusted-ca\") pod \"image-registry-544bcd5f5d-5jm4h\" (UID: \"1221dd0e-e994-44c0-8082-780934175e8e\") " pod="openshift-image-registry/image-registry-544bcd5f5d-5jm4h" Jul 27 09:27:04.191104 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:04.191098 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/1221dd0e-e994-44c0-8082-780934175e8e-ca-trust-extracted\") pod \"image-registry-544bcd5f5d-5jm4h\" (UID: \"1221dd0e-e994-44c0-8082-780934175e8e\") " pod="openshift-image-registry/image-registry-544bcd5f5d-5jm4h" Jul 27 09:27:04.226301 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:04.226270 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-q52r7_9b739ad1-657a-40f1-a932-0a94f3960ca4/serve-healthcheck-canary/0.log" Jul 27 09:27:04.226523 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:04.226436 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-848w4_2c043d82-d300-41ba-b138-60a03d65ea11/konnectivity-agent/0.log" limit=1024 Jul 27 09:27:04.291665 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:04.291628 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/1221dd0e-e994-44c0-8082-780934175e8e-installation-pull-secrets\") pod \"image-registry-544bcd5f5d-5jm4h\" (UID: \"1221dd0e-e994-44c0-8082-780934175e8e\") " pod="openshift-image-registry/image-registry-544bcd5f5d-5jm4h" Jul 27 09:27:04.291898 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:04.291684 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/1221dd0e-e994-44c0-8082-780934175e8e-trusted-ca\") pod \"image-registry-544bcd5f5d-5jm4h\" (UID: \"1221dd0e-e994-44c0-8082-780934175e8e\") " pod="openshift-image-registry/image-registry-544bcd5f5d-5jm4h" Jul 27 09:27:04.291898 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:04.291709 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/1221dd0e-e994-44c0-8082-780934175e8e-ca-trust-extracted\") pod \"image-registry-544bcd5f5d-5jm4h\" (UID: \"1221dd0e-e994-44c0-8082-780934175e8e\") " pod="openshift-image-registry/image-registry-544bcd5f5d-5jm4h" Jul 27 09:27:04.291898 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:04.291728 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/1221dd0e-e994-44c0-8082-780934175e8e-registry-certificates\") pod \"image-registry-544bcd5f5d-5jm4h\" (UID: \"1221dd0e-e994-44c0-8082-780934175e8e\") " pod="openshift-image-registry/image-registry-544bcd5f5d-5jm4h" Jul 27 09:27:04.291898 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:04.291810 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/1221dd0e-e994-44c0-8082-780934175e8e-image-registry-private-configuration\") pod \"image-registry-544bcd5f5d-5jm4h\" (UID: \"1221dd0e-e994-44c0-8082-780934175e8e\") " pod="openshift-image-registry/image-registry-544bcd5f5d-5jm4h" Jul 27 09:27:04.291898 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:04.291840 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1221dd0e-e994-44c0-8082-780934175e8e-registry-tls\") pod \"image-registry-544bcd5f5d-5jm4h\" (UID: \"1221dd0e-e994-44c0-8082-780934175e8e\") " pod="openshift-image-registry/image-registry-544bcd5f5d-5jm4h" Jul 27 09:27:04.291898 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:04.291883 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j7mcj\" (UniqueName: \"kubernetes.io/projected/1221dd0e-e994-44c0-8082-780934175e8e-kube-api-access-j7mcj\") pod \"image-registry-544bcd5f5d-5jm4h\" (UID: \"1221dd0e-e994-44c0-8082-780934175e8e\") " pod="openshift-image-registry/image-registry-544bcd5f5d-5jm4h" Jul 27 09:27:04.292207 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:04.291914 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/1221dd0e-e994-44c0-8082-780934175e8e-bound-sa-token\") pod \"image-registry-544bcd5f5d-5jm4h\" (UID: \"1221dd0e-e994-44c0-8082-780934175e8e\") " pod="openshift-image-registry/image-registry-544bcd5f5d-5jm4h" Jul 27 09:27:04.292207 ip-10-0-128-103 kubenswrapper[2565]: E0727 09:27:04.291998 2565 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 27 09:27:04.292207 ip-10-0-128-103 kubenswrapper[2565]: E0727 09:27:04.292020 2565 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-544bcd5f5d-5jm4h: secret "image-registry-tls" not found Jul 27 09:27:04.292207 ip-10-0-128-103 kubenswrapper[2565]: E0727 09:27:04.292079 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1221dd0e-e994-44c0-8082-780934175e8e-registry-tls podName:1221dd0e-e994-44c0-8082-780934175e8e nodeName:}" failed. No retries permitted until 2026-07-27 09:27:04.792058829 +0000 UTC m=+152.755844781 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/1221dd0e-e994-44c0-8082-780934175e8e-registry-tls") pod "image-registry-544bcd5f5d-5jm4h" (UID: "1221dd0e-e994-44c0-8082-780934175e8e") : secret "image-registry-tls" not found Jul 27 09:27:04.292207 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:04.292146 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/1221dd0e-e994-44c0-8082-780934175e8e-ca-trust-extracted\") pod \"image-registry-544bcd5f5d-5jm4h\" (UID: \"1221dd0e-e994-44c0-8082-780934175e8e\") " pod="openshift-image-registry/image-registry-544bcd5f5d-5jm4h" Jul 27 09:27:04.292623 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:04.292596 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/1221dd0e-e994-44c0-8082-780934175e8e-registry-certificates\") pod \"image-registry-544bcd5f5d-5jm4h\" (UID: \"1221dd0e-e994-44c0-8082-780934175e8e\") " pod="openshift-image-registry/image-registry-544bcd5f5d-5jm4h" Jul 27 09:27:04.292878 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:04.292862 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/1221dd0e-e994-44c0-8082-780934175e8e-trusted-ca\") pod \"image-registry-544bcd5f5d-5jm4h\" (UID: \"1221dd0e-e994-44c0-8082-780934175e8e\") " pod="openshift-image-registry/image-registry-544bcd5f5d-5jm4h" Jul 27 09:27:04.294193 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:04.294172 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/1221dd0e-e994-44c0-8082-780934175e8e-installation-pull-secrets\") pod \"image-registry-544bcd5f5d-5jm4h\" (UID: \"1221dd0e-e994-44c0-8082-780934175e8e\") " pod="openshift-image-registry/image-registry-544bcd5f5d-5jm4h" Jul 27 09:27:04.294283 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:04.294191 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/1221dd0e-e994-44c0-8082-780934175e8e-image-registry-private-configuration\") pod \"image-registry-544bcd5f5d-5jm4h\" (UID: \"1221dd0e-e994-44c0-8082-780934175e8e\") " pod="openshift-image-registry/image-registry-544bcd5f5d-5jm4h" Jul 27 09:27:04.299736 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:04.299710 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/1221dd0e-e994-44c0-8082-780934175e8e-bound-sa-token\") pod \"image-registry-544bcd5f5d-5jm4h\" (UID: \"1221dd0e-e994-44c0-8082-780934175e8e\") " pod="openshift-image-registry/image-registry-544bcd5f5d-5jm4h" Jul 27 09:27:04.299851 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:04.299794 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j7mcj\" (UniqueName: \"kubernetes.io/projected/1221dd0e-e994-44c0-8082-780934175e8e-kube-api-access-j7mcj\") pod \"image-registry-544bcd5f5d-5jm4h\" (UID: \"1221dd0e-e994-44c0-8082-780934175e8e\") " pod="openshift-image-registry/image-registry-544bcd5f5d-5jm4h" Jul 27 09:27:04.392571 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:04.392541 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a2f0dc08-339d-4aad-bae9-e13a8f594c70-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-xfvbm\" (UID: \"a2f0dc08-339d-4aad-bae9-e13a8f594c70\") " pod="openshift-insights/insights-runtime-extractor-xfvbm" Jul 27 09:27:04.392747 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:04.392587 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/a2f0dc08-339d-4aad-bae9-e13a8f594c70-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-xfvbm\" (UID: \"a2f0dc08-339d-4aad-bae9-e13a8f594c70\") " pod="openshift-insights/insights-runtime-extractor-xfvbm" Jul 27 09:27:04.392747 ip-10-0-128-103 kubenswrapper[2565]: E0727 09:27:04.392683 2565 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 27 09:27:04.392747 ip-10-0-128-103 kubenswrapper[2565]: E0727 09:27:04.392697 2565 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 27 09:27:04.392747 ip-10-0-128-103 kubenswrapper[2565]: E0727 09:27:04.392738 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/a2f0dc08-339d-4aad-bae9-e13a8f594c70-kube-rbac-proxy-cm podName:a2f0dc08-339d-4aad-bae9-e13a8f594c70 nodeName:}" failed. No retries permitted until 2026-07-27 09:27:08.392723124 +0000 UTC m=+156.356509058 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/a2f0dc08-339d-4aad-bae9-e13a8f594c70-kube-rbac-proxy-cm") pod "insights-runtime-extractor-xfvbm" (UID: "a2f0dc08-339d-4aad-bae9-e13a8f594c70") : configmap "kube-rbac-proxy" not found Jul 27 09:27:04.392747 ip-10-0-128-103 kubenswrapper[2565]: E0727 09:27:04.392750 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a2f0dc08-339d-4aad-bae9-e13a8f594c70-insights-runtime-extractor-tls podName:a2f0dc08-339d-4aad-bae9-e13a8f594c70 nodeName:}" failed. No retries permitted until 2026-07-27 09:27:08.392743989 +0000 UTC m=+156.356529925 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/a2f0dc08-339d-4aad-bae9-e13a8f594c70-insights-runtime-extractor-tls") pod "insights-runtime-extractor-xfvbm" (UID: "a2f0dc08-339d-4aad-bae9-e13a8f594c70") : secret "insights-runtime-extractor-tls" not found Jul 27 09:27:04.594598 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:04.594558 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/f3c02965-8034-4cc6-9cd6-eeff280ebb58-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-hsqzj\" (UID: \"f3c02965-8034-4cc6-9cd6-eeff280ebb58\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-hsqzj" Jul 27 09:27:04.594810 ip-10-0-128-103 kubenswrapper[2565]: E0727 09:27:04.594673 2565 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 27 09:27:04.594810 ip-10-0-128-103 kubenswrapper[2565]: E0727 09:27:04.594725 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f3c02965-8034-4cc6-9cd6-eeff280ebb58-samples-operator-tls podName:f3c02965-8034-4cc6-9cd6-eeff280ebb58 nodeName:}" failed. No retries permitted until 2026-07-27 09:27:12.594712363 +0000 UTC m=+160.558498298 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/f3c02965-8034-4cc6-9cd6-eeff280ebb58-samples-operator-tls") pod "cluster-samples-operator-6748489656-hsqzj" (UID: "f3c02965-8034-4cc6-9cd6-eeff280ebb58") : secret "samples-operator-tls" not found Jul 27 09:27:04.695828 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:04.695716 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/fb21c72b-5cea-49fa-9b28-d1362f405ab0-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-lgr4b\" (UID: \"fb21c72b-5cea-49fa-9b28-d1362f405ab0\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-lgr4b" Jul 27 09:27:04.696003 ip-10-0-128-103 kubenswrapper[2565]: E0727 09:27:04.695881 2565 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 27 09:27:04.696003 ip-10-0-128-103 kubenswrapper[2565]: E0727 09:27:04.695958 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fb21c72b-5cea-49fa-9b28-d1362f405ab0-cluster-monitoring-operator-tls podName:fb21c72b-5cea-49fa-9b28-d1362f405ab0 nodeName:}" failed. No retries permitted until 2026-07-27 09:27:12.695938317 +0000 UTC m=+160.659724275 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/fb21c72b-5cea-49fa-9b28-d1362f405ab0-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-lgr4b" (UID: "fb21c72b-5cea-49fa-9b28-d1362f405ab0") : secret "cluster-monitoring-operator-tls" not found Jul 27 09:27:04.796841 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:04.796806 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1221dd0e-e994-44c0-8082-780934175e8e-registry-tls\") pod \"image-registry-544bcd5f5d-5jm4h\" (UID: \"1221dd0e-e994-44c0-8082-780934175e8e\") " pod="openshift-image-registry/image-registry-544bcd5f5d-5jm4h" Jul 27 09:27:04.797023 ip-10-0-128-103 kubenswrapper[2565]: E0727 09:27:04.796969 2565 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 27 09:27:04.797023 ip-10-0-128-103 kubenswrapper[2565]: E0727 09:27:04.796989 2565 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-544bcd5f5d-5jm4h: secret "image-registry-tls" not found Jul 27 09:27:04.797102 ip-10-0-128-103 kubenswrapper[2565]: E0727 09:27:04.797043 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1221dd0e-e994-44c0-8082-780934175e8e-registry-tls podName:1221dd0e-e994-44c0-8082-780934175e8e nodeName:}" failed. No retries permitted until 2026-07-27 09:27:05.797029761 +0000 UTC m=+153.760815695 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/1221dd0e-e994-44c0-8082-780934175e8e-registry-tls") pod "image-registry-544bcd5f5d-5jm4h" (UID: "1221dd0e-e994-44c0-8082-780934175e8e") : secret "image-registry-tls" not found Jul 27 09:27:05.805870 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:05.805806 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1221dd0e-e994-44c0-8082-780934175e8e-registry-tls\") pod \"image-registry-544bcd5f5d-5jm4h\" (UID: \"1221dd0e-e994-44c0-8082-780934175e8e\") " pod="openshift-image-registry/image-registry-544bcd5f5d-5jm4h" Jul 27 09:27:05.806366 ip-10-0-128-103 kubenswrapper[2565]: E0727 09:27:05.805981 2565 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 27 09:27:05.806366 ip-10-0-128-103 kubenswrapper[2565]: E0727 09:27:05.806008 2565 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-544bcd5f5d-5jm4h: secret "image-registry-tls" not found Jul 27 09:27:05.806366 ip-10-0-128-103 kubenswrapper[2565]: E0727 09:27:05.806083 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1221dd0e-e994-44c0-8082-780934175e8e-registry-tls podName:1221dd0e-e994-44c0-8082-780934175e8e nodeName:}" failed. No retries permitted until 2026-07-27 09:27:07.806060921 +0000 UTC m=+155.769846857 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/1221dd0e-e994-44c0-8082-780934175e8e-registry-tls") pod "image-registry-544bcd5f5d-5jm4h" (UID: "1221dd0e-e994-44c0-8082-780934175e8e") : secret "image-registry-tls" not found Jul 27 09:27:06.023908 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:06.023861 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-848w4_2c043d82-d300-41ba-b138-60a03d65ea11/konnectivity-agent/0.log" limit=1024 Jul 27 09:27:07.305045 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:07.305004 2565 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-584dcb6b54-w7grd" Jul 27 09:27:07.305462 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:07.305370 2565 scope.go:117] "RemoveContainer" containerID="533b092dffb6a6b30e4f5626b68feede4e0431c4b9eb6e71433b9bf7df2f853e" Jul 27 09:27:07.305550 ip-10-0-128-103 kubenswrapper[2565]: E0727 09:27:07.305531 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-584dcb6b54-w7grd_openshift-console-operator(0badf8bd-7f64-46ef-872c-39f59dd11d0d)\"" pod="openshift-console-operator/console-operator-584dcb6b54-w7grd" podUID="0badf8bd-7f64-46ef-872c-39f59dd11d0d" Jul 27 09:27:07.679298 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:07.679262 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-848w4_2c043d82-d300-41ba-b138-60a03d65ea11/konnectivity-agent/0.log" limit=1024 Jul 27 09:27:07.780923 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:07.780885 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-848w4_2c043d82-d300-41ba-b138-60a03d65ea11/konnectivity-agent/0.log" limit=1024 Jul 27 09:27:07.820326 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:07.820294 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1221dd0e-e994-44c0-8082-780934175e8e-registry-tls\") pod \"image-registry-544bcd5f5d-5jm4h\" (UID: \"1221dd0e-e994-44c0-8082-780934175e8e\") " pod="openshift-image-registry/image-registry-544bcd5f5d-5jm4h" Jul 27 09:27:07.820440 ip-10-0-128-103 kubenswrapper[2565]: E0727 09:27:07.820410 2565 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 27 09:27:07.820440 ip-10-0-128-103 kubenswrapper[2565]: E0727 09:27:07.820424 2565 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-544bcd5f5d-5jm4h: secret "image-registry-tls" not found Jul 27 09:27:07.820512 ip-10-0-128-103 kubenswrapper[2565]: E0727 09:27:07.820480 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1221dd0e-e994-44c0-8082-780934175e8e-registry-tls podName:1221dd0e-e994-44c0-8082-780934175e8e nodeName:}" failed. No retries permitted until 2026-07-27 09:27:11.820462923 +0000 UTC m=+159.784248880 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/1221dd0e-e994-44c0-8082-780934175e8e-registry-tls") pod "image-registry-544bcd5f5d-5jm4h" (UID: "1221dd0e-e994-44c0-8082-780934175e8e") : secret "image-registry-tls" not found Jul 27 09:27:07.895139 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:07.895105 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-848w4_2c043d82-d300-41ba-b138-60a03d65ea11/konnectivity-agent/0.log" limit=1024 Jul 27 09:27:08.167709 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:08.167674 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-848w4_2c043d82-d300-41ba-b138-60a03d65ea11/konnectivity-agent/0.log" limit=1024 Jul 27 09:27:08.425672 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:08.425586 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a2f0dc08-339d-4aad-bae9-e13a8f594c70-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-xfvbm\" (UID: \"a2f0dc08-339d-4aad-bae9-e13a8f594c70\") " pod="openshift-insights/insights-runtime-extractor-xfvbm" Jul 27 09:27:08.425672 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:08.425630 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/a2f0dc08-339d-4aad-bae9-e13a8f594c70-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-xfvbm\" (UID: \"a2f0dc08-339d-4aad-bae9-e13a8f594c70\") " pod="openshift-insights/insights-runtime-extractor-xfvbm" Jul 27 09:27:08.426128 ip-10-0-128-103 kubenswrapper[2565]: E0727 09:27:08.425757 2565 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 27 09:27:08.426128 ip-10-0-128-103 kubenswrapper[2565]: E0727 09:27:08.425791 2565 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 27 09:27:08.426128 ip-10-0-128-103 kubenswrapper[2565]: E0727 09:27:08.425853 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/a2f0dc08-339d-4aad-bae9-e13a8f594c70-kube-rbac-proxy-cm podName:a2f0dc08-339d-4aad-bae9-e13a8f594c70 nodeName:}" failed. No retries permitted until 2026-07-27 09:27:16.425835717 +0000 UTC m=+164.389621671 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/a2f0dc08-339d-4aad-bae9-e13a8f594c70-kube-rbac-proxy-cm") pod "insights-runtime-extractor-xfvbm" (UID: "a2f0dc08-339d-4aad-bae9-e13a8f594c70") : configmap "kube-rbac-proxy" not found Jul 27 09:27:08.426128 ip-10-0-128-103 kubenswrapper[2565]: E0727 09:27:08.425872 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a2f0dc08-339d-4aad-bae9-e13a8f594c70-insights-runtime-extractor-tls podName:a2f0dc08-339d-4aad-bae9-e13a8f594c70 nodeName:}" failed. No retries permitted until 2026-07-27 09:27:16.425864508 +0000 UTC m=+164.389650443 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/a2f0dc08-339d-4aad-bae9-e13a8f594c70-insights-runtime-extractor-tls") pod "insights-runtime-extractor-xfvbm" (UID: "a2f0dc08-339d-4aad-bae9-e13a8f594c70") : secret "insights-runtime-extractor-tls" not found Jul 27 09:27:09.361760 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:09.361725 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-848w4_2c043d82-d300-41ba-b138-60a03d65ea11/konnectivity-agent/0.log" limit=1024 Jul 27 09:27:09.460871 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:09.460839 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-848w4_2c043d82-d300-41ba-b138-60a03d65ea11/konnectivity-agent/0.log" limit=1024 Jul 27 09:27:09.565503 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:09.565471 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-848w4_2c043d82-d300-41ba-b138-60a03d65ea11/konnectivity-agent/0.log" limit=1024 Jul 27 09:27:09.757032 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:09.756997 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-848w4_2c043d82-d300-41ba-b138-60a03d65ea11/konnectivity-agent/0.log" limit=1024 Jul 27 09:27:09.952530 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:09.952496 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-848w4_2c043d82-d300-41ba-b138-60a03d65ea11/konnectivity-agent/0.log" limit=1024 Jul 27 09:27:10.053096 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:10.053015 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-584dcb6b54-w7grd" Jul 27 09:27:10.053434 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:10.053420 2565 scope.go:117] "RemoveContainer" containerID="533b092dffb6a6b30e4f5626b68feede4e0431c4b9eb6e71433b9bf7df2f853e" Jul 27 09:27:10.053594 ip-10-0-128-103 kubenswrapper[2565]: E0727 09:27:10.053575 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-584dcb6b54-w7grd_openshift-console-operator(0badf8bd-7f64-46ef-872c-39f59dd11d0d)\"" pod="openshift-console-operator/console-operator-584dcb6b54-w7grd" podUID="0badf8bd-7f64-46ef-872c-39f59dd11d0d" Jul 27 09:27:10.156646 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:10.156614 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-848w4_2c043d82-d300-41ba-b138-60a03d65ea11/konnectivity-agent/0.log" limit=1024 Jul 27 09:27:10.375909 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:10.375875 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-848w4_2c043d82-d300-41ba-b138-60a03d65ea11/konnectivity-agent/0.log" limit=1024 Jul 27 09:27:10.483974 ip-10-0-128-103 kubenswrapper[2565]: E0727 09:27:10.483933 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-7lzb6" podUID="ef01a458-240c-4477-9ef9-d5b21026df4f" Jul 27 09:27:11.094540 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:11.094515 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-7lzb6" Jul 27 09:27:11.276074 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:11.276039 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-848w4_2c043d82-d300-41ba-b138-60a03d65ea11/konnectivity-agent/0.log" limit=1024 Jul 27 09:27:11.377532 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:11.377502 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-848w4_2c043d82-d300-41ba-b138-60a03d65ea11/konnectivity-agent/0.log" limit=1024 Jul 27 09:27:11.525165 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:11.525128 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-848w4_2c043d82-d300-41ba-b138-60a03d65ea11/konnectivity-agent/0.log" limit=1024 Jul 27 09:27:11.578173 ip-10-0-128-103 kubenswrapper[2565]: E0727 09:27:11.578134 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-trcl2" podUID="0d6ba35e-ec8a-4ec9-8520-0d32a5586a1d" Jul 27 09:27:11.625381 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:11.625350 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-848w4_2c043d82-d300-41ba-b138-60a03d65ea11/konnectivity-agent/0.log" limit=1024 Jul 27 09:27:11.759291 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:11.759216 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-848w4_2c043d82-d300-41ba-b138-60a03d65ea11/konnectivity-agent/0.log" limit=1024 Jul 27 09:27:11.854533 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:11.854492 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1221dd0e-e994-44c0-8082-780934175e8e-registry-tls\") pod \"image-registry-544bcd5f5d-5jm4h\" (UID: \"1221dd0e-e994-44c0-8082-780934175e8e\") " pod="openshift-image-registry/image-registry-544bcd5f5d-5jm4h" Jul 27 09:27:11.856963 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:11.856937 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1221dd0e-e994-44c0-8082-780934175e8e-registry-tls\") pod \"image-registry-544bcd5f5d-5jm4h\" (UID: \"1221dd0e-e994-44c0-8082-780934175e8e\") " pod="openshift-image-registry/image-registry-544bcd5f5d-5jm4h" Jul 27 09:27:11.900391 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:11.900349 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-544bcd5f5d-5jm4h" Jul 27 09:27:11.961691 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:11.961608 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-848w4_2c043d82-d300-41ba-b138-60a03d65ea11/konnectivity-agent/0.log" limit=1024 Jul 27 09:27:12.028415 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:12.028344 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-544bcd5f5d-5jm4h"] Jul 27 09:27:12.031587 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:27:12.031555 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1221dd0e_e994_44c0_8082_780934175e8e.slice/crio-39504e789e677839a9f8f0908754e09eb28da7cc17b6e1b5186ddf519bd64419 WatchSource:0}: Error finding container 39504e789e677839a9f8f0908754e09eb28da7cc17b6e1b5186ddf519bd64419: Status 404 returned error can't find the container with id 39504e789e677839a9f8f0908754e09eb28da7cc17b6e1b5186ddf519bd64419 Jul 27 09:27:12.031952 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:12.031928 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-848w4_2c043d82-d300-41ba-b138-60a03d65ea11/konnectivity-agent/0.log" limit=1024 Jul 27 09:27:12.099309 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:12.099276 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-544bcd5f5d-5jm4h" event={"ID":"1221dd0e-e994-44c0-8082-780934175e8e","Type":"ContainerStarted","Data":"591fe6d3bb76d5e34ef3a498f7b173c972892189e0c0b27f514956c9f0af60d1"} Jul 27 09:27:12.099439 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:12.099328 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-544bcd5f5d-5jm4h" event={"ID":"1221dd0e-e994-44c0-8082-780934175e8e","Type":"ContainerStarted","Data":"39504e789e677839a9f8f0908754e09eb28da7cc17b6e1b5186ddf519bd64419"} Jul 27 09:27:12.099439 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:12.099358 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-544bcd5f5d-5jm4h" Jul 27 09:27:12.118843 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:12.118793 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-544bcd5f5d-5jm4h" podStartSLOduration=8.118753767 podStartE2EDuration="8.118753767s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 09:27:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 09:27:12.117912208 +0000 UTC m=+160.081698186" watchObservedRunningTime="2026-07-27 09:27:12.118753767 +0000 UTC m=+160.082539723" Jul 27 09:27:12.158006 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:12.157976 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-848w4_2c043d82-d300-41ba-b138-60a03d65ea11/konnectivity-agent/0.log" limit=1024 Jul 27 09:27:12.661742 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:12.661694 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/f3c02965-8034-4cc6-9cd6-eeff280ebb58-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-hsqzj\" (UID: \"f3c02965-8034-4cc6-9cd6-eeff280ebb58\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-hsqzj" Jul 27 09:27:12.664176 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:12.664152 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/f3c02965-8034-4cc6-9cd6-eeff280ebb58-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-hsqzj\" (UID: \"f3c02965-8034-4cc6-9cd6-eeff280ebb58\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-hsqzj" Jul 27 09:27:12.707085 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:12.707053 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-hsqzj" Jul 27 09:27:12.763336 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:12.763252 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/fb21c72b-5cea-49fa-9b28-d1362f405ab0-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-lgr4b\" (UID: \"fb21c72b-5cea-49fa-9b28-d1362f405ab0\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-lgr4b" Jul 27 09:27:12.763473 ip-10-0-128-103 kubenswrapper[2565]: E0727 09:27:12.763414 2565 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 27 09:27:12.763526 ip-10-0-128-103 kubenswrapper[2565]: E0727 09:27:12.763504 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fb21c72b-5cea-49fa-9b28-d1362f405ab0-cluster-monitoring-operator-tls podName:fb21c72b-5cea-49fa-9b28-d1362f405ab0 nodeName:}" failed. No retries permitted until 2026-07-27 09:27:28.763479539 +0000 UTC m=+176.727265479 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/fb21c72b-5cea-49fa-9b28-d1362f405ab0-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-lgr4b" (UID: "fb21c72b-5cea-49fa-9b28-d1362f405ab0") : secret "cluster-monitoring-operator-tls" not found Jul 27 09:27:12.836061 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:12.836005 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-6748489656-hsqzj"] Jul 27 09:27:13.103623 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:13.103533 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-hsqzj" event={"ID":"f3c02965-8034-4cc6-9cd6-eeff280ebb58","Type":"ContainerStarted","Data":"4abcb6844eb73a077641d75b4abe657ebd4f2441e7143befb08d5da2b4bc10ce"} Jul 27 09:27:13.161563 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:13.161531 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-848w4_2c043d82-d300-41ba-b138-60a03d65ea11/konnectivity-agent/0.log" limit=1024 Jul 27 09:27:13.361379 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:13.361297 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-848w4_2c043d82-d300-41ba-b138-60a03d65ea11/konnectivity-agent/0.log" limit=1024 Jul 27 09:27:13.760999 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:13.760960 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-848w4_2c043d82-d300-41ba-b138-60a03d65ea11/konnectivity-agent/0.log" limit=1024 Jul 27 09:27:13.862437 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:13.862401 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-848w4_2c043d82-d300-41ba-b138-60a03d65ea11/konnectivity-agent/0.log" limit=1024 Jul 27 09:27:13.963288 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:13.963238 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-848w4_2c043d82-d300-41ba-b138-60a03d65ea11/konnectivity-agent/0.log" limit=1024 Jul 27 09:27:14.025417 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:14.025342 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-848w4_2c043d82-d300-41ba-b138-60a03d65ea11/konnectivity-agent/0.log" limit=1024 Jul 27 09:27:14.578456 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:14.578421 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-848w4_2c043d82-d300-41ba-b138-60a03d65ea11/konnectivity-agent/0.log" limit=1024 Jul 27 09:27:14.965032 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:14.965001 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-848w4_2c043d82-d300-41ba-b138-60a03d65ea11/konnectivity-agent/0.log" limit=1024 Jul 27 09:27:15.111201 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:15.111168 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-hsqzj" event={"ID":"f3c02965-8034-4cc6-9cd6-eeff280ebb58","Type":"ContainerStarted","Data":"44ee463a4da3c5b65944483028eae7b9ef5fcf00436ccc689aecfb989a9bb67b"} Jul 27 09:27:15.111341 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:15.111253 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-hsqzj" event={"ID":"f3c02965-8034-4cc6-9cd6-eeff280ebb58","Type":"ContainerStarted","Data":"09ef9eaf981c1de65f7fd1099a59f7dd23ec75de73fdb63dfcd8d2010fa55bdf"} Jul 27 09:27:15.129698 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:15.129635 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-hsqzj" podStartSLOduration=17.585454581 podStartE2EDuration="19.129618096s" totalImagesPullingTime="1.544163515s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 09:26:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 09:27:15.129238782 +0000 UTC m=+163.093024739" watchObservedRunningTime="2026-07-27 09:27:15.129618096 +0000 UTC m=+163.093404055" Jul 27 09:27:15.367838 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:15.367811 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-848w4_2c043d82-d300-41ba-b138-60a03d65ea11/konnectivity-agent/0.log" limit=1024 Jul 27 09:27:15.384068 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:15.384041 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ef01a458-240c-4477-9ef9-d5b21026df4f-metrics-tls\") pod \"dns-default-7lzb6\" (UID: \"ef01a458-240c-4477-9ef9-d5b21026df4f\") " pod="openshift-dns/dns-default-7lzb6" Jul 27 09:27:15.386332 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:15.386307 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ef01a458-240c-4477-9ef9-d5b21026df4f-metrics-tls\") pod \"dns-default-7lzb6\" (UID: \"ef01a458-240c-4477-9ef9-d5b21026df4f\") " pod="openshift-dns/dns-default-7lzb6" Jul 27 09:27:15.597921 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:15.597884 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-v4cq5\"" Jul 27 09:27:15.606845 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:15.606822 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-7lzb6" Jul 27 09:27:15.731119 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:15.731085 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-7lzb6"] Jul 27 09:27:15.734355 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:27:15.734317 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef01a458_240c_4477_9ef9_d5b21026df4f.slice/crio-6b981b36df7a2dda5d406b3dde17d6522ab0187ca724231735801966a51bfe3a WatchSource:0}: Error finding container 6b981b36df7a2dda5d406b3dde17d6522ab0187ca724231735801966a51bfe3a: Status 404 returned error can't find the container with id 6b981b36df7a2dda5d406b3dde17d6522ab0187ca724231735801966a51bfe3a Jul 27 09:27:15.763445 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:15.763413 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-848w4_2c043d82-d300-41ba-b138-60a03d65ea11/konnectivity-agent/0.log" limit=1024 Jul 27 09:27:15.958984 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:15.958894 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-848w4_2c043d82-d300-41ba-b138-60a03d65ea11/konnectivity-agent/0.log" limit=1024 Jul 27 09:27:16.115019 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:16.114983 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-7lzb6" event={"ID":"ef01a458-240c-4477-9ef9-d5b21026df4f","Type":"ContainerStarted","Data":"6b981b36df7a2dda5d406b3dde17d6522ab0187ca724231735801966a51bfe3a"} Jul 27 09:27:16.360992 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:16.360907 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-848w4_2c043d82-d300-41ba-b138-60a03d65ea11/konnectivity-agent/0.log" limit=1024 Jul 27 09:27:16.494690 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:16.494648 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a2f0dc08-339d-4aad-bae9-e13a8f594c70-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-xfvbm\" (UID: \"a2f0dc08-339d-4aad-bae9-e13a8f594c70\") " pod="openshift-insights/insights-runtime-extractor-xfvbm" Jul 27 09:27:16.494887 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:16.494705 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/a2f0dc08-339d-4aad-bae9-e13a8f594c70-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-xfvbm\" (UID: \"a2f0dc08-339d-4aad-bae9-e13a8f594c70\") " pod="openshift-insights/insights-runtime-extractor-xfvbm" Jul 27 09:27:16.494887 ip-10-0-128-103 kubenswrapper[2565]: E0727 09:27:16.494831 2565 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 27 09:27:16.494974 ip-10-0-128-103 kubenswrapper[2565]: E0727 09:27:16.494926 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a2f0dc08-339d-4aad-bae9-e13a8f594c70-insights-runtime-extractor-tls podName:a2f0dc08-339d-4aad-bae9-e13a8f594c70 nodeName:}" failed. No retries permitted until 2026-07-27 09:27:32.494909908 +0000 UTC m=+180.458695844 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/a2f0dc08-339d-4aad-bae9-e13a8f594c70-insights-runtime-extractor-tls") pod "insights-runtime-extractor-xfvbm" (UID: "a2f0dc08-339d-4aad-bae9-e13a8f594c70") : secret "insights-runtime-extractor-tls" not found Jul 27 09:27:16.495329 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:16.495304 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/a2f0dc08-339d-4aad-bae9-e13a8f594c70-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-xfvbm\" (UID: \"a2f0dc08-339d-4aad-bae9-e13a8f594c70\") " pod="openshift-insights/insights-runtime-extractor-xfvbm" Jul 27 09:27:16.963009 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:16.962962 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-848w4_2c043d82-d300-41ba-b138-60a03d65ea11/konnectivity-agent/0.log" limit=1024 Jul 27 09:27:17.360019 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:17.359987 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-848w4_2c043d82-d300-41ba-b138-60a03d65ea11/konnectivity-agent/0.log" limit=1024 Jul 27 09:27:17.760654 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:17.760622 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-848w4_2c043d82-d300-41ba-b138-60a03d65ea11/konnectivity-agent/0.log" limit=1024 Jul 27 09:27:17.959385 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:17.959347 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-848w4_2c043d82-d300-41ba-b138-60a03d65ea11/konnectivity-agent/0.log" limit=1024 Jul 27 09:27:18.125147 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:18.125106 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-7lzb6" event={"ID":"ef01a458-240c-4477-9ef9-d5b21026df4f","Type":"ContainerStarted","Data":"373c06d0ecd06dbb08d503bfe157e6e7aeefb05831781c46b4af4d9ef4f3a341"} Jul 27 09:27:18.125327 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:18.125169 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-7lzb6" event={"ID":"ef01a458-240c-4477-9ef9-d5b21026df4f","Type":"ContainerStarted","Data":"c9a4f0f7373e47b0c0868aecbdc80fcac4c676d7acd798ee0b11a97a9a8113b3"} Jul 27 09:27:18.125327 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:18.125211 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-7lzb6" Jul 27 09:27:18.146446 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:18.146396 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-7lzb6" podStartSLOduration=129.594379265 podStartE2EDuration="2m11.146378233s" totalImagesPullingTime="1.551998968s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 09:25:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 09:27:18.145631006 +0000 UTC m=+166.109416964" watchObservedRunningTime="2026-07-27 09:27:18.146378233 +0000 UTC m=+166.110164193" Jul 27 09:27:18.167644 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:18.167616 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-848w4_2c043d82-d300-41ba-b138-60a03d65ea11/konnectivity-agent/0.log" limit=1024 Jul 27 09:27:18.280391 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:18.280361 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-848w4_2c043d82-d300-41ba-b138-60a03d65ea11/konnectivity-agent/0.log" limit=1024 Jul 27 09:27:22.558495 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:22.558454 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-trcl2" Jul 27 09:27:25.558705 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:25.558627 2565 scope.go:117] "RemoveContainer" containerID="533b092dffb6a6b30e4f5626b68feede4e0431c4b9eb6e71433b9bf7df2f853e" Jul 27 09:27:25.703466 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:25.703422 2565 prober.go:120] "Probe failed" probeType="Readiness" pod="kube-system/konnectivity-agent-848w4" podUID="2c043d82-d300-41ba-b138-60a03d65ea11" containerName="konnectivity-agent" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 27 09:27:25.703642 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:25.703534 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-848w4" Jul 27 09:27:25.704270 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:25.704239 2565 prober.go:120] "Probe failed" probeType="Readiness" pod="kube-system/konnectivity-agent-848w4" podUID="2c043d82-d300-41ba-b138-60a03d65ea11" containerName="konnectivity-agent" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 27 09:27:26.154940 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:26.154910 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-w7grd_0badf8bd-7f64-46ef-872c-39f59dd11d0d/console-operator/2.log" Jul 27 09:27:26.155329 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:26.155314 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-w7grd_0badf8bd-7f64-46ef-872c-39f59dd11d0d/console-operator/1.log" Jul 27 09:27:26.155387 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:26.155347 2565 generic.go:356] "Generic (PLEG): container finished" podID="0badf8bd-7f64-46ef-872c-39f59dd11d0d" containerID="1491a240002f05c46d392fdf6a0e9ca99f3d5760eb568e9316bbe43b49bc5c3e" exitCode=255 Jul 27 09:27:26.155419 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:26.155384 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-584dcb6b54-w7grd" event={"ID":"0badf8bd-7f64-46ef-872c-39f59dd11d0d","Type":"ContainerDied","Data":"1491a240002f05c46d392fdf6a0e9ca99f3d5760eb568e9316bbe43b49bc5c3e"} Jul 27 09:27:26.155462 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:26.155452 2565 scope.go:117] "RemoveContainer" containerID="533b092dffb6a6b30e4f5626b68feede4e0431c4b9eb6e71433b9bf7df2f853e" Jul 27 09:27:26.155782 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:26.155743 2565 scope.go:117] "RemoveContainer" containerID="1491a240002f05c46d392fdf6a0e9ca99f3d5760eb568e9316bbe43b49bc5c3e" Jul 27 09:27:26.155965 ip-10-0-128-103 kubenswrapper[2565]: E0727 09:27:26.155941 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-584dcb6b54-w7grd_openshift-console-operator(0badf8bd-7f64-46ef-872c-39f59dd11d0d)\"" pod="openshift-console-operator/console-operator-584dcb6b54-w7grd" podUID="0badf8bd-7f64-46ef-872c-39f59dd11d0d" Jul 27 09:27:26.551944 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:26.551862 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-544bcd5f5d-5jm4h"] Jul 27 09:27:26.593630 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:26.593601 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-56f74b4f54-4jm6n"] Jul 27 09:27:26.598141 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:26.598110 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-56f74b4f54-4jm6n" Jul 27 09:27:26.613323 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:26.613286 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-56f74b4f54-4jm6n"] Jul 27 09:27:26.673865 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:26.673824 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/309e9463-20cb-4831-847a-c3421997a50f-image-registry-private-configuration\") pod \"image-registry-56f74b4f54-4jm6n\" (UID: \"309e9463-20cb-4831-847a-c3421997a50f\") " pod="openshift-image-registry/image-registry-56f74b4f54-4jm6n" Jul 27 09:27:26.673865 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:26.673862 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/309e9463-20cb-4831-847a-c3421997a50f-registry-certificates\") pod \"image-registry-56f74b4f54-4jm6n\" (UID: \"309e9463-20cb-4831-847a-c3421997a50f\") " pod="openshift-image-registry/image-registry-56f74b4f54-4jm6n" Jul 27 09:27:26.674073 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:26.673961 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/309e9463-20cb-4831-847a-c3421997a50f-installation-pull-secrets\") pod \"image-registry-56f74b4f54-4jm6n\" (UID: \"309e9463-20cb-4831-847a-c3421997a50f\") " pod="openshift-image-registry/image-registry-56f74b4f54-4jm6n" Jul 27 09:27:26.674073 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:26.674024 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/309e9463-20cb-4831-847a-c3421997a50f-registry-tls\") pod \"image-registry-56f74b4f54-4jm6n\" (UID: \"309e9463-20cb-4831-847a-c3421997a50f\") " pod="openshift-image-registry/image-registry-56f74b4f54-4jm6n" Jul 27 09:27:26.674073 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:26.674042 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/309e9463-20cb-4831-847a-c3421997a50f-bound-sa-token\") pod \"image-registry-56f74b4f54-4jm6n\" (UID: \"309e9463-20cb-4831-847a-c3421997a50f\") " pod="openshift-image-registry/image-registry-56f74b4f54-4jm6n" Jul 27 09:27:26.674073 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:26.674061 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/309e9463-20cb-4831-847a-c3421997a50f-ca-trust-extracted\") pod \"image-registry-56f74b4f54-4jm6n\" (UID: \"309e9463-20cb-4831-847a-c3421997a50f\") " pod="openshift-image-registry/image-registry-56f74b4f54-4jm6n" Jul 27 09:27:26.674194 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:26.674077 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/309e9463-20cb-4831-847a-c3421997a50f-trusted-ca\") pod \"image-registry-56f74b4f54-4jm6n\" (UID: \"309e9463-20cb-4831-847a-c3421997a50f\") " pod="openshift-image-registry/image-registry-56f74b4f54-4jm6n" Jul 27 09:27:26.674194 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:26.674107 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-svfq2\" (UniqueName: \"kubernetes.io/projected/309e9463-20cb-4831-847a-c3421997a50f-kube-api-access-svfq2\") pod \"image-registry-56f74b4f54-4jm6n\" (UID: \"309e9463-20cb-4831-847a-c3421997a50f\") " pod="openshift-image-registry/image-registry-56f74b4f54-4jm6n" Jul 27 09:27:26.774712 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:26.774675 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/309e9463-20cb-4831-847a-c3421997a50f-registry-tls\") pod \"image-registry-56f74b4f54-4jm6n\" (UID: \"309e9463-20cb-4831-847a-c3421997a50f\") " pod="openshift-image-registry/image-registry-56f74b4f54-4jm6n" Jul 27 09:27:26.774712 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:26.774712 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/309e9463-20cb-4831-847a-c3421997a50f-bound-sa-token\") pod \"image-registry-56f74b4f54-4jm6n\" (UID: \"309e9463-20cb-4831-847a-c3421997a50f\") " pod="openshift-image-registry/image-registry-56f74b4f54-4jm6n" Jul 27 09:27:26.774988 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:26.774733 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/309e9463-20cb-4831-847a-c3421997a50f-ca-trust-extracted\") pod \"image-registry-56f74b4f54-4jm6n\" (UID: \"309e9463-20cb-4831-847a-c3421997a50f\") " pod="openshift-image-registry/image-registry-56f74b4f54-4jm6n" Jul 27 09:27:26.774988 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:26.774750 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/309e9463-20cb-4831-847a-c3421997a50f-trusted-ca\") pod \"image-registry-56f74b4f54-4jm6n\" (UID: \"309e9463-20cb-4831-847a-c3421997a50f\") " pod="openshift-image-registry/image-registry-56f74b4f54-4jm6n" Jul 27 09:27:26.774988 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:26.774825 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-svfq2\" (UniqueName: \"kubernetes.io/projected/309e9463-20cb-4831-847a-c3421997a50f-kube-api-access-svfq2\") pod \"image-registry-56f74b4f54-4jm6n\" (UID: \"309e9463-20cb-4831-847a-c3421997a50f\") " pod="openshift-image-registry/image-registry-56f74b4f54-4jm6n" Jul 27 09:27:26.775167 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:26.775058 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/309e9463-20cb-4831-847a-c3421997a50f-image-registry-private-configuration\") pod \"image-registry-56f74b4f54-4jm6n\" (UID: \"309e9463-20cb-4831-847a-c3421997a50f\") " pod="openshift-image-registry/image-registry-56f74b4f54-4jm6n" Jul 27 09:27:26.775167 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:26.775106 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/309e9463-20cb-4831-847a-c3421997a50f-registry-certificates\") pod \"image-registry-56f74b4f54-4jm6n\" (UID: \"309e9463-20cb-4831-847a-c3421997a50f\") " pod="openshift-image-registry/image-registry-56f74b4f54-4jm6n" Jul 27 09:27:26.775276 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:26.775176 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/309e9463-20cb-4831-847a-c3421997a50f-installation-pull-secrets\") pod \"image-registry-56f74b4f54-4jm6n\" (UID: \"309e9463-20cb-4831-847a-c3421997a50f\") " pod="openshift-image-registry/image-registry-56f74b4f54-4jm6n" Jul 27 09:27:26.775276 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:26.775212 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/309e9463-20cb-4831-847a-c3421997a50f-ca-trust-extracted\") pod \"image-registry-56f74b4f54-4jm6n\" (UID: \"309e9463-20cb-4831-847a-c3421997a50f\") " pod="openshift-image-registry/image-registry-56f74b4f54-4jm6n" Jul 27 09:27:26.775938 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:26.775914 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/309e9463-20cb-4831-847a-c3421997a50f-registry-certificates\") pod \"image-registry-56f74b4f54-4jm6n\" (UID: \"309e9463-20cb-4831-847a-c3421997a50f\") " pod="openshift-image-registry/image-registry-56f74b4f54-4jm6n" Jul 27 09:27:26.776073 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:26.776034 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/309e9463-20cb-4831-847a-c3421997a50f-trusted-ca\") pod \"image-registry-56f74b4f54-4jm6n\" (UID: \"309e9463-20cb-4831-847a-c3421997a50f\") " pod="openshift-image-registry/image-registry-56f74b4f54-4jm6n" Jul 27 09:27:26.777429 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:26.777407 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/309e9463-20cb-4831-847a-c3421997a50f-image-registry-private-configuration\") pod \"image-registry-56f74b4f54-4jm6n\" (UID: \"309e9463-20cb-4831-847a-c3421997a50f\") " pod="openshift-image-registry/image-registry-56f74b4f54-4jm6n" Jul 27 09:27:26.777650 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:26.777633 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/309e9463-20cb-4831-847a-c3421997a50f-installation-pull-secrets\") pod \"image-registry-56f74b4f54-4jm6n\" (UID: \"309e9463-20cb-4831-847a-c3421997a50f\") " pod="openshift-image-registry/image-registry-56f74b4f54-4jm6n" Jul 27 09:27:26.777716 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:26.777699 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/309e9463-20cb-4831-847a-c3421997a50f-registry-tls\") pod \"image-registry-56f74b4f54-4jm6n\" (UID: \"309e9463-20cb-4831-847a-c3421997a50f\") " pod="openshift-image-registry/image-registry-56f74b4f54-4jm6n" Jul 27 09:27:26.782615 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:26.782592 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/309e9463-20cb-4831-847a-c3421997a50f-bound-sa-token\") pod \"image-registry-56f74b4f54-4jm6n\" (UID: \"309e9463-20cb-4831-847a-c3421997a50f\") " pod="openshift-image-registry/image-registry-56f74b4f54-4jm6n" Jul 27 09:27:26.782726 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:26.782709 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-svfq2\" (UniqueName: \"kubernetes.io/projected/309e9463-20cb-4831-847a-c3421997a50f-kube-api-access-svfq2\") pod \"image-registry-56f74b4f54-4jm6n\" (UID: \"309e9463-20cb-4831-847a-c3421997a50f\") " pod="openshift-image-registry/image-registry-56f74b4f54-4jm6n" Jul 27 09:27:26.907256 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:26.907228 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-56f74b4f54-4jm6n" Jul 27 09:27:27.046561 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:27.046520 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-56f74b4f54-4jm6n"] Jul 27 09:27:27.050133 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:27:27.050103 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod309e9463_20cb_4831_847a_c3421997a50f.slice/crio-9ca8b63d4904801b6633e6609b21eca4bf69ef627d6dae8f768ba73f54eb0908 WatchSource:0}: Error finding container 9ca8b63d4904801b6633e6609b21eca4bf69ef627d6dae8f768ba73f54eb0908: Status 404 returned error can't find the container with id 9ca8b63d4904801b6633e6609b21eca4bf69ef627d6dae8f768ba73f54eb0908 Jul 27 09:27:27.161828 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:27.161728 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-w7grd_0badf8bd-7f64-46ef-872c-39f59dd11d0d/console-operator/2.log" Jul 27 09:27:27.163397 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:27.163369 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-56f74b4f54-4jm6n" event={"ID":"309e9463-20cb-4831-847a-c3421997a50f","Type":"ContainerStarted","Data":"d6beb65faaab72589eef7841ee28d8fcc14bbcd54b2f03ab10cfc09a900a66c4"} Jul 27 09:27:27.163534 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:27.163418 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-56f74b4f54-4jm6n" event={"ID":"309e9463-20cb-4831-847a-c3421997a50f","Type":"ContainerStarted","Data":"9ca8b63d4904801b6633e6609b21eca4bf69ef627d6dae8f768ba73f54eb0908"} Jul 27 09:27:27.163534 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:27.163499 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-56f74b4f54-4jm6n" Jul 27 09:27:27.184755 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:27.184688 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-56f74b4f54-4jm6n" podStartSLOduration=1.184665517 podStartE2EDuration="1.184665517s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 09:27:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 09:27:27.183869371 +0000 UTC m=+175.147655353" watchObservedRunningTime="2026-07-27 09:27:27.184665517 +0000 UTC m=+175.148451478" Jul 27 09:27:27.305112 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:27.305068 2565 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-584dcb6b54-w7grd" Jul 27 09:27:27.305495 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:27.305477 2565 scope.go:117] "RemoveContainer" containerID="1491a240002f05c46d392fdf6a0e9ca99f3d5760eb568e9316bbe43b49bc5c3e" Jul 27 09:27:27.305677 ip-10-0-128-103 kubenswrapper[2565]: E0727 09:27:27.305657 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-584dcb6b54-w7grd_openshift-console-operator(0badf8bd-7f64-46ef-872c-39f59dd11d0d)\"" pod="openshift-console-operator/console-operator-584dcb6b54-w7grd" podUID="0badf8bd-7f64-46ef-872c-39f59dd11d0d" Jul 27 09:27:28.130637 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:28.130608 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-7lzb6" Jul 27 09:27:28.791018 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:28.790966 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/fb21c72b-5cea-49fa-9b28-d1362f405ab0-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-lgr4b\" (UID: \"fb21c72b-5cea-49fa-9b28-d1362f405ab0\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-lgr4b" Jul 27 09:27:28.793391 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:28.793369 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/fb21c72b-5cea-49fa-9b28-d1362f405ab0-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-lgr4b\" (UID: \"fb21c72b-5cea-49fa-9b28-d1362f405ab0\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-lgr4b" Jul 27 09:27:29.003256 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:29.003217 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-lgr4b" Jul 27 09:27:29.192066 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:29.192032 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-548b489cb8-lgr4b"] Jul 27 09:27:29.197550 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:27:29.197521 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfb21c72b_5cea_49fa_9b28_d1362f405ab0.slice/crio-3fe7eb6f8884a5d3eb7ff922725ee68e9bab3b7186f23837ff6319dccf109685 WatchSource:0}: Error finding container 3fe7eb6f8884a5d3eb7ff922725ee68e9bab3b7186f23837ff6319dccf109685: Status 404 returned error can't find the container with id 3fe7eb6f8884a5d3eb7ff922725ee68e9bab3b7186f23837ff6319dccf109685 Jul 27 09:27:30.052423 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:30.052382 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-584dcb6b54-w7grd" Jul 27 09:27:30.052930 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:30.052906 2565 scope.go:117] "RemoveContainer" containerID="1491a240002f05c46d392fdf6a0e9ca99f3d5760eb568e9316bbe43b49bc5c3e" Jul 27 09:27:30.053164 ip-10-0-128-103 kubenswrapper[2565]: E0727 09:27:30.053137 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-584dcb6b54-w7grd_openshift-console-operator(0badf8bd-7f64-46ef-872c-39f59dd11d0d)\"" pod="openshift-console-operator/console-operator-584dcb6b54-w7grd" podUID="0badf8bd-7f64-46ef-872c-39f59dd11d0d" Jul 27 09:27:30.173893 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:30.173853 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-lgr4b" event={"ID":"fb21c72b-5cea-49fa-9b28-d1362f405ab0","Type":"ContainerStarted","Data":"3fe7eb6f8884a5d3eb7ff922725ee68e9bab3b7186f23837ff6319dccf109685"} Jul 27 09:27:31.763511 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:31.763476 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-848w4_2c043d82-d300-41ba-b138-60a03d65ea11/konnectivity-agent/0.log" limit=1024 Jul 27 09:27:31.865360 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:31.865317 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-848w4_2c043d82-d300-41ba-b138-60a03d65ea11/konnectivity-agent/0.log" limit=1024 Jul 27 09:27:31.924953 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:31.924916 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-848w4_2c043d82-d300-41ba-b138-60a03d65ea11/konnectivity-agent/0.log" limit=1024 Jul 27 09:27:32.101537 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:32.101450 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-848w4_2c043d82-d300-41ba-b138-60a03d65ea11/konnectivity-agent/0.log" limit=1024 Jul 27 09:27:32.181855 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:32.181819 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-lgr4b" event={"ID":"fb21c72b-5cea-49fa-9b28-d1362f405ab0","Type":"ContainerStarted","Data":"1a1f8bacec073bbf6c819bc12f935a8a11644bce7b138852c0b84346f815f989"} Jul 27 09:27:32.200471 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:32.200419 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-lgr4b" podStartSLOduration=34.044479604 podStartE2EDuration="36.200403188s" totalImagesPullingTime="2.155923584s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 09:26:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 09:27:32.199829067 +0000 UTC m=+180.163615021" watchObservedRunningTime="2026-07-27 09:27:32.200403188 +0000 UTC m=+180.164189145" Jul 27 09:27:32.525048 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:32.525012 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a2f0dc08-339d-4aad-bae9-e13a8f594c70-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-xfvbm\" (UID: \"a2f0dc08-339d-4aad-bae9-e13a8f594c70\") " pod="openshift-insights/insights-runtime-extractor-xfvbm" Jul 27 09:27:32.527335 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:32.527309 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a2f0dc08-339d-4aad-bae9-e13a8f594c70-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-xfvbm\" (UID: \"a2f0dc08-339d-4aad-bae9-e13a8f594c70\") " pod="openshift-insights/insights-runtime-extractor-xfvbm" Jul 27 09:27:32.760777 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:32.760734 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-5ldlr\"" Jul 27 09:27:32.768154 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:32.768138 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-xfvbm" Jul 27 09:27:32.900531 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:32.900496 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-xfvbm"] Jul 27 09:27:32.903589 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:27:32.903561 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda2f0dc08_339d_4aad_bae9_e13a8f594c70.slice/crio-d6b0eefaab4084328dd6dc85d0db02a98fcea353bf01086d4df97661b286d0bc WatchSource:0}: Error finding container d6b0eefaab4084328dd6dc85d0db02a98fcea353bf01086d4df97661b286d0bc: Status 404 returned error can't find the container with id d6b0eefaab4084328dd6dc85d0db02a98fcea353bf01086d4df97661b286d0bc Jul 27 09:27:33.186700 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:33.186668 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-xfvbm" event={"ID":"a2f0dc08-339d-4aad-bae9-e13a8f594c70","Type":"ContainerStarted","Data":"d6b0eefaab4084328dd6dc85d0db02a98fcea353bf01086d4df97661b286d0bc"} Jul 27 09:27:34.908476 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:34.908436 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-848w4_2c043d82-d300-41ba-b138-60a03d65ea11/konnectivity-agent/0.log" limit=1024 Jul 27 09:27:34.951970 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:34.951936 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-84d69f794d-2v9b6"] Jul 27 09:27:34.954212 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:34.954187 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-84d69f794d-2v9b6" Jul 27 09:27:34.958059 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:34.957722 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\"" Jul 27 09:27:34.958059 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:34.957792 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\"" Jul 27 09:27:34.958059 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:34.957821 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-wx9zc\"" Jul 27 09:27:34.958059 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:34.957834 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 27 09:27:34.966606 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:34.966584 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-84d69f794d-2v9b6"] Jul 27 09:27:34.975175 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:34.975144 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-848w4_2c043d82-d300-41ba-b138-60a03d65ea11/konnectivity-agent/0.log" limit=1024 Jul 27 09:27:35.031170 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:35.031118 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-848w4_2c043d82-d300-41ba-b138-60a03d65ea11/konnectivity-agent/0.log" limit=1024 Jul 27 09:27:35.046068 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:35.046038 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/73c95e17-4ad0-41fb-a28f-23cc54ac8a7c-prometheus-operator-tls\") pod \"prometheus-operator-84d69f794d-2v9b6\" (UID: \"73c95e17-4ad0-41fb-a28f-23cc54ac8a7c\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-2v9b6" Jul 27 09:27:35.046068 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:35.046075 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/73c95e17-4ad0-41fb-a28f-23cc54ac8a7c-metrics-client-ca\") pod \"prometheus-operator-84d69f794d-2v9b6\" (UID: \"73c95e17-4ad0-41fb-a28f-23cc54ac8a7c\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-2v9b6" Jul 27 09:27:35.046286 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:35.046106 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/73c95e17-4ad0-41fb-a28f-23cc54ac8a7c-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-84d69f794d-2v9b6\" (UID: \"73c95e17-4ad0-41fb-a28f-23cc54ac8a7c\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-2v9b6" Jul 27 09:27:35.046286 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:35.046204 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gdp7r\" (UniqueName: \"kubernetes.io/projected/73c95e17-4ad0-41fb-a28f-23cc54ac8a7c-kube-api-access-gdp7r\") pod \"prometheus-operator-84d69f794d-2v9b6\" (UID: \"73c95e17-4ad0-41fb-a28f-23cc54ac8a7c\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-2v9b6" Jul 27 09:27:35.147299 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:35.147258 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/73c95e17-4ad0-41fb-a28f-23cc54ac8a7c-prometheus-operator-tls\") pod \"prometheus-operator-84d69f794d-2v9b6\" (UID: \"73c95e17-4ad0-41fb-a28f-23cc54ac8a7c\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-2v9b6" Jul 27 09:27:35.147299 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:35.147306 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/73c95e17-4ad0-41fb-a28f-23cc54ac8a7c-metrics-client-ca\") pod \"prometheus-operator-84d69f794d-2v9b6\" (UID: \"73c95e17-4ad0-41fb-a28f-23cc54ac8a7c\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-2v9b6" Jul 27 09:27:35.147545 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:35.147329 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/73c95e17-4ad0-41fb-a28f-23cc54ac8a7c-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-84d69f794d-2v9b6\" (UID: \"73c95e17-4ad0-41fb-a28f-23cc54ac8a7c\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-2v9b6" Jul 27 09:27:35.147545 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:35.147404 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gdp7r\" (UniqueName: \"kubernetes.io/projected/73c95e17-4ad0-41fb-a28f-23cc54ac8a7c-kube-api-access-gdp7r\") pod \"prometheus-operator-84d69f794d-2v9b6\" (UID: \"73c95e17-4ad0-41fb-a28f-23cc54ac8a7c\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-2v9b6" Jul 27 09:27:35.148139 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:35.148113 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/73c95e17-4ad0-41fb-a28f-23cc54ac8a7c-metrics-client-ca\") pod \"prometheus-operator-84d69f794d-2v9b6\" (UID: \"73c95e17-4ad0-41fb-a28f-23cc54ac8a7c\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-2v9b6" Jul 27 09:27:35.150094 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:35.150058 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/73c95e17-4ad0-41fb-a28f-23cc54ac8a7c-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-84d69f794d-2v9b6\" (UID: \"73c95e17-4ad0-41fb-a28f-23cc54ac8a7c\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-2v9b6" Jul 27 09:27:35.150339 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:35.150319 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/73c95e17-4ad0-41fb-a28f-23cc54ac8a7c-prometheus-operator-tls\") pod \"prometheus-operator-84d69f794d-2v9b6\" (UID: \"73c95e17-4ad0-41fb-a28f-23cc54ac8a7c\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-2v9b6" Jul 27 09:27:35.155712 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:35.155651 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gdp7r\" (UniqueName: \"kubernetes.io/projected/73c95e17-4ad0-41fb-a28f-23cc54ac8a7c-kube-api-access-gdp7r\") pod \"prometheus-operator-84d69f794d-2v9b6\" (UID: \"73c95e17-4ad0-41fb-a28f-23cc54ac8a7c\") " pod="openshift-monitoring/prometheus-operator-84d69f794d-2v9b6" Jul 27 09:27:35.266346 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:35.266265 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-84d69f794d-2v9b6" Jul 27 09:27:35.364147 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:35.364112 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-848w4_2c043d82-d300-41ba-b138-60a03d65ea11/konnectivity-agent/0.log" limit=1024 Jul 27 09:27:35.424969 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:35.424933 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-848w4_2c043d82-d300-41ba-b138-60a03d65ea11/konnectivity-agent/0.log" limit=1024 Jul 27 09:27:36.367811 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:36.367751 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-848w4_2c043d82-d300-41ba-b138-60a03d65ea11/konnectivity-agent/0.log" limit=1024 Jul 27 09:27:36.427575 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:36.427538 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-848w4_2c043d82-d300-41ba-b138-60a03d65ea11/konnectivity-agent/0.log" limit=1024 Jul 27 09:27:36.562569 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:36.562537 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-544bcd5f5d-5jm4h" Jul 27 09:27:37.808447 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:37.808420 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-84d69f794d-2v9b6"] Jul 27 09:27:37.811446 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:27:37.811413 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod73c95e17_4ad0_41fb_a28f_23cc54ac8a7c.slice/crio-5162ec913f4734bfd23a5dad3087322f861e71dec3b660f44eb11b2454e4817b WatchSource:0}: Error finding container 5162ec913f4734bfd23a5dad3087322f861e71dec3b660f44eb11b2454e4817b: Status 404 returned error can't find the container with id 5162ec913f4734bfd23a5dad3087322f861e71dec3b660f44eb11b2454e4817b Jul 27 09:27:37.832789 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:37.832752 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-848w4_2c043d82-d300-41ba-b138-60a03d65ea11/konnectivity-agent/0.log" limit=1024 Jul 27 09:27:37.934048 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:37.934009 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-848w4_2c043d82-d300-41ba-b138-60a03d65ea11/konnectivity-agent/0.log" limit=1024 Jul 27 09:27:38.204891 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:38.204850 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-xfvbm" event={"ID":"a2f0dc08-339d-4aad-bae9-e13a8f594c70","Type":"ContainerStarted","Data":"36ad54527fd45ac4b437737ba441c512e9184958f346502e9fe4d82bc119a0e7"} Jul 27 09:27:38.205874 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:38.205851 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-84d69f794d-2v9b6" event={"ID":"73c95e17-4ad0-41fb-a28f-23cc54ac8a7c","Type":"ContainerStarted","Data":"5162ec913f4734bfd23a5dad3087322f861e71dec3b660f44eb11b2454e4817b"} Jul 27 09:27:39.628116 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:39.628075 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-848w4_2c043d82-d300-41ba-b138-60a03d65ea11/konnectivity-agent/0.log" limit=1024 Jul 27 09:27:40.216995 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:40.216956 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-84d69f794d-2v9b6" event={"ID":"73c95e17-4ad0-41fb-a28f-23cc54ac8a7c","Type":"ContainerStarted","Data":"4b1d7abd94bfbcd56f806ff3bc80b2050dd575a34b37faf5fb819cf6d80803ad"} Jul 27 09:27:40.217184 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:40.217029 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-84d69f794d-2v9b6" event={"ID":"73c95e17-4ad0-41fb-a28f-23cc54ac8a7c","Type":"ContainerStarted","Data":"9b3a8bb072f43c57d406ce4a1ce67dccfc3ab412142c85ff10fb02a3d8efe7cd"} Jul 27 09:27:40.234124 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:40.234074 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-84d69f794d-2v9b6" podStartSLOduration=4.725761295 podStartE2EDuration="6.234054891s" totalImagesPullingTime="1.508293596s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 09:27:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 09:27:40.23283028 +0000 UTC m=+188.196616259" watchObservedRunningTime="2026-07-27 09:27:40.234054891 +0000 UTC m=+188.197840850" Jul 27 09:27:40.932056 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:40.932004 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-848w4_2c043d82-d300-41ba-b138-60a03d65ea11/konnectivity-agent/0.log" limit=1024 Jul 27 09:27:42.109571 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:42.109505 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-848w4_2c043d82-d300-41ba-b138-60a03d65ea11/konnectivity-agent/0.log" limit=1024 Jul 27 09:27:42.228578 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:42.228541 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-xfvbm" event={"ID":"a2f0dc08-339d-4aad-bae9-e13a8f594c70","Type":"ContainerStarted","Data":"9108cd072f53f164644de08e275cabf7207c165c415b3d77404d383a75bf22d6"} Jul 27 09:27:42.329440 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:42.329401 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-848w4_2c043d82-d300-41ba-b138-60a03d65ea11/konnectivity-agent/0.log" limit=1024 Jul 27 09:27:42.399991 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:42.399952 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-vwsrv"] Jul 27 09:27:42.403646 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:42.403613 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-vwsrv" Jul 27 09:27:42.407463 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:42.407282 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 27 09:27:42.407463 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:42.407331 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 27 09:27:42.407666 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:42.407583 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 27 09:27:42.407666 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:42.407655 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-fp6tz\"" Jul 27 09:27:42.428778 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:42.428748 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-56d9f944b4-mhq2g"] Jul 27 09:27:42.432069 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:42.432028 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-mhq2g" Jul 27 09:27:42.437116 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:42.437031 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\"" Jul 27 09:27:42.437116 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:42.437104 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-vnmfp\"" Jul 27 09:27:42.437303 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:42.437193 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\"" Jul 27 09:27:42.437303 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:42.437292 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\"" Jul 27 09:27:42.446617 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:42.446589 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-56d9f944b4-mhq2g"] Jul 27 09:27:42.447200 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:42.447171 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-848w4_2c043d82-d300-41ba-b138-60a03d65ea11/konnectivity-agent/0.log" limit=1024 Jul 27 09:27:42.509165 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:42.509130 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/45db6a76-b0a6-475e-a2c2-2e0bf01f8e17-kube-state-metrics-tls\") pod \"kube-state-metrics-56d9f944b4-mhq2g\" (UID: \"45db6a76-b0a6-475e-a2c2-2e0bf01f8e17\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-mhq2g" Jul 27 09:27:42.509420 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:42.509173 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/45db6a76-b0a6-475e-a2c2-2e0bf01f8e17-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56d9f944b4-mhq2g\" (UID: \"45db6a76-b0a6-475e-a2c2-2e0bf01f8e17\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-mhq2g" Jul 27 09:27:42.509420 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:42.509208 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/45db6a76-b0a6-475e-a2c2-2e0bf01f8e17-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56d9f944b4-mhq2g\" (UID: \"45db6a76-b0a6-475e-a2c2-2e0bf01f8e17\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-mhq2g" Jul 27 09:27:42.509420 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:42.509300 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d9769977-82c3-4556-a9f3-7a6ec82bdb75-sys\") pod \"node-exporter-vwsrv\" (UID: \"d9769977-82c3-4556-a9f3-7a6ec82bdb75\") " pod="openshift-monitoring/node-exporter-vwsrv" Jul 27 09:27:42.509420 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:42.509350 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2w6h6\" (UniqueName: \"kubernetes.io/projected/45db6a76-b0a6-475e-a2c2-2e0bf01f8e17-kube-api-access-2w6h6\") pod \"kube-state-metrics-56d9f944b4-mhq2g\" (UID: \"45db6a76-b0a6-475e-a2c2-2e0bf01f8e17\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-mhq2g" Jul 27 09:27:42.509420 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:42.509401 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d9769977-82c3-4556-a9f3-7a6ec82bdb75-metrics-client-ca\") pod \"node-exporter-vwsrv\" (UID: \"d9769977-82c3-4556-a9f3-7a6ec82bdb75\") " pod="openshift-monitoring/node-exporter-vwsrv" Jul 27 09:27:42.509687 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:42.509470 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d9769977-82c3-4556-a9f3-7a6ec82bdb75-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-vwsrv\" (UID: \"d9769977-82c3-4556-a9f3-7a6ec82bdb75\") " pod="openshift-monitoring/node-exporter-vwsrv" Jul 27 09:27:42.509687 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:42.509507 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/d9769977-82c3-4556-a9f3-7a6ec82bdb75-node-exporter-wtmp\") pod \"node-exporter-vwsrv\" (UID: \"d9769977-82c3-4556-a9f3-7a6ec82bdb75\") " pod="openshift-monitoring/node-exporter-vwsrv" Jul 27 09:27:42.509687 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:42.509561 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/d9769977-82c3-4556-a9f3-7a6ec82bdb75-node-exporter-tls\") pod \"node-exporter-vwsrv\" (UID: \"d9769977-82c3-4556-a9f3-7a6ec82bdb75\") " pod="openshift-monitoring/node-exporter-vwsrv" Jul 27 09:27:42.509687 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:42.509600 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/45db6a76-b0a6-475e-a2c2-2e0bf01f8e17-metrics-client-ca\") pod \"kube-state-metrics-56d9f944b4-mhq2g\" (UID: \"45db6a76-b0a6-475e-a2c2-2e0bf01f8e17\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-mhq2g" Jul 27 09:27:42.509687 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:42.509632 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/45db6a76-b0a6-475e-a2c2-2e0bf01f8e17-volume-directive-shadow\") pod \"kube-state-metrics-56d9f944b4-mhq2g\" (UID: \"45db6a76-b0a6-475e-a2c2-2e0bf01f8e17\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-mhq2g" Jul 27 09:27:42.509889 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:42.509746 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/d9769977-82c3-4556-a9f3-7a6ec82bdb75-node-exporter-accelerators-collector-config\") pod \"node-exporter-vwsrv\" (UID: \"d9769977-82c3-4556-a9f3-7a6ec82bdb75\") " pod="openshift-monitoring/node-exporter-vwsrv" Jul 27 09:27:42.509889 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:42.509785 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/d9769977-82c3-4556-a9f3-7a6ec82bdb75-root\") pod \"node-exporter-vwsrv\" (UID: \"d9769977-82c3-4556-a9f3-7a6ec82bdb75\") " pod="openshift-monitoring/node-exporter-vwsrv" Jul 27 09:27:42.509889 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:42.509830 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/d9769977-82c3-4556-a9f3-7a6ec82bdb75-node-exporter-textfile\") pod \"node-exporter-vwsrv\" (UID: \"d9769977-82c3-4556-a9f3-7a6ec82bdb75\") " pod="openshift-monitoring/node-exporter-vwsrv" Jul 27 09:27:42.509889 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:42.509860 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9xlhh\" (UniqueName: \"kubernetes.io/projected/d9769977-82c3-4556-a9f3-7a6ec82bdb75-kube-api-access-9xlhh\") pod \"node-exporter-vwsrv\" (UID: \"d9769977-82c3-4556-a9f3-7a6ec82bdb75\") " pod="openshift-monitoring/node-exporter-vwsrv" Jul 27 09:27:42.553661 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:42.553622 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-848w4_2c043d82-d300-41ba-b138-60a03d65ea11/konnectivity-agent/0.log" limit=1024 Jul 27 09:27:42.610722 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:42.610689 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/45db6a76-b0a6-475e-a2c2-2e0bf01f8e17-kube-state-metrics-tls\") pod \"kube-state-metrics-56d9f944b4-mhq2g\" (UID: \"45db6a76-b0a6-475e-a2c2-2e0bf01f8e17\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-mhq2g" Jul 27 09:27:42.610918 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:42.610738 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/45db6a76-b0a6-475e-a2c2-2e0bf01f8e17-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56d9f944b4-mhq2g\" (UID: \"45db6a76-b0a6-475e-a2c2-2e0bf01f8e17\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-mhq2g" Jul 27 09:27:42.610918 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:42.610784 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/45db6a76-b0a6-475e-a2c2-2e0bf01f8e17-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56d9f944b4-mhq2g\" (UID: \"45db6a76-b0a6-475e-a2c2-2e0bf01f8e17\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-mhq2g" Jul 27 09:27:42.610918 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:42.610833 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d9769977-82c3-4556-a9f3-7a6ec82bdb75-sys\") pod \"node-exporter-vwsrv\" (UID: \"d9769977-82c3-4556-a9f3-7a6ec82bdb75\") " pod="openshift-monitoring/node-exporter-vwsrv" Jul 27 09:27:42.610918 ip-10-0-128-103 kubenswrapper[2565]: E0727 09:27:42.610889 2565 secret.go:189] Couldn't get secret openshift-monitoring/kube-state-metrics-tls: secret "kube-state-metrics-tls" not found Jul 27 09:27:42.611119 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:42.610901 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d9769977-82c3-4556-a9f3-7a6ec82bdb75-sys\") pod \"node-exporter-vwsrv\" (UID: \"d9769977-82c3-4556-a9f3-7a6ec82bdb75\") " pod="openshift-monitoring/node-exporter-vwsrv" Jul 27 09:27:42.611119 ip-10-0-128-103 kubenswrapper[2565]: E0727 09:27:42.610970 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/45db6a76-b0a6-475e-a2c2-2e0bf01f8e17-kube-state-metrics-tls podName:45db6a76-b0a6-475e-a2c2-2e0bf01f8e17 nodeName:}" failed. No retries permitted until 2026-07-27 09:27:43.110947145 +0000 UTC m=+191.074733100 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-state-metrics-tls" (UniqueName: "kubernetes.io/secret/45db6a76-b0a6-475e-a2c2-2e0bf01f8e17-kube-state-metrics-tls") pod "kube-state-metrics-56d9f944b4-mhq2g" (UID: "45db6a76-b0a6-475e-a2c2-2e0bf01f8e17") : secret "kube-state-metrics-tls" not found Jul 27 09:27:42.611119 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:42.611014 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2w6h6\" (UniqueName: \"kubernetes.io/projected/45db6a76-b0a6-475e-a2c2-2e0bf01f8e17-kube-api-access-2w6h6\") pod \"kube-state-metrics-56d9f944b4-mhq2g\" (UID: \"45db6a76-b0a6-475e-a2c2-2e0bf01f8e17\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-mhq2g" Jul 27 09:27:42.611119 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:42.611063 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d9769977-82c3-4556-a9f3-7a6ec82bdb75-metrics-client-ca\") pod \"node-exporter-vwsrv\" (UID: \"d9769977-82c3-4556-a9f3-7a6ec82bdb75\") " pod="openshift-monitoring/node-exporter-vwsrv" Jul 27 09:27:42.611378 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:42.611124 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d9769977-82c3-4556-a9f3-7a6ec82bdb75-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-vwsrv\" (UID: \"d9769977-82c3-4556-a9f3-7a6ec82bdb75\") " pod="openshift-monitoring/node-exporter-vwsrv" Jul 27 09:27:42.611378 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:42.611175 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/d9769977-82c3-4556-a9f3-7a6ec82bdb75-node-exporter-wtmp\") pod \"node-exporter-vwsrv\" (UID: \"d9769977-82c3-4556-a9f3-7a6ec82bdb75\") " pod="openshift-monitoring/node-exporter-vwsrv" Jul 27 09:27:42.611378 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:42.611203 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/d9769977-82c3-4556-a9f3-7a6ec82bdb75-node-exporter-tls\") pod \"node-exporter-vwsrv\" (UID: \"d9769977-82c3-4556-a9f3-7a6ec82bdb75\") " pod="openshift-monitoring/node-exporter-vwsrv" Jul 27 09:27:42.611378 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:42.611239 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/45db6a76-b0a6-475e-a2c2-2e0bf01f8e17-metrics-client-ca\") pod \"kube-state-metrics-56d9f944b4-mhq2g\" (UID: \"45db6a76-b0a6-475e-a2c2-2e0bf01f8e17\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-mhq2g" Jul 27 09:27:42.611378 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:42.611269 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/45db6a76-b0a6-475e-a2c2-2e0bf01f8e17-volume-directive-shadow\") pod \"kube-state-metrics-56d9f944b4-mhq2g\" (UID: \"45db6a76-b0a6-475e-a2c2-2e0bf01f8e17\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-mhq2g" Jul 27 09:27:42.611378 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:42.611357 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/d9769977-82c3-4556-a9f3-7a6ec82bdb75-node-exporter-accelerators-collector-config\") pod \"node-exporter-vwsrv\" (UID: \"d9769977-82c3-4556-a9f3-7a6ec82bdb75\") " pod="openshift-monitoring/node-exporter-vwsrv" Jul 27 09:27:42.611754 ip-10-0-128-103 kubenswrapper[2565]: E0727 09:27:42.611386 2565 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Jul 27 09:27:42.611754 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:42.611430 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/d9769977-82c3-4556-a9f3-7a6ec82bdb75-root\") pod \"node-exporter-vwsrv\" (UID: \"d9769977-82c3-4556-a9f3-7a6ec82bdb75\") " pod="openshift-monitoring/node-exporter-vwsrv" Jul 27 09:27:42.611754 ip-10-0-128-103 kubenswrapper[2565]: E0727 09:27:42.611445 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d9769977-82c3-4556-a9f3-7a6ec82bdb75-node-exporter-tls podName:d9769977-82c3-4556-a9f3-7a6ec82bdb75 nodeName:}" failed. No retries permitted until 2026-07-27 09:27:43.111425354 +0000 UTC m=+191.075211297 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/d9769977-82c3-4556-a9f3-7a6ec82bdb75-node-exporter-tls") pod "node-exporter-vwsrv" (UID: "d9769977-82c3-4556-a9f3-7a6ec82bdb75") : secret "node-exporter-tls" not found Jul 27 09:27:42.611928 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:42.611755 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/45db6a76-b0a6-475e-a2c2-2e0bf01f8e17-volume-directive-shadow\") pod \"kube-state-metrics-56d9f944b4-mhq2g\" (UID: \"45db6a76-b0a6-475e-a2c2-2e0bf01f8e17\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-mhq2g" Jul 27 09:27:42.611928 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:42.611890 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/d9769977-82c3-4556-a9f3-7a6ec82bdb75-node-exporter-wtmp\") pod \"node-exporter-vwsrv\" (UID: \"d9769977-82c3-4556-a9f3-7a6ec82bdb75\") " pod="openshift-monitoring/node-exporter-vwsrv" Jul 27 09:27:42.611928 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:42.611387 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/d9769977-82c3-4556-a9f3-7a6ec82bdb75-root\") pod \"node-exporter-vwsrv\" (UID: \"d9769977-82c3-4556-a9f3-7a6ec82bdb75\") " pod="openshift-monitoring/node-exporter-vwsrv" Jul 27 09:27:42.611928 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:42.611888 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/45db6a76-b0a6-475e-a2c2-2e0bf01f8e17-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56d9f944b4-mhq2g\" (UID: \"45db6a76-b0a6-475e-a2c2-2e0bf01f8e17\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-mhq2g" Jul 27 09:27:42.612184 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:42.611955 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/d9769977-82c3-4556-a9f3-7a6ec82bdb75-node-exporter-textfile\") pod \"node-exporter-vwsrv\" (UID: \"d9769977-82c3-4556-a9f3-7a6ec82bdb75\") " pod="openshift-monitoring/node-exporter-vwsrv" Jul 27 09:27:42.612184 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:42.611987 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9xlhh\" (UniqueName: \"kubernetes.io/projected/d9769977-82c3-4556-a9f3-7a6ec82bdb75-kube-api-access-9xlhh\") pod \"node-exporter-vwsrv\" (UID: \"d9769977-82c3-4556-a9f3-7a6ec82bdb75\") " pod="openshift-monitoring/node-exporter-vwsrv" Jul 27 09:27:42.612184 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:42.612049 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/45db6a76-b0a6-475e-a2c2-2e0bf01f8e17-metrics-client-ca\") pod \"kube-state-metrics-56d9f944b4-mhq2g\" (UID: \"45db6a76-b0a6-475e-a2c2-2e0bf01f8e17\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-mhq2g" Jul 27 09:27:42.612343 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:42.612224 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/d9769977-82c3-4556-a9f3-7a6ec82bdb75-node-exporter-textfile\") pod \"node-exporter-vwsrv\" (UID: \"d9769977-82c3-4556-a9f3-7a6ec82bdb75\") " pod="openshift-monitoring/node-exporter-vwsrv" Jul 27 09:27:42.612343 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:42.612273 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/d9769977-82c3-4556-a9f3-7a6ec82bdb75-node-exporter-accelerators-collector-config\") pod \"node-exporter-vwsrv\" (UID: \"d9769977-82c3-4556-a9f3-7a6ec82bdb75\") " pod="openshift-monitoring/node-exporter-vwsrv" Jul 27 09:27:42.612441 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:42.612400 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d9769977-82c3-4556-a9f3-7a6ec82bdb75-metrics-client-ca\") pod \"node-exporter-vwsrv\" (UID: \"d9769977-82c3-4556-a9f3-7a6ec82bdb75\") " pod="openshift-monitoring/node-exporter-vwsrv" Jul 27 09:27:42.613861 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:42.613759 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d9769977-82c3-4556-a9f3-7a6ec82bdb75-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-vwsrv\" (UID: \"d9769977-82c3-4556-a9f3-7a6ec82bdb75\") " pod="openshift-monitoring/node-exporter-vwsrv" Jul 27 09:27:42.614389 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:42.614365 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/45db6a76-b0a6-475e-a2c2-2e0bf01f8e17-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56d9f944b4-mhq2g\" (UID: \"45db6a76-b0a6-475e-a2c2-2e0bf01f8e17\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-mhq2g" Jul 27 09:27:42.621402 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:42.621353 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2w6h6\" (UniqueName: \"kubernetes.io/projected/45db6a76-b0a6-475e-a2c2-2e0bf01f8e17-kube-api-access-2w6h6\") pod \"kube-state-metrics-56d9f944b4-mhq2g\" (UID: \"45db6a76-b0a6-475e-a2c2-2e0bf01f8e17\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-mhq2g" Jul 27 09:27:42.622976 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:42.622949 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9xlhh\" (UniqueName: \"kubernetes.io/projected/d9769977-82c3-4556-a9f3-7a6ec82bdb75-kube-api-access-9xlhh\") pod \"node-exporter-vwsrv\" (UID: \"d9769977-82c3-4556-a9f3-7a6ec82bdb75\") " pod="openshift-monitoring/node-exporter-vwsrv" Jul 27 09:27:42.655072 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:42.654983 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-848w4_2c043d82-d300-41ba-b138-60a03d65ea11/konnectivity-agent/0.log" limit=1024 Jul 27 09:27:42.776277 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:42.776238 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-848w4_2c043d82-d300-41ba-b138-60a03d65ea11/konnectivity-agent/0.log" limit=1024 Jul 27 09:27:42.851668 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:42.851630 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-848w4_2c043d82-d300-41ba-b138-60a03d65ea11/konnectivity-agent/0.log" limit=1024 Jul 27 09:27:42.921316 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:42.921224 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-848w4_2c043d82-d300-41ba-b138-60a03d65ea11/konnectivity-agent/0.log" limit=1024 Jul 27 09:27:43.117089 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:43.117048 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/d9769977-82c3-4556-a9f3-7a6ec82bdb75-node-exporter-tls\") pod \"node-exporter-vwsrv\" (UID: \"d9769977-82c3-4556-a9f3-7a6ec82bdb75\") " pod="openshift-monitoring/node-exporter-vwsrv" Jul 27 09:27:43.117564 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:43.117141 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/45db6a76-b0a6-475e-a2c2-2e0bf01f8e17-kube-state-metrics-tls\") pod \"kube-state-metrics-56d9f944b4-mhq2g\" (UID: \"45db6a76-b0a6-475e-a2c2-2e0bf01f8e17\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-mhq2g" Jul 27 09:27:43.117564 ip-10-0-128-103 kubenswrapper[2565]: E0727 09:27:43.117209 2565 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Jul 27 09:27:43.117564 ip-10-0-128-103 kubenswrapper[2565]: E0727 09:27:43.117297 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d9769977-82c3-4556-a9f3-7a6ec82bdb75-node-exporter-tls podName:d9769977-82c3-4556-a9f3-7a6ec82bdb75 nodeName:}" failed. No retries permitted until 2026-07-27 09:27:44.117270482 +0000 UTC m=+192.081056422 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/d9769977-82c3-4556-a9f3-7a6ec82bdb75-node-exporter-tls") pod "node-exporter-vwsrv" (UID: "d9769977-82c3-4556-a9f3-7a6ec82bdb75") : secret "node-exporter-tls" not found Jul 27 09:27:43.120106 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:43.120061 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/45db6a76-b0a6-475e-a2c2-2e0bf01f8e17-kube-state-metrics-tls\") pod \"kube-state-metrics-56d9f944b4-mhq2g\" (UID: \"45db6a76-b0a6-475e-a2c2-2e0bf01f8e17\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-mhq2g" Jul 27 09:27:43.272761 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:43.272677 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-848w4_2c043d82-d300-41ba-b138-60a03d65ea11/konnectivity-agent/0.log" limit=1024 Jul 27 09:27:43.343116 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:43.343082 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-mhq2g" Jul 27 09:27:43.370219 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:43.370179 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-848w4_2c043d82-d300-41ba-b138-60a03d65ea11/konnectivity-agent/0.log" limit=1024 Jul 27 09:27:43.465683 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:43.465645 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-848w4_2c043d82-d300-41ba-b138-60a03d65ea11/konnectivity-agent/0.log" limit=1024 Jul 27 09:27:43.678784 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:43.678706 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-848w4_2c043d82-d300-41ba-b138-60a03d65ea11/konnectivity-agent/0.log" limit=1024 Jul 27 09:27:44.126833 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:44.126801 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/d9769977-82c3-4556-a9f3-7a6ec82bdb75-node-exporter-tls\") pod \"node-exporter-vwsrv\" (UID: \"d9769977-82c3-4556-a9f3-7a6ec82bdb75\") " pod="openshift-monitoring/node-exporter-vwsrv" Jul 27 09:27:44.129476 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:44.129449 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/d9769977-82c3-4556-a9f3-7a6ec82bdb75-node-exporter-tls\") pod \"node-exporter-vwsrv\" (UID: \"d9769977-82c3-4556-a9f3-7a6ec82bdb75\") " pod="openshift-monitoring/node-exporter-vwsrv" Jul 27 09:27:44.214526 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:44.214484 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-vwsrv" Jul 27 09:27:44.558663 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:44.558577 2565 scope.go:117] "RemoveContainer" containerID="1491a240002f05c46d392fdf6a0e9ca99f3d5760eb568e9316bbe43b49bc5c3e" Jul 27 09:27:44.558856 ip-10-0-128-103 kubenswrapper[2565]: E0727 09:27:44.558808 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-584dcb6b54-w7grd_openshift-console-operator(0badf8bd-7f64-46ef-872c-39f59dd11d0d)\"" pod="openshift-console-operator/console-operator-584dcb6b54-w7grd" podUID="0badf8bd-7f64-46ef-872c-39f59dd11d0d" Jul 27 09:27:44.685671 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:44.685618 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-848w4_2c043d82-d300-41ba-b138-60a03d65ea11/konnectivity-agent/0.log" limit=1024 Jul 27 09:27:44.780585 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:44.780548 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-848w4_2c043d82-d300-41ba-b138-60a03d65ea11/konnectivity-agent/0.log" limit=1024 Jul 27 09:27:45.029659 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:27:45.029624 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd9769977_82c3_4556_a9f3_7a6ec82bdb75.slice/crio-ab0566abdfed4aee69b4aa7e20f38987f59b7ad959e4ca714bbf0e9ced53d1df WatchSource:0}: Error finding container ab0566abdfed4aee69b4aa7e20f38987f59b7ad959e4ca714bbf0e9ced53d1df: Status 404 returned error can't find the container with id ab0566abdfed4aee69b4aa7e20f38987f59b7ad959e4ca714bbf0e9ced53d1df Jul 27 09:27:45.162241 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:45.162092 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-56d9f944b4-mhq2g"] Jul 27 09:27:45.164838 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:27:45.164814 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod45db6a76_b0a6_475e_a2c2_2e0bf01f8e17.slice/crio-807741bda643f0b140f2c1059df25c694e14501fdb934e3b9dc33b37bc4a8343 WatchSource:0}: Error finding container 807741bda643f0b140f2c1059df25c694e14501fdb934e3b9dc33b37bc4a8343: Status 404 returned error can't find the container with id 807741bda643f0b140f2c1059df25c694e14501fdb934e3b9dc33b37bc4a8343 Jul 27 09:27:45.240825 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:45.240793 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-vwsrv" event={"ID":"d9769977-82c3-4556-a9f3-7a6ec82bdb75","Type":"ContainerStarted","Data":"ab0566abdfed4aee69b4aa7e20f38987f59b7ad959e4ca714bbf0e9ced53d1df"} Jul 27 09:27:45.242817 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:45.242793 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-xfvbm" event={"ID":"a2f0dc08-339d-4aad-bae9-e13a8f594c70","Type":"ContainerStarted","Data":"3c7732d36e581dabfb8bbf6c662e75cb9b625f3c880db21815f2d0e0f8614a74"} Jul 27 09:27:45.243871 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:45.243849 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-mhq2g" event={"ID":"45db6a76-b0a6-475e-a2c2-2e0bf01f8e17","Type":"ContainerStarted","Data":"807741bda643f0b140f2c1059df25c694e14501fdb934e3b9dc33b37bc4a8343"} Jul 27 09:27:45.263009 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:45.262958 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-xfvbm" podStartSLOduration=33.230603575 podStartE2EDuration="45.26294021s" totalImagesPullingTime="12.032336635s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-27 09:27:00 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 09:27:45.261903071 +0000 UTC m=+193.225689051" watchObservedRunningTime="2026-07-27 09:27:45.26294021 +0000 UTC m=+193.226726167" Jul 27 09:27:45.328308 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:45.328215 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-848w4_2c043d82-d300-41ba-b138-60a03d65ea11/konnectivity-agent/0.log" limit=1024 Jul 27 09:27:45.447262 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:45.447225 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-848w4_2c043d82-d300-41ba-b138-60a03d65ea11/konnectivity-agent/0.log" limit=1024 Jul 27 09:27:45.544347 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:45.544311 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-848w4_2c043d82-d300-41ba-b138-60a03d65ea11/konnectivity-agent/0.log" limit=1024 Jul 27 09:27:45.768923 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:45.768834 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-848w4_2c043d82-d300-41ba-b138-60a03d65ea11/konnectivity-agent/0.log" limit=1024 Jul 27 09:27:46.249928 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:46.249888 2565 generic.go:356] "Generic (PLEG): container finished" podID="d9769977-82c3-4556-a9f3-7a6ec82bdb75" containerID="8594a97d1c2247099e3d7b05c2e3f55dff0e933d93073ead4ad5b103246ff271" exitCode=0 Jul 27 09:27:46.250346 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:46.249978 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-vwsrv" event={"ID":"d9769977-82c3-4556-a9f3-7a6ec82bdb75","Type":"ContainerDied","Data":"8594a97d1c2247099e3d7b05c2e3f55dff0e933d93073ead4ad5b103246ff271"} Jul 27 09:27:46.704926 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:46.704854 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-848f65bd86-bb84d"] Jul 27 09:27:46.709078 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:46.709057 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-848f65bd86-bb84d" Jul 27 09:27:46.711622 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:46.711596 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\"" Jul 27 09:27:46.711781 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:46.711746 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\"" Jul 27 09:27:46.711916 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:46.711897 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-faf93g3ga1rdp\"" Jul 27 09:27:46.711997 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:46.711982 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\"" Jul 27 09:27:46.711997 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:46.711994 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jul 27 09:27:46.712216 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:46.712199 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-l84jj\"" Jul 27 09:27:46.718338 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:46.717997 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-848f65bd86-bb84d"] Jul 27 09:27:46.851055 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:46.851009 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/0aa1aaa9-b87e-4497-9ae2-11b00f96bf39-secret-metrics-server-tls\") pod \"metrics-server-848f65bd86-bb84d\" (UID: \"0aa1aaa9-b87e-4497-9ae2-11b00f96bf39\") " pod="openshift-monitoring/metrics-server-848f65bd86-bb84d" Jul 27 09:27:46.851212 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:46.851062 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/0aa1aaa9-b87e-4497-9ae2-11b00f96bf39-secret-metrics-server-client-certs\") pod \"metrics-server-848f65bd86-bb84d\" (UID: \"0aa1aaa9-b87e-4497-9ae2-11b00f96bf39\") " pod="openshift-monitoring/metrics-server-848f65bd86-bb84d" Jul 27 09:27:46.851212 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:46.851137 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0aa1aaa9-b87e-4497-9ae2-11b00f96bf39-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-848f65bd86-bb84d\" (UID: \"0aa1aaa9-b87e-4497-9ae2-11b00f96bf39\") " pod="openshift-monitoring/metrics-server-848f65bd86-bb84d" Jul 27 09:27:46.851280 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:46.851219 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/0aa1aaa9-b87e-4497-9ae2-11b00f96bf39-client-ca-bundle\") pod \"metrics-server-848f65bd86-bb84d\" (UID: \"0aa1aaa9-b87e-4497-9ae2-11b00f96bf39\") " pod="openshift-monitoring/metrics-server-848f65bd86-bb84d" Jul 27 09:27:46.851340 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:46.851289 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/0aa1aaa9-b87e-4497-9ae2-11b00f96bf39-audit-log\") pod \"metrics-server-848f65bd86-bb84d\" (UID: \"0aa1aaa9-b87e-4497-9ae2-11b00f96bf39\") " pod="openshift-monitoring/metrics-server-848f65bd86-bb84d" Jul 27 09:27:46.851340 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:46.851318 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/0aa1aaa9-b87e-4497-9ae2-11b00f96bf39-metrics-server-audit-profiles\") pod \"metrics-server-848f65bd86-bb84d\" (UID: \"0aa1aaa9-b87e-4497-9ae2-11b00f96bf39\") " pod="openshift-monitoring/metrics-server-848f65bd86-bb84d" Jul 27 09:27:46.851414 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:46.851341 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qmshv\" (UniqueName: \"kubernetes.io/projected/0aa1aaa9-b87e-4497-9ae2-11b00f96bf39-kube-api-access-qmshv\") pod \"metrics-server-848f65bd86-bb84d\" (UID: \"0aa1aaa9-b87e-4497-9ae2-11b00f96bf39\") " pod="openshift-monitoring/metrics-server-848f65bd86-bb84d" Jul 27 09:27:46.918059 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:46.918020 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-848w4_2c043d82-d300-41ba-b138-60a03d65ea11/konnectivity-agent/0.log" limit=1024 Jul 27 09:27:46.952116 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:46.952075 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/0aa1aaa9-b87e-4497-9ae2-11b00f96bf39-client-ca-bundle\") pod \"metrics-server-848f65bd86-bb84d\" (UID: \"0aa1aaa9-b87e-4497-9ae2-11b00f96bf39\") " pod="openshift-monitoring/metrics-server-848f65bd86-bb84d" Jul 27 09:27:46.952303 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:46.952147 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/0aa1aaa9-b87e-4497-9ae2-11b00f96bf39-audit-log\") pod \"metrics-server-848f65bd86-bb84d\" (UID: \"0aa1aaa9-b87e-4497-9ae2-11b00f96bf39\") " pod="openshift-monitoring/metrics-server-848f65bd86-bb84d" Jul 27 09:27:46.952303 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:46.952172 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/0aa1aaa9-b87e-4497-9ae2-11b00f96bf39-metrics-server-audit-profiles\") pod \"metrics-server-848f65bd86-bb84d\" (UID: \"0aa1aaa9-b87e-4497-9ae2-11b00f96bf39\") " pod="openshift-monitoring/metrics-server-848f65bd86-bb84d" Jul 27 09:27:46.952303 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:46.952197 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qmshv\" (UniqueName: \"kubernetes.io/projected/0aa1aaa9-b87e-4497-9ae2-11b00f96bf39-kube-api-access-qmshv\") pod \"metrics-server-848f65bd86-bb84d\" (UID: \"0aa1aaa9-b87e-4497-9ae2-11b00f96bf39\") " pod="openshift-monitoring/metrics-server-848f65bd86-bb84d" Jul 27 09:27:46.952303 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:46.952232 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/0aa1aaa9-b87e-4497-9ae2-11b00f96bf39-secret-metrics-server-tls\") pod \"metrics-server-848f65bd86-bb84d\" (UID: \"0aa1aaa9-b87e-4497-9ae2-11b00f96bf39\") " pod="openshift-monitoring/metrics-server-848f65bd86-bb84d" Jul 27 09:27:46.952303 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:46.952260 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/0aa1aaa9-b87e-4497-9ae2-11b00f96bf39-secret-metrics-server-client-certs\") pod \"metrics-server-848f65bd86-bb84d\" (UID: \"0aa1aaa9-b87e-4497-9ae2-11b00f96bf39\") " pod="openshift-monitoring/metrics-server-848f65bd86-bb84d" Jul 27 09:27:46.952303 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:46.952297 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0aa1aaa9-b87e-4497-9ae2-11b00f96bf39-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-848f65bd86-bb84d\" (UID: \"0aa1aaa9-b87e-4497-9ae2-11b00f96bf39\") " pod="openshift-monitoring/metrics-server-848f65bd86-bb84d" Jul 27 09:27:46.953103 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:46.953080 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/0aa1aaa9-b87e-4497-9ae2-11b00f96bf39-audit-log\") pod \"metrics-server-848f65bd86-bb84d\" (UID: \"0aa1aaa9-b87e-4497-9ae2-11b00f96bf39\") " pod="openshift-monitoring/metrics-server-848f65bd86-bb84d" Jul 27 09:27:46.953411 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:46.953384 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0aa1aaa9-b87e-4497-9ae2-11b00f96bf39-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-848f65bd86-bb84d\" (UID: \"0aa1aaa9-b87e-4497-9ae2-11b00f96bf39\") " pod="openshift-monitoring/metrics-server-848f65bd86-bb84d" Jul 27 09:27:46.953649 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:46.953625 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/0aa1aaa9-b87e-4497-9ae2-11b00f96bf39-metrics-server-audit-profiles\") pod \"metrics-server-848f65bd86-bb84d\" (UID: \"0aa1aaa9-b87e-4497-9ae2-11b00f96bf39\") " pod="openshift-monitoring/metrics-server-848f65bd86-bb84d" Jul 27 09:27:46.954916 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:46.954892 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/0aa1aaa9-b87e-4497-9ae2-11b00f96bf39-secret-metrics-server-tls\") pod \"metrics-server-848f65bd86-bb84d\" (UID: \"0aa1aaa9-b87e-4497-9ae2-11b00f96bf39\") " pod="openshift-monitoring/metrics-server-848f65bd86-bb84d" Jul 27 09:27:46.955203 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:46.955162 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/0aa1aaa9-b87e-4497-9ae2-11b00f96bf39-client-ca-bundle\") pod \"metrics-server-848f65bd86-bb84d\" (UID: \"0aa1aaa9-b87e-4497-9ae2-11b00f96bf39\") " pod="openshift-monitoring/metrics-server-848f65bd86-bb84d" Jul 27 09:27:46.955203 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:46.955175 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/0aa1aaa9-b87e-4497-9ae2-11b00f96bf39-secret-metrics-server-client-certs\") pod \"metrics-server-848f65bd86-bb84d\" (UID: \"0aa1aaa9-b87e-4497-9ae2-11b00f96bf39\") " pod="openshift-monitoring/metrics-server-848f65bd86-bb84d" Jul 27 09:27:46.962238 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:46.962215 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qmshv\" (UniqueName: \"kubernetes.io/projected/0aa1aaa9-b87e-4497-9ae2-11b00f96bf39-kube-api-access-qmshv\") pod \"metrics-server-848f65bd86-bb84d\" (UID: \"0aa1aaa9-b87e-4497-9ae2-11b00f96bf39\") " pod="openshift-monitoring/metrics-server-848f65bd86-bb84d" Jul 27 09:27:47.025094 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:47.025060 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-848w4_2c043d82-d300-41ba-b138-60a03d65ea11/konnectivity-agent/0.log" limit=1024 Jul 27 09:27:47.027946 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:47.027923 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-848f65bd86-bb84d" Jul 27 09:27:47.166031 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:47.165999 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-848w4_2c043d82-d300-41ba-b138-60a03d65ea11/konnectivity-agent/0.log" limit=1024 Jul 27 09:27:47.172980 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:47.172952 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-848f65bd86-bb84d"] Jul 27 09:27:47.176646 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:27:47.176616 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0aa1aaa9_b87e_4497_9ae2_11b00f96bf39.slice/crio-c242d086ecb93600c8768d0a5c24cc263e3286abe1b53037c4e8ccd4be890697 WatchSource:0}: Error finding container c242d086ecb93600c8768d0a5c24cc263e3286abe1b53037c4e8ccd4be890697: Status 404 returned error can't find the container with id c242d086ecb93600c8768d0a5c24cc263e3286abe1b53037c4e8ccd4be890697 Jul 27 09:27:47.256433 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:47.256332 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-mhq2g" event={"ID":"45db6a76-b0a6-475e-a2c2-2e0bf01f8e17","Type":"ContainerStarted","Data":"e24aec52291d63590478e6b88be37b5963e00107447422065269befa5827721e"} Jul 27 09:27:47.256433 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:47.256408 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-mhq2g" event={"ID":"45db6a76-b0a6-475e-a2c2-2e0bf01f8e17","Type":"ContainerStarted","Data":"eeb65369ea3fa65958c116bcc121bb670c8476749db9f0b010b5664617aa2d9e"} Jul 27 09:27:47.256433 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:47.256436 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-mhq2g" event={"ID":"45db6a76-b0a6-475e-a2c2-2e0bf01f8e17","Type":"ContainerStarted","Data":"8cf94b201d548d9585be4a2b31284a93f9c4ddcdb1e7e075b3c4dbad9ef95ca8"} Jul 27 09:27:47.258393 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:47.258370 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-vwsrv" event={"ID":"d9769977-82c3-4556-a9f3-7a6ec82bdb75","Type":"ContainerStarted","Data":"ffb8fc62645cce2fbb474698307d3b71a65b785aef4ef11cbc3624ebd1ac0c26"} Jul 27 09:27:47.258532 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:47.258424 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-vwsrv" event={"ID":"d9769977-82c3-4556-a9f3-7a6ec82bdb75","Type":"ContainerStarted","Data":"9ae96602ef7d357a74ed0eff45c54fdd53ddbaf8db4e02788fce68625687a09f"} Jul 27 09:27:47.259524 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:47.259504 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-848f65bd86-bb84d" event={"ID":"0aa1aaa9-b87e-4497-9ae2-11b00f96bf39","Type":"ContainerStarted","Data":"c242d086ecb93600c8768d0a5c24cc263e3286abe1b53037c4e8ccd4be890697"} Jul 27 09:27:47.280925 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:47.280868 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-mhq2g" podStartSLOduration=3.94939468 podStartE2EDuration="5.28084409s" totalImagesPullingTime="1.33144941s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 09:27:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 09:27:47.280045614 +0000 UTC m=+195.243831593" watchObservedRunningTime="2026-07-27 09:27:47.28084409 +0000 UTC m=+195.244630048" Jul 27 09:27:47.300964 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:47.300912 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-vwsrv" podStartSLOduration=4.290556885 podStartE2EDuration="5.300896298s" totalImagesPullingTime="939.704242ms" totalInitContainerRuntime="70.635171ms" isStatefulPod=true podCreationTimestamp="2026-07-27 09:27:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 09:27:47.299725228 +0000 UTC m=+195.263511179" watchObservedRunningTime="2026-07-27 09:27:47.300896298 +0000 UTC m=+195.264682255" Jul 27 09:27:47.610028 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:47.609943 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-848w4_2c043d82-d300-41ba-b138-60a03d65ea11/konnectivity-agent/0.log" limit=1024 Jul 27 09:27:48.170909 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:48.170881 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-56f74b4f54-4jm6n" Jul 27 09:27:48.323549 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:48.323511 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-848w4_2c043d82-d300-41ba-b138-60a03d65ea11/konnectivity-agent/0.log" limit=1024 Jul 27 09:27:48.423682 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:48.423591 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-848w4_2c043d82-d300-41ba-b138-60a03d65ea11/konnectivity-agent/0.log" limit=1024 Jul 27 09:27:48.522951 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:48.522865 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-848w4_2c043d82-d300-41ba-b138-60a03d65ea11/konnectivity-agent/0.log" limit=1024 Jul 27 09:27:48.640298 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:48.640253 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-848w4_2c043d82-d300-41ba-b138-60a03d65ea11/konnectivity-agent/0.log" limit=1024 Jul 27 09:27:49.269447 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:49.269410 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-848f65bd86-bb84d" event={"ID":"0aa1aaa9-b87e-4497-9ae2-11b00f96bf39","Type":"ContainerStarted","Data":"2e21ee38e611760ce626807b876f801f10ed1fb3ec2a7bf1608c20bc8f25e408"} Jul 27 09:27:49.287865 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:49.287815 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-848f65bd86-bb84d" podStartSLOduration=1.846348224 podStartE2EDuration="3.287797712s" totalImagesPullingTime="1.441449488s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 09:27:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 09:27:49.287150124 +0000 UTC m=+197.250936082" watchObservedRunningTime="2026-07-27 09:27:49.287797712 +0000 UTC m=+197.251583669" Jul 27 09:27:51.572665 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:51.572603 2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-544bcd5f5d-5jm4h" podUID="1221dd0e-e994-44c0-8082-780934175e8e" containerName="registry" containerID="cri-o://591fe6d3bb76d5e34ef3a498f7b173c972892189e0c0b27f514956c9f0af60d1" gracePeriod=30 Jul 27 09:27:51.805217 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:51.805192 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-544bcd5f5d-5jm4h" Jul 27 09:27:51.894785 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:51.894732 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/1221dd0e-e994-44c0-8082-780934175e8e-registry-certificates\") pod \"1221dd0e-e994-44c0-8082-780934175e8e\" (UID: \"1221dd0e-e994-44c0-8082-780934175e8e\") " Jul 27 09:27:51.894962 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:51.894818 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-j7mcj\" (UniqueName: \"kubernetes.io/projected/1221dd0e-e994-44c0-8082-780934175e8e-kube-api-access-j7mcj\") pod \"1221dd0e-e994-44c0-8082-780934175e8e\" (UID: \"1221dd0e-e994-44c0-8082-780934175e8e\") " Jul 27 09:27:51.894962 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:51.894835 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/1221dd0e-e994-44c0-8082-780934175e8e-bound-sa-token\") pod \"1221dd0e-e994-44c0-8082-780934175e8e\" (UID: \"1221dd0e-e994-44c0-8082-780934175e8e\") " Jul 27 09:27:51.894962 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:51.894856 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/1221dd0e-e994-44c0-8082-780934175e8e-installation-pull-secrets\") pod \"1221dd0e-e994-44c0-8082-780934175e8e\" (UID: \"1221dd0e-e994-44c0-8082-780934175e8e\") " Jul 27 09:27:51.894962 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:51.894883 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/1221dd0e-e994-44c0-8082-780934175e8e-ca-trust-extracted\") pod \"1221dd0e-e994-44c0-8082-780934175e8e\" (UID: \"1221dd0e-e994-44c0-8082-780934175e8e\") " Jul 27 09:27:51.894962 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:51.894921 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/1221dd0e-e994-44c0-8082-780934175e8e-trusted-ca\") pod \"1221dd0e-e994-44c0-8082-780934175e8e\" (UID: \"1221dd0e-e994-44c0-8082-780934175e8e\") " Jul 27 09:27:51.894962 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:51.894953 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/1221dd0e-e994-44c0-8082-780934175e8e-image-registry-private-configuration\") pod \"1221dd0e-e994-44c0-8082-780934175e8e\" (UID: \"1221dd0e-e994-44c0-8082-780934175e8e\") " Jul 27 09:27:51.895272 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:51.894991 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1221dd0e-e994-44c0-8082-780934175e8e-registry-tls\") pod \"1221dd0e-e994-44c0-8082-780934175e8e\" (UID: \"1221dd0e-e994-44c0-8082-780934175e8e\") " Jul 27 09:27:51.895272 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:51.895186 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1221dd0e-e994-44c0-8082-780934175e8e-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "1221dd0e-e994-44c0-8082-780934175e8e" (UID: "1221dd0e-e994-44c0-8082-780934175e8e"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 27 09:27:51.895487 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:51.895437 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1221dd0e-e994-44c0-8082-780934175e8e-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "1221dd0e-e994-44c0-8082-780934175e8e" (UID: "1221dd0e-e994-44c0-8082-780934175e8e"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 27 09:27:51.897342 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:51.897317 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1221dd0e-e994-44c0-8082-780934175e8e-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "1221dd0e-e994-44c0-8082-780934175e8e" (UID: "1221dd0e-e994-44c0-8082-780934175e8e"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 27 09:27:51.897544 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:51.897524 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1221dd0e-e994-44c0-8082-780934175e8e-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "1221dd0e-e994-44c0-8082-780934175e8e" (UID: "1221dd0e-e994-44c0-8082-780934175e8e"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 27 09:27:51.897616 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:51.897533 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1221dd0e-e994-44c0-8082-780934175e8e-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "1221dd0e-e994-44c0-8082-780934175e8e" (UID: "1221dd0e-e994-44c0-8082-780934175e8e"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 27 09:27:51.897616 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:51.897549 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1221dd0e-e994-44c0-8082-780934175e8e-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "1221dd0e-e994-44c0-8082-780934175e8e" (UID: "1221dd0e-e994-44c0-8082-780934175e8e"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 27 09:27:51.897616 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:51.897541 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1221dd0e-e994-44c0-8082-780934175e8e-kube-api-access-j7mcj" (OuterVolumeSpecName: "kube-api-access-j7mcj") pod "1221dd0e-e994-44c0-8082-780934175e8e" (UID: "1221dd0e-e994-44c0-8082-780934175e8e"). InnerVolumeSpecName "kube-api-access-j7mcj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 27 09:27:51.906463 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:51.906431 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1221dd0e-e994-44c0-8082-780934175e8e-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "1221dd0e-e994-44c0-8082-780934175e8e" (UID: "1221dd0e-e994-44c0-8082-780934175e8e"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 27 09:27:51.996034 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:51.996000 2565 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/1221dd0e-e994-44c0-8082-780934175e8e-trusted-ca\") on node \"ip-10-0-128-103.ec2.internal\" DevicePath \"\"" Jul 27 09:27:51.996034 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:51.996028 2565 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/1221dd0e-e994-44c0-8082-780934175e8e-image-registry-private-configuration\") on node \"ip-10-0-128-103.ec2.internal\" DevicePath \"\"" Jul 27 09:27:51.996034 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:51.996039 2565 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1221dd0e-e994-44c0-8082-780934175e8e-registry-tls\") on node \"ip-10-0-128-103.ec2.internal\" DevicePath \"\"" Jul 27 09:27:51.996248 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:51.996049 2565 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/1221dd0e-e994-44c0-8082-780934175e8e-registry-certificates\") on node \"ip-10-0-128-103.ec2.internal\" DevicePath \"\"" Jul 27 09:27:51.996248 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:51.996058 2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j7mcj\" (UniqueName: \"kubernetes.io/projected/1221dd0e-e994-44c0-8082-780934175e8e-kube-api-access-j7mcj\") on node \"ip-10-0-128-103.ec2.internal\" DevicePath \"\"" Jul 27 09:27:51.996248 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:51.996067 2565 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/1221dd0e-e994-44c0-8082-780934175e8e-bound-sa-token\") on node \"ip-10-0-128-103.ec2.internal\" DevicePath \"\"" Jul 27 09:27:51.996248 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:51.996076 2565 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/1221dd0e-e994-44c0-8082-780934175e8e-installation-pull-secrets\") on node \"ip-10-0-128-103.ec2.internal\" DevicePath \"\"" Jul 27 09:27:51.996248 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:51.996083 2565 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/1221dd0e-e994-44c0-8082-780934175e8e-ca-trust-extracted\") on node \"ip-10-0-128-103.ec2.internal\" DevicePath \"\"" Jul 27 09:27:52.281384 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:52.281297 2565 generic.go:356] "Generic (PLEG): container finished" podID="1221dd0e-e994-44c0-8082-780934175e8e" containerID="591fe6d3bb76d5e34ef3a498f7b173c972892189e0c0b27f514956c9f0af60d1" exitCode=0 Jul 27 09:27:52.281384 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:52.281339 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-544bcd5f5d-5jm4h" Jul 27 09:27:52.281384 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:52.281366 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-544bcd5f5d-5jm4h" event={"ID":"1221dd0e-e994-44c0-8082-780934175e8e","Type":"ContainerDied","Data":"591fe6d3bb76d5e34ef3a498f7b173c972892189e0c0b27f514956c9f0af60d1"} Jul 27 09:27:52.281609 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:52.281408 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-544bcd5f5d-5jm4h" event={"ID":"1221dd0e-e994-44c0-8082-780934175e8e","Type":"ContainerDied","Data":"39504e789e677839a9f8f0908754e09eb28da7cc17b6e1b5186ddf519bd64419"} Jul 27 09:27:52.281609 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:52.281427 2565 scope.go:117] "RemoveContainer" containerID="591fe6d3bb76d5e34ef3a498f7b173c972892189e0c0b27f514956c9f0af60d1" Jul 27 09:27:52.289664 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:52.289646 2565 scope.go:117] "RemoveContainer" containerID="591fe6d3bb76d5e34ef3a498f7b173c972892189e0c0b27f514956c9f0af60d1" Jul 27 09:27:52.289972 ip-10-0-128-103 kubenswrapper[2565]: E0727 09:27:52.289940 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"591fe6d3bb76d5e34ef3a498f7b173c972892189e0c0b27f514956c9f0af60d1\": container with ID starting with 591fe6d3bb76d5e34ef3a498f7b173c972892189e0c0b27f514956c9f0af60d1 not found: ID does not exist" containerID="591fe6d3bb76d5e34ef3a498f7b173c972892189e0c0b27f514956c9f0af60d1" Jul 27 09:27:52.290042 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:52.289981 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"591fe6d3bb76d5e34ef3a498f7b173c972892189e0c0b27f514956c9f0af60d1"} err="failed to get container status \"591fe6d3bb76d5e34ef3a498f7b173c972892189e0c0b27f514956c9f0af60d1\": rpc error: code = NotFound desc = could not find container \"591fe6d3bb76d5e34ef3a498f7b173c972892189e0c0b27f514956c9f0af60d1\": container with ID starting with 591fe6d3bb76d5e34ef3a498f7b173c972892189e0c0b27f514956c9f0af60d1 not found: ID does not exist" Jul 27 09:27:52.309397 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:52.309343 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-544bcd5f5d-5jm4h"] Jul 27 09:27:52.310983 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:52.310957 2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-544bcd5f5d-5jm4h"] Jul 27 09:27:52.562618 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:52.562527 2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1221dd0e-e994-44c0-8082-780934175e8e" path="/var/lib/kubelet/pods/1221dd0e-e994-44c0-8082-780934175e8e/volumes" Jul 27 09:27:55.558310 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:55.558271 2565 scope.go:117] "RemoveContainer" containerID="1491a240002f05c46d392fdf6a0e9ca99f3d5760eb568e9316bbe43b49bc5c3e" Jul 27 09:27:55.705107 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:55.705081 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-848w4" Jul 27 09:27:55.772489 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:55.772451 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-848w4_2c043d82-d300-41ba-b138-60a03d65ea11/konnectivity-agent/0.log" limit=1024 Jul 27 09:27:56.298070 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:56.298042 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-w7grd_0badf8bd-7f64-46ef-872c-39f59dd11d0d/console-operator/2.log" Jul 27 09:27:56.298233 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:56.298103 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-584dcb6b54-w7grd" event={"ID":"0badf8bd-7f64-46ef-872c-39f59dd11d0d","Type":"ContainerStarted","Data":"cef01693c766ebbb895b42ef13aa96659ef1c5798b593d98415a384f3ec665ca"} Jul 27 09:27:56.298472 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:56.298447 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-584dcb6b54-w7grd" Jul 27 09:27:56.758618 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:56.758587 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-584dcb6b54-w7grd" Jul 27 09:27:56.955798 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:56.955744 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-7c97dc4555-v8rpf"] Jul 27 09:27:56.956178 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:56.956161 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1221dd0e-e994-44c0-8082-780934175e8e" containerName="registry" Jul 27 09:27:56.956222 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:56.956182 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="1221dd0e-e994-44c0-8082-780934175e8e" containerName="registry" Jul 27 09:27:56.956273 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:56.956262 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="1221dd0e-e994-44c0-8082-780934175e8e" containerName="registry" Jul 27 09:27:56.959405 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:56.959376 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-7c97dc4555-v8rpf" Jul 27 09:27:56.961247 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:56.961206 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-848w4_2c043d82-d300-41ba-b138-60a03d65ea11/konnectivity-agent/0.log" limit=1024 Jul 27 09:27:56.963135 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:56.963117 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 27 09:27:56.963658 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:56.963618 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 27 09:27:56.963658 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:56.963641 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-mjjgk\"" Jul 27 09:27:56.969265 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:56.969242 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-7c97dc4555-v8rpf"] Jul 27 09:27:57.039985 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:57.039900 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-848w4_2c043d82-d300-41ba-b138-60a03d65ea11/konnectivity-agent/0.log" limit=1024 Jul 27 09:27:57.041871 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:57.041847 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dk2h9\" (UniqueName: \"kubernetes.io/projected/3e10f8fb-0a12-4c28-8758-deaee169ebaf-kube-api-access-dk2h9\") pod \"downloads-7c97dc4555-v8rpf\" (UID: \"3e10f8fb-0a12-4c28-8758-deaee169ebaf\") " pod="openshift-console/downloads-7c97dc4555-v8rpf" Jul 27 09:27:57.140607 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:57.140574 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-848w4_2c043d82-d300-41ba-b138-60a03d65ea11/konnectivity-agent/0.log" limit=1024 Jul 27 09:27:57.142653 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:57.142626 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dk2h9\" (UniqueName: \"kubernetes.io/projected/3e10f8fb-0a12-4c28-8758-deaee169ebaf-kube-api-access-dk2h9\") pod \"downloads-7c97dc4555-v8rpf\" (UID: \"3e10f8fb-0a12-4c28-8758-deaee169ebaf\") " pod="openshift-console/downloads-7c97dc4555-v8rpf" Jul 27 09:27:57.151009 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:57.150985 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dk2h9\" (UniqueName: \"kubernetes.io/projected/3e10f8fb-0a12-4c28-8758-deaee169ebaf-kube-api-access-dk2h9\") pod \"downloads-7c97dc4555-v8rpf\" (UID: \"3e10f8fb-0a12-4c28-8758-deaee169ebaf\") " pod="openshift-console/downloads-7c97dc4555-v8rpf" Jul 27 09:27:57.241504 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:57.241462 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-848w4_2c043d82-d300-41ba-b138-60a03d65ea11/konnectivity-agent/0.log" limit=1024 Jul 27 09:27:57.272128 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:57.272096 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-7c97dc4555-v8rpf" Jul 27 09:27:57.303831 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:57.303733 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-848w4_2c043d82-d300-41ba-b138-60a03d65ea11/konnectivity-agent/0.log" limit=1024 Jul 27 09:27:57.408237 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:57.408204 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-7c97dc4555-v8rpf"] Jul 27 09:27:57.412330 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:27:57.412302 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3e10f8fb_0a12_4c28_8758_deaee169ebaf.slice/crio-172eec6a58911f1b74b64955e3a2612a6ed6827e445797c84479bf69582d78a9 WatchSource:0}: Error finding container 172eec6a58911f1b74b64955e3a2612a6ed6827e445797c84479bf69582d78a9: Status 404 returned error can't find the container with id 172eec6a58911f1b74b64955e3a2612a6ed6827e445797c84479bf69582d78a9 Jul 27 09:27:57.619459 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:57.619413 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-848w4_2c043d82-d300-41ba-b138-60a03d65ea11/konnectivity-agent/0.log" limit=1024 Jul 27 09:27:57.720300 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:57.720264 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-848w4_2c043d82-d300-41ba-b138-60a03d65ea11/konnectivity-agent/0.log" limit=1024 Jul 27 09:27:57.827602 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:57.827571 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-848w4_2c043d82-d300-41ba-b138-60a03d65ea11/konnectivity-agent/0.log" limit=1024 Jul 27 09:27:58.020142 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:58.020043 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-848w4_2c043d82-d300-41ba-b138-60a03d65ea11/konnectivity-agent/0.log" limit=1024 Jul 27 09:27:58.120982 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:58.120947 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-848w4_2c043d82-d300-41ba-b138-60a03d65ea11/konnectivity-agent/0.log" limit=1024 Jul 27 09:27:58.315713 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:58.315616 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-7c97dc4555-v8rpf" event={"ID":"3e10f8fb-0a12-4c28-8758-deaee169ebaf","Type":"ContainerStarted","Data":"172eec6a58911f1b74b64955e3a2612a6ed6827e445797c84479bf69582d78a9"} Jul 27 09:27:58.429074 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:58.428985 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-848w4_2c043d82-d300-41ba-b138-60a03d65ea11/konnectivity-agent/0.log" limit=1024 Jul 27 09:27:58.624518 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:58.624474 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-848w4_2c043d82-d300-41ba-b138-60a03d65ea11/konnectivity-agent/0.log" limit=1024 Jul 27 09:27:58.821742 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:58.821644 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-848w4_2c043d82-d300-41ba-b138-60a03d65ea11/konnectivity-agent/0.log" limit=1024 Jul 27 09:27:59.646096 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:59.646054 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-848w4_2c043d82-d300-41ba-b138-60a03d65ea11/konnectivity-agent/0.log" limit=1024 Jul 27 09:27:59.706101 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:27:59.706069 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-848w4_2c043d82-d300-41ba-b138-60a03d65ea11/konnectivity-agent/0.log" limit=1024 Jul 27 09:28:01.151832 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:28:01.151788 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-848w4_2c043d82-d300-41ba-b138-60a03d65ea11/konnectivity-agent/0.log" limit=1024 Jul 27 09:28:01.213059 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:28:01.213023 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-848w4_2c043d82-d300-41ba-b138-60a03d65ea11/konnectivity-agent/0.log" limit=1024 Jul 27 09:28:06.871457 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:28:06.871415 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-6665866d46-vn7rh"] Jul 27 09:28:06.875958 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:28:06.875930 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6665866d46-vn7rh" Jul 27 09:28:06.878476 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:28:06.878451 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 27 09:28:06.878677 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:28:06.878655 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-d6tcp\"" Jul 27 09:28:06.878677 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:28:06.878674 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 27 09:28:06.878959 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:28:06.878792 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 27 09:28:06.879444 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:28:06.879417 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 27 09:28:06.879444 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:28:06.879435 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 27 09:28:06.888511 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:28:06.888487 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6665866d46-vn7rh"] Jul 27 09:28:06.932946 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:28:06.932915 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a0119ea8-ea1f-44b9-963c-8f361779b4e7-service-ca\") pod \"console-6665866d46-vn7rh\" (UID: \"a0119ea8-ea1f-44b9-963c-8f361779b4e7\") " pod="openshift-console/console-6665866d46-vn7rh" Jul 27 09:28:06.932946 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:28:06.932964 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a0119ea8-ea1f-44b9-963c-8f361779b4e7-oauth-serving-cert\") pod \"console-6665866d46-vn7rh\" (UID: \"a0119ea8-ea1f-44b9-963c-8f361779b4e7\") " pod="openshift-console/console-6665866d46-vn7rh" Jul 27 09:28:06.933177 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:28:06.933024 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a0119ea8-ea1f-44b9-963c-8f361779b4e7-console-config\") pod \"console-6665866d46-vn7rh\" (UID: \"a0119ea8-ea1f-44b9-963c-8f361779b4e7\") " pod="openshift-console/console-6665866d46-vn7rh" Jul 27 09:28:06.933177 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:28:06.933053 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a0119ea8-ea1f-44b9-963c-8f361779b4e7-console-serving-cert\") pod \"console-6665866d46-vn7rh\" (UID: \"a0119ea8-ea1f-44b9-963c-8f361779b4e7\") " pod="openshift-console/console-6665866d46-vn7rh" Jul 27 09:28:06.933177 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:28:06.933114 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a0119ea8-ea1f-44b9-963c-8f361779b4e7-console-oauth-config\") pod \"console-6665866d46-vn7rh\" (UID: \"a0119ea8-ea1f-44b9-963c-8f361779b4e7\") " pod="openshift-console/console-6665866d46-vn7rh" Jul 27 09:28:06.933177 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:28:06.933171 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5r8th\" (UniqueName: \"kubernetes.io/projected/a0119ea8-ea1f-44b9-963c-8f361779b4e7-kube-api-access-5r8th\") pod \"console-6665866d46-vn7rh\" (UID: \"a0119ea8-ea1f-44b9-963c-8f361779b4e7\") " pod="openshift-console/console-6665866d46-vn7rh" Jul 27 09:28:07.028086 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:28:07.028048 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-848f65bd86-bb84d" Jul 27 09:28:07.028267 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:28:07.028131 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-848f65bd86-bb84d" Jul 27 09:28:07.033989 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:28:07.033957 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a0119ea8-ea1f-44b9-963c-8f361779b4e7-console-oauth-config\") pod \"console-6665866d46-vn7rh\" (UID: \"a0119ea8-ea1f-44b9-963c-8f361779b4e7\") " pod="openshift-console/console-6665866d46-vn7rh" Jul 27 09:28:07.034160 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:28:07.033996 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5r8th\" (UniqueName: \"kubernetes.io/projected/a0119ea8-ea1f-44b9-963c-8f361779b4e7-kube-api-access-5r8th\") pod \"console-6665866d46-vn7rh\" (UID: \"a0119ea8-ea1f-44b9-963c-8f361779b4e7\") " pod="openshift-console/console-6665866d46-vn7rh" Jul 27 09:28:07.034160 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:28:07.034111 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a0119ea8-ea1f-44b9-963c-8f361779b4e7-service-ca\") pod \"console-6665866d46-vn7rh\" (UID: \"a0119ea8-ea1f-44b9-963c-8f361779b4e7\") " pod="openshift-console/console-6665866d46-vn7rh" Jul 27 09:28:07.034276 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:28:07.034164 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a0119ea8-ea1f-44b9-963c-8f361779b4e7-oauth-serving-cert\") pod \"console-6665866d46-vn7rh\" (UID: \"a0119ea8-ea1f-44b9-963c-8f361779b4e7\") " pod="openshift-console/console-6665866d46-vn7rh" Jul 27 09:28:07.034806 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:28:07.034551 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a0119ea8-ea1f-44b9-963c-8f361779b4e7-console-config\") pod \"console-6665866d46-vn7rh\" (UID: \"a0119ea8-ea1f-44b9-963c-8f361779b4e7\") " pod="openshift-console/console-6665866d46-vn7rh" Jul 27 09:28:07.034806 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:28:07.034612 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a0119ea8-ea1f-44b9-963c-8f361779b4e7-console-serving-cert\") pod \"console-6665866d46-vn7rh\" (UID: \"a0119ea8-ea1f-44b9-963c-8f361779b4e7\") " pod="openshift-console/console-6665866d46-vn7rh" Jul 27 09:28:07.034992 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:28:07.034842 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a0119ea8-ea1f-44b9-963c-8f361779b4e7-oauth-serving-cert\") pod \"console-6665866d46-vn7rh\" (UID: \"a0119ea8-ea1f-44b9-963c-8f361779b4e7\") " pod="openshift-console/console-6665866d46-vn7rh" Jul 27 09:28:07.035161 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:28:07.035137 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a0119ea8-ea1f-44b9-963c-8f361779b4e7-console-config\") pod \"console-6665866d46-vn7rh\" (UID: \"a0119ea8-ea1f-44b9-963c-8f361779b4e7\") " pod="openshift-console/console-6665866d46-vn7rh" Jul 27 09:28:07.035368 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:28:07.035344 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a0119ea8-ea1f-44b9-963c-8f361779b4e7-service-ca\") pod \"console-6665866d46-vn7rh\" (UID: \"a0119ea8-ea1f-44b9-963c-8f361779b4e7\") " pod="openshift-console/console-6665866d46-vn7rh" Jul 27 09:28:07.037258 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:28:07.037236 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a0119ea8-ea1f-44b9-963c-8f361779b4e7-console-oauth-config\") pod \"console-6665866d46-vn7rh\" (UID: \"a0119ea8-ea1f-44b9-963c-8f361779b4e7\") " pod="openshift-console/console-6665866d46-vn7rh" Jul 27 09:28:07.037476 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:28:07.037457 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a0119ea8-ea1f-44b9-963c-8f361779b4e7-console-serving-cert\") pod \"console-6665866d46-vn7rh\" (UID: \"a0119ea8-ea1f-44b9-963c-8f361779b4e7\") " pod="openshift-console/console-6665866d46-vn7rh" Jul 27 09:28:07.043309 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:28:07.043287 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5r8th\" (UniqueName: \"kubernetes.io/projected/a0119ea8-ea1f-44b9-963c-8f361779b4e7-kube-api-access-5r8th\") pod \"console-6665866d46-vn7rh\" (UID: \"a0119ea8-ea1f-44b9-963c-8f361779b4e7\") " pod="openshift-console/console-6665866d46-vn7rh" Jul 27 09:28:07.187466 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:28:07.187365 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6665866d46-vn7rh" Jul 27 09:28:08.852630 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:28:08.852598 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-848w4_2c043d82-d300-41ba-b138-60a03d65ea11/konnectivity-agent/0.log" limit=1024 Jul 27 09:28:11.259290 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:28:11.259250 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-848w4_2c043d82-d300-41ba-b138-60a03d65ea11/konnectivity-agent/0.log" limit=1024 Jul 27 09:28:12.051633 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:28:12.051586 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-848w4_2c043d82-d300-41ba-b138-60a03d65ea11/konnectivity-agent/0.log" limit=1024 Jul 27 09:28:12.651251 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:28:12.651212 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-848w4_2c043d82-d300-41ba-b138-60a03d65ea11/konnectivity-agent/0.log" limit=1024 Jul 27 09:28:12.715521 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:28:12.715486 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-848w4_2c043d82-d300-41ba-b138-60a03d65ea11/konnectivity-agent/0.log" limit=1024 Jul 27 09:28:13.610637 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:28:13.610605 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6665866d46-vn7rh"] Jul 27 09:28:13.613683 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:28:13.613659 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda0119ea8_ea1f_44b9_963c_8f361779b4e7.slice/crio-ec0d90390cb3db4b4a57a80c445ef45222174f3a75c7fc4f91059bdf882dd7c6 WatchSource:0}: Error finding container ec0d90390cb3db4b4a57a80c445ef45222174f3a75c7fc4f91059bdf882dd7c6: Status 404 returned error can't find the container with id ec0d90390cb3db4b4a57a80c445ef45222174f3a75c7fc4f91059bdf882dd7c6 Jul 27 09:28:14.380392 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:28:14.380137 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-7c97dc4555-v8rpf" event={"ID":"3e10f8fb-0a12-4c28-8758-deaee169ebaf","Type":"ContainerStarted","Data":"86f527ff962a7b30dbf47c298630be247cd50c0d540d7cb5d50d311f0ec63ca4"} Jul 27 09:28:14.380392 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:28:14.380212 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-7c97dc4555-v8rpf" Jul 27 09:28:14.381698 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:28:14.381670 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6665866d46-vn7rh" event={"ID":"a0119ea8-ea1f-44b9-963c-8f361779b4e7","Type":"ContainerStarted","Data":"ec0d90390cb3db4b4a57a80c445ef45222174f3a75c7fc4f91059bdf882dd7c6"} Jul 27 09:28:14.381939 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:28:14.381911 2565 patch_prober.go:28] interesting pod/downloads-7c97dc4555-v8rpf container/download-server namespace/openshift-console: Readiness probe status=failure output="Get \"http://10.133.0.19:8080/\": dial tcp 10.133.0.19:8080: connect: connection refused" start-of-body= Jul 27 09:28:14.382044 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:28:14.381968 2565 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-console/downloads-7c97dc4555-v8rpf" podUID="3e10f8fb-0a12-4c28-8758-deaee169ebaf" containerName="download-server" probeResult="failure" output="Get \"http://10.133.0.19:8080/\": dial tcp 10.133.0.19:8080: connect: connection refused" Jul 27 09:28:14.398703 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:28:14.398645 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-7c97dc4555-v8rpf" podStartSLOduration=1.920410157 podStartE2EDuration="18.39862836s" totalImagesPullingTime="16.478218203s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 09:27:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 09:28:14.397442995 +0000 UTC m=+222.361228973" watchObservedRunningTime="2026-07-27 09:28:14.39862836 +0000 UTC m=+222.362414320" Jul 27 09:28:15.400777 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:28:15.400721 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-7c97dc4555-v8rpf" Jul 27 09:28:15.654200 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:28:15.654106 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-7c49b99956-dkcst"] Jul 27 09:28:15.664363 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:28:15.664332 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7c49b99956-dkcst" Jul 27 09:28:15.672422 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:28:15.672343 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jul 27 09:28:15.676791 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:28:15.675615 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7c49b99956-dkcst"] Jul 27 09:28:15.725156 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:28:15.725114 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/13306691-0229-4af9-a3f8-a0d5c09b9164-oauth-serving-cert\") pod \"console-7c49b99956-dkcst\" (UID: \"13306691-0229-4af9-a3f8-a0d5c09b9164\") " pod="openshift-console/console-7c49b99956-dkcst" Jul 27 09:28:15.725423 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:28:15.725189 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8t9z2\" (UniqueName: \"kubernetes.io/projected/13306691-0229-4af9-a3f8-a0d5c09b9164-kube-api-access-8t9z2\") pod \"console-7c49b99956-dkcst\" (UID: \"13306691-0229-4af9-a3f8-a0d5c09b9164\") " pod="openshift-console/console-7c49b99956-dkcst" Jul 27 09:28:15.725423 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:28:15.725230 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/13306691-0229-4af9-a3f8-a0d5c09b9164-console-serving-cert\") pod \"console-7c49b99956-dkcst\" (UID: \"13306691-0229-4af9-a3f8-a0d5c09b9164\") " pod="openshift-console/console-7c49b99956-dkcst" Jul 27 09:28:15.725423 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:28:15.725275 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/13306691-0229-4af9-a3f8-a0d5c09b9164-console-oauth-config\") pod \"console-7c49b99956-dkcst\" (UID: \"13306691-0229-4af9-a3f8-a0d5c09b9164\") " pod="openshift-console/console-7c49b99956-dkcst" Jul 27 09:28:15.725423 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:28:15.725320 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/13306691-0229-4af9-a3f8-a0d5c09b9164-trusted-ca-bundle\") pod \"console-7c49b99956-dkcst\" (UID: \"13306691-0229-4af9-a3f8-a0d5c09b9164\") " pod="openshift-console/console-7c49b99956-dkcst" Jul 27 09:28:15.725423 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:28:15.725356 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/13306691-0229-4af9-a3f8-a0d5c09b9164-service-ca\") pod \"console-7c49b99956-dkcst\" (UID: \"13306691-0229-4af9-a3f8-a0d5c09b9164\") " pod="openshift-console/console-7c49b99956-dkcst" Jul 27 09:28:15.725698 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:28:15.725443 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/13306691-0229-4af9-a3f8-a0d5c09b9164-console-config\") pod \"console-7c49b99956-dkcst\" (UID: \"13306691-0229-4af9-a3f8-a0d5c09b9164\") " pod="openshift-console/console-7c49b99956-dkcst" Jul 27 09:28:15.826315 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:28:15.826169 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/13306691-0229-4af9-a3f8-a0d5c09b9164-oauth-serving-cert\") pod \"console-7c49b99956-dkcst\" (UID: \"13306691-0229-4af9-a3f8-a0d5c09b9164\") " pod="openshift-console/console-7c49b99956-dkcst" Jul 27 09:28:15.826315 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:28:15.826246 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8t9z2\" (UniqueName: \"kubernetes.io/projected/13306691-0229-4af9-a3f8-a0d5c09b9164-kube-api-access-8t9z2\") pod \"console-7c49b99956-dkcst\" (UID: \"13306691-0229-4af9-a3f8-a0d5c09b9164\") " pod="openshift-console/console-7c49b99956-dkcst" Jul 27 09:28:15.826315 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:28:15.826280 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/13306691-0229-4af9-a3f8-a0d5c09b9164-console-serving-cert\") pod \"console-7c49b99956-dkcst\" (UID: \"13306691-0229-4af9-a3f8-a0d5c09b9164\") " pod="openshift-console/console-7c49b99956-dkcst" Jul 27 09:28:15.826315 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:28:15.826315 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/13306691-0229-4af9-a3f8-a0d5c09b9164-console-oauth-config\") pod \"console-7c49b99956-dkcst\" (UID: \"13306691-0229-4af9-a3f8-a0d5c09b9164\") " pod="openshift-console/console-7c49b99956-dkcst" Jul 27 09:28:15.826653 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:28:15.826365 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/13306691-0229-4af9-a3f8-a0d5c09b9164-trusted-ca-bundle\") pod \"console-7c49b99956-dkcst\" (UID: \"13306691-0229-4af9-a3f8-a0d5c09b9164\") " pod="openshift-console/console-7c49b99956-dkcst" Jul 27 09:28:15.826653 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:28:15.826406 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/13306691-0229-4af9-a3f8-a0d5c09b9164-service-ca\") pod \"console-7c49b99956-dkcst\" (UID: \"13306691-0229-4af9-a3f8-a0d5c09b9164\") " pod="openshift-console/console-7c49b99956-dkcst" Jul 27 09:28:15.826653 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:28:15.826434 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/13306691-0229-4af9-a3f8-a0d5c09b9164-console-config\") pod \"console-7c49b99956-dkcst\" (UID: \"13306691-0229-4af9-a3f8-a0d5c09b9164\") " pod="openshift-console/console-7c49b99956-dkcst" Jul 27 09:28:15.827155 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:28:15.827097 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/13306691-0229-4af9-a3f8-a0d5c09b9164-oauth-serving-cert\") pod \"console-7c49b99956-dkcst\" (UID: \"13306691-0229-4af9-a3f8-a0d5c09b9164\") " pod="openshift-console/console-7c49b99956-dkcst" Jul 27 09:28:15.827272 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:28:15.827227 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/13306691-0229-4af9-a3f8-a0d5c09b9164-console-config\") pod \"console-7c49b99956-dkcst\" (UID: \"13306691-0229-4af9-a3f8-a0d5c09b9164\") " pod="openshift-console/console-7c49b99956-dkcst" Jul 27 09:28:15.827690 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:28:15.827659 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/13306691-0229-4af9-a3f8-a0d5c09b9164-trusted-ca-bundle\") pod \"console-7c49b99956-dkcst\" (UID: \"13306691-0229-4af9-a3f8-a0d5c09b9164\") " pod="openshift-console/console-7c49b99956-dkcst" Jul 27 09:28:15.828277 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:28:15.828220 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/13306691-0229-4af9-a3f8-a0d5c09b9164-service-ca\") pod \"console-7c49b99956-dkcst\" (UID: \"13306691-0229-4af9-a3f8-a0d5c09b9164\") " pod="openshift-console/console-7c49b99956-dkcst" Jul 27 09:28:15.829681 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:28:15.829647 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/13306691-0229-4af9-a3f8-a0d5c09b9164-console-oauth-config\") pod \"console-7c49b99956-dkcst\" (UID: \"13306691-0229-4af9-a3f8-a0d5c09b9164\") " pod="openshift-console/console-7c49b99956-dkcst" Jul 27 09:28:15.830100 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:28:15.830061 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/13306691-0229-4af9-a3f8-a0d5c09b9164-console-serving-cert\") pod \"console-7c49b99956-dkcst\" (UID: \"13306691-0229-4af9-a3f8-a0d5c09b9164\") " pod="openshift-console/console-7c49b99956-dkcst" Jul 27 09:28:15.836256 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:28:15.836228 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8t9z2\" (UniqueName: \"kubernetes.io/projected/13306691-0229-4af9-a3f8-a0d5c09b9164-kube-api-access-8t9z2\") pod \"console-7c49b99956-dkcst\" (UID: \"13306691-0229-4af9-a3f8-a0d5c09b9164\") " pod="openshift-console/console-7c49b99956-dkcst" Jul 27 09:28:15.861662 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:28:15.861559 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-848w4_2c043d82-d300-41ba-b138-60a03d65ea11/konnectivity-agent/0.log" limit=1024 Jul 27 09:28:15.978267 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:28:15.978033 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7c49b99956-dkcst" Jul 27 09:28:17.220088 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:28:17.220055 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7c49b99956-dkcst"] Jul 27 09:28:17.223666 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:28:17.223632 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod13306691_0229_4af9_a3f8_a0d5c09b9164.slice/crio-6907638b6b921ba170db80fe3f026b7fe3a3f3dbcc1e71efc812cf13777aa9cf WatchSource:0}: Error finding container 6907638b6b921ba170db80fe3f026b7fe3a3f3dbcc1e71efc812cf13777aa9cf: Status 404 returned error can't find the container with id 6907638b6b921ba170db80fe3f026b7fe3a3f3dbcc1e71efc812cf13777aa9cf Jul 27 09:28:17.397572 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:28:17.397533 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7c49b99956-dkcst" event={"ID":"13306691-0229-4af9-a3f8-a0d5c09b9164","Type":"ContainerStarted","Data":"6907638b6b921ba170db80fe3f026b7fe3a3f3dbcc1e71efc812cf13777aa9cf"} Jul 27 09:28:18.403896 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:28:18.403859 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6665866d46-vn7rh" event={"ID":"a0119ea8-ea1f-44b9-963c-8f361779b4e7","Type":"ContainerStarted","Data":"5aa937e641e0728d67692e63fd13982a8c60f09dd169b82ce879717739400cb7"} Jul 27 09:28:18.405717 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:28:18.405682 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7c49b99956-dkcst" event={"ID":"13306691-0229-4af9-a3f8-a0d5c09b9164","Type":"ContainerStarted","Data":"18d96fe4f89cd71b3cf47ff89b82449ca282c428d80f69c366bf13d807418c25"} Jul 27 09:28:18.422586 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:28:18.422481 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-6665866d46-vn7rh" podStartSLOduration=8.608021150999999 podStartE2EDuration="12.422456472s" totalImagesPullingTime="3.814435321s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 09:28:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 09:28:18.422206814 +0000 UTC m=+226.385992802" watchObservedRunningTime="2026-07-27 09:28:18.422456472 +0000 UTC m=+226.386242428" Jul 27 09:28:18.441555 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:28:18.441491 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-7c49b99956-dkcst" podStartSLOduration=2.920437633 podStartE2EDuration="3.441469547s" totalImagesPullingTime="521.031914ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 09:28:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 09:28:18.440909123 +0000 UTC m=+226.404695105" watchObservedRunningTime="2026-07-27 09:28:18.441469547 +0000 UTC m=+226.405255503" Jul 27 09:28:19.335289 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:28:19.335251 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-848w4_2c043d82-d300-41ba-b138-60a03d65ea11/konnectivity-agent/0.log" limit=1024 Jul 27 09:28:19.436776 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:28:19.436714 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-848w4_2c043d82-d300-41ba-b138-60a03d65ea11/konnectivity-agent/0.log" limit=1024 Jul 27 09:28:19.490217 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:28:19.490181 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-848w4_2c043d82-d300-41ba-b138-60a03d65ea11/konnectivity-agent/0.log" limit=1024 Jul 27 09:28:21.420034 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:28:21.420000 2565 generic.go:356] "Generic (PLEG): container finished" podID="85bf3791-57a8-4f6a-aba7-85b8a44e170c" containerID="f1bd60cf2ef9c6ea383212107d2a8dda47942a509e44f370b68a866d3755e0e7" exitCode=0 Jul 27 09:28:21.420518 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:28:21.420081 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-b5jrj" event={"ID":"85bf3791-57a8-4f6a-aba7-85b8a44e170c","Type":"ContainerDied","Data":"f1bd60cf2ef9c6ea383212107d2a8dda47942a509e44f370b68a866d3755e0e7"} Jul 27 09:28:21.420614 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:28:21.420521 2565 scope.go:117] "RemoveContainer" containerID="f1bd60cf2ef9c6ea383212107d2a8dda47942a509e44f370b68a866d3755e0e7" Jul 27 09:28:21.421939 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:28:21.421911 2565 generic.go:356] "Generic (PLEG): container finished" podID="fd3e4e31-3c65-472d-bd64-e3bd969db33e" containerID="4dc76c2e2bb6a749bcb3cbcbd9b6df5bbf2230ab735fe7ab99917f180320dd2b" exitCode=0 Jul 27 09:28:21.422052 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:28:21.421957 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-pjm9w" event={"ID":"fd3e4e31-3c65-472d-bd64-e3bd969db33e","Type":"ContainerDied","Data":"4dc76c2e2bb6a749bcb3cbcbd9b6df5bbf2230ab735fe7ab99917f180320dd2b"} Jul 27 09:28:21.422243 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:28:21.422227 2565 scope.go:117] "RemoveContainer" containerID="4dc76c2e2bb6a749bcb3cbcbd9b6df5bbf2230ab735fe7ab99917f180320dd2b" Jul 27 09:28:22.428986 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:28:22.428946 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-pjm9w" event={"ID":"fd3e4e31-3c65-472d-bd64-e3bd969db33e","Type":"ContainerStarted","Data":"4e706c4cd32630b7f0587808d4b7aec8d9f258e504bd4093b4577aa7f7a8c670"} Jul 27 09:28:22.431212 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:28:22.431187 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-b5jrj" event={"ID":"85bf3791-57a8-4f6a-aba7-85b8a44e170c","Type":"ContainerStarted","Data":"07195a9b1c6257a720e12380ecb33d1a20a4531bbb23cd730b9d01d400780fcf"} Jul 27 09:28:25.979325 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:28:25.979278 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-7c49b99956-dkcst" Jul 27 09:28:25.979841 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:28:25.979350 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7c49b99956-dkcst" Jul 27 09:28:25.984337 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:28:25.984308 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-7c49b99956-dkcst" Jul 27 09:28:26.298558 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:28:26.298434 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-848w4_2c043d82-d300-41ba-b138-60a03d65ea11/konnectivity-agent/0.log" limit=1024 Jul 27 09:28:26.377018 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:28:26.376981 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-848w4_2c043d82-d300-41ba-b138-60a03d65ea11/konnectivity-agent/0.log" limit=1024 Jul 27 09:28:26.450816 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:28:26.450785 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-7c49b99956-dkcst" Jul 27 09:28:26.501315 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:28:26.501276 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6665866d46-vn7rh"] Jul 27 09:28:26.604386 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:28:26.604299 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-848w4_2c043d82-d300-41ba-b138-60a03d65ea11/konnectivity-agent/0.log" limit=1024 Jul 27 09:28:26.661350 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:28:26.661313 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-848w4_2c043d82-d300-41ba-b138-60a03d65ea11/konnectivity-agent/0.log" limit=1024 Jul 27 09:28:27.034499 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:28:27.034466 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-848f65bd86-bb84d" Jul 27 09:28:27.038374 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:28:27.038349 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-848f65bd86-bb84d" Jul 27 09:28:27.049151 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:28:27.049103 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-848w4_2c043d82-d300-41ba-b138-60a03d65ea11/konnectivity-agent/0.log" limit=1024 Jul 27 09:28:27.149900 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:28:27.149856 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-848w4_2c043d82-d300-41ba-b138-60a03d65ea11/konnectivity-agent/0.log" limit=1024 Jul 27 09:28:27.188331 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:28:27.188296 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6665866d46-vn7rh" Jul 27 09:28:27.640573 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:28:27.640539 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-848w4_2c043d82-d300-41ba-b138-60a03d65ea11/konnectivity-agent/0.log" limit=1024 Jul 27 09:28:27.743613 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:28:27.743577 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-848w4_2c043d82-d300-41ba-b138-60a03d65ea11/konnectivity-agent/0.log" limit=1024 Jul 27 09:28:27.887570 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:28:27.887536 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-848w4_2c043d82-d300-41ba-b138-60a03d65ea11/konnectivity-agent/0.log" limit=1024 Jul 27 09:28:35.449927 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:28:35.449865 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-848w4_2c043d82-d300-41ba-b138-60a03d65ea11/konnectivity-agent/0.log" limit=1024 Jul 27 09:28:42.644118 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:28:42.644082 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-848w4_2c043d82-d300-41ba-b138-60a03d65ea11/konnectivity-agent/0.log" limit=1024 Jul 27 09:28:42.706745 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:28:42.706711 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-848w4_2c043d82-d300-41ba-b138-60a03d65ea11/konnectivity-agent/0.log" limit=1024 Jul 27 09:28:44.295688 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:28:44.295653 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0d6ba35e-ec8a-4ec9-8520-0d32a5586a1d-metrics-certs\") pod \"network-metrics-daemon-trcl2\" (UID: \"0d6ba35e-ec8a-4ec9-8520-0d32a5586a1d\") " pod="openshift-multus/network-metrics-daemon-trcl2" Jul 27 09:28:44.297942 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:28:44.297916 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0d6ba35e-ec8a-4ec9-8520-0d32a5586a1d-metrics-certs\") pod \"network-metrics-daemon-trcl2\" (UID: \"0d6ba35e-ec8a-4ec9-8520-0d32a5586a1d\") " pod="openshift-multus/network-metrics-daemon-trcl2" Jul 27 09:28:44.461791 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:28:44.461750 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-fpm4f\"" Jul 27 09:28:44.469669 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:28:44.469641 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-trcl2" Jul 27 09:28:44.710503 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:28:44.710405 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-trcl2"] Jul 27 09:28:44.719380 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:28:44.719341 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0d6ba35e_ec8a_4ec9_8520_0d32a5586a1d.slice/crio-aa5b506a8f78baf9d218be600cb167521b45e2d64f44e8a564e6afd8f725097b WatchSource:0}: Error finding container aa5b506a8f78baf9d218be600cb167521b45e2d64f44e8a564e6afd8f725097b: Status 404 returned error can't find the container with id aa5b506a8f78baf9d218be600cb167521b45e2d64f44e8a564e6afd8f725097b Jul 27 09:28:45.527041 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:28:45.527004 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-trcl2" event={"ID":"0d6ba35e-ec8a-4ec9-8520-0d32a5586a1d","Type":"ContainerStarted","Data":"aa5b506a8f78baf9d218be600cb167521b45e2d64f44e8a564e6afd8f725097b"} Jul 27 09:28:45.556840 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:28:45.556777 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-848w4_2c043d82-d300-41ba-b138-60a03d65ea11/konnectivity-agent/0.log" limit=1024 Jul 27 09:28:45.622082 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:28:45.622047 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-848w4_2c043d82-d300-41ba-b138-60a03d65ea11/konnectivity-agent/0.log" limit=1024 Jul 27 09:28:46.532630 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:28:46.532590 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-trcl2" event={"ID":"0d6ba35e-ec8a-4ec9-8520-0d32a5586a1d","Type":"ContainerStarted","Data":"243baac32ad09cc3bae0e2d21876feaa39cb595183b0636ac9c1e7d88383b812"} Jul 27 09:28:46.533059 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:28:46.532658 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-trcl2" event={"ID":"0d6ba35e-ec8a-4ec9-8520-0d32a5586a1d","Type":"ContainerStarted","Data":"8ab8e764f4ba76e5b67d853b378d32d0400fad1d502d96ff9aabfa8e49ebf785"} Jul 27 09:28:46.550428 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:28:46.550377 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-trcl2" podStartSLOduration=252.210839345 podStartE2EDuration="4m13.550360237s" totalImagesPullingTime="1.339520892s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 09:24:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 09:28:46.549274308 +0000 UTC m=+254.513060290" watchObservedRunningTime="2026-07-27 09:28:46.550360237 +0000 UTC m=+254.514146193" Jul 27 09:28:46.636922 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:28:46.636889 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-848w4_2c043d82-d300-41ba-b138-60a03d65ea11/konnectivity-agent/0.log" limit=1024 Jul 27 09:28:46.738255 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:28:46.738218 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-848w4_2c043d82-d300-41ba-b138-60a03d65ea11/konnectivity-agent/0.log" limit=1024 Jul 27 09:28:51.521103 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:28:51.521053 2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-6665866d46-vn7rh" podUID="a0119ea8-ea1f-44b9-963c-8f361779b4e7" containerName="console" containerID="cri-o://5aa937e641e0728d67692e63fd13982a8c60f09dd169b82ce879717739400cb7" gracePeriod=15 Jul 27 09:28:51.796117 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:28:51.796044 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6665866d46-vn7rh" Jul 27 09:28:51.871692 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:28:51.871656 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a0119ea8-ea1f-44b9-963c-8f361779b4e7-console-oauth-config\") pod \"a0119ea8-ea1f-44b9-963c-8f361779b4e7\" (UID: \"a0119ea8-ea1f-44b9-963c-8f361779b4e7\") " Jul 27 09:28:51.871918 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:28:51.871704 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a0119ea8-ea1f-44b9-963c-8f361779b4e7-service-ca\") pod \"a0119ea8-ea1f-44b9-963c-8f361779b4e7\" (UID: \"a0119ea8-ea1f-44b9-963c-8f361779b4e7\") " Jul 27 09:28:51.871918 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:28:51.871722 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a0119ea8-ea1f-44b9-963c-8f361779b4e7-oauth-serving-cert\") pod \"a0119ea8-ea1f-44b9-963c-8f361779b4e7\" (UID: \"a0119ea8-ea1f-44b9-963c-8f361779b4e7\") " Jul 27 09:28:51.871918 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:28:51.871753 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a0119ea8-ea1f-44b9-963c-8f361779b4e7-console-config\") pod \"a0119ea8-ea1f-44b9-963c-8f361779b4e7\" (UID: \"a0119ea8-ea1f-44b9-963c-8f361779b4e7\") " Jul 27 09:28:51.871918 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:28:51.871794 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a0119ea8-ea1f-44b9-963c-8f361779b4e7-console-serving-cert\") pod \"a0119ea8-ea1f-44b9-963c-8f361779b4e7\" (UID: \"a0119ea8-ea1f-44b9-963c-8f361779b4e7\") " Jul 27 09:28:51.871918 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:28:51.871839 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5r8th\" (UniqueName: \"kubernetes.io/projected/a0119ea8-ea1f-44b9-963c-8f361779b4e7-kube-api-access-5r8th\") pod \"a0119ea8-ea1f-44b9-963c-8f361779b4e7\" (UID: \"a0119ea8-ea1f-44b9-963c-8f361779b4e7\") " Jul 27 09:28:51.872222 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:28:51.872186 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a0119ea8-ea1f-44b9-963c-8f361779b4e7-service-ca" (OuterVolumeSpecName: "service-ca") pod "a0119ea8-ea1f-44b9-963c-8f361779b4e7" (UID: "a0119ea8-ea1f-44b9-963c-8f361779b4e7"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 27 09:28:51.872336 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:28:51.872198 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a0119ea8-ea1f-44b9-963c-8f361779b4e7-console-config" (OuterVolumeSpecName: "console-config") pod "a0119ea8-ea1f-44b9-963c-8f361779b4e7" (UID: "a0119ea8-ea1f-44b9-963c-8f361779b4e7"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 27 09:28:51.872336 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:28:51.872197 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a0119ea8-ea1f-44b9-963c-8f361779b4e7-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "a0119ea8-ea1f-44b9-963c-8f361779b4e7" (UID: "a0119ea8-ea1f-44b9-963c-8f361779b4e7"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 27 09:28:51.874169 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:28:51.874139 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a0119ea8-ea1f-44b9-963c-8f361779b4e7-kube-api-access-5r8th" (OuterVolumeSpecName: "kube-api-access-5r8th") pod "a0119ea8-ea1f-44b9-963c-8f361779b4e7" (UID: "a0119ea8-ea1f-44b9-963c-8f361779b4e7"). InnerVolumeSpecName "kube-api-access-5r8th". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 27 09:28:51.874169 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:28:51.874157 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a0119ea8-ea1f-44b9-963c-8f361779b4e7-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "a0119ea8-ea1f-44b9-963c-8f361779b4e7" (UID: "a0119ea8-ea1f-44b9-963c-8f361779b4e7"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 27 09:28:51.874295 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:28:51.874140 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a0119ea8-ea1f-44b9-963c-8f361779b4e7-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "a0119ea8-ea1f-44b9-963c-8f361779b4e7" (UID: "a0119ea8-ea1f-44b9-963c-8f361779b4e7"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 27 09:28:51.972744 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:28:51.972707 2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5r8th\" (UniqueName: \"kubernetes.io/projected/a0119ea8-ea1f-44b9-963c-8f361779b4e7-kube-api-access-5r8th\") on node \"ip-10-0-128-103.ec2.internal\" DevicePath \"\"" Jul 27 09:28:51.972744 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:28:51.972739 2565 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a0119ea8-ea1f-44b9-963c-8f361779b4e7-console-oauth-config\") on node \"ip-10-0-128-103.ec2.internal\" DevicePath \"\"" Jul 27 09:28:51.972744 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:28:51.972748 2565 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a0119ea8-ea1f-44b9-963c-8f361779b4e7-service-ca\") on node \"ip-10-0-128-103.ec2.internal\" DevicePath \"\"" Jul 27 09:28:51.972993 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:28:51.972759 2565 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a0119ea8-ea1f-44b9-963c-8f361779b4e7-oauth-serving-cert\") on node \"ip-10-0-128-103.ec2.internal\" DevicePath \"\"" Jul 27 09:28:51.972993 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:28:51.972788 2565 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a0119ea8-ea1f-44b9-963c-8f361779b4e7-console-config\") on node \"ip-10-0-128-103.ec2.internal\" DevicePath \"\"" Jul 27 09:28:51.972993 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:28:51.972797 2565 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a0119ea8-ea1f-44b9-963c-8f361779b4e7-console-serving-cert\") on node \"ip-10-0-128-103.ec2.internal\" DevicePath \"\"" Jul 27 09:28:52.556534 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:28:52.556495 2565 generic.go:356] "Generic (PLEG): container finished" podID="a0119ea8-ea1f-44b9-963c-8f361779b4e7" containerID="5aa937e641e0728d67692e63fd13982a8c60f09dd169b82ce879717739400cb7" exitCode=0 Jul 27 09:28:52.556534 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:28:52.556525 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6665866d46-vn7rh" Jul 27 09:28:52.557063 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:28:52.556573 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6665866d46-vn7rh" event={"ID":"a0119ea8-ea1f-44b9-963c-8f361779b4e7","Type":"ContainerDied","Data":"5aa937e641e0728d67692e63fd13982a8c60f09dd169b82ce879717739400cb7"} Jul 27 09:28:52.557063 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:28:52.556627 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6665866d46-vn7rh" event={"ID":"a0119ea8-ea1f-44b9-963c-8f361779b4e7","Type":"ContainerDied","Data":"ec0d90390cb3db4b4a57a80c445ef45222174f3a75c7fc4f91059bdf882dd7c6"} Jul 27 09:28:52.557063 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:28:52.556652 2565 scope.go:117] "RemoveContainer" containerID="5aa937e641e0728d67692e63fd13982a8c60f09dd169b82ce879717739400cb7" Jul 27 09:28:52.566058 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:28:52.566038 2565 scope.go:117] "RemoveContainer" containerID="5aa937e641e0728d67692e63fd13982a8c60f09dd169b82ce879717739400cb7" Jul 27 09:28:52.566393 ip-10-0-128-103 kubenswrapper[2565]: E0727 09:28:52.566364 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5aa937e641e0728d67692e63fd13982a8c60f09dd169b82ce879717739400cb7\": container with ID starting with 5aa937e641e0728d67692e63fd13982a8c60f09dd169b82ce879717739400cb7 not found: ID does not exist" containerID="5aa937e641e0728d67692e63fd13982a8c60f09dd169b82ce879717739400cb7" Jul 27 09:28:52.566491 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:28:52.566402 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5aa937e641e0728d67692e63fd13982a8c60f09dd169b82ce879717739400cb7"} err="failed to get container status \"5aa937e641e0728d67692e63fd13982a8c60f09dd169b82ce879717739400cb7\": rpc error: code = NotFound desc = could not find container \"5aa937e641e0728d67692e63fd13982a8c60f09dd169b82ce879717739400cb7\": container with ID starting with 5aa937e641e0728d67692e63fd13982a8c60f09dd169b82ce879717739400cb7 not found: ID does not exist" Jul 27 09:28:52.580692 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:28:52.580663 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6665866d46-vn7rh"] Jul 27 09:28:52.584355 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:28:52.584329 2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-6665866d46-vn7rh"] Jul 27 09:28:53.779780 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:28:53.779726 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-848w4_2c043d82-d300-41ba-b138-60a03d65ea11/konnectivity-agent/0.log" limit=1024 Jul 27 09:28:53.854545 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:28:53.854508 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-848w4_2c043d82-d300-41ba-b138-60a03d65ea11/konnectivity-agent/0.log" limit=1024 Jul 27 09:28:53.918233 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:28:53.918187 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-848w4_2c043d82-d300-41ba-b138-60a03d65ea11/konnectivity-agent/0.log" limit=1024 Jul 27 09:28:54.566904 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:28:54.566810 2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a0119ea8-ea1f-44b9-963c-8f361779b4e7" path="/var/lib/kubelet/pods/a0119ea8-ea1f-44b9-963c-8f361779b4e7/volumes" Jul 27 09:29:01.684838 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:29:01.684798 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-848w4_2c043d82-d300-41ba-b138-60a03d65ea11/konnectivity-agent/0.log" limit=1024 Jul 27 09:29:01.906078 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:29:01.906038 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-848w4_2c043d82-d300-41ba-b138-60a03d65ea11/konnectivity-agent/0.log" limit=1024 Jul 27 09:29:06.702982 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:29:06.702947 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-8666cc54cb-cx9bx"] Jul 27 09:29:06.703365 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:29:06.703223 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a0119ea8-ea1f-44b9-963c-8f361779b4e7" containerName="console" Jul 27 09:29:06.703365 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:29:06.703233 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="a0119ea8-ea1f-44b9-963c-8f361779b4e7" containerName="console" Jul 27 09:29:06.703365 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:29:06.703286 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="a0119ea8-ea1f-44b9-963c-8f361779b4e7" containerName="console" Jul 27 09:29:06.705421 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:29:06.705398 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-8666cc54cb-cx9bx" Jul 27 09:29:06.708075 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:29:06.708054 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\"" Jul 27 09:29:06.708276 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:29:06.708257 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\"" Jul 27 09:29:06.708420 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:29:06.708171 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\"" Jul 27 09:29:06.708556 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:29:06.708054 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-6zrpz\"" Jul 27 09:29:06.708631 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:29:06.708561 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\"" Jul 27 09:29:06.708885 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:29:06.708865 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\"" Jul 27 09:29:06.723963 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:29:06.723934 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\"" Jul 27 09:29:06.724341 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:29:06.724313 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-8666cc54cb-cx9bx"] Jul 27 09:29:06.802266 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:29:06.802224 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f598fec9-8aac-4221-a931-0127a29b7617-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-8666cc54cb-cx9bx\" (UID: \"f598fec9-8aac-4221-a931-0127a29b7617\") " pod="openshift-monitoring/telemeter-client-8666cc54cb-cx9bx" Jul 27 09:29:06.802266 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:29:06.802267 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/f598fec9-8aac-4221-a931-0127a29b7617-secret-telemeter-client\") pod \"telemeter-client-8666cc54cb-cx9bx\" (UID: \"f598fec9-8aac-4221-a931-0127a29b7617\") " pod="openshift-monitoring/telemeter-client-8666cc54cb-cx9bx" Jul 27 09:29:06.802474 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:29:06.802292 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f598fec9-8aac-4221-a931-0127a29b7617-serving-certs-ca-bundle\") pod \"telemeter-client-8666cc54cb-cx9bx\" (UID: \"f598fec9-8aac-4221-a931-0127a29b7617\") " pod="openshift-monitoring/telemeter-client-8666cc54cb-cx9bx" Jul 27 09:29:06.802474 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:29:06.802417 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f598fec9-8aac-4221-a931-0127a29b7617-metrics-client-ca\") pod \"telemeter-client-8666cc54cb-cx9bx\" (UID: \"f598fec9-8aac-4221-a931-0127a29b7617\") " pod="openshift-monitoring/telemeter-client-8666cc54cb-cx9bx" Jul 27 09:29:06.802544 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:29:06.802473 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qr8t8\" (UniqueName: \"kubernetes.io/projected/f598fec9-8aac-4221-a931-0127a29b7617-kube-api-access-qr8t8\") pod \"telemeter-client-8666cc54cb-cx9bx\" (UID: \"f598fec9-8aac-4221-a931-0127a29b7617\") " pod="openshift-monitoring/telemeter-client-8666cc54cb-cx9bx" Jul 27 09:29:06.802544 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:29:06.802500 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/f598fec9-8aac-4221-a931-0127a29b7617-telemeter-client-tls\") pod \"telemeter-client-8666cc54cb-cx9bx\" (UID: \"f598fec9-8aac-4221-a931-0127a29b7617\") " pod="openshift-monitoring/telemeter-client-8666cc54cb-cx9bx" Jul 27 09:29:06.802544 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:29:06.802518 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f598fec9-8aac-4221-a931-0127a29b7617-telemeter-trusted-ca-bundle\") pod \"telemeter-client-8666cc54cb-cx9bx\" (UID: \"f598fec9-8aac-4221-a931-0127a29b7617\") " pod="openshift-monitoring/telemeter-client-8666cc54cb-cx9bx" Jul 27 09:29:06.802544 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:29:06.802538 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/f598fec9-8aac-4221-a931-0127a29b7617-federate-client-tls\") pod \"telemeter-client-8666cc54cb-cx9bx\" (UID: \"f598fec9-8aac-4221-a931-0127a29b7617\") " pod="openshift-monitoring/telemeter-client-8666cc54cb-cx9bx" Jul 27 09:29:06.903003 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:29:06.902962 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qr8t8\" (UniqueName: \"kubernetes.io/projected/f598fec9-8aac-4221-a931-0127a29b7617-kube-api-access-qr8t8\") pod \"telemeter-client-8666cc54cb-cx9bx\" (UID: \"f598fec9-8aac-4221-a931-0127a29b7617\") " pod="openshift-monitoring/telemeter-client-8666cc54cb-cx9bx" Jul 27 09:29:06.903003 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:29:06.903006 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/f598fec9-8aac-4221-a931-0127a29b7617-telemeter-client-tls\") pod \"telemeter-client-8666cc54cb-cx9bx\" (UID: \"f598fec9-8aac-4221-a931-0127a29b7617\") " pod="openshift-monitoring/telemeter-client-8666cc54cb-cx9bx" Jul 27 09:29:06.903260 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:29:06.903066 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f598fec9-8aac-4221-a931-0127a29b7617-telemeter-trusted-ca-bundle\") pod \"telemeter-client-8666cc54cb-cx9bx\" (UID: \"f598fec9-8aac-4221-a931-0127a29b7617\") " pod="openshift-monitoring/telemeter-client-8666cc54cb-cx9bx" Jul 27 09:29:06.903260 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:29:06.903104 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/f598fec9-8aac-4221-a931-0127a29b7617-federate-client-tls\") pod \"telemeter-client-8666cc54cb-cx9bx\" (UID: \"f598fec9-8aac-4221-a931-0127a29b7617\") " pod="openshift-monitoring/telemeter-client-8666cc54cb-cx9bx" Jul 27 09:29:06.903260 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:29:06.903171 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f598fec9-8aac-4221-a931-0127a29b7617-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-8666cc54cb-cx9bx\" (UID: \"f598fec9-8aac-4221-a931-0127a29b7617\") " pod="openshift-monitoring/telemeter-client-8666cc54cb-cx9bx" Jul 27 09:29:06.903260 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:29:06.903217 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/f598fec9-8aac-4221-a931-0127a29b7617-secret-telemeter-client\") pod \"telemeter-client-8666cc54cb-cx9bx\" (UID: \"f598fec9-8aac-4221-a931-0127a29b7617\") " pod="openshift-monitoring/telemeter-client-8666cc54cb-cx9bx" Jul 27 09:29:06.903260 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:29:06.903256 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f598fec9-8aac-4221-a931-0127a29b7617-serving-certs-ca-bundle\") pod \"telemeter-client-8666cc54cb-cx9bx\" (UID: \"f598fec9-8aac-4221-a931-0127a29b7617\") " pod="openshift-monitoring/telemeter-client-8666cc54cb-cx9bx" Jul 27 09:29:06.903500 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:29:06.903309 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f598fec9-8aac-4221-a931-0127a29b7617-metrics-client-ca\") pod \"telemeter-client-8666cc54cb-cx9bx\" (UID: \"f598fec9-8aac-4221-a931-0127a29b7617\") " pod="openshift-monitoring/telemeter-client-8666cc54cb-cx9bx" Jul 27 09:29:06.904295 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:29:06.904012 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f598fec9-8aac-4221-a931-0127a29b7617-metrics-client-ca\") pod \"telemeter-client-8666cc54cb-cx9bx\" (UID: \"f598fec9-8aac-4221-a931-0127a29b7617\") " pod="openshift-monitoring/telemeter-client-8666cc54cb-cx9bx" Jul 27 09:29:06.904295 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:29:06.904168 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f598fec9-8aac-4221-a931-0127a29b7617-telemeter-trusted-ca-bundle\") pod \"telemeter-client-8666cc54cb-cx9bx\" (UID: \"f598fec9-8aac-4221-a931-0127a29b7617\") " pod="openshift-monitoring/telemeter-client-8666cc54cb-cx9bx" Jul 27 09:29:06.904295 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:29:06.904246 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f598fec9-8aac-4221-a931-0127a29b7617-serving-certs-ca-bundle\") pod \"telemeter-client-8666cc54cb-cx9bx\" (UID: \"f598fec9-8aac-4221-a931-0127a29b7617\") " pod="openshift-monitoring/telemeter-client-8666cc54cb-cx9bx" Jul 27 09:29:06.907601 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:29:06.906927 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/f598fec9-8aac-4221-a931-0127a29b7617-secret-telemeter-client\") pod \"telemeter-client-8666cc54cb-cx9bx\" (UID: \"f598fec9-8aac-4221-a931-0127a29b7617\") " pod="openshift-monitoring/telemeter-client-8666cc54cb-cx9bx" Jul 27 09:29:06.907601 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:29:06.907003 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/f598fec9-8aac-4221-a931-0127a29b7617-federate-client-tls\") pod \"telemeter-client-8666cc54cb-cx9bx\" (UID: \"f598fec9-8aac-4221-a931-0127a29b7617\") " pod="openshift-monitoring/telemeter-client-8666cc54cb-cx9bx" Jul 27 09:29:06.907601 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:29:06.907279 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f598fec9-8aac-4221-a931-0127a29b7617-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-8666cc54cb-cx9bx\" (UID: \"f598fec9-8aac-4221-a931-0127a29b7617\") " pod="openshift-monitoring/telemeter-client-8666cc54cb-cx9bx" Jul 27 09:29:06.907877 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:29:06.907658 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/f598fec9-8aac-4221-a931-0127a29b7617-telemeter-client-tls\") pod \"telemeter-client-8666cc54cb-cx9bx\" (UID: \"f598fec9-8aac-4221-a931-0127a29b7617\") " pod="openshift-monitoring/telemeter-client-8666cc54cb-cx9bx" Jul 27 09:29:06.912274 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:29:06.912235 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qr8t8\" (UniqueName: \"kubernetes.io/projected/f598fec9-8aac-4221-a931-0127a29b7617-kube-api-access-qr8t8\") pod \"telemeter-client-8666cc54cb-cx9bx\" (UID: \"f598fec9-8aac-4221-a931-0127a29b7617\") " pod="openshift-monitoring/telemeter-client-8666cc54cb-cx9bx" Jul 27 09:29:07.023356 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:29:07.023274 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-8666cc54cb-cx9bx" Jul 27 09:29:07.182706 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:29:07.182665 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-8666cc54cb-cx9bx"] Jul 27 09:29:07.185610 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:29:07.185557 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf598fec9_8aac_4221_a931_0127a29b7617.slice/crio-bedaa034bfa65e7971d0d862a02e95e659d3180cb25ee6e2e4b6f39dee0f047e WatchSource:0}: Error finding container bedaa034bfa65e7971d0d862a02e95e659d3180cb25ee6e2e4b6f39dee0f047e: Status 404 returned error can't find the container with id bedaa034bfa65e7971d0d862a02e95e659d3180cb25ee6e2e4b6f39dee0f047e Jul 27 09:29:07.619435 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:29:07.619398 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-8666cc54cb-cx9bx" event={"ID":"f598fec9-8aac-4221-a931-0127a29b7617","Type":"ContainerStarted","Data":"bedaa034bfa65e7971d0d862a02e95e659d3180cb25ee6e2e4b6f39dee0f047e"} Jul 27 09:29:09.630207 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:29:09.630165 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-8666cc54cb-cx9bx" event={"ID":"f598fec9-8aac-4221-a931-0127a29b7617","Type":"ContainerStarted","Data":"c71f223f3cec6659ce9bc547633b89e88dc8a83db22428fa1d6a374dba9cc261"} Jul 27 09:29:10.636120 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:29:10.636084 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-8666cc54cb-cx9bx" event={"ID":"f598fec9-8aac-4221-a931-0127a29b7617","Type":"ContainerStarted","Data":"1b9ffd65f0a5a60fd30d7975c8dc7bc7fdf8a0d174580ce0a99bf9f04b45a8b8"} Jul 27 09:29:10.636513 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:29:10.636145 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-8666cc54cb-cx9bx" event={"ID":"f598fec9-8aac-4221-a931-0127a29b7617","Type":"ContainerStarted","Data":"8f52975b0c27d13700c76fe958560a5bc6ab4d6576731173e9a025643462f925"} Jul 27 09:29:10.659283 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:29:10.659234 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-8666cc54cb-cx9bx" podStartSLOduration=2.120453595 podStartE2EDuration="4.659216704s" totalImagesPullingTime="2.538763109s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 09:29:06 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 09:29:10.658681209 +0000 UTC m=+278.622467191" watchObservedRunningTime="2026-07-27 09:29:10.659216704 +0000 UTC m=+278.623002660" Jul 27 09:29:11.438854 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:29:11.438821 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-7dffb9899c-jnpmr"] Jul 27 09:29:11.441157 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:29:11.441136 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7dffb9899c-jnpmr" Jul 27 09:29:11.453248 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:29:11.453215 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7dffb9899c-jnpmr"] Jul 27 09:29:11.492015 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:29:11.491982 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-848w4_2c043d82-d300-41ba-b138-60a03d65ea11/konnectivity-agent/0.log" limit=1024 Jul 27 09:29:11.544020 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:29:11.543981 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b6629a29-0698-4cf5-81b3-0f94bd835a3a-console-serving-cert\") pod \"console-7dffb9899c-jnpmr\" (UID: \"b6629a29-0698-4cf5-81b3-0f94bd835a3a\") " pod="openshift-console/console-7dffb9899c-jnpmr" Jul 27 09:29:11.544020 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:29:11.544025 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ljb5x\" (UniqueName: \"kubernetes.io/projected/b6629a29-0698-4cf5-81b3-0f94bd835a3a-kube-api-access-ljb5x\") pod \"console-7dffb9899c-jnpmr\" (UID: \"b6629a29-0698-4cf5-81b3-0f94bd835a3a\") " pod="openshift-console/console-7dffb9899c-jnpmr" Jul 27 09:29:11.544290 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:29:11.544050 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b6629a29-0698-4cf5-81b3-0f94bd835a3a-oauth-serving-cert\") pod \"console-7dffb9899c-jnpmr\" (UID: \"b6629a29-0698-4cf5-81b3-0f94bd835a3a\") " pod="openshift-console/console-7dffb9899c-jnpmr" Jul 27 09:29:11.544290 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:29:11.544179 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b6629a29-0698-4cf5-81b3-0f94bd835a3a-trusted-ca-bundle\") pod \"console-7dffb9899c-jnpmr\" (UID: \"b6629a29-0698-4cf5-81b3-0f94bd835a3a\") " pod="openshift-console/console-7dffb9899c-jnpmr" Jul 27 09:29:11.544290 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:29:11.544228 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b6629a29-0698-4cf5-81b3-0f94bd835a3a-service-ca\") pod \"console-7dffb9899c-jnpmr\" (UID: \"b6629a29-0698-4cf5-81b3-0f94bd835a3a\") " pod="openshift-console/console-7dffb9899c-jnpmr" Jul 27 09:29:11.544290 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:29:11.544277 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b6629a29-0698-4cf5-81b3-0f94bd835a3a-console-config\") pod \"console-7dffb9899c-jnpmr\" (UID: \"b6629a29-0698-4cf5-81b3-0f94bd835a3a\") " pod="openshift-console/console-7dffb9899c-jnpmr" Jul 27 09:29:11.544446 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:29:11.544309 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b6629a29-0698-4cf5-81b3-0f94bd835a3a-console-oauth-config\") pod \"console-7dffb9899c-jnpmr\" (UID: \"b6629a29-0698-4cf5-81b3-0f94bd835a3a\") " pod="openshift-console/console-7dffb9899c-jnpmr" Jul 27 09:29:11.549006 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:29:11.548964 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-848w4_2c043d82-d300-41ba-b138-60a03d65ea11/konnectivity-agent/0.log" limit=1024 Jul 27 09:29:11.645498 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:29:11.645465 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b6629a29-0698-4cf5-81b3-0f94bd835a3a-trusted-ca-bundle\") pod \"console-7dffb9899c-jnpmr\" (UID: \"b6629a29-0698-4cf5-81b3-0f94bd835a3a\") " pod="openshift-console/console-7dffb9899c-jnpmr" Jul 27 09:29:11.646057 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:29:11.645512 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b6629a29-0698-4cf5-81b3-0f94bd835a3a-service-ca\") pod \"console-7dffb9899c-jnpmr\" (UID: \"b6629a29-0698-4cf5-81b3-0f94bd835a3a\") " pod="openshift-console/console-7dffb9899c-jnpmr" Jul 27 09:29:11.646057 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:29:11.645545 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b6629a29-0698-4cf5-81b3-0f94bd835a3a-console-config\") pod \"console-7dffb9899c-jnpmr\" (UID: \"b6629a29-0698-4cf5-81b3-0f94bd835a3a\") " pod="openshift-console/console-7dffb9899c-jnpmr" Jul 27 09:29:11.646057 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:29:11.645579 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b6629a29-0698-4cf5-81b3-0f94bd835a3a-console-oauth-config\") pod \"console-7dffb9899c-jnpmr\" (UID: \"b6629a29-0698-4cf5-81b3-0f94bd835a3a\") " pod="openshift-console/console-7dffb9899c-jnpmr" Jul 27 09:29:11.646057 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:29:11.645960 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b6629a29-0698-4cf5-81b3-0f94bd835a3a-console-serving-cert\") pod \"console-7dffb9899c-jnpmr\" (UID: \"b6629a29-0698-4cf5-81b3-0f94bd835a3a\") " pod="openshift-console/console-7dffb9899c-jnpmr" Jul 27 09:29:11.646283 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:29:11.646111 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ljb5x\" (UniqueName: \"kubernetes.io/projected/b6629a29-0698-4cf5-81b3-0f94bd835a3a-kube-api-access-ljb5x\") pod \"console-7dffb9899c-jnpmr\" (UID: \"b6629a29-0698-4cf5-81b3-0f94bd835a3a\") " pod="openshift-console/console-7dffb9899c-jnpmr" Jul 27 09:29:11.646283 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:29:11.646156 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b6629a29-0698-4cf5-81b3-0f94bd835a3a-oauth-serving-cert\") pod \"console-7dffb9899c-jnpmr\" (UID: \"b6629a29-0698-4cf5-81b3-0f94bd835a3a\") " pod="openshift-console/console-7dffb9899c-jnpmr" Jul 27 09:29:11.646400 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:29:11.646374 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b6629a29-0698-4cf5-81b3-0f94bd835a3a-console-config\") pod \"console-7dffb9899c-jnpmr\" (UID: \"b6629a29-0698-4cf5-81b3-0f94bd835a3a\") " pod="openshift-console/console-7dffb9899c-jnpmr" Jul 27 09:29:11.646486 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:29:11.646463 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b6629a29-0698-4cf5-81b3-0f94bd835a3a-service-ca\") pod \"console-7dffb9899c-jnpmr\" (UID: \"b6629a29-0698-4cf5-81b3-0f94bd835a3a\") " pod="openshift-console/console-7dffb9899c-jnpmr" Jul 27 09:29:11.646531 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:29:11.646514 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b6629a29-0698-4cf5-81b3-0f94bd835a3a-trusted-ca-bundle\") pod \"console-7dffb9899c-jnpmr\" (UID: \"b6629a29-0698-4cf5-81b3-0f94bd835a3a\") " pod="openshift-console/console-7dffb9899c-jnpmr" Jul 27 09:29:11.646724 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:29:11.646708 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b6629a29-0698-4cf5-81b3-0f94bd835a3a-oauth-serving-cert\") pod \"console-7dffb9899c-jnpmr\" (UID: \"b6629a29-0698-4cf5-81b3-0f94bd835a3a\") " pod="openshift-console/console-7dffb9899c-jnpmr" Jul 27 09:29:11.648111 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:29:11.648090 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b6629a29-0698-4cf5-81b3-0f94bd835a3a-console-oauth-config\") pod \"console-7dffb9899c-jnpmr\" (UID: \"b6629a29-0698-4cf5-81b3-0f94bd835a3a\") " pod="openshift-console/console-7dffb9899c-jnpmr" Jul 27 09:29:11.648345 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:29:11.648324 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b6629a29-0698-4cf5-81b3-0f94bd835a3a-console-serving-cert\") pod \"console-7dffb9899c-jnpmr\" (UID: \"b6629a29-0698-4cf5-81b3-0f94bd835a3a\") " pod="openshift-console/console-7dffb9899c-jnpmr" Jul 27 09:29:11.654545 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:29:11.654522 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ljb5x\" (UniqueName: \"kubernetes.io/projected/b6629a29-0698-4cf5-81b3-0f94bd835a3a-kube-api-access-ljb5x\") pod \"console-7dffb9899c-jnpmr\" (UID: \"b6629a29-0698-4cf5-81b3-0f94bd835a3a\") " pod="openshift-console/console-7dffb9899c-jnpmr" Jul 27 09:29:11.751995 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:29:11.751897 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7dffb9899c-jnpmr" Jul 27 09:29:11.901636 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:29:11.901442 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7dffb9899c-jnpmr"] Jul 27 09:29:11.904298 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:29:11.904259 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb6629a29_0698_4cf5_81b3_0f94bd835a3a.slice/crio-fc93762eef132c03d0382a892c33bcbc614723962b39c528c1608ab9c6c403f3 WatchSource:0}: Error finding container fc93762eef132c03d0382a892c33bcbc614723962b39c528c1608ab9c6c403f3: Status 404 returned error can't find the container with id fc93762eef132c03d0382a892c33bcbc614723962b39c528c1608ab9c6c403f3 Jul 27 09:29:12.647965 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:29:12.647928 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7dffb9899c-jnpmr" event={"ID":"b6629a29-0698-4cf5-81b3-0f94bd835a3a","Type":"ContainerStarted","Data":"1d4d767e4f950de06378baf2c3732aad80e581e554f3574691373c3f59415c64"} Jul 27 09:29:12.648351 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:29:12.647988 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7dffb9899c-jnpmr" event={"ID":"b6629a29-0698-4cf5-81b3-0f94bd835a3a","Type":"ContainerStarted","Data":"fc93762eef132c03d0382a892c33bcbc614723962b39c528c1608ab9c6c403f3"} Jul 27 09:29:12.669098 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:29:12.669050 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-7dffb9899c-jnpmr" podStartSLOduration=1.669034115 podStartE2EDuration="1.669034115s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 09:29:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 09:29:12.66756814 +0000 UTC m=+280.631354102" watchObservedRunningTime="2026-07-27 09:29:12.669034115 +0000 UTC m=+280.632820072" Jul 27 09:29:14.458276 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:29:14.458225 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-848w4_2c043d82-d300-41ba-b138-60a03d65ea11/konnectivity-agent/0.log" limit=1024 Jul 27 09:29:17.633484 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:29:17.633434 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-848w4_2c043d82-d300-41ba-b138-60a03d65ea11/konnectivity-agent/0.log" limit=1024 Jul 27 09:29:17.690913 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:29:17.690878 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-848w4_2c043d82-d300-41ba-b138-60a03d65ea11/konnectivity-agent/0.log" limit=1024 Jul 27 09:29:21.752106 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:29:21.752067 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-7dffb9899c-jnpmr" Jul 27 09:29:21.752467 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:29:21.752122 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7dffb9899c-jnpmr" Jul 27 09:29:21.757180 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:29:21.757155 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-7dffb9899c-jnpmr" Jul 27 09:29:22.689531 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:29:22.689495 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-7dffb9899c-jnpmr" Jul 27 09:29:22.738054 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:29:22.738022 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7c49b99956-dkcst"] Jul 27 09:29:22.895680 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:29:22.895631 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-848w4_2c043d82-d300-41ba-b138-60a03d65ea11/konnectivity-agent/0.log" limit=1024 Jul 27 09:29:22.990005 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:29:22.989913 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-848w4_2c043d82-d300-41ba-b138-60a03d65ea11/konnectivity-agent/0.log" limit=1024 Jul 27 09:29:23.048406 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:29:23.048364 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-848w4_2c043d82-d300-41ba-b138-60a03d65ea11/konnectivity-agent/0.log" limit=1024 Jul 27 09:29:32.391059 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:29:32.391028 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-w7grd_0badf8bd-7f64-46ef-872c-39f59dd11d0d/console-operator/2.log" Jul 27 09:29:32.393870 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:29:32.393844 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-w7grd_0badf8bd-7f64-46ef-872c-39f59dd11d0d/console-operator/2.log" Jul 27 09:29:32.395911 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:29:32.395889 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-mqhsn_5c031f05-bf4a-4ff7-81ad-381d31b163ff/ovn-acl-logging/0.log" Jul 27 09:29:32.398844 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:29:32.398818 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-mqhsn_5c031f05-bf4a-4ff7-81ad-381d31b163ff/ovn-acl-logging/0.log" Jul 27 09:29:32.407676 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:29:32.407655 2565 kubelet.go:1648] "Image garbage collection succeeded" Jul 27 09:29:47.758930 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:29:47.758861 2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-7c49b99956-dkcst" podUID="13306691-0229-4af9-a3f8-a0d5c09b9164" containerName="console" containerID="cri-o://18d96fe4f89cd71b3cf47ff89b82449ca282c428d80f69c366bf13d807418c25" gracePeriod=15 Jul 27 09:29:47.995925 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:29:47.995900 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7c49b99956-dkcst" Jul 27 09:29:48.061273 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:29:48.061183 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/13306691-0229-4af9-a3f8-a0d5c09b9164-trusted-ca-bundle\") pod \"13306691-0229-4af9-a3f8-a0d5c09b9164\" (UID: \"13306691-0229-4af9-a3f8-a0d5c09b9164\") " Jul 27 09:29:48.061273 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:29:48.061233 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/13306691-0229-4af9-a3f8-a0d5c09b9164-console-config\") pod \"13306691-0229-4af9-a3f8-a0d5c09b9164\" (UID: \"13306691-0229-4af9-a3f8-a0d5c09b9164\") " Jul 27 09:29:48.061273 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:29:48.061262 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/13306691-0229-4af9-a3f8-a0d5c09b9164-service-ca\") pod \"13306691-0229-4af9-a3f8-a0d5c09b9164\" (UID: \"13306691-0229-4af9-a3f8-a0d5c09b9164\") " Jul 27 09:29:48.061524 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:29:48.061285 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8t9z2\" (UniqueName: \"kubernetes.io/projected/13306691-0229-4af9-a3f8-a0d5c09b9164-kube-api-access-8t9z2\") pod \"13306691-0229-4af9-a3f8-a0d5c09b9164\" (UID: \"13306691-0229-4af9-a3f8-a0d5c09b9164\") " Jul 27 09:29:48.061524 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:29:48.061306 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/13306691-0229-4af9-a3f8-a0d5c09b9164-oauth-serving-cert\") pod \"13306691-0229-4af9-a3f8-a0d5c09b9164\" (UID: \"13306691-0229-4af9-a3f8-a0d5c09b9164\") " Jul 27 09:29:48.061524 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:29:48.061326 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/13306691-0229-4af9-a3f8-a0d5c09b9164-console-serving-cert\") pod \"13306691-0229-4af9-a3f8-a0d5c09b9164\" (UID: \"13306691-0229-4af9-a3f8-a0d5c09b9164\") " Jul 27 09:29:48.061524 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:29:48.061378 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/13306691-0229-4af9-a3f8-a0d5c09b9164-console-oauth-config\") pod \"13306691-0229-4af9-a3f8-a0d5c09b9164\" (UID: \"13306691-0229-4af9-a3f8-a0d5c09b9164\") " Jul 27 09:29:48.061730 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:29:48.061631 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/13306691-0229-4af9-a3f8-a0d5c09b9164-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "13306691-0229-4af9-a3f8-a0d5c09b9164" (UID: "13306691-0229-4af9-a3f8-a0d5c09b9164"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 27 09:29:48.061730 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:29:48.061697 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/13306691-0229-4af9-a3f8-a0d5c09b9164-console-config" (OuterVolumeSpecName: "console-config") pod "13306691-0229-4af9-a3f8-a0d5c09b9164" (UID: "13306691-0229-4af9-a3f8-a0d5c09b9164"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 27 09:29:48.061730 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:29:48.061700 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/13306691-0229-4af9-a3f8-a0d5c09b9164-service-ca" (OuterVolumeSpecName: "service-ca") pod "13306691-0229-4af9-a3f8-a0d5c09b9164" (UID: "13306691-0229-4af9-a3f8-a0d5c09b9164"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 27 09:29:48.062071 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:29:48.062036 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/13306691-0229-4af9-a3f8-a0d5c09b9164-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "13306691-0229-4af9-a3f8-a0d5c09b9164" (UID: "13306691-0229-4af9-a3f8-a0d5c09b9164"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 27 09:29:48.063660 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:29:48.063635 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/13306691-0229-4af9-a3f8-a0d5c09b9164-kube-api-access-8t9z2" (OuterVolumeSpecName: "kube-api-access-8t9z2") pod "13306691-0229-4af9-a3f8-a0d5c09b9164" (UID: "13306691-0229-4af9-a3f8-a0d5c09b9164"). InnerVolumeSpecName "kube-api-access-8t9z2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 27 09:29:48.063777 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:29:48.063629 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/13306691-0229-4af9-a3f8-a0d5c09b9164-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "13306691-0229-4af9-a3f8-a0d5c09b9164" (UID: "13306691-0229-4af9-a3f8-a0d5c09b9164"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 27 09:29:48.063777 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:29:48.063654 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/13306691-0229-4af9-a3f8-a0d5c09b9164-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "13306691-0229-4af9-a3f8-a0d5c09b9164" (UID: "13306691-0229-4af9-a3f8-a0d5c09b9164"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 27 09:29:48.162243 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:29:48.162207 2565 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/13306691-0229-4af9-a3f8-a0d5c09b9164-console-oauth-config\") on node \"ip-10-0-128-103.ec2.internal\" DevicePath \"\"" Jul 27 09:29:48.162243 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:29:48.162236 2565 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/13306691-0229-4af9-a3f8-a0d5c09b9164-trusted-ca-bundle\") on node \"ip-10-0-128-103.ec2.internal\" DevicePath \"\"" Jul 27 09:29:48.162243 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:29:48.162247 2565 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/13306691-0229-4af9-a3f8-a0d5c09b9164-console-config\") on node \"ip-10-0-128-103.ec2.internal\" DevicePath \"\"" Jul 27 09:29:48.162464 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:29:48.162256 2565 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/13306691-0229-4af9-a3f8-a0d5c09b9164-service-ca\") on node \"ip-10-0-128-103.ec2.internal\" DevicePath \"\"" Jul 27 09:29:48.162464 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:29:48.162267 2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8t9z2\" (UniqueName: \"kubernetes.io/projected/13306691-0229-4af9-a3f8-a0d5c09b9164-kube-api-access-8t9z2\") on node \"ip-10-0-128-103.ec2.internal\" DevicePath \"\"" Jul 27 09:29:48.162464 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:29:48.162284 2565 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/13306691-0229-4af9-a3f8-a0d5c09b9164-oauth-serving-cert\") on node \"ip-10-0-128-103.ec2.internal\" DevicePath \"\"" Jul 27 09:29:48.162464 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:29:48.162292 2565 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/13306691-0229-4af9-a3f8-a0d5c09b9164-console-serving-cert\") on node \"ip-10-0-128-103.ec2.internal\" DevicePath \"\"" Jul 27 09:29:48.788629 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:29:48.788588 2565 generic.go:356] "Generic (PLEG): container finished" podID="13306691-0229-4af9-a3f8-a0d5c09b9164" containerID="18d96fe4f89cd71b3cf47ff89b82449ca282c428d80f69c366bf13d807418c25" exitCode=0 Jul 27 09:29:48.789048 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:29:48.788647 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7c49b99956-dkcst" Jul 27 09:29:48.789048 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:29:48.788673 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7c49b99956-dkcst" event={"ID":"13306691-0229-4af9-a3f8-a0d5c09b9164","Type":"ContainerDied","Data":"18d96fe4f89cd71b3cf47ff89b82449ca282c428d80f69c366bf13d807418c25"} Jul 27 09:29:48.789048 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:29:48.788742 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7c49b99956-dkcst" event={"ID":"13306691-0229-4af9-a3f8-a0d5c09b9164","Type":"ContainerDied","Data":"6907638b6b921ba170db80fe3f026b7fe3a3f3dbcc1e71efc812cf13777aa9cf"} Jul 27 09:29:48.789048 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:29:48.788762 2565 scope.go:117] "RemoveContainer" containerID="18d96fe4f89cd71b3cf47ff89b82449ca282c428d80f69c366bf13d807418c25" Jul 27 09:29:48.796957 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:29:48.796938 2565 scope.go:117] "RemoveContainer" containerID="18d96fe4f89cd71b3cf47ff89b82449ca282c428d80f69c366bf13d807418c25" Jul 27 09:29:48.797220 ip-10-0-128-103 kubenswrapper[2565]: E0727 09:29:48.797201 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"18d96fe4f89cd71b3cf47ff89b82449ca282c428d80f69c366bf13d807418c25\": container with ID starting with 18d96fe4f89cd71b3cf47ff89b82449ca282c428d80f69c366bf13d807418c25 not found: ID does not exist" containerID="18d96fe4f89cd71b3cf47ff89b82449ca282c428d80f69c366bf13d807418c25" Jul 27 09:29:48.797272 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:29:48.797232 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"18d96fe4f89cd71b3cf47ff89b82449ca282c428d80f69c366bf13d807418c25"} err="failed to get container status \"18d96fe4f89cd71b3cf47ff89b82449ca282c428d80f69c366bf13d807418c25\": rpc error: code = NotFound desc = could not find container \"18d96fe4f89cd71b3cf47ff89b82449ca282c428d80f69c366bf13d807418c25\": container with ID starting with 18d96fe4f89cd71b3cf47ff89b82449ca282c428d80f69c366bf13d807418c25 not found: ID does not exist" Jul 27 09:29:48.805994 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:29:48.805952 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7c49b99956-dkcst"] Jul 27 09:29:48.811536 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:29:48.811501 2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-7c49b99956-dkcst"] Jul 27 09:29:50.562821 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:29:50.562793 2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="13306691-0229-4af9-a3f8-a0d5c09b9164" path="/var/lib/kubelet/pods/13306691-0229-4af9-a3f8-a0d5c09b9164/volumes" Jul 27 09:30:01.627547 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:30:01.627514 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-848w4_2c043d82-d300-41ba-b138-60a03d65ea11/konnectivity-agent/0.log" limit=1024 Jul 27 09:30:01.785871 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:30:01.785839 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-848w4_2c043d82-d300-41ba-b138-60a03d65ea11/konnectivity-agent/0.log" limit=1024 Jul 27 09:30:16.904079 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:30:16.904046 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-848w4_2c043d82-d300-41ba-b138-60a03d65ea11/konnectivity-agent/0.log" limit=1024 Jul 27 09:30:16.959140 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:30:16.959100 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-848w4_2c043d82-d300-41ba-b138-60a03d65ea11/konnectivity-agent/0.log" limit=1024 Jul 27 09:30:28.870824 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:30:28.870732 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-848w4_2c043d82-d300-41ba-b138-60a03d65ea11/konnectivity-agent/0.log" limit=1024 Jul 27 09:30:28.931887 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:30:28.931852 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-848w4_2c043d82-d300-41ba-b138-60a03d65ea11/konnectivity-agent/0.log" limit=1024 Jul 27 09:30:37.517711 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:30:37.517669 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-848w4_2c043d82-d300-41ba-b138-60a03d65ea11/konnectivity-agent/0.log" limit=1024 Jul 27 09:30:42.692000 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:30:42.691961 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-848w4_2c043d82-d300-41ba-b138-60a03d65ea11/konnectivity-agent/0.log" limit=1024 Jul 27 09:30:42.793818 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:30:42.793790 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-848w4_2c043d82-d300-41ba-b138-60a03d65ea11/konnectivity-agent/0.log" limit=1024 Jul 27 09:30:56.128537 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:30:56.128504 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-cf67b94c4-7cpbh"] Jul 27 09:30:56.129108 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:30:56.128821 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="13306691-0229-4af9-a3f8-a0d5c09b9164" containerName="console" Jul 27 09:30:56.129108 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:30:56.128834 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="13306691-0229-4af9-a3f8-a0d5c09b9164" containerName="console" Jul 27 09:30:56.129108 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:30:56.128910 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="13306691-0229-4af9-a3f8-a0d5c09b9164" containerName="console" Jul 27 09:30:56.131661 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:30:56.131633 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-cf67b94c4-7cpbh" Jul 27 09:30:56.144092 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:30:56.144069 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-cf67b94c4-7cpbh"] Jul 27 09:30:56.176747 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:30:56.176717 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-848w4_2c043d82-d300-41ba-b138-60a03d65ea11/konnectivity-agent/0.log" limit=1024 Jul 27 09:30:56.231857 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:30:56.231822 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-848w4_2c043d82-d300-41ba-b138-60a03d65ea11/konnectivity-agent/0.log" limit=1024 Jul 27 09:30:56.232810 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:30:56.232760 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/44a40905-5ef0-41be-9da8-6862bdc8ed22-oauth-serving-cert\") pod \"console-cf67b94c4-7cpbh\" (UID: \"44a40905-5ef0-41be-9da8-6862bdc8ed22\") " pod="openshift-console/console-cf67b94c4-7cpbh" Jul 27 09:30:56.232894 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:30:56.232869 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/44a40905-5ef0-41be-9da8-6862bdc8ed22-console-config\") pod \"console-cf67b94c4-7cpbh\" (UID: \"44a40905-5ef0-41be-9da8-6862bdc8ed22\") " pod="openshift-console/console-cf67b94c4-7cpbh" Jul 27 09:30:56.232957 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:30:56.232936 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/44a40905-5ef0-41be-9da8-6862bdc8ed22-console-oauth-config\") pod \"console-cf67b94c4-7cpbh\" (UID: \"44a40905-5ef0-41be-9da8-6862bdc8ed22\") " pod="openshift-console/console-cf67b94c4-7cpbh" Jul 27 09:30:56.232997 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:30:56.232972 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/44a40905-5ef0-41be-9da8-6862bdc8ed22-console-serving-cert\") pod \"console-cf67b94c4-7cpbh\" (UID: \"44a40905-5ef0-41be-9da8-6862bdc8ed22\") " pod="openshift-console/console-cf67b94c4-7cpbh" Jul 27 09:30:56.232997 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:30:56.232991 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/44a40905-5ef0-41be-9da8-6862bdc8ed22-service-ca\") pod \"console-cf67b94c4-7cpbh\" (UID: \"44a40905-5ef0-41be-9da8-6862bdc8ed22\") " pod="openshift-console/console-cf67b94c4-7cpbh" Jul 27 09:30:56.233070 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:30:56.233018 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wjq27\" (UniqueName: \"kubernetes.io/projected/44a40905-5ef0-41be-9da8-6862bdc8ed22-kube-api-access-wjq27\") pod \"console-cf67b94c4-7cpbh\" (UID: \"44a40905-5ef0-41be-9da8-6862bdc8ed22\") " pod="openshift-console/console-cf67b94c4-7cpbh" Jul 27 09:30:56.233113 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:30:56.233071 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/44a40905-5ef0-41be-9da8-6862bdc8ed22-trusted-ca-bundle\") pod \"console-cf67b94c4-7cpbh\" (UID: \"44a40905-5ef0-41be-9da8-6862bdc8ed22\") " pod="openshift-console/console-cf67b94c4-7cpbh" Jul 27 09:30:56.333642 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:30:56.333603 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/44a40905-5ef0-41be-9da8-6862bdc8ed22-trusted-ca-bundle\") pod \"console-cf67b94c4-7cpbh\" (UID: \"44a40905-5ef0-41be-9da8-6862bdc8ed22\") " pod="openshift-console/console-cf67b94c4-7cpbh" Jul 27 09:30:56.333642 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:30:56.333644 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/44a40905-5ef0-41be-9da8-6862bdc8ed22-oauth-serving-cert\") pod \"console-cf67b94c4-7cpbh\" (UID: \"44a40905-5ef0-41be-9da8-6862bdc8ed22\") " pod="openshift-console/console-cf67b94c4-7cpbh" Jul 27 09:30:56.333921 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:30:56.333745 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/44a40905-5ef0-41be-9da8-6862bdc8ed22-console-config\") pod \"console-cf67b94c4-7cpbh\" (UID: \"44a40905-5ef0-41be-9da8-6862bdc8ed22\") " pod="openshift-console/console-cf67b94c4-7cpbh" Jul 27 09:30:56.333921 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:30:56.333803 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/44a40905-5ef0-41be-9da8-6862bdc8ed22-console-oauth-config\") pod \"console-cf67b94c4-7cpbh\" (UID: \"44a40905-5ef0-41be-9da8-6862bdc8ed22\") " pod="openshift-console/console-cf67b94c4-7cpbh" Jul 27 09:30:56.333921 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:30:56.333824 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/44a40905-5ef0-41be-9da8-6862bdc8ed22-console-serving-cert\") pod \"console-cf67b94c4-7cpbh\" (UID: \"44a40905-5ef0-41be-9da8-6862bdc8ed22\") " pod="openshift-console/console-cf67b94c4-7cpbh" Jul 27 09:30:56.333921 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:30:56.333847 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/44a40905-5ef0-41be-9da8-6862bdc8ed22-service-ca\") pod \"console-cf67b94c4-7cpbh\" (UID: \"44a40905-5ef0-41be-9da8-6862bdc8ed22\") " pod="openshift-console/console-cf67b94c4-7cpbh" Jul 27 09:30:56.333921 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:30:56.333888 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wjq27\" (UniqueName: \"kubernetes.io/projected/44a40905-5ef0-41be-9da8-6862bdc8ed22-kube-api-access-wjq27\") pod \"console-cf67b94c4-7cpbh\" (UID: \"44a40905-5ef0-41be-9da8-6862bdc8ed22\") " pod="openshift-console/console-cf67b94c4-7cpbh" Jul 27 09:30:56.334576 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:30:56.334545 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/44a40905-5ef0-41be-9da8-6862bdc8ed22-oauth-serving-cert\") pod \"console-cf67b94c4-7cpbh\" (UID: \"44a40905-5ef0-41be-9da8-6862bdc8ed22\") " pod="openshift-console/console-cf67b94c4-7cpbh" Jul 27 09:30:56.334785 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:30:56.334567 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/44a40905-5ef0-41be-9da8-6862bdc8ed22-trusted-ca-bundle\") pod \"console-cf67b94c4-7cpbh\" (UID: \"44a40905-5ef0-41be-9da8-6862bdc8ed22\") " pod="openshift-console/console-cf67b94c4-7cpbh" Jul 27 09:30:56.334915 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:30:56.334602 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/44a40905-5ef0-41be-9da8-6862bdc8ed22-service-ca\") pod \"console-cf67b94c4-7cpbh\" (UID: \"44a40905-5ef0-41be-9da8-6862bdc8ed22\") " pod="openshift-console/console-cf67b94c4-7cpbh" Jul 27 09:30:56.334985 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:30:56.334959 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/44a40905-5ef0-41be-9da8-6862bdc8ed22-console-config\") pod \"console-cf67b94c4-7cpbh\" (UID: \"44a40905-5ef0-41be-9da8-6862bdc8ed22\") " pod="openshift-console/console-cf67b94c4-7cpbh" Jul 27 09:30:56.336364 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:30:56.336341 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/44a40905-5ef0-41be-9da8-6862bdc8ed22-console-oauth-config\") pod \"console-cf67b94c4-7cpbh\" (UID: \"44a40905-5ef0-41be-9da8-6862bdc8ed22\") " pod="openshift-console/console-cf67b94c4-7cpbh" Jul 27 09:30:56.336454 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:30:56.336424 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/44a40905-5ef0-41be-9da8-6862bdc8ed22-console-serving-cert\") pod \"console-cf67b94c4-7cpbh\" (UID: \"44a40905-5ef0-41be-9da8-6862bdc8ed22\") " pod="openshift-console/console-cf67b94c4-7cpbh" Jul 27 09:30:56.342376 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:30:56.342349 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wjq27\" (UniqueName: \"kubernetes.io/projected/44a40905-5ef0-41be-9da8-6862bdc8ed22-kube-api-access-wjq27\") pod \"console-cf67b94c4-7cpbh\" (UID: \"44a40905-5ef0-41be-9da8-6862bdc8ed22\") " pod="openshift-console/console-cf67b94c4-7cpbh" Jul 27 09:30:56.441046 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:30:56.440947 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-cf67b94c4-7cpbh" Jul 27 09:30:56.576996 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:30:56.576970 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-cf67b94c4-7cpbh"] Jul 27 09:30:56.579488 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:30:56.579453 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod44a40905_5ef0_41be_9da8_6862bdc8ed22.slice/crio-2c405d938f3f10b1af5e195eb6038b8ca9686ff82c50cd7625eec886beaea93d WatchSource:0}: Error finding container 2c405d938f3f10b1af5e195eb6038b8ca9686ff82c50cd7625eec886beaea93d: Status 404 returned error can't find the container with id 2c405d938f3f10b1af5e195eb6038b8ca9686ff82c50cd7625eec886beaea93d Jul 27 09:30:57.054538 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:30:57.054505 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-cf67b94c4-7cpbh" event={"ID":"44a40905-5ef0-41be-9da8-6862bdc8ed22","Type":"ContainerStarted","Data":"de46ab06b062db0c1f046e3660c2db7605f6c2f92171dab48ebc8cbdac4605a0"} Jul 27 09:30:57.054876 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:30:57.054554 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-cf67b94c4-7cpbh" event={"ID":"44a40905-5ef0-41be-9da8-6862bdc8ed22","Type":"ContainerStarted","Data":"2c405d938f3f10b1af5e195eb6038b8ca9686ff82c50cd7625eec886beaea93d"} Jul 27 09:30:57.078620 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:30:57.078569 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-cf67b94c4-7cpbh" podStartSLOduration=1.078555218 podStartE2EDuration="1.078555218s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 09:30:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 09:30:57.077916719 +0000 UTC m=+385.041702710" watchObservedRunningTime="2026-07-27 09:30:57.078555218 +0000 UTC m=+385.042341172" Jul 27 09:31:06.442144 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:31:06.442105 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-cf67b94c4-7cpbh" Jul 27 09:31:06.442548 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:31:06.442187 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-cf67b94c4-7cpbh" Jul 27 09:31:06.446835 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:31:06.446808 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-cf67b94c4-7cpbh" Jul 27 09:31:07.095458 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:31:07.095426 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-cf67b94c4-7cpbh" Jul 27 09:31:07.142925 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:31:07.142894 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7dffb9899c-jnpmr"] Jul 27 09:31:07.255838 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:31:07.255781 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-848w4_2c043d82-d300-41ba-b138-60a03d65ea11/konnectivity-agent/0.log" limit=1024 Jul 27 09:31:07.322525 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:31:07.322498 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-848w4_2c043d82-d300-41ba-b138-60a03d65ea11/konnectivity-agent/0.log" limit=1024 Jul 27 09:31:07.389543 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:31:07.389496 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-848w4_2c043d82-d300-41ba-b138-60a03d65ea11/konnectivity-agent/0.log" limit=1024 Jul 27 09:31:32.163981 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:31:32.163915 2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-7dffb9899c-jnpmr" podUID="b6629a29-0698-4cf5-81b3-0f94bd835a3a" containerName="console" containerID="cri-o://1d4d767e4f950de06378baf2c3732aad80e581e554f3574691373c3f59415c64" gracePeriod=15 Jul 27 09:31:32.400296 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:31:32.400270 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7dffb9899c-jnpmr" Jul 27 09:31:32.524027 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:31:32.523951 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b6629a29-0698-4cf5-81b3-0f94bd835a3a-console-oauth-config\") pod \"b6629a29-0698-4cf5-81b3-0f94bd835a3a\" (UID: \"b6629a29-0698-4cf5-81b3-0f94bd835a3a\") " Jul 27 09:31:32.524027 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:31:32.523992 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b6629a29-0698-4cf5-81b3-0f94bd835a3a-service-ca\") pod \"b6629a29-0698-4cf5-81b3-0f94bd835a3a\" (UID: \"b6629a29-0698-4cf5-81b3-0f94bd835a3a\") " Jul 27 09:31:32.524249 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:31:32.524041 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ljb5x\" (UniqueName: \"kubernetes.io/projected/b6629a29-0698-4cf5-81b3-0f94bd835a3a-kube-api-access-ljb5x\") pod \"b6629a29-0698-4cf5-81b3-0f94bd835a3a\" (UID: \"b6629a29-0698-4cf5-81b3-0f94bd835a3a\") " Jul 27 09:31:32.524249 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:31:32.524073 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b6629a29-0698-4cf5-81b3-0f94bd835a3a-oauth-serving-cert\") pod \"b6629a29-0698-4cf5-81b3-0f94bd835a3a\" (UID: \"b6629a29-0698-4cf5-81b3-0f94bd835a3a\") " Jul 27 09:31:32.524249 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:31:32.524103 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b6629a29-0698-4cf5-81b3-0f94bd835a3a-console-config\") pod \"b6629a29-0698-4cf5-81b3-0f94bd835a3a\" (UID: \"b6629a29-0698-4cf5-81b3-0f94bd835a3a\") " Jul 27 09:31:32.524249 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:31:32.524157 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b6629a29-0698-4cf5-81b3-0f94bd835a3a-console-serving-cert\") pod \"b6629a29-0698-4cf5-81b3-0f94bd835a3a\" (UID: \"b6629a29-0698-4cf5-81b3-0f94bd835a3a\") " Jul 27 09:31:32.524249 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:31:32.524190 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b6629a29-0698-4cf5-81b3-0f94bd835a3a-trusted-ca-bundle\") pod \"b6629a29-0698-4cf5-81b3-0f94bd835a3a\" (UID: \"b6629a29-0698-4cf5-81b3-0f94bd835a3a\") " Jul 27 09:31:32.524505 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:31:32.524485 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b6629a29-0698-4cf5-81b3-0f94bd835a3a-service-ca" (OuterVolumeSpecName: "service-ca") pod "b6629a29-0698-4cf5-81b3-0f94bd835a3a" (UID: "b6629a29-0698-4cf5-81b3-0f94bd835a3a"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 27 09:31:32.524569 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:31:32.524550 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b6629a29-0698-4cf5-81b3-0f94bd835a3a-console-config" (OuterVolumeSpecName: "console-config") pod "b6629a29-0698-4cf5-81b3-0f94bd835a3a" (UID: "b6629a29-0698-4cf5-81b3-0f94bd835a3a"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 27 09:31:32.524637 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:31:32.524615 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b6629a29-0698-4cf5-81b3-0f94bd835a3a-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "b6629a29-0698-4cf5-81b3-0f94bd835a3a" (UID: "b6629a29-0698-4cf5-81b3-0f94bd835a3a"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 27 09:31:32.524904 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:31:32.524881 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b6629a29-0698-4cf5-81b3-0f94bd835a3a-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "b6629a29-0698-4cf5-81b3-0f94bd835a3a" (UID: "b6629a29-0698-4cf5-81b3-0f94bd835a3a"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 27 09:31:32.526242 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:31:32.526210 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b6629a29-0698-4cf5-81b3-0f94bd835a3a-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "b6629a29-0698-4cf5-81b3-0f94bd835a3a" (UID: "b6629a29-0698-4cf5-81b3-0f94bd835a3a"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 27 09:31:32.526355 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:31:32.526323 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b6629a29-0698-4cf5-81b3-0f94bd835a3a-kube-api-access-ljb5x" (OuterVolumeSpecName: "kube-api-access-ljb5x") pod "b6629a29-0698-4cf5-81b3-0f94bd835a3a" (UID: "b6629a29-0698-4cf5-81b3-0f94bd835a3a"). InnerVolumeSpecName "kube-api-access-ljb5x". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 27 09:31:32.526355 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:31:32.526331 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b6629a29-0698-4cf5-81b3-0f94bd835a3a-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "b6629a29-0698-4cf5-81b3-0f94bd835a3a" (UID: "b6629a29-0698-4cf5-81b3-0f94bd835a3a"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 27 09:31:32.631544 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:31:32.627660 2565 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b6629a29-0698-4cf5-81b3-0f94bd835a3a-console-serving-cert\") on node \"ip-10-0-128-103.ec2.internal\" DevicePath \"\"" Jul 27 09:31:32.631544 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:31:32.627699 2565 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b6629a29-0698-4cf5-81b3-0f94bd835a3a-trusted-ca-bundle\") on node \"ip-10-0-128-103.ec2.internal\" DevicePath \"\"" Jul 27 09:31:32.631544 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:31:32.627720 2565 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b6629a29-0698-4cf5-81b3-0f94bd835a3a-console-oauth-config\") on node \"ip-10-0-128-103.ec2.internal\" DevicePath \"\"" Jul 27 09:31:32.631544 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:31:32.627735 2565 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b6629a29-0698-4cf5-81b3-0f94bd835a3a-service-ca\") on node \"ip-10-0-128-103.ec2.internal\" DevicePath \"\"" Jul 27 09:31:32.631544 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:31:32.627751 2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ljb5x\" (UniqueName: \"kubernetes.io/projected/b6629a29-0698-4cf5-81b3-0f94bd835a3a-kube-api-access-ljb5x\") on node \"ip-10-0-128-103.ec2.internal\" DevicePath \"\"" Jul 27 09:31:32.631544 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:31:32.627791 2565 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b6629a29-0698-4cf5-81b3-0f94bd835a3a-oauth-serving-cert\") on node \"ip-10-0-128-103.ec2.internal\" DevicePath \"\"" Jul 27 09:31:32.631544 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:31:32.627811 2565 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b6629a29-0698-4cf5-81b3-0f94bd835a3a-console-config\") on node \"ip-10-0-128-103.ec2.internal\" DevicePath \"\"" Jul 27 09:31:33.193263 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:31:33.193228 2565 generic.go:356] "Generic (PLEG): container finished" podID="b6629a29-0698-4cf5-81b3-0f94bd835a3a" containerID="1d4d767e4f950de06378baf2c3732aad80e581e554f3574691373c3f59415c64" exitCode=0 Jul 27 09:31:33.193668 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:31:33.193278 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7dffb9899c-jnpmr" Jul 27 09:31:33.193668 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:31:33.193305 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7dffb9899c-jnpmr" event={"ID":"b6629a29-0698-4cf5-81b3-0f94bd835a3a","Type":"ContainerDied","Data":"1d4d767e4f950de06378baf2c3732aad80e581e554f3574691373c3f59415c64"} Jul 27 09:31:33.193668 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:31:33.193378 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7dffb9899c-jnpmr" event={"ID":"b6629a29-0698-4cf5-81b3-0f94bd835a3a","Type":"ContainerDied","Data":"fc93762eef132c03d0382a892c33bcbc614723962b39c528c1608ab9c6c403f3"} Jul 27 09:31:33.193668 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:31:33.193397 2565 scope.go:117] "RemoveContainer" containerID="1d4d767e4f950de06378baf2c3732aad80e581e554f3574691373c3f59415c64" Jul 27 09:31:33.201580 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:31:33.201562 2565 scope.go:117] "RemoveContainer" containerID="1d4d767e4f950de06378baf2c3732aad80e581e554f3574691373c3f59415c64" Jul 27 09:31:33.201901 ip-10-0-128-103 kubenswrapper[2565]: E0727 09:31:33.201877 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1d4d767e4f950de06378baf2c3732aad80e581e554f3574691373c3f59415c64\": container with ID starting with 1d4d767e4f950de06378baf2c3732aad80e581e554f3574691373c3f59415c64 not found: ID does not exist" containerID="1d4d767e4f950de06378baf2c3732aad80e581e554f3574691373c3f59415c64" Jul 27 09:31:33.201958 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:31:33.201912 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1d4d767e4f950de06378baf2c3732aad80e581e554f3574691373c3f59415c64"} err="failed to get container status \"1d4d767e4f950de06378baf2c3732aad80e581e554f3574691373c3f59415c64\": rpc error: code = NotFound desc = could not find container \"1d4d767e4f950de06378baf2c3732aad80e581e554f3574691373c3f59415c64\": container with ID starting with 1d4d767e4f950de06378baf2c3732aad80e581e554f3574691373c3f59415c64 not found: ID does not exist" Jul 27 09:31:33.209384 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:31:33.209356 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7dffb9899c-jnpmr"] Jul 27 09:31:33.212710 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:31:33.212689 2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-7dffb9899c-jnpmr"] Jul 27 09:31:34.563008 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:31:34.562972 2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b6629a29-0698-4cf5-81b3-0f94bd835a3a" path="/var/lib/kubelet/pods/b6629a29-0698-4cf5-81b3-0f94bd835a3a/volumes" Jul 27 09:31:38.498294 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:31:38.498256 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-848w4_2c043d82-d300-41ba-b138-60a03d65ea11/konnectivity-agent/0.log" limit=1024 Jul 27 09:31:39.095038 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:31:39.094999 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ljb58"] Jul 27 09:31:39.095389 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:31:39.095370 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b6629a29-0698-4cf5-81b3-0f94bd835a3a" containerName="console" Jul 27 09:31:39.095481 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:31:39.095392 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="b6629a29-0698-4cf5-81b3-0f94bd835a3a" containerName="console" Jul 27 09:31:39.095537 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:31:39.095498 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="b6629a29-0698-4cf5-81b3-0f94bd835a3a" containerName="console" Jul 27 09:31:39.100006 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:31:39.099981 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ljb58" Jul 27 09:31:39.102718 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:31:39.102689 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 27 09:31:39.102857 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:31:39.102791 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-zhl5j\"" Jul 27 09:31:39.103525 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:31:39.103508 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 27 09:31:39.108423 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:31:39.108399 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ljb58"] Jul 27 09:31:39.178334 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:31:39.178301 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/54c2720b-c34e-4798-8aa8-63627dfb095f-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ljb58\" (UID: \"54c2720b-c34e-4798-8aa8-63627dfb095f\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ljb58" Jul 27 09:31:39.178334 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:31:39.178344 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tbv6w\" (UniqueName: \"kubernetes.io/projected/54c2720b-c34e-4798-8aa8-63627dfb095f-kube-api-access-tbv6w\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ljb58\" (UID: \"54c2720b-c34e-4798-8aa8-63627dfb095f\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ljb58" Jul 27 09:31:39.178572 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:31:39.178485 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/54c2720b-c34e-4798-8aa8-63627dfb095f-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ljb58\" (UID: \"54c2720b-c34e-4798-8aa8-63627dfb095f\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ljb58" Jul 27 09:31:39.279067 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:31:39.279032 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/54c2720b-c34e-4798-8aa8-63627dfb095f-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ljb58\" (UID: \"54c2720b-c34e-4798-8aa8-63627dfb095f\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ljb58" Jul 27 09:31:39.279235 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:31:39.279072 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tbv6w\" (UniqueName: \"kubernetes.io/projected/54c2720b-c34e-4798-8aa8-63627dfb095f-kube-api-access-tbv6w\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ljb58\" (UID: \"54c2720b-c34e-4798-8aa8-63627dfb095f\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ljb58" Jul 27 09:31:39.279235 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:31:39.279144 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/54c2720b-c34e-4798-8aa8-63627dfb095f-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ljb58\" (UID: \"54c2720b-c34e-4798-8aa8-63627dfb095f\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ljb58" Jul 27 09:31:39.279449 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:31:39.279423 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/54c2720b-c34e-4798-8aa8-63627dfb095f-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ljb58\" (UID: \"54c2720b-c34e-4798-8aa8-63627dfb095f\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ljb58" Jul 27 09:31:39.279575 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:31:39.279461 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/54c2720b-c34e-4798-8aa8-63627dfb095f-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ljb58\" (UID: \"54c2720b-c34e-4798-8aa8-63627dfb095f\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ljb58" Jul 27 09:31:39.287602 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:31:39.287579 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tbv6w\" (UniqueName: \"kubernetes.io/projected/54c2720b-c34e-4798-8aa8-63627dfb095f-kube-api-access-tbv6w\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ljb58\" (UID: \"54c2720b-c34e-4798-8aa8-63627dfb095f\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ljb58" Jul 27 09:31:39.409216 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:31:39.409171 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ljb58" Jul 27 09:31:39.541084 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:31:39.541051 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ljb58"] Jul 27 09:31:39.543994 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:31:39.543962 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod54c2720b_c34e_4798_8aa8_63627dfb095f.slice/crio-26e6c5f0bdfbdfe94e0881c98b027d16da9bd4fec63380d0cab3e1fa92723ece WatchSource:0}: Error finding container 26e6c5f0bdfbdfe94e0881c98b027d16da9bd4fec63380d0cab3e1fa92723ece: Status 404 returned error can't find the container with id 26e6c5f0bdfbdfe94e0881c98b027d16da9bd4fec63380d0cab3e1fa92723ece Jul 27 09:31:39.546041 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:31:39.546026 2565 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 27 09:31:40.221781 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:31:40.221728 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ljb58" event={"ID":"54c2720b-c34e-4798-8aa8-63627dfb095f","Type":"ContainerStarted","Data":"26e6c5f0bdfbdfe94e0881c98b027d16da9bd4fec63380d0cab3e1fa92723ece"} Jul 27 09:31:41.117900 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:31:41.117863 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-848w4_2c043d82-d300-41ba-b138-60a03d65ea11/konnectivity-agent/0.log" limit=1024 Jul 27 09:31:45.250341 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:31:45.250248 2565 generic.go:356] "Generic (PLEG): container finished" podID="54c2720b-c34e-4798-8aa8-63627dfb095f" containerID="bbbbdf7587321d62726aab594a547ab98d0c729e1f407182693ddf6ec5f9fe56" exitCode=0 Jul 27 09:31:45.250341 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:31:45.250313 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ljb58" event={"ID":"54c2720b-c34e-4798-8aa8-63627dfb095f","Type":"ContainerDied","Data":"bbbbdf7587321d62726aab594a547ab98d0c729e1f407182693ddf6ec5f9fe56"} Jul 27 09:31:48.267850 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:31:48.267812 2565 generic.go:356] "Generic (PLEG): container finished" podID="54c2720b-c34e-4798-8aa8-63627dfb095f" containerID="3b08c0e973d2a7b22681b93494ea0d9a7b396fa4d02c737979428bd0641c136d" exitCode=0 Jul 27 09:31:48.268320 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:31:48.267880 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ljb58" event={"ID":"54c2720b-c34e-4798-8aa8-63627dfb095f","Type":"ContainerDied","Data":"3b08c0e973d2a7b22681b93494ea0d9a7b396fa4d02c737979428bd0641c136d"} Jul 27 09:31:55.301212 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:31:55.301173 2565 generic.go:356] "Generic (PLEG): container finished" podID="54c2720b-c34e-4798-8aa8-63627dfb095f" containerID="1fd3568af7caa95f28354aa276df8371c980da1805a0292543e931db0bd11070" exitCode=0 Jul 27 09:31:55.301655 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:31:55.301258 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ljb58" event={"ID":"54c2720b-c34e-4798-8aa8-63627dfb095f","Type":"ContainerDied","Data":"1fd3568af7caa95f28354aa276df8371c980da1805a0292543e931db0bd11070"} Jul 27 09:31:56.429751 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:31:56.429726 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ljb58" Jul 27 09:31:56.537400 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:31:56.537366 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/54c2720b-c34e-4798-8aa8-63627dfb095f-util\") pod \"54c2720b-c34e-4798-8aa8-63627dfb095f\" (UID: \"54c2720b-c34e-4798-8aa8-63627dfb095f\") " Jul 27 09:31:56.537578 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:31:56.537419 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tbv6w\" (UniqueName: \"kubernetes.io/projected/54c2720b-c34e-4798-8aa8-63627dfb095f-kube-api-access-tbv6w\") pod \"54c2720b-c34e-4798-8aa8-63627dfb095f\" (UID: \"54c2720b-c34e-4798-8aa8-63627dfb095f\") " Jul 27 09:31:56.537578 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:31:56.537480 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/54c2720b-c34e-4798-8aa8-63627dfb095f-bundle\") pod \"54c2720b-c34e-4798-8aa8-63627dfb095f\" (UID: \"54c2720b-c34e-4798-8aa8-63627dfb095f\") " Jul 27 09:31:56.538187 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:31:56.538158 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/54c2720b-c34e-4798-8aa8-63627dfb095f-bundle" (OuterVolumeSpecName: "bundle") pod "54c2720b-c34e-4798-8aa8-63627dfb095f" (UID: "54c2720b-c34e-4798-8aa8-63627dfb095f"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 27 09:31:56.539631 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:31:56.539606 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/54c2720b-c34e-4798-8aa8-63627dfb095f-kube-api-access-tbv6w" (OuterVolumeSpecName: "kube-api-access-tbv6w") pod "54c2720b-c34e-4798-8aa8-63627dfb095f" (UID: "54c2720b-c34e-4798-8aa8-63627dfb095f"). InnerVolumeSpecName "kube-api-access-tbv6w". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 27 09:31:56.542423 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:31:56.542394 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/54c2720b-c34e-4798-8aa8-63627dfb095f-util" (OuterVolumeSpecName: "util") pod "54c2720b-c34e-4798-8aa8-63627dfb095f" (UID: "54c2720b-c34e-4798-8aa8-63627dfb095f"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 27 09:31:56.638918 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:31:56.638882 2565 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/54c2720b-c34e-4798-8aa8-63627dfb095f-bundle\") on node \"ip-10-0-128-103.ec2.internal\" DevicePath \"\"" Jul 27 09:31:56.638918 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:31:56.638916 2565 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/54c2720b-c34e-4798-8aa8-63627dfb095f-util\") on node \"ip-10-0-128-103.ec2.internal\" DevicePath \"\"" Jul 27 09:31:56.638918 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:31:56.638930 2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tbv6w\" (UniqueName: \"kubernetes.io/projected/54c2720b-c34e-4798-8aa8-63627dfb095f-kube-api-access-tbv6w\") on node \"ip-10-0-128-103.ec2.internal\" DevicePath \"\"" Jul 27 09:31:57.311202 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:31:57.311168 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ljb58" Jul 27 09:31:57.311202 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:31:57.311181 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ljb58" event={"ID":"54c2720b-c34e-4798-8aa8-63627dfb095f","Type":"ContainerDied","Data":"26e6c5f0bdfbdfe94e0881c98b027d16da9bd4fec63380d0cab3e1fa92723ece"} Jul 27 09:31:57.311418 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:31:57.311257 2565 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="26e6c5f0bdfbdfe94e0881c98b027d16da9bd4fec63380d0cab3e1fa92723ece" Jul 27 09:32:00.128638 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:32:00.128605 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-848w4_2c043d82-d300-41ba-b138-60a03d65ea11/konnectivity-agent/0.log" limit=1024 Jul 27 09:32:01.106356 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:32:01.106312 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-848w4_2c043d82-d300-41ba-b138-60a03d65ea11/konnectivity-agent/0.log" limit=1024 Jul 27 09:32:02.131026 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:32:02.130988 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-56b4598547-q9tz2"] Jul 27 09:32:02.131436 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:32:02.131278 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="54c2720b-c34e-4798-8aa8-63627dfb095f" containerName="pull" Jul 27 09:32:02.131436 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:32:02.131288 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="54c2720b-c34e-4798-8aa8-63627dfb095f" containerName="pull" Jul 27 09:32:02.131436 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:32:02.131303 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="54c2720b-c34e-4798-8aa8-63627dfb095f" containerName="extract" Jul 27 09:32:02.131436 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:32:02.131309 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="54c2720b-c34e-4798-8aa8-63627dfb095f" containerName="extract" Jul 27 09:32:02.131436 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:32:02.131320 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="54c2720b-c34e-4798-8aa8-63627dfb095f" containerName="util" Jul 27 09:32:02.131436 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:32:02.131325 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="54c2720b-c34e-4798-8aa8-63627dfb095f" containerName="util" Jul 27 09:32:02.131436 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:32:02.131378 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="54c2720b-c34e-4798-8aa8-63627dfb095f" containerName="extract" Jul 27 09:32:02.162313 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:32:02.162279 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-56b4598547-q9tz2"] Jul 27 09:32:02.162472 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:32:02.162398 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-56b4598547-q9tz2" Jul 27 09:32:02.165005 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:32:02.164985 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\"" Jul 27 09:32:02.165136 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:32:02.165023 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-jmzgf\"" Jul 27 09:32:02.165136 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:32:02.165100 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\"" Jul 27 09:32:02.291739 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:32:02.291697 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/024471b5-0cf4-4da5-80dc-6eb33e2bd98f-tmp\") pod \"cert-manager-operator-controller-manager-56b4598547-q9tz2\" (UID: \"024471b5-0cf4-4da5-80dc-6eb33e2bd98f\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-56b4598547-q9tz2" Jul 27 09:32:02.291945 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:32:02.291752 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-688st\" (UniqueName: \"kubernetes.io/projected/024471b5-0cf4-4da5-80dc-6eb33e2bd98f-kube-api-access-688st\") pod \"cert-manager-operator-controller-manager-56b4598547-q9tz2\" (UID: \"024471b5-0cf4-4da5-80dc-6eb33e2bd98f\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-56b4598547-q9tz2" Jul 27 09:32:02.393173 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:32:02.393083 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/024471b5-0cf4-4da5-80dc-6eb33e2bd98f-tmp\") pod \"cert-manager-operator-controller-manager-56b4598547-q9tz2\" (UID: \"024471b5-0cf4-4da5-80dc-6eb33e2bd98f\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-56b4598547-q9tz2" Jul 27 09:32:02.393173 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:32:02.393141 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-688st\" (UniqueName: \"kubernetes.io/projected/024471b5-0cf4-4da5-80dc-6eb33e2bd98f-kube-api-access-688st\") pod \"cert-manager-operator-controller-manager-56b4598547-q9tz2\" (UID: \"024471b5-0cf4-4da5-80dc-6eb33e2bd98f\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-56b4598547-q9tz2" Jul 27 09:32:02.393459 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:32:02.393437 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/024471b5-0cf4-4da5-80dc-6eb33e2bd98f-tmp\") pod \"cert-manager-operator-controller-manager-56b4598547-q9tz2\" (UID: \"024471b5-0cf4-4da5-80dc-6eb33e2bd98f\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-56b4598547-q9tz2" Jul 27 09:32:02.402068 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:32:02.402040 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-688st\" (UniqueName: \"kubernetes.io/projected/024471b5-0cf4-4da5-80dc-6eb33e2bd98f-kube-api-access-688st\") pod \"cert-manager-operator-controller-manager-56b4598547-q9tz2\" (UID: \"024471b5-0cf4-4da5-80dc-6eb33e2bd98f\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-56b4598547-q9tz2" Jul 27 09:32:02.471856 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:32:02.471819 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-56b4598547-q9tz2" Jul 27 09:32:02.632805 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:32:02.632779 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-56b4598547-q9tz2"] Jul 27 09:32:02.635451 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:32:02.635425 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod024471b5_0cf4_4da5_80dc_6eb33e2bd98f.slice/crio-c5ad568f3e663e9ab5f88b6951c72b88bb5895c2d3d1955716046864636948b1 WatchSource:0}: Error finding container c5ad568f3e663e9ab5f88b6951c72b88bb5895c2d3d1955716046864636948b1: Status 404 returned error can't find the container with id c5ad568f3e663e9ab5f88b6951c72b88bb5895c2d3d1955716046864636948b1 Jul 27 09:32:03.338515 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:32:03.338472 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-56b4598547-q9tz2" event={"ID":"024471b5-0cf4-4da5-80dc-6eb33e2bd98f","Type":"ContainerStarted","Data":"c5ad568f3e663e9ab5f88b6951c72b88bb5895c2d3d1955716046864636948b1"} Jul 27 09:32:03.723359 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:32:03.723314 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-848w4_2c043d82-d300-41ba-b138-60a03d65ea11/konnectivity-agent/0.log" limit=1024 Jul 27 09:32:03.775529 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:32:03.775482 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-848w4_2c043d82-d300-41ba-b138-60a03d65ea11/konnectivity-agent/0.log" limit=1024 Jul 27 09:32:05.295029 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:32:05.294995 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-848w4_2c043d82-d300-41ba-b138-60a03d65ea11/konnectivity-agent/0.log" limit=1024 Jul 27 09:32:05.347745 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:32:05.347702 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-56b4598547-q9tz2" event={"ID":"024471b5-0cf4-4da5-80dc-6eb33e2bd98f","Type":"ContainerStarted","Data":"dc0c0fb88ac8aae997e25dd1f8af19485b0a92130e1dafd46607d0a2a1fa3356"} Jul 27 09:32:05.375643 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:32:05.375587 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-56b4598547-q9tz2" podStartSLOduration=1.4691369650000001 podStartE2EDuration="3.375570233s" totalImagesPullingTime="1.906433268s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 09:32:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 09:32:05.374298245 +0000 UTC m=+453.338084230" watchObservedRunningTime="2026-07-27 09:32:05.375570233 +0000 UTC m=+453.339356189" Jul 27 09:32:05.380438 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:32:05.380411 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-848w4_2c043d82-d300-41ba-b138-60a03d65ea11/konnectivity-agent/0.log" limit=1024 Jul 27 09:32:07.750902 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:32:07.750865 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-848w4_2c043d82-d300-41ba-b138-60a03d65ea11/konnectivity-agent/0.log" limit=1024 Jul 27 09:32:08.111407 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:32:08.111319 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-848w4_2c043d82-d300-41ba-b138-60a03d65ea11/konnectivity-agent/0.log" limit=1024 Jul 27 09:32:08.171483 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:32:08.171455 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-848w4_2c043d82-d300-41ba-b138-60a03d65ea11/konnectivity-agent/0.log" limit=1024 Jul 27 09:32:08.923355 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:32:08.923309 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-848w4_2c043d82-d300-41ba-b138-60a03d65ea11/konnectivity-agent/0.log" limit=1024 Jul 27 09:32:08.983964 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:32:08.983930 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-848w4_2c043d82-d300-41ba-b138-60a03d65ea11/konnectivity-agent/0.log" limit=1024 Jul 27 09:32:12.280407 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:32:12.280372 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-hfh6d"] Jul 27 09:32:12.282593 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:32:12.282576 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-hfh6d" Jul 27 09:32:12.285061 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:32:12.285041 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Jul 27 09:32:12.286026 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:32:12.286003 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-pdp66\"" Jul 27 09:32:12.286026 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:32:12.286010 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Jul 27 09:32:12.291314 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:32:12.291287 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-hfh6d"] Jul 27 09:32:12.364285 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:32:12.364250 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/87be943b-5d4b-40a8-80b6-0e781c8218c8-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-hfh6d\" (UID: \"87be943b-5d4b-40a8-80b6-0e781c8218c8\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-hfh6d" Jul 27 09:32:12.364460 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:32:12.364329 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mktbt\" (UniqueName: \"kubernetes.io/projected/87be943b-5d4b-40a8-80b6-0e781c8218c8-kube-api-access-mktbt\") pod \"cert-manager-cainjector-559cdd8c87-hfh6d\" (UID: \"87be943b-5d4b-40a8-80b6-0e781c8218c8\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-hfh6d" Jul 27 09:32:12.465592 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:32:12.465558 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mktbt\" (UniqueName: \"kubernetes.io/projected/87be943b-5d4b-40a8-80b6-0e781c8218c8-kube-api-access-mktbt\") pod \"cert-manager-cainjector-559cdd8c87-hfh6d\" (UID: \"87be943b-5d4b-40a8-80b6-0e781c8218c8\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-hfh6d" Jul 27 09:32:12.465757 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:32:12.465607 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/87be943b-5d4b-40a8-80b6-0e781c8218c8-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-hfh6d\" (UID: \"87be943b-5d4b-40a8-80b6-0e781c8218c8\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-hfh6d" Jul 27 09:32:12.474531 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:32:12.474498 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/87be943b-5d4b-40a8-80b6-0e781c8218c8-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-hfh6d\" (UID: \"87be943b-5d4b-40a8-80b6-0e781c8218c8\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-hfh6d" Jul 27 09:32:12.474721 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:32:12.474699 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mktbt\" (UniqueName: \"kubernetes.io/projected/87be943b-5d4b-40a8-80b6-0e781c8218c8-kube-api-access-mktbt\") pod \"cert-manager-cainjector-559cdd8c87-hfh6d\" (UID: \"87be943b-5d4b-40a8-80b6-0e781c8218c8\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-hfh6d" Jul 27 09:32:12.605725 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:32:12.605631 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-hfh6d" Jul 27 09:32:12.740344 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:32:12.740184 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-hfh6d"] Jul 27 09:32:12.743004 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:32:12.742974 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod87be943b_5d4b_40a8_80b6_0e781c8218c8.slice/crio-d3c0c392ef9033029a078a3a248788a1bfd0961e3579822e70e0dfe10bf46931 WatchSource:0}: Error finding container d3c0c392ef9033029a078a3a248788a1bfd0961e3579822e70e0dfe10bf46931: Status 404 returned error can't find the container with id d3c0c392ef9033029a078a3a248788a1bfd0961e3579822e70e0dfe10bf46931 Jul 27 09:32:13.384460 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:32:13.384425 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-hfh6d" event={"ID":"87be943b-5d4b-40a8-80b6-0e781c8218c8","Type":"ContainerStarted","Data":"d3c0c392ef9033029a078a3a248788a1bfd0961e3579822e70e0dfe10bf46931"} Jul 27 09:32:14.920805 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:32:14.920759 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-848w4_2c043d82-d300-41ba-b138-60a03d65ea11/konnectivity-agent/0.log" limit=1024 Jul 27 09:32:16.005421 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:32:16.005388 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-4xd74"] Jul 27 09:32:16.007714 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:32:16.007693 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-4xd74" Jul 27 09:32:16.009961 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:32:16.009935 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-vr9td\"" Jul 27 09:32:16.018841 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:32:16.018813 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-4xd74"] Jul 27 09:32:16.096282 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:32:16.096239 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/c1522a98-e626-4ce4-b8eb-a2ee7430167c-bound-sa-token\") pod \"cert-manager-55b4d954d5-4xd74\" (UID: \"c1522a98-e626-4ce4-b8eb-a2ee7430167c\") " pod="cert-manager/cert-manager-55b4d954d5-4xd74" Jul 27 09:32:16.096282 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:32:16.096278 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8tk9t\" (UniqueName: \"kubernetes.io/projected/c1522a98-e626-4ce4-b8eb-a2ee7430167c-kube-api-access-8tk9t\") pod \"cert-manager-55b4d954d5-4xd74\" (UID: \"c1522a98-e626-4ce4-b8eb-a2ee7430167c\") " pod="cert-manager/cert-manager-55b4d954d5-4xd74" Jul 27 09:32:16.197347 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:32:16.197302 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/c1522a98-e626-4ce4-b8eb-a2ee7430167c-bound-sa-token\") pod \"cert-manager-55b4d954d5-4xd74\" (UID: \"c1522a98-e626-4ce4-b8eb-a2ee7430167c\") " pod="cert-manager/cert-manager-55b4d954d5-4xd74" Jul 27 09:32:16.197526 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:32:16.197354 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8tk9t\" (UniqueName: \"kubernetes.io/projected/c1522a98-e626-4ce4-b8eb-a2ee7430167c-kube-api-access-8tk9t\") pod \"cert-manager-55b4d954d5-4xd74\" (UID: \"c1522a98-e626-4ce4-b8eb-a2ee7430167c\") " pod="cert-manager/cert-manager-55b4d954d5-4xd74" Jul 27 09:32:16.206566 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:32:16.206534 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/c1522a98-e626-4ce4-b8eb-a2ee7430167c-bound-sa-token\") pod \"cert-manager-55b4d954d5-4xd74\" (UID: \"c1522a98-e626-4ce4-b8eb-a2ee7430167c\") " pod="cert-manager/cert-manager-55b4d954d5-4xd74" Jul 27 09:32:16.206711 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:32:16.206617 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8tk9t\" (UniqueName: \"kubernetes.io/projected/c1522a98-e626-4ce4-b8eb-a2ee7430167c-kube-api-access-8tk9t\") pod \"cert-manager-55b4d954d5-4xd74\" (UID: \"c1522a98-e626-4ce4-b8eb-a2ee7430167c\") " pod="cert-manager/cert-manager-55b4d954d5-4xd74" Jul 27 09:32:16.319706 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:32:16.319624 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-4xd74" Jul 27 09:32:17.293878 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:32:17.293843 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-4xd74"] Jul 27 09:32:17.297036 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:32:17.296993 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc1522a98_e626_4ce4_b8eb_a2ee7430167c.slice/crio-ab16e4c3dc6dc6d4e2c0a07d9d192e14377ff77f11a03a6e18886c77daaa21e2 WatchSource:0}: Error finding container ab16e4c3dc6dc6d4e2c0a07d9d192e14377ff77f11a03a6e18886c77daaa21e2: Status 404 returned error can't find the container with id ab16e4c3dc6dc6d4e2c0a07d9d192e14377ff77f11a03a6e18886c77daaa21e2 Jul 27 09:32:17.411966 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:32:17.411846 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-hfh6d" event={"ID":"87be943b-5d4b-40a8-80b6-0e781c8218c8","Type":"ContainerStarted","Data":"fda11e52f7ec23deaba8f5df66783a16a3006ab9c39330966a0f56589224975b"} Jul 27 09:32:17.413944 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:32:17.413837 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-4xd74" event={"ID":"c1522a98-e626-4ce4-b8eb-a2ee7430167c","Type":"ContainerStarted","Data":"499d8efec0972a14a7ee0ac1b7121aa3343940a72d10ade206245d3cc55c1ad0"} Jul 27 09:32:17.414107 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:32:17.414004 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-4xd74" event={"ID":"c1522a98-e626-4ce4-b8eb-a2ee7430167c","Type":"ContainerStarted","Data":"ab16e4c3dc6dc6d4e2c0a07d9d192e14377ff77f11a03a6e18886c77daaa21e2"} Jul 27 09:32:17.428522 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:32:17.428458 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-hfh6d" podStartSLOduration=0.980327291 podStartE2EDuration="5.428439821s" totalImagesPullingTime="4.44811253s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 09:32:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 09:32:17.426927125 +0000 UTC m=+465.390713082" watchObservedRunningTime="2026-07-27 09:32:17.428439821 +0000 UTC m=+465.392225777" Jul 27 09:32:17.446193 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:32:17.446127 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-4xd74" podStartSLOduration=2.4461011040000002 podStartE2EDuration="2.446101104s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 09:32:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 09:32:17.444433117 +0000 UTC m=+465.408219075" watchObservedRunningTime="2026-07-27 09:32:17.446101104 +0000 UTC m=+465.409887063" Jul 27 09:32:17.644828 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:32:17.644793 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-848w4_2c043d82-d300-41ba-b138-60a03d65ea11/konnectivity-agent/0.log" limit=1024 Jul 27 09:32:17.744710 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:32:17.744612 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-848w4_2c043d82-d300-41ba-b138-60a03d65ea11/konnectivity-agent/0.log" limit=1024 Jul 27 09:32:17.984121 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:32:17.984089 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-848w4_2c043d82-d300-41ba-b138-60a03d65ea11/konnectivity-agent/0.log" limit=1024 Jul 27 09:32:18.084359 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:32:18.084278 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-848w4_2c043d82-d300-41ba-b138-60a03d65ea11/konnectivity-agent/0.log" limit=1024 Jul 27 09:32:18.538942 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:32:18.538906 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/c2ca89134faa49158137edb0141b62ea0c6a854657aff316cf72d9c78e9p4hr"] Jul 27 09:32:18.541417 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:32:18.541398 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c2ca89134faa49158137edb0141b62ea0c6a854657aff316cf72d9c78e9p4hr" Jul 27 09:32:18.543784 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:32:18.543751 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 27 09:32:18.544705 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:32:18.544686 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-zhl5j\"" Jul 27 09:32:18.544835 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:32:18.544712 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 27 09:32:18.550621 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:32:18.550594 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/c2ca89134faa49158137edb0141b62ea0c6a854657aff316cf72d9c78e9p4hr"] Jul 27 09:32:18.620823 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:32:18.620758 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1862fdad-dfe8-437d-ad01-2f6651bcc9b9-bundle\") pod \"c2ca89134faa49158137edb0141b62ea0c6a854657aff316cf72d9c78e9p4hr\" (UID: \"1862fdad-dfe8-437d-ad01-2f6651bcc9b9\") " pod="openshift-marketplace/c2ca89134faa49158137edb0141b62ea0c6a854657aff316cf72d9c78e9p4hr" Jul 27 09:32:18.620992 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:32:18.620852 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5h6jb\" (UniqueName: \"kubernetes.io/projected/1862fdad-dfe8-437d-ad01-2f6651bcc9b9-kube-api-access-5h6jb\") pod \"c2ca89134faa49158137edb0141b62ea0c6a854657aff316cf72d9c78e9p4hr\" (UID: \"1862fdad-dfe8-437d-ad01-2f6651bcc9b9\") " pod="openshift-marketplace/c2ca89134faa49158137edb0141b62ea0c6a854657aff316cf72d9c78e9p4hr" Jul 27 09:32:18.620992 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:32:18.620947 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1862fdad-dfe8-437d-ad01-2f6651bcc9b9-util\") pod \"c2ca89134faa49158137edb0141b62ea0c6a854657aff316cf72d9c78e9p4hr\" (UID: \"1862fdad-dfe8-437d-ad01-2f6651bcc9b9\") " pod="openshift-marketplace/c2ca89134faa49158137edb0141b62ea0c6a854657aff316cf72d9c78e9p4hr" Jul 27 09:32:18.721658 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:32:18.721616 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1862fdad-dfe8-437d-ad01-2f6651bcc9b9-util\") pod \"c2ca89134faa49158137edb0141b62ea0c6a854657aff316cf72d9c78e9p4hr\" (UID: \"1862fdad-dfe8-437d-ad01-2f6651bcc9b9\") " pod="openshift-marketplace/c2ca89134faa49158137edb0141b62ea0c6a854657aff316cf72d9c78e9p4hr" Jul 27 09:32:18.721897 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:32:18.721687 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1862fdad-dfe8-437d-ad01-2f6651bcc9b9-bundle\") pod \"c2ca89134faa49158137edb0141b62ea0c6a854657aff316cf72d9c78e9p4hr\" (UID: \"1862fdad-dfe8-437d-ad01-2f6651bcc9b9\") " pod="openshift-marketplace/c2ca89134faa49158137edb0141b62ea0c6a854657aff316cf72d9c78e9p4hr" Jul 27 09:32:18.721897 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:32:18.721713 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5h6jb\" (UniqueName: \"kubernetes.io/projected/1862fdad-dfe8-437d-ad01-2f6651bcc9b9-kube-api-access-5h6jb\") pod \"c2ca89134faa49158137edb0141b62ea0c6a854657aff316cf72d9c78e9p4hr\" (UID: \"1862fdad-dfe8-437d-ad01-2f6651bcc9b9\") " pod="openshift-marketplace/c2ca89134faa49158137edb0141b62ea0c6a854657aff316cf72d9c78e9p4hr" Jul 27 09:32:18.722055 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:32:18.722030 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1862fdad-dfe8-437d-ad01-2f6651bcc9b9-util\") pod \"c2ca89134faa49158137edb0141b62ea0c6a854657aff316cf72d9c78e9p4hr\" (UID: \"1862fdad-dfe8-437d-ad01-2f6651bcc9b9\") " pod="openshift-marketplace/c2ca89134faa49158137edb0141b62ea0c6a854657aff316cf72d9c78e9p4hr" Jul 27 09:32:18.722119 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:32:18.722096 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1862fdad-dfe8-437d-ad01-2f6651bcc9b9-bundle\") pod \"c2ca89134faa49158137edb0141b62ea0c6a854657aff316cf72d9c78e9p4hr\" (UID: \"1862fdad-dfe8-437d-ad01-2f6651bcc9b9\") " pod="openshift-marketplace/c2ca89134faa49158137edb0141b62ea0c6a854657aff316cf72d9c78e9p4hr" Jul 27 09:32:18.730566 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:32:18.730530 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5h6jb\" (UniqueName: \"kubernetes.io/projected/1862fdad-dfe8-437d-ad01-2f6651bcc9b9-kube-api-access-5h6jb\") pod \"c2ca89134faa49158137edb0141b62ea0c6a854657aff316cf72d9c78e9p4hr\" (UID: \"1862fdad-dfe8-437d-ad01-2f6651bcc9b9\") " pod="openshift-marketplace/c2ca89134faa49158137edb0141b62ea0c6a854657aff316cf72d9c78e9p4hr" Jul 27 09:32:18.851150 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:32:18.851077 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c2ca89134faa49158137edb0141b62ea0c6a854657aff316cf72d9c78e9p4hr" Jul 27 09:32:18.982537 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:32:18.982504 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/c2ca89134faa49158137edb0141b62ea0c6a854657aff316cf72d9c78e9p4hr"] Jul 27 09:32:18.984979 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:32:18.984954 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1862fdad_dfe8_437d_ad01_2f6651bcc9b9.slice/crio-45dfe237187dbe1834b35f125c4d21eeca544e992a7d50607a48ee15a5e1c378 WatchSource:0}: Error finding container 45dfe237187dbe1834b35f125c4d21eeca544e992a7d50607a48ee15a5e1c378: Status 404 returned error can't find the container with id 45dfe237187dbe1834b35f125c4d21eeca544e992a7d50607a48ee15a5e1c378 Jul 27 09:32:19.423583 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:32:19.423546 2565 generic.go:356] "Generic (PLEG): container finished" podID="1862fdad-dfe8-437d-ad01-2f6651bcc9b9" containerID="458709f5d039c65d7c28474c8705637f7e266cdebbce7760da21707548797a8c" exitCode=0 Jul 27 09:32:19.423796 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:32:19.423613 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c2ca89134faa49158137edb0141b62ea0c6a854657aff316cf72d9c78e9p4hr" event={"ID":"1862fdad-dfe8-437d-ad01-2f6651bcc9b9","Type":"ContainerDied","Data":"458709f5d039c65d7c28474c8705637f7e266cdebbce7760da21707548797a8c"} Jul 27 09:32:19.423796 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:32:19.423652 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c2ca89134faa49158137edb0141b62ea0c6a854657aff316cf72d9c78e9p4hr" event={"ID":"1862fdad-dfe8-437d-ad01-2f6651bcc9b9","Type":"ContainerStarted","Data":"45dfe237187dbe1834b35f125c4d21eeca544e992a7d50607a48ee15a5e1c378"} Jul 27 09:32:22.441270 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:32:22.441231 2565 generic.go:356] "Generic (PLEG): container finished" podID="1862fdad-dfe8-437d-ad01-2f6651bcc9b9" containerID="40514e169f8851e59322bae913b2c5d7d3a65ae0642833e409d425edfc3175cb" exitCode=0 Jul 27 09:32:22.441693 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:32:22.441281 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c2ca89134faa49158137edb0141b62ea0c6a854657aff316cf72d9c78e9p4hr" event={"ID":"1862fdad-dfe8-437d-ad01-2f6651bcc9b9","Type":"ContainerDied","Data":"40514e169f8851e59322bae913b2c5d7d3a65ae0642833e409d425edfc3175cb"} Jul 27 09:32:23.447726 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:32:23.447693 2565 generic.go:356] "Generic (PLEG): container finished" podID="1862fdad-dfe8-437d-ad01-2f6651bcc9b9" containerID="35479a345f6e4af8d521bd9f8aedd2ccdfab78d2c2b47264d0dcf4e9d97632e1" exitCode=0 Jul 27 09:32:23.448121 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:32:23.447801 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c2ca89134faa49158137edb0141b62ea0c6a854657aff316cf72d9c78e9p4hr" event={"ID":"1862fdad-dfe8-437d-ad01-2f6651bcc9b9","Type":"ContainerDied","Data":"35479a345f6e4af8d521bd9f8aedd2ccdfab78d2c2b47264d0dcf4e9d97632e1"} Jul 27 09:32:24.581300 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:32:24.581273 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c2ca89134faa49158137edb0141b62ea0c6a854657aff316cf72d9c78e9p4hr" Jul 27 09:32:24.676485 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:32:24.676449 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1862fdad-dfe8-437d-ad01-2f6651bcc9b9-util\") pod \"1862fdad-dfe8-437d-ad01-2f6651bcc9b9\" (UID: \"1862fdad-dfe8-437d-ad01-2f6651bcc9b9\") " Jul 27 09:32:24.676681 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:32:24.676517 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5h6jb\" (UniqueName: \"kubernetes.io/projected/1862fdad-dfe8-437d-ad01-2f6651bcc9b9-kube-api-access-5h6jb\") pod \"1862fdad-dfe8-437d-ad01-2f6651bcc9b9\" (UID: \"1862fdad-dfe8-437d-ad01-2f6651bcc9b9\") " Jul 27 09:32:24.676681 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:32:24.676546 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1862fdad-dfe8-437d-ad01-2f6651bcc9b9-bundle\") pod \"1862fdad-dfe8-437d-ad01-2f6651bcc9b9\" (UID: \"1862fdad-dfe8-437d-ad01-2f6651bcc9b9\") " Jul 27 09:32:24.676994 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:32:24.676971 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1862fdad-dfe8-437d-ad01-2f6651bcc9b9-bundle" (OuterVolumeSpecName: "bundle") pod "1862fdad-dfe8-437d-ad01-2f6651bcc9b9" (UID: "1862fdad-dfe8-437d-ad01-2f6651bcc9b9"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 27 09:32:24.678654 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:32:24.678627 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1862fdad-dfe8-437d-ad01-2f6651bcc9b9-kube-api-access-5h6jb" (OuterVolumeSpecName: "kube-api-access-5h6jb") pod "1862fdad-dfe8-437d-ad01-2f6651bcc9b9" (UID: "1862fdad-dfe8-437d-ad01-2f6651bcc9b9"). InnerVolumeSpecName "kube-api-access-5h6jb". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 27 09:32:24.681244 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:32:24.681219 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1862fdad-dfe8-437d-ad01-2f6651bcc9b9-util" (OuterVolumeSpecName: "util") pod "1862fdad-dfe8-437d-ad01-2f6651bcc9b9" (UID: "1862fdad-dfe8-437d-ad01-2f6651bcc9b9"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 27 09:32:24.777507 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:32:24.777410 2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5h6jb\" (UniqueName: \"kubernetes.io/projected/1862fdad-dfe8-437d-ad01-2f6651bcc9b9-kube-api-access-5h6jb\") on node \"ip-10-0-128-103.ec2.internal\" DevicePath \"\"" Jul 27 09:32:24.777507 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:32:24.777446 2565 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1862fdad-dfe8-437d-ad01-2f6651bcc9b9-bundle\") on node \"ip-10-0-128-103.ec2.internal\" DevicePath \"\"" Jul 27 09:32:24.777507 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:32:24.777459 2565 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1862fdad-dfe8-437d-ad01-2f6651bcc9b9-util\") on node \"ip-10-0-128-103.ec2.internal\" DevicePath \"\"" Jul 27 09:32:25.458888 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:32:25.458852 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c2ca89134faa49158137edb0141b62ea0c6a854657aff316cf72d9c78e9p4hr" event={"ID":"1862fdad-dfe8-437d-ad01-2f6651bcc9b9","Type":"ContainerDied","Data":"45dfe237187dbe1834b35f125c4d21eeca544e992a7d50607a48ee15a5e1c378"} Jul 27 09:32:25.458888 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:32:25.458889 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c2ca89134faa49158137edb0141b62ea0c6a854657aff316cf72d9c78e9p4hr" Jul 27 09:32:25.459099 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:32:25.458903 2565 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="45dfe237187dbe1834b35f125c4d21eeca544e992a7d50607a48ee15a5e1c378" Jul 27 09:32:28.751977 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:32:28.751943 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-848w4_2c043d82-d300-41ba-b138-60a03d65ea11/konnectivity-agent/0.log" limit=1024 Jul 27 09:32:28.847591 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:32:28.847556 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-848w4_2c043d82-d300-41ba-b138-60a03d65ea11/konnectivity-agent/0.log" limit=1024 Jul 27 09:32:40.977576 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:32:40.977524 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-848w4_2c043d82-d300-41ba-b138-60a03d65ea11/konnectivity-agent/0.log" limit=1024 Jul 27 09:32:41.385934 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:32:41.385889 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-848w4_2c043d82-d300-41ba-b138-60a03d65ea11/konnectivity-agent/0.log" limit=1024 Jul 27 09:32:41.484639 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:32:41.484599 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-848w4_2c043d82-d300-41ba-b138-60a03d65ea11/konnectivity-agent/0.log" limit=1024 Jul 27 09:32:43.121721 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:32:43.121687 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-jobset-operator/jobset-controller-manager-659557f77c-4cczh"] Jul 27 09:32:43.122137 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:32:43.122060 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1862fdad-dfe8-437d-ad01-2f6651bcc9b9" containerName="extract" Jul 27 09:32:43.122137 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:32:43.122073 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="1862fdad-dfe8-437d-ad01-2f6651bcc9b9" containerName="extract" Jul 27 09:32:43.122137 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:32:43.122085 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1862fdad-dfe8-437d-ad01-2f6651bcc9b9" containerName="util" Jul 27 09:32:43.122137 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:32:43.122090 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="1862fdad-dfe8-437d-ad01-2f6651bcc9b9" containerName="util" Jul 27 09:32:43.122137 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:32:43.122105 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1862fdad-dfe8-437d-ad01-2f6651bcc9b9" containerName="pull" Jul 27 09:32:43.122137 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:32:43.122111 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="1862fdad-dfe8-437d-ad01-2f6651bcc9b9" containerName="pull" Jul 27 09:32:43.122367 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:32:43.122166 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="1862fdad-dfe8-437d-ad01-2f6651bcc9b9" containerName="extract" Jul 27 09:32:43.127689 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:32:43.127667 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-jobset-operator/jobset-controller-manager-659557f77c-4cczh" Jul 27 09:32:43.130124 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:32:43.130102 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-jobset-operator\"/\"jobset-manager-config\"" Jul 27 09:32:43.131062 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:32:43.131025 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-jobset-operator\"/\"kube-root-ca.crt\"" Jul 27 09:32:43.131062 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:32:43.131033 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-jobset-operator\"/\"webhook-server-cert\"" Jul 27 09:32:43.131247 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:32:43.131084 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-jobset-operator\"/\"metrics-server-cert\"" Jul 27 09:32:43.131247 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:32:43.131041 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-jobset-operator\"/\"openshift-service-ca.crt\"" Jul 27 09:32:43.131247 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:32:43.131032 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-jobset-operator\"/\"jobset-controller-manager-dockercfg-q4gdl\"" Jul 27 09:32:43.133722 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:32:43.133701 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-jobset-operator/jobset-controller-manager-659557f77c-4cczh"] Jul 27 09:32:43.235484 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:32:43.235450 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/85d82827-9d5c-47df-9d62-23baa0706e51-cert\") pod \"jobset-controller-manager-659557f77c-4cczh\" (UID: \"85d82827-9d5c-47df-9d62-23baa0706e51\") " pod="openshift-jobset-operator/jobset-controller-manager-659557f77c-4cczh" Jul 27 09:32:43.235667 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:32:43.235511 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/85d82827-9d5c-47df-9d62-23baa0706e51-manager-config\") pod \"jobset-controller-manager-659557f77c-4cczh\" (UID: \"85d82827-9d5c-47df-9d62-23baa0706e51\") " pod="openshift-jobset-operator/jobset-controller-manager-659557f77c-4cczh" Jul 27 09:32:43.235667 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:32:43.235550 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/85d82827-9d5c-47df-9d62-23baa0706e51-metrics-certs\") pod \"jobset-controller-manager-659557f77c-4cczh\" (UID: \"85d82827-9d5c-47df-9d62-23baa0706e51\") " pod="openshift-jobset-operator/jobset-controller-manager-659557f77c-4cczh" Jul 27 09:32:43.235667 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:32:43.235581 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z7vnp\" (UniqueName: \"kubernetes.io/projected/85d82827-9d5c-47df-9d62-23baa0706e51-kube-api-access-z7vnp\") pod \"jobset-controller-manager-659557f77c-4cczh\" (UID: \"85d82827-9d5c-47df-9d62-23baa0706e51\") " pod="openshift-jobset-operator/jobset-controller-manager-659557f77c-4cczh" Jul 27 09:32:43.336571 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:32:43.336537 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/85d82827-9d5c-47df-9d62-23baa0706e51-cert\") pod \"jobset-controller-manager-659557f77c-4cczh\" (UID: \"85d82827-9d5c-47df-9d62-23baa0706e51\") " pod="openshift-jobset-operator/jobset-controller-manager-659557f77c-4cczh" Jul 27 09:32:43.336722 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:32:43.336581 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/85d82827-9d5c-47df-9d62-23baa0706e51-manager-config\") pod \"jobset-controller-manager-659557f77c-4cczh\" (UID: \"85d82827-9d5c-47df-9d62-23baa0706e51\") " pod="openshift-jobset-operator/jobset-controller-manager-659557f77c-4cczh" Jul 27 09:32:43.336722 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:32:43.336606 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/85d82827-9d5c-47df-9d62-23baa0706e51-metrics-certs\") pod \"jobset-controller-manager-659557f77c-4cczh\" (UID: \"85d82827-9d5c-47df-9d62-23baa0706e51\") " pod="openshift-jobset-operator/jobset-controller-manager-659557f77c-4cczh" Jul 27 09:32:43.336819 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:32:43.336804 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z7vnp\" (UniqueName: \"kubernetes.io/projected/85d82827-9d5c-47df-9d62-23baa0706e51-kube-api-access-z7vnp\") pod \"jobset-controller-manager-659557f77c-4cczh\" (UID: \"85d82827-9d5c-47df-9d62-23baa0706e51\") " pod="openshift-jobset-operator/jobset-controller-manager-659557f77c-4cczh" Jul 27 09:32:43.337317 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:32:43.337293 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/85d82827-9d5c-47df-9d62-23baa0706e51-manager-config\") pod \"jobset-controller-manager-659557f77c-4cczh\" (UID: \"85d82827-9d5c-47df-9d62-23baa0706e51\") " pod="openshift-jobset-operator/jobset-controller-manager-659557f77c-4cczh" Jul 27 09:32:43.339147 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:32:43.339127 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/85d82827-9d5c-47df-9d62-23baa0706e51-cert\") pod \"jobset-controller-manager-659557f77c-4cczh\" (UID: \"85d82827-9d5c-47df-9d62-23baa0706e51\") " pod="openshift-jobset-operator/jobset-controller-manager-659557f77c-4cczh" Jul 27 09:32:43.339239 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:32:43.339195 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/85d82827-9d5c-47df-9d62-23baa0706e51-metrics-certs\") pod \"jobset-controller-manager-659557f77c-4cczh\" (UID: \"85d82827-9d5c-47df-9d62-23baa0706e51\") " pod="openshift-jobset-operator/jobset-controller-manager-659557f77c-4cczh" Jul 27 09:32:43.345883 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:32:43.345862 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z7vnp\" (UniqueName: \"kubernetes.io/projected/85d82827-9d5c-47df-9d62-23baa0706e51-kube-api-access-z7vnp\") pod \"jobset-controller-manager-659557f77c-4cczh\" (UID: \"85d82827-9d5c-47df-9d62-23baa0706e51\") " pod="openshift-jobset-operator/jobset-controller-manager-659557f77c-4cczh" Jul 27 09:32:43.439129 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:32:43.439037 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-jobset-operator/jobset-controller-manager-659557f77c-4cczh" Jul 27 09:32:43.503182 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:32:43.503134 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-848w4_2c043d82-d300-41ba-b138-60a03d65ea11/konnectivity-agent/0.log" limit=1024 Jul 27 09:32:43.576677 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:32:43.576616 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-jobset-operator/jobset-controller-manager-659557f77c-4cczh"] Jul 27 09:32:43.579510 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:32:43.579487 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod85d82827_9d5c_47df_9d62_23baa0706e51.slice/crio-ed4c59578e73f82904e3e0d21f9d0c7675814c9d012b9cb0277e34a9d8903166 WatchSource:0}: Error finding container ed4c59578e73f82904e3e0d21f9d0c7675814c9d012b9cb0277e34a9d8903166: Status 404 returned error can't find the container with id ed4c59578e73f82904e3e0d21f9d0c7675814c9d012b9cb0277e34a9d8903166 Jul 27 09:32:44.550930 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:32:44.550888 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-jobset-operator/jobset-controller-manager-659557f77c-4cczh" event={"ID":"85d82827-9d5c-47df-9d62-23baa0706e51","Type":"ContainerStarted","Data":"ed4c59578e73f82904e3e0d21f9d0c7675814c9d012b9cb0277e34a9d8903166"} Jul 27 09:32:46.569373 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:32:46.569334 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-jobset-operator/jobset-controller-manager-659557f77c-4cczh" event={"ID":"85d82827-9d5c-47df-9d62-23baa0706e51","Type":"ContainerStarted","Data":"85bb6a0272d971061b4de550324e96858500eb0b3be34a02b350c5dac4ca2dd0"} Jul 27 09:32:46.569791 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:32:46.569397 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-jobset-operator/jobset-controller-manager-659557f77c-4cczh" Jul 27 09:32:46.586601 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:32:46.586523 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-jobset-operator/jobset-controller-manager-659557f77c-4cczh" podStartSLOduration=1.288602425 podStartE2EDuration="3.586508259s" totalImagesPullingTime="2.297905834s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 09:32:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 09:32:46.585810027 +0000 UTC m=+494.549596014" watchObservedRunningTime="2026-07-27 09:32:46.586508259 +0000 UTC m=+494.550294214" Jul 27 09:32:51.060026 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:32:51.059984 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-848w4_2c043d82-d300-41ba-b138-60a03d65ea11/konnectivity-agent/0.log" limit=1024 Jul 27 09:32:57.579218 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:32:57.579188 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-jobset-operator/jobset-controller-manager-659557f77c-4cczh" Jul 27 09:33:37.625458 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:33:37.625354 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-848w4_2c043d82-d300-41ba-b138-60a03d65ea11/konnectivity-agent/0.log" limit=1024 Jul 27 09:33:37.740535 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:33:37.740490 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-848w4_2c043d82-d300-41ba-b138-60a03d65ea11/konnectivity-agent/0.log" limit=1024 Jul 27 09:33:37.818394 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:33:37.818360 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-848w4_2c043d82-d300-41ba-b138-60a03d65ea11/konnectivity-agent/0.log" limit=1024 Jul 27 09:34:27.723299 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:34:27.723259 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-848w4_2c043d82-d300-41ba-b138-60a03d65ea11/konnectivity-agent/0.log" limit=1024 Jul 27 09:34:27.845349 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:34:27.845309 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-848w4_2c043d82-d300-41ba-b138-60a03d65ea11/konnectivity-agent/0.log" limit=1024 Jul 27 09:34:29.965700 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:34:29.965669 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-848w4_2c043d82-d300-41ba-b138-60a03d65ea11/konnectivity-agent/0.log" limit=1024 Jul 27 09:34:30.365874 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:34:30.365845 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-84b8475cb-bmbsk"] Jul 27 09:34:30.369047 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:34:30.369026 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-84b8475cb-bmbsk" Jul 27 09:34:30.381399 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:34:30.381373 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-84b8475cb-bmbsk"] Jul 27 09:34:30.421579 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:34:30.421541 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-848w4_2c043d82-d300-41ba-b138-60a03d65ea11/konnectivity-agent/0.log" limit=1024 Jul 27 09:34:30.432506 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:34:30.432471 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b69b1104-f7d2-4f48-b9ee-003693681adb-oauth-serving-cert\") pod \"console-84b8475cb-bmbsk\" (UID: \"b69b1104-f7d2-4f48-b9ee-003693681adb\") " pod="openshift-console/console-84b8475cb-bmbsk" Jul 27 09:34:30.432680 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:34:30.432513 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b69b1104-f7d2-4f48-b9ee-003693681adb-console-config\") pod \"console-84b8475cb-bmbsk\" (UID: \"b69b1104-f7d2-4f48-b9ee-003693681adb\") " pod="openshift-console/console-84b8475cb-bmbsk" Jul 27 09:34:30.432680 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:34:30.432538 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b69b1104-f7d2-4f48-b9ee-003693681adb-trusted-ca-bundle\") pod \"console-84b8475cb-bmbsk\" (UID: \"b69b1104-f7d2-4f48-b9ee-003693681adb\") " pod="openshift-console/console-84b8475cb-bmbsk" Jul 27 09:34:30.432680 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:34:30.432613 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-srphh\" (UniqueName: \"kubernetes.io/projected/b69b1104-f7d2-4f48-b9ee-003693681adb-kube-api-access-srphh\") pod \"console-84b8475cb-bmbsk\" (UID: \"b69b1104-f7d2-4f48-b9ee-003693681adb\") " pod="openshift-console/console-84b8475cb-bmbsk" Jul 27 09:34:30.432680 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:34:30.432666 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b69b1104-f7d2-4f48-b9ee-003693681adb-console-oauth-config\") pod \"console-84b8475cb-bmbsk\" (UID: \"b69b1104-f7d2-4f48-b9ee-003693681adb\") " pod="openshift-console/console-84b8475cb-bmbsk" Jul 27 09:34:30.437625 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:34:30.432699 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b69b1104-f7d2-4f48-b9ee-003693681adb-console-serving-cert\") pod \"console-84b8475cb-bmbsk\" (UID: \"b69b1104-f7d2-4f48-b9ee-003693681adb\") " pod="openshift-console/console-84b8475cb-bmbsk" Jul 27 09:34:30.437825 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:34:30.437746 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b69b1104-f7d2-4f48-b9ee-003693681adb-service-ca\") pod \"console-84b8475cb-bmbsk\" (UID: \"b69b1104-f7d2-4f48-b9ee-003693681adb\") " pod="openshift-console/console-84b8475cb-bmbsk" Jul 27 09:34:30.521004 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:34:30.520971 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-848w4_2c043d82-d300-41ba-b138-60a03d65ea11/konnectivity-agent/0.log" limit=1024 Jul 27 09:34:30.538961 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:34:30.538928 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b69b1104-f7d2-4f48-b9ee-003693681adb-console-serving-cert\") pod \"console-84b8475cb-bmbsk\" (UID: \"b69b1104-f7d2-4f48-b9ee-003693681adb\") " pod="openshift-console/console-84b8475cb-bmbsk" Jul 27 09:34:30.539137 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:34:30.538980 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b69b1104-f7d2-4f48-b9ee-003693681adb-service-ca\") pod \"console-84b8475cb-bmbsk\" (UID: \"b69b1104-f7d2-4f48-b9ee-003693681adb\") " pod="openshift-console/console-84b8475cb-bmbsk" Jul 27 09:34:30.539137 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:34:30.539007 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b69b1104-f7d2-4f48-b9ee-003693681adb-oauth-serving-cert\") pod \"console-84b8475cb-bmbsk\" (UID: \"b69b1104-f7d2-4f48-b9ee-003693681adb\") " pod="openshift-console/console-84b8475cb-bmbsk" Jul 27 09:34:30.539137 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:34:30.539029 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b69b1104-f7d2-4f48-b9ee-003693681adb-console-config\") pod \"console-84b8475cb-bmbsk\" (UID: \"b69b1104-f7d2-4f48-b9ee-003693681adb\") " pod="openshift-console/console-84b8475cb-bmbsk" Jul 27 09:34:30.539319 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:34:30.539208 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b69b1104-f7d2-4f48-b9ee-003693681adb-trusted-ca-bundle\") pod \"console-84b8475cb-bmbsk\" (UID: \"b69b1104-f7d2-4f48-b9ee-003693681adb\") " pod="openshift-console/console-84b8475cb-bmbsk" Jul 27 09:34:30.539376 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:34:30.539318 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-srphh\" (UniqueName: \"kubernetes.io/projected/b69b1104-f7d2-4f48-b9ee-003693681adb-kube-api-access-srphh\") pod \"console-84b8475cb-bmbsk\" (UID: \"b69b1104-f7d2-4f48-b9ee-003693681adb\") " pod="openshift-console/console-84b8475cb-bmbsk" Jul 27 09:34:30.539376 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:34:30.539359 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b69b1104-f7d2-4f48-b9ee-003693681adb-console-oauth-config\") pod \"console-84b8475cb-bmbsk\" (UID: \"b69b1104-f7d2-4f48-b9ee-003693681adb\") " pod="openshift-console/console-84b8475cb-bmbsk" Jul 27 09:34:30.539845 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:34:30.539810 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b69b1104-f7d2-4f48-b9ee-003693681adb-console-config\") pod \"console-84b8475cb-bmbsk\" (UID: \"b69b1104-f7d2-4f48-b9ee-003693681adb\") " pod="openshift-console/console-84b8475cb-bmbsk" Jul 27 09:34:30.539845 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:34:30.539827 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b69b1104-f7d2-4f48-b9ee-003693681adb-oauth-serving-cert\") pod \"console-84b8475cb-bmbsk\" (UID: \"b69b1104-f7d2-4f48-b9ee-003693681adb\") " pod="openshift-console/console-84b8475cb-bmbsk" Jul 27 09:34:30.539982 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:34:30.539921 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b69b1104-f7d2-4f48-b9ee-003693681adb-service-ca\") pod \"console-84b8475cb-bmbsk\" (UID: \"b69b1104-f7d2-4f48-b9ee-003693681adb\") " pod="openshift-console/console-84b8475cb-bmbsk" Jul 27 09:34:30.540024 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:34:30.539986 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b69b1104-f7d2-4f48-b9ee-003693681adb-trusted-ca-bundle\") pod \"console-84b8475cb-bmbsk\" (UID: \"b69b1104-f7d2-4f48-b9ee-003693681adb\") " pod="openshift-console/console-84b8475cb-bmbsk" Jul 27 09:34:30.542159 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:34:30.542070 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b69b1104-f7d2-4f48-b9ee-003693681adb-console-serving-cert\") pod \"console-84b8475cb-bmbsk\" (UID: \"b69b1104-f7d2-4f48-b9ee-003693681adb\") " pod="openshift-console/console-84b8475cb-bmbsk" Jul 27 09:34:30.542260 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:34:30.542168 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b69b1104-f7d2-4f48-b9ee-003693681adb-console-oauth-config\") pod \"console-84b8475cb-bmbsk\" (UID: \"b69b1104-f7d2-4f48-b9ee-003693681adb\") " pod="openshift-console/console-84b8475cb-bmbsk" Jul 27 09:34:30.549429 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:34:30.549408 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-srphh\" (UniqueName: \"kubernetes.io/projected/b69b1104-f7d2-4f48-b9ee-003693681adb-kube-api-access-srphh\") pod \"console-84b8475cb-bmbsk\" (UID: \"b69b1104-f7d2-4f48-b9ee-003693681adb\") " pod="openshift-console/console-84b8475cb-bmbsk" Jul 27 09:34:30.679149 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:34:30.679072 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-84b8475cb-bmbsk" Jul 27 09:34:30.682419 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:34:30.682394 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-848w4_2c043d82-d300-41ba-b138-60a03d65ea11/konnectivity-agent/0.log" limit=1024 Jul 27 09:34:30.824857 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:34:30.824829 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-84b8475cb-bmbsk"] Jul 27 09:34:30.827315 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:34:30.827276 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb69b1104_f7d2_4f48_b9ee_003693681adb.slice/crio-3fafc5bec4d8dd09a6279ee3ef15a25d3b98b5c1a93d332fd96a735311d10112 WatchSource:0}: Error finding container 3fafc5bec4d8dd09a6279ee3ef15a25d3b98b5c1a93d332fd96a735311d10112: Status 404 returned error can't find the container with id 3fafc5bec4d8dd09a6279ee3ef15a25d3b98b5c1a93d332fd96a735311d10112 Jul 27 09:34:31.041569 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:34:31.041468 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-84b8475cb-bmbsk" event={"ID":"b69b1104-f7d2-4f48-b9ee-003693681adb","Type":"ContainerStarted","Data":"5c8248be1b0b2a252f7b4e97557f15a7986187eccc2f84739a84fd5c9891f0c2"} Jul 27 09:34:31.041569 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:34:31.041520 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-84b8475cb-bmbsk" event={"ID":"b69b1104-f7d2-4f48-b9ee-003693681adb","Type":"ContainerStarted","Data":"3fafc5bec4d8dd09a6279ee3ef15a25d3b98b5c1a93d332fd96a735311d10112"} Jul 27 09:34:31.061538 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:34:31.061476 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-84b8475cb-bmbsk" podStartSLOduration=1.061457805 podStartE2EDuration="1.061457805s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 09:34:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 09:34:31.059854398 +0000 UTC m=+599.023640376" watchObservedRunningTime="2026-07-27 09:34:31.061457805 +0000 UTC m=+599.025243762" Jul 27 09:34:31.158660 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:34:31.158619 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-848w4_2c043d82-d300-41ba-b138-60a03d65ea11/konnectivity-agent/0.log" limit=1024 Jul 27 09:34:31.213373 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:34:31.213329 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-848w4_2c043d82-d300-41ba-b138-60a03d65ea11/konnectivity-agent/0.log" limit=1024 Jul 27 09:34:32.426221 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:34:32.426198 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-w7grd_0badf8bd-7f64-46ef-872c-39f59dd11d0d/console-operator/2.log" Jul 27 09:34:32.426647 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:34:32.426198 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-w7grd_0badf8bd-7f64-46ef-872c-39f59dd11d0d/console-operator/2.log" Jul 27 09:34:32.430050 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:34:32.430029 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-mqhsn_5c031f05-bf4a-4ff7-81ad-381d31b163ff/ovn-acl-logging/0.log" Jul 27 09:34:32.430190 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:34:32.430033 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-mqhsn_5c031f05-bf4a-4ff7-81ad-381d31b163ff/ovn-acl-logging/0.log" Jul 27 09:34:33.591111 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:34:33.591077 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-848w4_2c043d82-d300-41ba-b138-60a03d65ea11/konnectivity-agent/0.log" limit=1024 Jul 27 09:34:40.679880 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:34:40.679838 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-84b8475cb-bmbsk" Jul 27 09:34:40.680812 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:34:40.680791 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-84b8475cb-bmbsk" Jul 27 09:34:40.686591 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:34:40.686567 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-84b8475cb-bmbsk" Jul 27 09:34:41.094850 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:34:41.094749 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-84b8475cb-bmbsk" Jul 27 09:34:41.143205 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:34:41.143172 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-cf67b94c4-7cpbh"] Jul 27 09:34:41.250902 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:34:41.250871 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-848w4_2c043d82-d300-41ba-b138-60a03d65ea11/konnectivity-agent/0.log" limit=1024 Jul 27 09:34:41.312200 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:34:41.312167 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-848w4_2c043d82-d300-41ba-b138-60a03d65ea11/konnectivity-agent/0.log" limit=1024 Jul 27 09:34:41.370144 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:34:41.370106 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-848w4_2c043d82-d300-41ba-b138-60a03d65ea11/konnectivity-agent/0.log" limit=1024 Jul 27 09:34:41.432512 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:34:41.432463 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-848w4_2c043d82-d300-41ba-b138-60a03d65ea11/konnectivity-agent/0.log" limit=1024 Jul 27 09:34:43.238139 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:34:43.238092 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-848w4_2c043d82-d300-41ba-b138-60a03d65ea11/konnectivity-agent/0.log" limit=1024 Jul 27 09:34:51.724371 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:34:51.724329 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-848w4_2c043d82-d300-41ba-b138-60a03d65ea11/konnectivity-agent/0.log" limit=1024 Jul 27 09:35:06.163415 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:35:06.163351 2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-cf67b94c4-7cpbh" podUID="44a40905-5ef0-41be-9da8-6862bdc8ed22" containerName="console" containerID="cri-o://de46ab06b062db0c1f046e3660c2db7605f6c2f92171dab48ebc8cbdac4605a0" gracePeriod=15 Jul 27 09:35:06.415678 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:35:06.415616 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-cf67b94c4-7cpbh" Jul 27 09:35:06.447708 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:35:06.447678 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/44a40905-5ef0-41be-9da8-6862bdc8ed22-oauth-serving-cert\") pod \"44a40905-5ef0-41be-9da8-6862bdc8ed22\" (UID: \"44a40905-5ef0-41be-9da8-6862bdc8ed22\") " Jul 27 09:35:06.447890 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:35:06.447720 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/44a40905-5ef0-41be-9da8-6862bdc8ed22-service-ca\") pod \"44a40905-5ef0-41be-9da8-6862bdc8ed22\" (UID: \"44a40905-5ef0-41be-9da8-6862bdc8ed22\") " Jul 27 09:35:06.447890 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:35:06.447747 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/44a40905-5ef0-41be-9da8-6862bdc8ed22-console-oauth-config\") pod \"44a40905-5ef0-41be-9da8-6862bdc8ed22\" (UID: \"44a40905-5ef0-41be-9da8-6862bdc8ed22\") " Jul 27 09:35:06.447890 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:35:06.447829 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/44a40905-5ef0-41be-9da8-6862bdc8ed22-console-serving-cert\") pod \"44a40905-5ef0-41be-9da8-6862bdc8ed22\" (UID: \"44a40905-5ef0-41be-9da8-6862bdc8ed22\") " Jul 27 09:35:06.447890 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:35:06.447856 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/44a40905-5ef0-41be-9da8-6862bdc8ed22-console-config\") pod \"44a40905-5ef0-41be-9da8-6862bdc8ed22\" (UID: \"44a40905-5ef0-41be-9da8-6862bdc8ed22\") " Jul 27 09:35:06.447890 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:35:06.447886 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/44a40905-5ef0-41be-9da8-6862bdc8ed22-trusted-ca-bundle\") pod \"44a40905-5ef0-41be-9da8-6862bdc8ed22\" (UID: \"44a40905-5ef0-41be-9da8-6862bdc8ed22\") " Jul 27 09:35:06.448155 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:35:06.447914 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wjq27\" (UniqueName: \"kubernetes.io/projected/44a40905-5ef0-41be-9da8-6862bdc8ed22-kube-api-access-wjq27\") pod \"44a40905-5ef0-41be-9da8-6862bdc8ed22\" (UID: \"44a40905-5ef0-41be-9da8-6862bdc8ed22\") " Jul 27 09:35:06.448155 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:35:06.448107 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/44a40905-5ef0-41be-9da8-6862bdc8ed22-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "44a40905-5ef0-41be-9da8-6862bdc8ed22" (UID: "44a40905-5ef0-41be-9da8-6862bdc8ed22"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 27 09:35:06.448303 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:35:06.448151 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/44a40905-5ef0-41be-9da8-6862bdc8ed22-service-ca" (OuterVolumeSpecName: "service-ca") pod "44a40905-5ef0-41be-9da8-6862bdc8ed22" (UID: "44a40905-5ef0-41be-9da8-6862bdc8ed22"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 27 09:35:06.448303 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:35:06.448292 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/44a40905-5ef0-41be-9da8-6862bdc8ed22-console-config" (OuterVolumeSpecName: "console-config") pod "44a40905-5ef0-41be-9da8-6862bdc8ed22" (UID: "44a40905-5ef0-41be-9da8-6862bdc8ed22"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 27 09:35:06.448456 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:35:06.448321 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/44a40905-5ef0-41be-9da8-6862bdc8ed22-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "44a40905-5ef0-41be-9da8-6862bdc8ed22" (UID: "44a40905-5ef0-41be-9da8-6862bdc8ed22"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 27 09:35:06.450123 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:35:06.450097 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/44a40905-5ef0-41be-9da8-6862bdc8ed22-kube-api-access-wjq27" (OuterVolumeSpecName: "kube-api-access-wjq27") pod "44a40905-5ef0-41be-9da8-6862bdc8ed22" (UID: "44a40905-5ef0-41be-9da8-6862bdc8ed22"). InnerVolumeSpecName "kube-api-access-wjq27". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 27 09:35:06.450215 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:35:06.450164 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/44a40905-5ef0-41be-9da8-6862bdc8ed22-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "44a40905-5ef0-41be-9da8-6862bdc8ed22" (UID: "44a40905-5ef0-41be-9da8-6862bdc8ed22"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 27 09:35:06.450417 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:35:06.450393 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/44a40905-5ef0-41be-9da8-6862bdc8ed22-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "44a40905-5ef0-41be-9da8-6862bdc8ed22" (UID: "44a40905-5ef0-41be-9da8-6862bdc8ed22"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 27 09:35:06.549065 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:35:06.549026 2565 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/44a40905-5ef0-41be-9da8-6862bdc8ed22-service-ca\") on node \"ip-10-0-128-103.ec2.internal\" DevicePath \"\"" Jul 27 09:35:06.549065 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:35:06.549058 2565 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/44a40905-5ef0-41be-9da8-6862bdc8ed22-console-oauth-config\") on node \"ip-10-0-128-103.ec2.internal\" DevicePath \"\"" Jul 27 09:35:06.549065 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:35:06.549069 2565 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/44a40905-5ef0-41be-9da8-6862bdc8ed22-console-serving-cert\") on node \"ip-10-0-128-103.ec2.internal\" DevicePath \"\"" Jul 27 09:35:06.549065 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:35:06.549080 2565 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/44a40905-5ef0-41be-9da8-6862bdc8ed22-console-config\") on node \"ip-10-0-128-103.ec2.internal\" DevicePath \"\"" Jul 27 09:35:06.549343 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:35:06.549089 2565 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/44a40905-5ef0-41be-9da8-6862bdc8ed22-trusted-ca-bundle\") on node \"ip-10-0-128-103.ec2.internal\" DevicePath \"\"" Jul 27 09:35:06.549343 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:35:06.549097 2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wjq27\" (UniqueName: \"kubernetes.io/projected/44a40905-5ef0-41be-9da8-6862bdc8ed22-kube-api-access-wjq27\") on node \"ip-10-0-128-103.ec2.internal\" DevicePath \"\"" Jul 27 09:35:06.549343 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:35:06.549106 2565 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/44a40905-5ef0-41be-9da8-6862bdc8ed22-oauth-serving-cert\") on node \"ip-10-0-128-103.ec2.internal\" DevicePath \"\"" Jul 27 09:35:07.209658 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:35:07.209619 2565 generic.go:356] "Generic (PLEG): container finished" podID="44a40905-5ef0-41be-9da8-6862bdc8ed22" containerID="de46ab06b062db0c1f046e3660c2db7605f6c2f92171dab48ebc8cbdac4605a0" exitCode=0 Jul 27 09:35:07.210105 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:35:07.209673 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-cf67b94c4-7cpbh" Jul 27 09:35:07.210105 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:35:07.209695 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-cf67b94c4-7cpbh" event={"ID":"44a40905-5ef0-41be-9da8-6862bdc8ed22","Type":"ContainerDied","Data":"de46ab06b062db0c1f046e3660c2db7605f6c2f92171dab48ebc8cbdac4605a0"} Jul 27 09:35:07.210105 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:35:07.209787 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-cf67b94c4-7cpbh" event={"ID":"44a40905-5ef0-41be-9da8-6862bdc8ed22","Type":"ContainerDied","Data":"2c405d938f3f10b1af5e195eb6038b8ca9686ff82c50cd7625eec886beaea93d"} Jul 27 09:35:07.210105 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:35:07.209810 2565 scope.go:117] "RemoveContainer" containerID="de46ab06b062db0c1f046e3660c2db7605f6c2f92171dab48ebc8cbdac4605a0" Jul 27 09:35:07.218172 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:35:07.218156 2565 scope.go:117] "RemoveContainer" containerID="de46ab06b062db0c1f046e3660c2db7605f6c2f92171dab48ebc8cbdac4605a0" Jul 27 09:35:07.218462 ip-10-0-128-103 kubenswrapper[2565]: E0727 09:35:07.218436 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"de46ab06b062db0c1f046e3660c2db7605f6c2f92171dab48ebc8cbdac4605a0\": container with ID starting with de46ab06b062db0c1f046e3660c2db7605f6c2f92171dab48ebc8cbdac4605a0 not found: ID does not exist" containerID="de46ab06b062db0c1f046e3660c2db7605f6c2f92171dab48ebc8cbdac4605a0" Jul 27 09:35:07.218530 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:35:07.218475 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"de46ab06b062db0c1f046e3660c2db7605f6c2f92171dab48ebc8cbdac4605a0"} err="failed to get container status \"de46ab06b062db0c1f046e3660c2db7605f6c2f92171dab48ebc8cbdac4605a0\": rpc error: code = NotFound desc = could not find container \"de46ab06b062db0c1f046e3660c2db7605f6c2f92171dab48ebc8cbdac4605a0\": container with ID starting with de46ab06b062db0c1f046e3660c2db7605f6c2f92171dab48ebc8cbdac4605a0 not found: ID does not exist" Jul 27 09:35:07.226675 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:35:07.226651 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-cf67b94c4-7cpbh"] Jul 27 09:35:07.230527 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:35:07.230505 2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-cf67b94c4-7cpbh"] Jul 27 09:35:08.563445 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:35:08.563411 2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="44a40905-5ef0-41be-9da8-6862bdc8ed22" path="/var/lib/kubelet/pods/44a40905-5ef0-41be-9da8-6862bdc8ed22/volumes" Jul 27 09:36:01.016299 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:36:01.016257 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-848w4_2c043d82-d300-41ba-b138-60a03d65ea11/konnectivity-agent/0.log" limit=1024 Jul 27 09:36:01.146729 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:36:01.146691 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-848w4_2c043d82-d300-41ba-b138-60a03d65ea11/konnectivity-agent/0.log" limit=1024 Jul 27 09:36:01.225365 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:36:01.225334 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-848w4_2c043d82-d300-41ba-b138-60a03d65ea11/konnectivity-agent/0.log" limit=1024 Jul 27 09:36:01.425523 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:36:01.425481 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-848w4_2c043d82-d300-41ba-b138-60a03d65ea11/konnectivity-agent/0.log" limit=1024 Jul 27 09:36:01.483217 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:36:01.483179 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-848w4_2c043d82-d300-41ba-b138-60a03d65ea11/konnectivity-agent/0.log" limit=1024 Jul 27 09:36:02.836284 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:36:02.836245 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["trainer-operator-system/trainer-operator-controller-manager-57ffcf598-jnfnb"] Jul 27 09:36:02.836680 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:36:02.836559 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="44a40905-5ef0-41be-9da8-6862bdc8ed22" containerName="console" Jul 27 09:36:02.836680 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:36:02.836569 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="44a40905-5ef0-41be-9da8-6862bdc8ed22" containerName="console" Jul 27 09:36:02.836680 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:36:02.836632 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="44a40905-5ef0-41be-9da8-6862bdc8ed22" containerName="console" Jul 27 09:36:02.839191 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:36:02.839174 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="trainer-operator-system/trainer-operator-controller-manager-57ffcf598-jnfnb" Jul 27 09:36:02.841813 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:36:02.841785 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"trainer-operator-system\"/\"kube-root-ca.crt\"" Jul 27 09:36:02.841947 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:36:02.841785 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"trainer-operator-system\"/\"openshift-service-ca.crt\"" Jul 27 09:36:02.842548 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:36:02.842528 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"trainer-operator-system\"/\"trainer-operator-controller-manager-dockercfg-8nvwb\"" Jul 27 09:36:02.852335 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:36:02.852305 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["trainer-operator-system/trainer-operator-controller-manager-57ffcf598-jnfnb"] Jul 27 09:36:02.946563 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:36:02.946526 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qbj48\" (UniqueName: \"kubernetes.io/projected/c8d0d6a1-71e0-4f0c-9ae6-cd609db495a5-kube-api-access-qbj48\") pod \"trainer-operator-controller-manager-57ffcf598-jnfnb\" (UID: \"c8d0d6a1-71e0-4f0c-9ae6-cd609db495a5\") " pod="trainer-operator-system/trainer-operator-controller-manager-57ffcf598-jnfnb" Jul 27 09:36:02.946737 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:36:02.946570 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests-work\" (UniqueName: \"kubernetes.io/empty-dir/c8d0d6a1-71e0-4f0c-9ae6-cd609db495a5-manifests-work\") pod \"trainer-operator-controller-manager-57ffcf598-jnfnb\" (UID: \"c8d0d6a1-71e0-4f0c-9ae6-cd609db495a5\") " pod="trainer-operator-system/trainer-operator-controller-manager-57ffcf598-jnfnb" Jul 27 09:36:02.946737 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:36:02.946600 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c8d0d6a1-71e0-4f0c-9ae6-cd609db495a5-tmp\") pod \"trainer-operator-controller-manager-57ffcf598-jnfnb\" (UID: \"c8d0d6a1-71e0-4f0c-9ae6-cd609db495a5\") " pod="trainer-operator-system/trainer-operator-controller-manager-57ffcf598-jnfnb" Jul 27 09:36:03.047903 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:36:03.047851 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qbj48\" (UniqueName: \"kubernetes.io/projected/c8d0d6a1-71e0-4f0c-9ae6-cd609db495a5-kube-api-access-qbj48\") pod \"trainer-operator-controller-manager-57ffcf598-jnfnb\" (UID: \"c8d0d6a1-71e0-4f0c-9ae6-cd609db495a5\") " pod="trainer-operator-system/trainer-operator-controller-manager-57ffcf598-jnfnb" Jul 27 09:36:03.048069 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:36:03.047918 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests-work\" (UniqueName: \"kubernetes.io/empty-dir/c8d0d6a1-71e0-4f0c-9ae6-cd609db495a5-manifests-work\") pod \"trainer-operator-controller-manager-57ffcf598-jnfnb\" (UID: \"c8d0d6a1-71e0-4f0c-9ae6-cd609db495a5\") " pod="trainer-operator-system/trainer-operator-controller-manager-57ffcf598-jnfnb" Jul 27 09:36:03.048069 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:36:03.047950 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c8d0d6a1-71e0-4f0c-9ae6-cd609db495a5-tmp\") pod \"trainer-operator-controller-manager-57ffcf598-jnfnb\" (UID: \"c8d0d6a1-71e0-4f0c-9ae6-cd609db495a5\") " pod="trainer-operator-system/trainer-operator-controller-manager-57ffcf598-jnfnb" Jul 27 09:36:03.048261 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:36:03.048243 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests-work\" (UniqueName: \"kubernetes.io/empty-dir/c8d0d6a1-71e0-4f0c-9ae6-cd609db495a5-manifests-work\") pod \"trainer-operator-controller-manager-57ffcf598-jnfnb\" (UID: \"c8d0d6a1-71e0-4f0c-9ae6-cd609db495a5\") " pod="trainer-operator-system/trainer-operator-controller-manager-57ffcf598-jnfnb" Jul 27 09:36:03.048297 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:36:03.048277 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c8d0d6a1-71e0-4f0c-9ae6-cd609db495a5-tmp\") pod \"trainer-operator-controller-manager-57ffcf598-jnfnb\" (UID: \"c8d0d6a1-71e0-4f0c-9ae6-cd609db495a5\") " pod="trainer-operator-system/trainer-operator-controller-manager-57ffcf598-jnfnb" Jul 27 09:36:03.057121 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:36:03.057097 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qbj48\" (UniqueName: \"kubernetes.io/projected/c8d0d6a1-71e0-4f0c-9ae6-cd609db495a5-kube-api-access-qbj48\") pod \"trainer-operator-controller-manager-57ffcf598-jnfnb\" (UID: \"c8d0d6a1-71e0-4f0c-9ae6-cd609db495a5\") " pod="trainer-operator-system/trainer-operator-controller-manager-57ffcf598-jnfnb" Jul 27 09:36:03.149166 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:36:03.149117 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="trainer-operator-system/trainer-operator-controller-manager-57ffcf598-jnfnb" Jul 27 09:36:03.282361 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:36:03.282336 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["trainer-operator-system/trainer-operator-controller-manager-57ffcf598-jnfnb"] Jul 27 09:36:03.285222 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:36:03.285192 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc8d0d6a1_71e0_4f0c_9ae6_cd609db495a5.slice/crio-6d907e078426e2b06883ba960d825847d845ee8954e6097763a610ad5c00bd25 WatchSource:0}: Error finding container 6d907e078426e2b06883ba960d825847d845ee8954e6097763a610ad5c00bd25: Status 404 returned error can't find the container with id 6d907e078426e2b06883ba960d825847d845ee8954e6097763a610ad5c00bd25 Jul 27 09:36:03.464836 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:36:03.464743 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="trainer-operator-system/trainer-operator-controller-manager-57ffcf598-jnfnb" event={"ID":"c8d0d6a1-71e0-4f0c-9ae6-cd609db495a5","Type":"ContainerStarted","Data":"6d907e078426e2b06883ba960d825847d845ee8954e6097763a610ad5c00bd25"} Jul 27 09:36:04.259885 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:36:04.259845 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-848w4_2c043d82-d300-41ba-b138-60a03d65ea11/konnectivity-agent/0.log" limit=1024 Jul 27 09:36:04.329223 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:36:04.329184 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-848w4_2c043d82-d300-41ba-b138-60a03d65ea11/konnectivity-agent/0.log" limit=1024 Jul 27 09:36:06.481950 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:36:06.481908 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="trainer-operator-system/trainer-operator-controller-manager-57ffcf598-jnfnb" event={"ID":"c8d0d6a1-71e0-4f0c-9ae6-cd609db495a5","Type":"ContainerStarted","Data":"499df4f7ca2845c9caa02d730ef36245e23b2159156d237ac7e940d90674f4fb"} Jul 27 09:36:06.482413 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:36:06.482047 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="trainer-operator-system/trainer-operator-controller-manager-57ffcf598-jnfnb" Jul 27 09:36:06.499909 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:36:06.499856 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="trainer-operator-system/trainer-operator-controller-manager-57ffcf598-jnfnb" podStartSLOduration=1.7567855140000002 podStartE2EDuration="4.499839453s" totalImagesPullingTime="2.743053939s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 09:36:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 09:36:06.497934402 +0000 UTC m=+694.461720380" watchObservedRunningTime="2026-07-27 09:36:06.499839453 +0000 UTC m=+694.463625410" Jul 27 09:36:17.489446 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:36:17.489410 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="trainer-operator-system/trainer-operator-controller-manager-57ffcf598-jnfnb" Jul 27 09:36:59.729973 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:36:59.729891 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-848w4_2c043d82-d300-41ba-b138-60a03d65ea11/konnectivity-agent/0.log" limit=1024 Jul 27 09:37:01.122334 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:37:01.122292 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-848w4_2c043d82-d300-41ba-b138-60a03d65ea11/konnectivity-agent/0.log" limit=1024 Jul 27 09:37:02.432327 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:37:02.432291 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-rqz85_204c97b6-2efe-4db7-bc9f-a5653fed77c7/global-pull-secret-syncer/0.log" Jul 27 09:37:02.486898 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:37:02.486863 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-848w4_2c043d82-d300-41ba-b138-60a03d65ea11/konnectivity-agent/0.log" Jul 27 09:37:02.567325 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:37:02.567297 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-128-103.ec2.internal_f9f6fe57433891594e1efa7d6875336f/haproxy/0.log" Jul 27 09:37:05.170567 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:37:05.170530 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-848w4_2c043d82-d300-41ba-b138-60a03d65ea11/konnectivity-agent/0.log" limit=1024 Jul 27 09:37:05.228949 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:37:05.228894 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-848w4_2c043d82-d300-41ba-b138-60a03d65ea11/konnectivity-agent/0.log" limit=1024 Jul 27 09:37:05.897705 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:37:05.897663 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-548b489cb8-lgr4b_fb21c72b-5cea-49fa-9b28-d1362f405ab0/cluster-monitoring-operator/0.log" Jul 27 09:37:05.942060 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:37:05.942025 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56d9f944b4-mhq2g_45db6a76-b0a6-475e-a2c2-2e0bf01f8e17/kube-state-metrics/0.log" Jul 27 09:37:05.973688 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:37:05.973657 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56d9f944b4-mhq2g_45db6a76-b0a6-475e-a2c2-2e0bf01f8e17/kube-rbac-proxy-main/0.log" Jul 27 09:37:06.001373 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:37:06.001348 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56d9f944b4-mhq2g_45db6a76-b0a6-475e-a2c2-2e0bf01f8e17/kube-rbac-proxy-self/0.log" Jul 27 09:37:06.040506 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:37:06.040474 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-848f65bd86-bb84d_0aa1aaa9-b87e-4497-9ae2-11b00f96bf39/metrics-server/0.log" Jul 27 09:37:06.290510 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:37:06.290432 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-vwsrv_d9769977-82c3-4556-a9f3-7a6ec82bdb75/node-exporter/0.log" Jul 27 09:37:06.318137 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:37:06.318106 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-vwsrv_d9769977-82c3-4556-a9f3-7a6ec82bdb75/kube-rbac-proxy/0.log" Jul 27 09:37:06.344172 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:37:06.344144 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-vwsrv_d9769977-82c3-4556-a9f3-7a6ec82bdb75/init-textfile/0.log" Jul 27 09:37:06.670816 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:37:06.670787 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-84d69f794d-2v9b6_73c95e17-4ad0-41fb-a28f-23cc54ac8a7c/prometheus-operator/0.log" Jul 27 09:37:06.690739 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:37:06.690709 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-84d69f794d-2v9b6_73c95e17-4ad0-41fb-a28f-23cc54ac8a7c/kube-rbac-proxy/0.log" Jul 27 09:37:06.753615 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:37:06.753574 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-8666cc54cb-cx9bx_f598fec9-8aac-4221-a931-0127a29b7617/telemeter-client/0.log" Jul 27 09:37:06.778640 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:37:06.778612 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-8666cc54cb-cx9bx_f598fec9-8aac-4221-a931-0127a29b7617/reload/0.log" Jul 27 09:37:06.804134 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:37:06.804102 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-8666cc54cb-cx9bx_f598fec9-8aac-4221-a931-0127a29b7617/kube-rbac-proxy/0.log" Jul 27 09:37:08.793585 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:37:08.793559 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-w7grd_0badf8bd-7f64-46ef-872c-39f59dd11d0d/console-operator/2.log" Jul 27 09:37:08.801305 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:37:08.801276 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-w7grd_0badf8bd-7f64-46ef-872c-39f59dd11d0d/console-operator/3.log" Jul 27 09:37:09.222678 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:37:09.222652 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-84b8475cb-bmbsk_b69b1104-f7d2-4f48-b9ee-003693681adb/console/0.log" Jul 27 09:37:09.253965 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:37:09.253934 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-7c97dc4555-v8rpf_3e10f8fb-0a12-4c28-8758-deaee169ebaf/download-server/0.log" Jul 27 09:37:10.336255 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:37:10.336223 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-l9jqc/perf-node-gather-daemonset-6b9sg"] Jul 27 09:37:10.343406 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:37:10.343366 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-l9jqc/perf-node-gather-daemonset-6b9sg" Jul 27 09:37:10.346267 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:37:10.346237 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-l9jqc\"/\"openshift-service-ca.crt\"" Jul 27 09:37:10.346412 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:37:10.346248 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-l9jqc\"/\"kube-root-ca.crt\"" Jul 27 09:37:10.346840 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:37:10.346826 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-l9jqc\"/\"default-dockercfg-nt7h7\"" Jul 27 09:37:10.350229 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:37:10.350204 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-l9jqc/perf-node-gather-daemonset-6b9sg"] Jul 27 09:37:10.436880 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:37:10.436842 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/34e192cd-bb15-4315-8661-3de19b76f3e8-podres\") pod \"perf-node-gather-daemonset-6b9sg\" (UID: \"34e192cd-bb15-4315-8661-3de19b76f3e8\") " pod="openshift-must-gather-l9jqc/perf-node-gather-daemonset-6b9sg" Jul 27 09:37:10.436880 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:37:10.436884 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/34e192cd-bb15-4315-8661-3de19b76f3e8-proc\") pod \"perf-node-gather-daemonset-6b9sg\" (UID: \"34e192cd-bb15-4315-8661-3de19b76f3e8\") " pod="openshift-must-gather-l9jqc/perf-node-gather-daemonset-6b9sg" Jul 27 09:37:10.437107 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:37:10.436910 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/34e192cd-bb15-4315-8661-3de19b76f3e8-lib-modules\") pod \"perf-node-gather-daemonset-6b9sg\" (UID: \"34e192cd-bb15-4315-8661-3de19b76f3e8\") " pod="openshift-must-gather-l9jqc/perf-node-gather-daemonset-6b9sg" Jul 27 09:37:10.437107 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:37:10.436972 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/34e192cd-bb15-4315-8661-3de19b76f3e8-sys\") pod \"perf-node-gather-daemonset-6b9sg\" (UID: \"34e192cd-bb15-4315-8661-3de19b76f3e8\") " pod="openshift-must-gather-l9jqc/perf-node-gather-daemonset-6b9sg" Jul 27 09:37:10.437107 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:37:10.437045 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-llzpv\" (UniqueName: \"kubernetes.io/projected/34e192cd-bb15-4315-8661-3de19b76f3e8-kube-api-access-llzpv\") pod \"perf-node-gather-daemonset-6b9sg\" (UID: \"34e192cd-bb15-4315-8661-3de19b76f3e8\") " pod="openshift-must-gather-l9jqc/perf-node-gather-daemonset-6b9sg" Jul 27 09:37:10.537659 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:37:10.537609 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-llzpv\" (UniqueName: \"kubernetes.io/projected/34e192cd-bb15-4315-8661-3de19b76f3e8-kube-api-access-llzpv\") pod \"perf-node-gather-daemonset-6b9sg\" (UID: \"34e192cd-bb15-4315-8661-3de19b76f3e8\") " pod="openshift-must-gather-l9jqc/perf-node-gather-daemonset-6b9sg" Jul 27 09:37:10.537659 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:37:10.537669 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/34e192cd-bb15-4315-8661-3de19b76f3e8-podres\") pod \"perf-node-gather-daemonset-6b9sg\" (UID: \"34e192cd-bb15-4315-8661-3de19b76f3e8\") " pod="openshift-must-gather-l9jqc/perf-node-gather-daemonset-6b9sg" Jul 27 09:37:10.537894 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:37:10.537692 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/34e192cd-bb15-4315-8661-3de19b76f3e8-proc\") pod \"perf-node-gather-daemonset-6b9sg\" (UID: \"34e192cd-bb15-4315-8661-3de19b76f3e8\") " pod="openshift-must-gather-l9jqc/perf-node-gather-daemonset-6b9sg" Jul 27 09:37:10.537894 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:37:10.537714 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/34e192cd-bb15-4315-8661-3de19b76f3e8-lib-modules\") pod \"perf-node-gather-daemonset-6b9sg\" (UID: \"34e192cd-bb15-4315-8661-3de19b76f3e8\") " pod="openshift-must-gather-l9jqc/perf-node-gather-daemonset-6b9sg" Jul 27 09:37:10.537894 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:37:10.537735 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/34e192cd-bb15-4315-8661-3de19b76f3e8-sys\") pod \"perf-node-gather-daemonset-6b9sg\" (UID: \"34e192cd-bb15-4315-8661-3de19b76f3e8\") " pod="openshift-must-gather-l9jqc/perf-node-gather-daemonset-6b9sg" Jul 27 09:37:10.537894 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:37:10.537827 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/34e192cd-bb15-4315-8661-3de19b76f3e8-proc\") pod \"perf-node-gather-daemonset-6b9sg\" (UID: \"34e192cd-bb15-4315-8661-3de19b76f3e8\") " pod="openshift-must-gather-l9jqc/perf-node-gather-daemonset-6b9sg" Jul 27 09:37:10.537894 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:37:10.537859 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/34e192cd-bb15-4315-8661-3de19b76f3e8-sys\") pod \"perf-node-gather-daemonset-6b9sg\" (UID: \"34e192cd-bb15-4315-8661-3de19b76f3e8\") " pod="openshift-must-gather-l9jqc/perf-node-gather-daemonset-6b9sg" Jul 27 09:37:10.538069 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:37:10.537891 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/34e192cd-bb15-4315-8661-3de19b76f3e8-podres\") pod \"perf-node-gather-daemonset-6b9sg\" (UID: \"34e192cd-bb15-4315-8661-3de19b76f3e8\") " pod="openshift-must-gather-l9jqc/perf-node-gather-daemonset-6b9sg" Jul 27 09:37:10.538069 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:37:10.537894 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/34e192cd-bb15-4315-8661-3de19b76f3e8-lib-modules\") pod \"perf-node-gather-daemonset-6b9sg\" (UID: \"34e192cd-bb15-4315-8661-3de19b76f3e8\") " pod="openshift-must-gather-l9jqc/perf-node-gather-daemonset-6b9sg" Jul 27 09:37:10.546239 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:37:10.546214 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-llzpv\" (UniqueName: \"kubernetes.io/projected/34e192cd-bb15-4315-8661-3de19b76f3e8-kube-api-access-llzpv\") pod \"perf-node-gather-daemonset-6b9sg\" (UID: \"34e192cd-bb15-4315-8661-3de19b76f3e8\") " pod="openshift-must-gather-l9jqc/perf-node-gather-daemonset-6b9sg" Jul 27 09:37:10.637513 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:37:10.637475 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-7lzb6_ef01a458-240c-4477-9ef9-d5b21026df4f/dns/0.log" Jul 27 09:37:10.655115 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:37:10.655087 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-l9jqc/perf-node-gather-daemonset-6b9sg" Jul 27 09:37:10.658942 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:37:10.658917 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-7lzb6_ef01a458-240c-4477-9ef9-d5b21026df4f/kube-rbac-proxy/0.log" Jul 27 09:37:10.789229 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:37:10.789103 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-l9jqc/perf-node-gather-daemonset-6b9sg"] Jul 27 09:37:10.792026 ip-10-0-128-103 kubenswrapper[2565]: W0727 09:37:10.791992 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod34e192cd_bb15_4315_8661_3de19b76f3e8.slice/crio-8b6922d613a171832f85b8d821d875a1051b75599c585c8f137ca503566236f2 WatchSource:0}: Error finding container 8b6922d613a171832f85b8d821d875a1051b75599c585c8f137ca503566236f2: Status 404 returned error can't find the container with id 8b6922d613a171832f85b8d821d875a1051b75599c585c8f137ca503566236f2 Jul 27 09:37:10.831931 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:37:10.831899 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-r2wsn_199af0d6-b2fc-4951-941f-948e403d3865/dns-node-resolver/0.log" Jul 27 09:37:11.212880 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:37:11.212840 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-56f74b4f54-4jm6n_309e9463-20cb-4831-847a-c3421997a50f/registry/0.log" Jul 27 09:37:11.234814 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:37:11.234786 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-bmklx_996c29bd-3969-4268-a795-4f4d21881d48/node-ca/0.log" Jul 27 09:37:11.780994 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:37:11.780962 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-l9jqc/perf-node-gather-daemonset-6b9sg" event={"ID":"34e192cd-bb15-4315-8661-3de19b76f3e8","Type":"ContainerStarted","Data":"05b27671fc8d84dadf8656db12a8544ac1fc7585705e136a7e75125b1ae18361"} Jul 27 09:37:11.781375 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:37:11.781016 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-l9jqc/perf-node-gather-daemonset-6b9sg" event={"ID":"34e192cd-bb15-4315-8661-3de19b76f3e8","Type":"ContainerStarted","Data":"8b6922d613a171832f85b8d821d875a1051b75599c585c8f137ca503566236f2"} Jul 27 09:37:11.781375 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:37:11.781087 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-l9jqc/perf-node-gather-daemonset-6b9sg" Jul 27 09:37:12.315342 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:37:12.315311 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-q52r7_9b739ad1-657a-40f1-a932-0a94f3960ca4/serve-healthcheck-canary/0.log" Jul 27 09:37:12.876600 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:37:12.876573 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-xfvbm_a2f0dc08-339d-4aad-bae9-e13a8f594c70/kube-rbac-proxy/0.log" Jul 27 09:37:12.902737 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:37:12.902708 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-xfvbm_a2f0dc08-339d-4aad-bae9-e13a8f594c70/exporter/0.log" Jul 27 09:37:12.924481 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:37:12.924401 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-xfvbm_a2f0dc08-339d-4aad-bae9-e13a8f594c70/extractor/0.log" Jul 27 09:37:13.655723 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:37:13.655688 2565 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-848w4_2c043d82-d300-41ba-b138-60a03d65ea11/konnectivity-agent/0.log" limit=1024 Jul 27 09:37:14.551623 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:37:14.551592 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-jobset-operator_jobset-controller-manager-659557f77c-4cczh_85d82827-9d5c-47df-9d62-23baa0706e51/manager/0.log" Jul 27 09:37:17.794937 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:37:17.794910 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-l9jqc/perf-node-gather-daemonset-6b9sg" Jul 27 09:37:17.812559 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:37:17.812507 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-l9jqc/perf-node-gather-daemonset-6b9sg" podStartSLOduration=7.812490582 podStartE2EDuration="7.812490582s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-27 09:37:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 09:37:11.797892627 +0000 UTC m=+759.761678604" watchObservedRunningTime="2026-07-27 09:37:17.812490582 +0000 UTC m=+765.776276577" Jul 27 09:37:18.010302 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:37:18.010267 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-6f4d6fb6bc-b5jrj_85bf3791-57a8-4f6a-aba7-85b8a44e170c/kube-storage-version-migrator-operator/1.log" Jul 27 09:37:18.012017 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:37:18.011970 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-6f4d6fb6bc-b5jrj_85bf3791-57a8-4f6a-aba7-85b8a44e170c/kube-storage-version-migrator-operator/0.log" Jul 27 09:37:18.957571 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:37:18.957543 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-54xcx_d0206685-d6b8-43ea-a691-ca3c1acd237a/kube-multus-additional-cni-plugins/0.log" Jul 27 09:37:18.981581 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:37:18.981497 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-54xcx_d0206685-d6b8-43ea-a691-ca3c1acd237a/egress-router-binary-copy/0.log" Jul 27 09:37:19.005896 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:37:19.005866 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-54xcx_d0206685-d6b8-43ea-a691-ca3c1acd237a/cni-plugins/0.log" Jul 27 09:37:19.037483 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:37:19.037452 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-54xcx_d0206685-d6b8-43ea-a691-ca3c1acd237a/bond-cni-plugin/0.log" Jul 27 09:37:19.064327 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:37:19.064302 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-54xcx_d0206685-d6b8-43ea-a691-ca3c1acd237a/routeoverride-cni/0.log" Jul 27 09:37:19.090270 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:37:19.090231 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-54xcx_d0206685-d6b8-43ea-a691-ca3c1acd237a/whereabouts-cni-bincopy/0.log" Jul 27 09:37:19.117804 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:37:19.117748 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-54xcx_d0206685-d6b8-43ea-a691-ca3c1acd237a/whereabouts-cni/0.log" Jul 27 09:37:19.505627 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:37:19.505590 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-jmgz6_29406f41-dc4d-4f8a-9627-9589659ae0d4/kube-multus/0.log" Jul 27 09:37:19.624785 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:37:19.624749 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-trcl2_0d6ba35e-ec8a-4ec9-8520-0d32a5586a1d/network-metrics-daemon/0.log" Jul 27 09:37:19.645647 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:37:19.645562 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-trcl2_0d6ba35e-ec8a-4ec9-8520-0d32a5586a1d/kube-rbac-proxy/0.log" Jul 27 09:37:20.648216 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:37:20.648176 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-mqhsn_5c031f05-bf4a-4ff7-81ad-381d31b163ff/ovn-controller/0.log" Jul 27 09:37:20.665302 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:37:20.665272 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-mqhsn_5c031f05-bf4a-4ff7-81ad-381d31b163ff/ovn-acl-logging/1.log" Jul 27 09:37:20.665462 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:37:20.665312 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-mqhsn_5c031f05-bf4a-4ff7-81ad-381d31b163ff/ovn-acl-logging/0.log" Jul 27 09:37:20.689834 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:37:20.689801 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-mqhsn_5c031f05-bf4a-4ff7-81ad-381d31b163ff/kube-rbac-proxy-node/0.log" Jul 27 09:37:20.715445 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:37:20.715415 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-mqhsn_5c031f05-bf4a-4ff7-81ad-381d31b163ff/kube-rbac-proxy-ovn-metrics/0.log" Jul 27 09:37:20.733287 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:37:20.733261 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-mqhsn_5c031f05-bf4a-4ff7-81ad-381d31b163ff/northd/0.log" Jul 27 09:37:20.756649 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:37:20.756615 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-mqhsn_5c031f05-bf4a-4ff7-81ad-381d31b163ff/nbdb/0.log" Jul 27 09:37:20.784924 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:37:20.784894 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-mqhsn_5c031f05-bf4a-4ff7-81ad-381d31b163ff/sbdb/0.log" Jul 27 09:37:20.850501 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:37:20.850468 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-mqhsn_5c031f05-bf4a-4ff7-81ad-381d31b163ff/ovnkube-controller/0.log" Jul 27 09:37:22.044386 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:37:22.044359 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-diagnostics_network-check-target-5zqgm_91816deb-5261-44b7-857a-04cf036feffa/network-check-target-container/0.log" Jul 27 09:37:23.129017 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:37:23.128992 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-operator_iptables-alerter-rcftn_6fd3f632-9d80-4e6e-8d4f-f5314c0f174a/iptables-alerter/0.log" Jul 27 09:37:23.885129 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:37:23.885095 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-node-tuning-operator_tuned-bk5w8_9cc406f3-cdff-4cde-ac7b-ff31abb20e0d/tuned/0.log" Jul 27 09:37:25.560646 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:37:25.560617 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-samples-operator_cluster-samples-operator-6748489656-hsqzj_f3c02965-8034-4cc6-9cd6-eeff280ebb58/cluster-samples-operator/0.log" Jul 27 09:37:25.577316 ip-10-0-128-103 kubenswrapper[2565]: I0727 09:37:25.577288 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-samples-operator_cluster-samples-operator-6748489656-hsqzj_f3c02965-8034-4cc6-9cd6-eeff280ebb58/cluster-samples-operator-watch/0.log"