Jul 01 16:26:23.346873 ip-10-0-130-227 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory Jul 01 16:26:23.346881 ip-10-0-130-227 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory Jul 01 16:26:23.346888 ip-10-0-130-227 systemd[1]: kubelet.service: Failed with result 'resources'. Jul 01 16:26:23.347096 ip-10-0-130-227 systemd[1]: Failed to start Kubernetes Kubelet. Jul 01 16:26:34.640683 ip-10-0-130-227 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found. Jul 01 16:26:34.640699 ip-10-0-130-227 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot f241e63db9d84736a911dafee810b9be -- Jul 01 16:28:41.311988 ip-10-0-130-227 systemd[1]: Starting Kubernetes Kubelet... Jul 01 16:28:42.768522 ip-10-0-130-227 kubenswrapper[2545]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 01 16:28:42.768522 ip-10-0-130-227 kubenswrapper[2545]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 01 16:28:42.768522 ip-10-0-130-227 kubenswrapper[2545]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 01 16:28:42.768522 ip-10-0-130-227 kubenswrapper[2545]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 01 16:28:42.768522 ip-10-0-130-227 kubenswrapper[2545]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 01 16:28:42.770232 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:42.770149 2545 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 01 16:28:42.775864 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.775849 2545 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 01 16:28:42.775901 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.775866 2545 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 01 16:28:42.775901 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.775870 2545 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 01 16:28:42.775901 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.775873 2545 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 01 16:28:42.775901 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.775876 2545 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 01 16:28:42.775901 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.775879 2545 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 01 16:28:42.775901 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.775883 2545 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 01 16:28:42.775901 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.775886 2545 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 01 16:28:42.775901 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.775889 2545 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 01 16:28:42.775901 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.775892 2545 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 01 16:28:42.775901 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.775895 2545 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 01 16:28:42.775901 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.775898 2545 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 01 16:28:42.775901 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.775901 2545 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 01 16:28:42.775901 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.775903 2545 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 01 16:28:42.775901 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.775906 2545 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 01 16:28:42.776214 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.775909 2545 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 01 16:28:42.776214 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.775912 2545 feature_gate.go:328] unrecognized feature gate: Example Jul 01 16:28:42.776214 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.775915 2545 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 01 16:28:42.776214 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.775918 2545 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 01 16:28:42.776214 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.775929 2545 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 01 16:28:42.776214 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.775932 2545 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 01 16:28:42.776214 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.775935 2545 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 01 16:28:42.776214 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.775938 2545 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 01 16:28:42.776214 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.775940 2545 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 01 16:28:42.776214 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.775943 2545 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 01 16:28:42.776214 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.775945 2545 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 01 16:28:42.776214 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.775948 2545 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 01 16:28:42.776214 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.775950 2545 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 01 16:28:42.776214 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.775953 2545 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 01 16:28:42.776214 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.775956 2545 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 01 16:28:42.776214 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.775958 2545 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 01 16:28:42.776214 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.775960 2545 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 01 16:28:42.776214 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.775963 2545 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 01 16:28:42.776214 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.775966 2545 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 01 16:28:42.776659 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.775968 2545 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 01 16:28:42.776659 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.775971 2545 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 01 16:28:42.776659 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.775974 2545 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 01 16:28:42.776659 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.775976 2545 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 01 16:28:42.776659 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.775980 2545 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 01 16:28:42.776659 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.775982 2545 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 01 16:28:42.776659 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.775985 2545 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 01 16:28:42.776659 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.775987 2545 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 01 16:28:42.776659 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.775991 2545 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 01 16:28:42.776659 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.775993 2545 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 01 16:28:42.776659 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.775996 2545 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 01 16:28:42.776659 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.775998 2545 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 01 16:28:42.776659 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.776002 2545 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 01 16:28:42.776659 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.776006 2545 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 01 16:28:42.776659 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.776009 2545 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 01 16:28:42.776659 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.776013 2545 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 01 16:28:42.776659 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.776016 2545 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 01 16:28:42.776659 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.776018 2545 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 01 16:28:42.776659 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.776021 2545 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 01 16:28:42.776659 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.776024 2545 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 01 16:28:42.777148 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.776027 2545 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 01 16:28:42.777148 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.776031 2545 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 01 16:28:42.777148 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.776034 2545 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 01 16:28:42.777148 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.776037 2545 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 01 16:28:42.777148 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.776040 2545 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 01 16:28:42.777148 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.776042 2545 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 01 16:28:42.777148 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.776045 2545 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 01 16:28:42.777148 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.776048 2545 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 01 16:28:42.777148 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.776051 2545 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 01 16:28:42.777148 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.776054 2545 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 01 16:28:42.777148 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.776056 2545 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 01 16:28:42.777148 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.776059 2545 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 01 16:28:42.777148 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.776062 2545 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 01 16:28:42.777148 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.776064 2545 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 01 16:28:42.777148 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.776067 2545 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 01 16:28:42.777148 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.776069 2545 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 01 16:28:42.777148 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.776072 2545 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 01 16:28:42.777148 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.776077 2545 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 01 16:28:42.777148 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.776079 2545 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 01 16:28:42.777628 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.776082 2545 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 01 16:28:42.777628 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.776084 2545 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 01 16:28:42.777628 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.776088 2545 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 01 16:28:42.777628 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.776091 2545 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 01 16:28:42.777628 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.776094 2545 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 01 16:28:42.777628 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.776096 2545 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 01 16:28:42.777628 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.776099 2545 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 01 16:28:42.777628 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.776102 2545 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 01 16:28:42.777628 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.776105 2545 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 01 16:28:42.777628 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.776107 2545 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 01 16:28:42.777628 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.776110 2545 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 01 16:28:42.777628 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.776112 2545 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 01 16:28:42.777628 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.776115 2545 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 01 16:28:42.777628 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.776117 2545 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 01 16:28:42.777628 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.776120 2545 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 01 16:28:42.777628 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.776124 2545 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 01 16:28:42.777628 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.776128 2545 feature_gate.go:328] unrecognized feature gate: Example2 Jul 01 16:28:42.777628 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.776131 2545 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 01 16:28:42.777628 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.776133 2545 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 01 16:28:42.777628 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.776136 2545 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 01 16:28:42.778109 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.776138 2545 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 01 16:28:42.778109 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.776141 2545 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 01 16:28:42.778109 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.776143 2545 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 01 16:28:42.778109 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.776146 2545 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 01 16:28:42.778109 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.776148 2545 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 01 16:28:42.778109 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.776151 2545 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 01 16:28:42.778109 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.776920 2545 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 01 16:28:42.778109 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.776932 2545 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 01 16:28:42.778109 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.776935 2545 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 01 16:28:42.778109 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.776939 2545 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 01 16:28:42.778109 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.776942 2545 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 01 16:28:42.778109 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.776945 2545 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 01 16:28:42.778109 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.776948 2545 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 01 16:28:42.778109 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.776950 2545 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 01 16:28:42.778109 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.776953 2545 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 01 16:28:42.778109 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.776955 2545 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 01 16:28:42.778109 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.776958 2545 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 01 16:28:42.778109 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.776960 2545 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 01 16:28:42.778109 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.776963 2545 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 01 16:28:42.778109 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.776966 2545 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 01 16:28:42.778595 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.776968 2545 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 01 16:28:42.778595 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.776970 2545 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 01 16:28:42.778595 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.776973 2545 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 01 16:28:42.778595 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.776976 2545 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 01 16:28:42.778595 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.776978 2545 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 01 16:28:42.778595 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.776981 2545 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 01 16:28:42.778595 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.776983 2545 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 01 16:28:42.778595 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.776986 2545 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 01 16:28:42.778595 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.776989 2545 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 01 16:28:42.778595 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.776991 2545 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 01 16:28:42.778595 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.776994 2545 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 01 16:28:42.778595 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.776996 2545 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 01 16:28:42.778595 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.777001 2545 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 01 16:28:42.778595 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.777005 2545 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 01 16:28:42.778595 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.777007 2545 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 01 16:28:42.778595 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.777010 2545 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 01 16:28:42.778595 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.777013 2545 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 01 16:28:42.778595 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.777015 2545 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 01 16:28:42.778595 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.777018 2545 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 01 16:28:42.779069 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.777025 2545 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 01 16:28:42.779069 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.777029 2545 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 01 16:28:42.779069 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.777033 2545 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 01 16:28:42.779069 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.777036 2545 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 01 16:28:42.779069 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.777039 2545 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 01 16:28:42.779069 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.777042 2545 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 01 16:28:42.779069 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.777045 2545 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 01 16:28:42.779069 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.777048 2545 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 01 16:28:42.779069 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.777051 2545 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 01 16:28:42.779069 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.777054 2545 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 01 16:28:42.779069 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.777056 2545 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 01 16:28:42.779069 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.777059 2545 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 01 16:28:42.779069 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.777061 2545 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 01 16:28:42.779069 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.777064 2545 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 01 16:28:42.779069 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.777066 2545 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 01 16:28:42.779069 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.777069 2545 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 01 16:28:42.779069 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.777071 2545 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 01 16:28:42.779069 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.777073 2545 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 01 16:28:42.779069 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.777076 2545 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 01 16:28:42.779069 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.777078 2545 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 01 16:28:42.779548 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.777081 2545 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 01 16:28:42.779548 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.777084 2545 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 01 16:28:42.779548 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.777086 2545 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 01 16:28:42.779548 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.777089 2545 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 01 16:28:42.779548 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.777091 2545 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 01 16:28:42.779548 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.777094 2545 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 01 16:28:42.779548 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.777096 2545 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 01 16:28:42.779548 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.777099 2545 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 01 16:28:42.779548 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.777101 2545 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 01 16:28:42.779548 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.777103 2545 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 01 16:28:42.779548 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.777106 2545 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 01 16:28:42.779548 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.777111 2545 feature_gate.go:328] unrecognized feature gate: Example2 Jul 01 16:28:42.779548 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.777113 2545 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 01 16:28:42.779548 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.777116 2545 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 01 16:28:42.779548 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.777119 2545 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 01 16:28:42.779548 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.777122 2545 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 01 16:28:42.779548 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.777124 2545 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 01 16:28:42.779548 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.777127 2545 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 01 16:28:42.779548 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.777129 2545 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 01 16:28:42.779548 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.777132 2545 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 01 16:28:42.780051 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.777134 2545 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 01 16:28:42.780051 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.777137 2545 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 01 16:28:42.780051 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.777139 2545 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 01 16:28:42.780051 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.777141 2545 feature_gate.go:328] unrecognized feature gate: Example Jul 01 16:28:42.780051 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.777145 2545 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 01 16:28:42.780051 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.777147 2545 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 01 16:28:42.780051 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.777150 2545 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 01 16:28:42.780051 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.777152 2545 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 01 16:28:42.780051 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.777155 2545 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 01 16:28:42.780051 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.777158 2545 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 01 16:28:42.780051 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.777160 2545 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 01 16:28:42.780051 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.777163 2545 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 01 16:28:42.780051 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.777166 2545 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 01 16:28:42.780051 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.777169 2545 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 01 16:28:42.780051 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.777171 2545 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 01 16:28:42.780051 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.777176 2545 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 01 16:28:42.780051 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.777180 2545 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 01 16:28:42.780051 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.777183 2545 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 01 16:28:42.780051 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.777185 2545 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 01 16:28:42.780051 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.777189 2545 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 01 16:28:42.780518 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.777192 2545 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 01 16:28:42.780518 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.777194 2545 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 01 16:28:42.780518 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.777197 2545 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 01 16:28:42.780518 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.777201 2545 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 01 16:28:42.780518 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.777203 2545 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 01 16:28:42.780518 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.777206 2545 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 01 16:28:42.780518 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.777581 2545 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 01 16:28:42.780518 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.777586 2545 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 01 16:28:42.780518 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.777589 2545 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 01 16:28:42.780518 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.777592 2545 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 01 16:28:42.780518 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.777595 2545 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 01 16:28:42.780518 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.777598 2545 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 01 16:28:42.780518 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.777601 2545 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 01 16:28:42.780518 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.777604 2545 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 01 16:28:42.780518 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.777607 2545 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 01 16:28:42.780518 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.777609 2545 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 01 16:28:42.780518 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.777612 2545 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 01 16:28:42.780518 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.777614 2545 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 01 16:28:42.780518 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.777617 2545 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 01 16:28:42.780518 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.777620 2545 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 01 16:28:42.780998 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.777623 2545 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 01 16:28:42.780998 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.777625 2545 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 01 16:28:42.780998 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.777628 2545 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 01 16:28:42.780998 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.777630 2545 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 01 16:28:42.780998 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.777635 2545 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 01 16:28:42.780998 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.777638 2545 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 01 16:28:42.780998 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.777641 2545 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 01 16:28:42.780998 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.777644 2545 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 01 16:28:42.780998 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.777647 2545 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 01 16:28:42.780998 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.777650 2545 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 01 16:28:42.780998 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.777652 2545 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 01 16:28:42.780998 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.777655 2545 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 01 16:28:42.780998 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.777657 2545 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 01 16:28:42.780998 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.777660 2545 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 01 16:28:42.780998 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.777662 2545 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 01 16:28:42.780998 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.777665 2545 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 01 16:28:42.780998 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.777669 2545 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 01 16:28:42.780998 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.777673 2545 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 01 16:28:42.780998 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.777676 2545 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 01 16:28:42.781463 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.777679 2545 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 01 16:28:42.781463 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.777681 2545 feature_gate.go:328] unrecognized feature gate: Example2 Jul 01 16:28:42.781463 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.777684 2545 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 01 16:28:42.781463 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.777687 2545 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 01 16:28:42.781463 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.777689 2545 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 01 16:28:42.781463 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.777692 2545 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 01 16:28:42.781463 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.777694 2545 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 01 16:28:42.781463 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.777697 2545 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 01 16:28:42.781463 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.777699 2545 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 01 16:28:42.781463 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.777702 2545 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 01 16:28:42.781463 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.777704 2545 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 01 16:28:42.781463 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.777725 2545 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 01 16:28:42.781463 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.777728 2545 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 01 16:28:42.781463 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.777731 2545 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 01 16:28:42.781463 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.777733 2545 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 01 16:28:42.781463 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.777736 2545 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 01 16:28:42.781463 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.777739 2545 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 01 16:28:42.781463 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.777741 2545 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 01 16:28:42.781463 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.777744 2545 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 01 16:28:42.781463 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.777746 2545 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 01 16:28:42.781938 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.777749 2545 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 01 16:28:42.781938 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.777751 2545 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 01 16:28:42.781938 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.777754 2545 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 01 16:28:42.781938 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.777757 2545 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 01 16:28:42.781938 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.777760 2545 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 01 16:28:42.781938 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.777762 2545 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 01 16:28:42.781938 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.777765 2545 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 01 16:28:42.781938 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.777768 2545 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 01 16:28:42.781938 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.777771 2545 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 01 16:28:42.781938 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.777774 2545 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 01 16:28:42.781938 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.777777 2545 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 01 16:28:42.781938 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.777779 2545 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 01 16:28:42.781938 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.777782 2545 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 01 16:28:42.781938 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.777785 2545 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 01 16:28:42.781938 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.777787 2545 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 01 16:28:42.781938 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.777790 2545 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 01 16:28:42.781938 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.777793 2545 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 01 16:28:42.781938 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.777795 2545 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 01 16:28:42.781938 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.777798 2545 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 01 16:28:42.782378 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.777800 2545 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 01 16:28:42.782378 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.777803 2545 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 01 16:28:42.782378 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.777805 2545 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 01 16:28:42.782378 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.777808 2545 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 01 16:28:42.782378 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.777810 2545 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 01 16:28:42.782378 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.777813 2545 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 01 16:28:42.782378 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.777818 2545 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 01 16:28:42.782378 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.777821 2545 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 01 16:28:42.782378 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.777824 2545 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 01 16:28:42.782378 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.777827 2545 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 01 16:28:42.782378 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.777830 2545 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 01 16:28:42.782378 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.777832 2545 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 01 16:28:42.782378 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.777835 2545 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 01 16:28:42.782378 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.777838 2545 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 01 16:28:42.782378 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.777840 2545 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 01 16:28:42.782378 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.777843 2545 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 01 16:28:42.782378 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.777845 2545 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 01 16:28:42.782378 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.777848 2545 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 01 16:28:42.782378 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.777850 2545 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 01 16:28:42.782901 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.777853 2545 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 01 16:28:42.782901 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.777855 2545 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 01 16:28:42.782901 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.777858 2545 feature_gate.go:328] unrecognized feature gate: Example Jul 01 16:28:42.782901 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.777861 2545 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 01 16:28:42.782901 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.777864 2545 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 01 16:28:42.782901 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.777866 2545 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 01 16:28:42.782901 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.777869 2545 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 01 16:28:42.782901 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.777871 2545 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 01 16:28:42.782901 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:42.777938 2545 flags.go:64] FLAG: --address="0.0.0.0" Jul 01 16:28:42.782901 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:42.777945 2545 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 01 16:28:42.782901 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:42.777951 2545 flags.go:64] FLAG: --anonymous-auth="true" Jul 01 16:28:42.782901 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:42.777956 2545 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 01 16:28:42.782901 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:42.777960 2545 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 01 16:28:42.782901 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:42.777963 2545 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 01 16:28:42.782901 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:42.777967 2545 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 01 16:28:42.782901 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:42.777970 2545 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 01 16:28:42.782901 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:42.777973 2545 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 01 16:28:42.782901 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:42.777977 2545 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 01 16:28:42.782901 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:42.777980 2545 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 01 16:28:42.782901 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:42.777999 2545 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 01 16:28:42.782901 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:42.778003 2545 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 01 16:28:42.782901 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:42.778006 2545 flags.go:64] FLAG: --cgroup-root="" Jul 01 16:28:42.783406 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:42.778009 2545 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 01 16:28:42.783406 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:42.778013 2545 flags.go:64] FLAG: --client-ca-file="" Jul 01 16:28:42.783406 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:42.778016 2545 flags.go:64] FLAG: --cloud-provider="external" Jul 01 16:28:42.783406 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:42.778019 2545 flags.go:64] FLAG: --cluster-dns="[]" Jul 01 16:28:42.783406 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:42.778022 2545 flags.go:64] FLAG: --cluster-domain="" Jul 01 16:28:42.783406 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:42.778025 2545 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 01 16:28:42.783406 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:42.778028 2545 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 01 16:28:42.783406 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:42.778031 2545 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 01 16:28:42.783406 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:42.778035 2545 flags.go:64] FLAG: --container-log-max-files="5" Jul 01 16:28:42.783406 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:42.778038 2545 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 01 16:28:42.783406 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:42.778041 2545 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 01 16:28:42.783406 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:42.778044 2545 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 01 16:28:42.783406 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:42.778047 2545 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 01 16:28:42.783406 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:42.778050 2545 flags.go:64] FLAG: --contention-profiling="false" Jul 01 16:28:42.783406 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:42.778055 2545 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 01 16:28:42.783406 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:42.778057 2545 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 01 16:28:42.783406 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:42.778060 2545 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 01 16:28:42.783406 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:42.778063 2545 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 01 16:28:42.783406 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:42.778068 2545 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 01 16:28:42.783406 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:42.778071 2545 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 01 16:28:42.783406 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:42.778074 2545 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 01 16:28:42.783406 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:42.778076 2545 flags.go:64] FLAG: --enable-load-reader="false" Jul 01 16:28:42.783406 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:42.778080 2545 flags.go:64] FLAG: --enable-server="true" Jul 01 16:28:42.783406 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:42.778083 2545 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 01 16:28:42.783406 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:42.778087 2545 flags.go:64] FLAG: --event-burst="100" Jul 01 16:28:42.784034 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:42.778090 2545 flags.go:64] FLAG: --event-qps="50" Jul 01 16:28:42.784034 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:42.778093 2545 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 01 16:28:42.784034 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:42.778096 2545 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 01 16:28:42.784034 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:42.778098 2545 flags.go:64] FLAG: --eviction-hard="" Jul 01 16:28:42.784034 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:42.778104 2545 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 01 16:28:42.784034 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:42.778107 2545 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 01 16:28:42.784034 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:42.778110 2545 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 01 16:28:42.784034 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:42.778113 2545 flags.go:64] FLAG: --eviction-soft="" Jul 01 16:28:42.784034 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:42.778116 2545 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 01 16:28:42.784034 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:42.778120 2545 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 01 16:28:42.784034 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:42.778123 2545 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 01 16:28:42.784034 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:42.778126 2545 flags.go:64] FLAG: --experimental-mounter-path="" Jul 01 16:28:42.784034 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:42.778128 2545 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 01 16:28:42.784034 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:42.778131 2545 flags.go:64] FLAG: --fail-swap-on="true" Jul 01 16:28:42.784034 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:42.778134 2545 flags.go:64] FLAG: --feature-gates="" Jul 01 16:28:42.784034 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:42.778138 2545 flags.go:64] FLAG: --file-check-frequency="20s" Jul 01 16:28:42.784034 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:42.778141 2545 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 01 16:28:42.784034 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:42.778144 2545 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 01 16:28:42.784034 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:42.778147 2545 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 01 16:28:42.784034 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:42.778150 2545 flags.go:64] FLAG: --healthz-port="10248" Jul 01 16:28:42.784034 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:42.778153 2545 flags.go:64] FLAG: --help="false" Jul 01 16:28:42.784034 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:42.778156 2545 flags.go:64] FLAG: --hostname-override="ip-10-0-130-227.ec2.internal" Jul 01 16:28:42.784034 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:42.778159 2545 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 01 16:28:42.784034 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:42.778162 2545 flags.go:64] FLAG: --http-check-frequency="20s" Jul 01 16:28:42.784605 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:42.778165 2545 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 01 16:28:42.784605 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:42.778168 2545 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 01 16:28:42.784605 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:42.778172 2545 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 01 16:28:42.784605 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:42.778175 2545 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 01 16:28:42.784605 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:42.778177 2545 flags.go:64] FLAG: --image-service-endpoint="" Jul 01 16:28:42.784605 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:42.778180 2545 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 01 16:28:42.784605 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:42.778183 2545 flags.go:64] FLAG: --kube-api-burst="100" Jul 01 16:28:42.784605 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:42.778186 2545 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 01 16:28:42.784605 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:42.778189 2545 flags.go:64] FLAG: --kube-api-qps="50" Jul 01 16:28:42.784605 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:42.778191 2545 flags.go:64] FLAG: --kube-reserved="" Jul 01 16:28:42.784605 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:42.778194 2545 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 01 16:28:42.784605 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:42.778197 2545 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 01 16:28:42.784605 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:42.778201 2545 flags.go:64] FLAG: --kubelet-cgroups="" Jul 01 16:28:42.784605 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:42.778204 2545 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 01 16:28:42.784605 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:42.778207 2545 flags.go:64] FLAG: --lock-file="" Jul 01 16:28:42.784605 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:42.778209 2545 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 01 16:28:42.784605 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:42.778212 2545 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 01 16:28:42.784605 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:42.778215 2545 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 01 16:28:42.784605 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:42.778220 2545 flags.go:64] FLAG: --log-json-split-stream="false" Jul 01 16:28:42.784605 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:42.778223 2545 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 01 16:28:42.784605 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:42.778226 2545 flags.go:64] FLAG: --log-text-split-stream="false" Jul 01 16:28:42.784605 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:42.778229 2545 flags.go:64] FLAG: --logging-format="text" Jul 01 16:28:42.784605 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:42.778231 2545 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 01 16:28:42.785154 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:42.778235 2545 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 01 16:28:42.785154 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:42.778237 2545 flags.go:64] FLAG: --manifest-url="" Jul 01 16:28:42.785154 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:42.778242 2545 flags.go:64] FLAG: --manifest-url-header="" Jul 01 16:28:42.785154 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:42.778246 2545 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 01 16:28:42.785154 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:42.778249 2545 flags.go:64] FLAG: --max-open-files="1000000" Jul 01 16:28:42.785154 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:42.778253 2545 flags.go:64] FLAG: --max-pods="110" Jul 01 16:28:42.785154 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:42.778256 2545 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 01 16:28:42.785154 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:42.778258 2545 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 01 16:28:42.785154 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:42.778261 2545 flags.go:64] FLAG: --memory-manager-policy="None" Jul 01 16:28:42.785154 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:42.778264 2545 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 01 16:28:42.785154 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:42.778267 2545 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 01 16:28:42.785154 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:42.778270 2545 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 01 16:28:42.785154 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:42.778273 2545 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 01 16:28:42.785154 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:42.778280 2545 flags.go:64] FLAG: --node-status-max-images="50" Jul 01 16:28:42.785154 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:42.778283 2545 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 01 16:28:42.785154 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:42.778286 2545 flags.go:64] FLAG: --oom-score-adj="-999" Jul 01 16:28:42.785154 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:42.778289 2545 flags.go:64] FLAG: --pod-cidr="" Jul 01 16:28:42.785154 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:42.778292 2545 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8a2632cd31d78133be7b5fb6999469b2acf9b98d077c7f6ae5d0d94083d76b04" Jul 01 16:28:42.785154 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:42.778296 2545 flags.go:64] FLAG: --pod-manifest-path="" Jul 01 16:28:42.785154 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:42.778299 2545 flags.go:64] FLAG: --pod-max-pids="-1" Jul 01 16:28:42.785154 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:42.778302 2545 flags.go:64] FLAG: --pods-per-core="0" Jul 01 16:28:42.785154 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:42.778307 2545 flags.go:64] FLAG: --port="10250" Jul 01 16:28:42.785154 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:42.778310 2545 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 01 16:28:42.785154 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:42.778313 2545 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-05561a69babf06c8e" Jul 01 16:28:42.785750 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:42.778316 2545 flags.go:64] FLAG: --qos-reserved="" Jul 01 16:28:42.785750 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:42.778319 2545 flags.go:64] FLAG: --read-only-port="10255" Jul 01 16:28:42.785750 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:42.778321 2545 flags.go:64] FLAG: --register-node="true" Jul 01 16:28:42.785750 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:42.778324 2545 flags.go:64] FLAG: --register-with-taints="" Jul 01 16:28:42.785750 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:42.778328 2545 flags.go:64] FLAG: --registry-burst="10" Jul 01 16:28:42.785750 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:42.778331 2545 flags.go:64] FLAG: --registry-qps="5" Jul 01 16:28:42.785750 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:42.778333 2545 flags.go:64] FLAG: --reserved-cpus="" Jul 01 16:28:42.785750 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:42.778336 2545 flags.go:64] FLAG: --reserved-memory="" Jul 01 16:28:42.785750 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:42.778340 2545 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 01 16:28:42.785750 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:42.778343 2545 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 01 16:28:42.785750 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:42.778346 2545 flags.go:64] FLAG: --rotate-certificates="false" Jul 01 16:28:42.785750 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:42.778350 2545 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 01 16:28:42.785750 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:42.778353 2545 flags.go:64] FLAG: --runonce="false" Jul 01 16:28:42.785750 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:42.778356 2545 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 01 16:28:42.785750 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:42.778359 2545 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 01 16:28:42.785750 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:42.778362 2545 flags.go:64] FLAG: --seccomp-default="false" Jul 01 16:28:42.785750 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:42.778364 2545 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 01 16:28:42.785750 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:42.778367 2545 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 01 16:28:42.785750 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:42.778370 2545 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 01 16:28:42.785750 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:42.778373 2545 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 01 16:28:42.785750 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:42.778376 2545 flags.go:64] FLAG: --storage-driver-password="root" Jul 01 16:28:42.785750 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:42.778379 2545 flags.go:64] FLAG: --storage-driver-secure="false" Jul 01 16:28:42.785750 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:42.778382 2545 flags.go:64] FLAG: --storage-driver-table="stats" Jul 01 16:28:42.785750 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:42.778385 2545 flags.go:64] FLAG: --storage-driver-user="root" Jul 01 16:28:42.785750 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:42.778388 2545 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 01 16:28:42.785750 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:42.778390 2545 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 01 16:28:42.786350 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:42.778393 2545 flags.go:64] FLAG: --system-cgroups="" Jul 01 16:28:42.786350 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:42.778396 2545 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 01 16:28:42.786350 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:42.778401 2545 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 01 16:28:42.786350 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:42.778405 2545 flags.go:64] FLAG: --tls-cert-file="" Jul 01 16:28:42.786350 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:42.778407 2545 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 01 16:28:42.786350 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:42.778411 2545 flags.go:64] FLAG: --tls-min-version="" Jul 01 16:28:42.786350 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:42.778413 2545 flags.go:64] FLAG: --tls-private-key-file="" Jul 01 16:28:42.786350 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:42.778416 2545 flags.go:64] FLAG: --topology-manager-policy="none" Jul 01 16:28:42.786350 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:42.778419 2545 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 01 16:28:42.786350 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:42.778422 2545 flags.go:64] FLAG: --topology-manager-scope="container" Jul 01 16:28:42.786350 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:42.778424 2545 flags.go:64] FLAG: --v="2" Jul 01 16:28:42.786350 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:42.778428 2545 flags.go:64] FLAG: --version="false" Jul 01 16:28:42.786350 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:42.778432 2545 flags.go:64] FLAG: --vmodule="" Jul 01 16:28:42.786350 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:42.778435 2545 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 01 16:28:42.786350 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:42.778438 2545 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 01 16:28:42.786350 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.778565 2545 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 01 16:28:42.786350 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.778569 2545 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 01 16:28:42.786350 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.778574 2545 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 01 16:28:42.786350 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.778577 2545 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 01 16:28:42.786350 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.778580 2545 feature_gate.go:328] unrecognized feature gate: Example2 Jul 01 16:28:42.786350 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.778583 2545 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 01 16:28:42.786350 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.778585 2545 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 01 16:28:42.786350 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.778588 2545 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 01 16:28:42.786350 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.778592 2545 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 01 16:28:42.787000 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.778595 2545 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 01 16:28:42.787000 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.778597 2545 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 01 16:28:42.787000 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.778600 2545 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 01 16:28:42.787000 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.778603 2545 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 01 16:28:42.787000 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.778608 2545 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 01 16:28:42.787000 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.778611 2545 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 01 16:28:42.787000 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.778614 2545 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 01 16:28:42.787000 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.778616 2545 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 01 16:28:42.787000 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.778619 2545 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 01 16:28:42.787000 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.778622 2545 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 01 16:28:42.787000 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.778624 2545 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 01 16:28:42.787000 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.778628 2545 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 01 16:28:42.787000 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.778630 2545 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 01 16:28:42.787000 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.778633 2545 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 01 16:28:42.787000 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.778635 2545 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 01 16:28:42.787000 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.778638 2545 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 01 16:28:42.787000 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.778640 2545 feature_gate.go:328] unrecognized feature gate: Example Jul 01 16:28:42.787000 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.778643 2545 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 01 16:28:42.787000 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.778645 2545 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 01 16:28:42.787479 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.778648 2545 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 01 16:28:42.787479 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.778651 2545 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 01 16:28:42.787479 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.778655 2545 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 01 16:28:42.787479 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.778659 2545 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 01 16:28:42.787479 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.778662 2545 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 01 16:28:42.787479 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.778665 2545 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 01 16:28:42.787479 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.778668 2545 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 01 16:28:42.787479 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.778671 2545 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 01 16:28:42.787479 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.778674 2545 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 01 16:28:42.787479 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.778676 2545 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 01 16:28:42.787479 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.778679 2545 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 01 16:28:42.787479 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.778681 2545 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 01 16:28:42.787479 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.778684 2545 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 01 16:28:42.787479 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.778687 2545 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 01 16:28:42.787479 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.778689 2545 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 01 16:28:42.787479 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.778692 2545 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 01 16:28:42.787479 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.778694 2545 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 01 16:28:42.787479 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.778697 2545 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 01 16:28:42.787479 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.778705 2545 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 01 16:28:42.787937 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.778725 2545 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 01 16:28:42.787937 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.778729 2545 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 01 16:28:42.787937 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.778732 2545 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 01 16:28:42.787937 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.778735 2545 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 01 16:28:42.787937 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.778737 2545 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 01 16:28:42.787937 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.778741 2545 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 01 16:28:42.787937 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.778744 2545 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 01 16:28:42.787937 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.778746 2545 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 01 16:28:42.787937 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.778749 2545 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 01 16:28:42.787937 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.778752 2545 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 01 16:28:42.787937 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.778755 2545 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 01 16:28:42.787937 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.778757 2545 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 01 16:28:42.787937 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.778760 2545 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 01 16:28:42.787937 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.778762 2545 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 01 16:28:42.787937 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.778765 2545 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 01 16:28:42.787937 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.778767 2545 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 01 16:28:42.787937 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.778770 2545 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 01 16:28:42.787937 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.778772 2545 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 01 16:28:42.787937 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.778775 2545 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 01 16:28:42.788394 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.778778 2545 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 01 16:28:42.788394 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.778780 2545 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 01 16:28:42.788394 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.778783 2545 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 01 16:28:42.788394 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.778785 2545 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 01 16:28:42.788394 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.778788 2545 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 01 16:28:42.788394 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.778790 2545 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 01 16:28:42.788394 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.778793 2545 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 01 16:28:42.788394 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.778796 2545 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 01 16:28:42.788394 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.778798 2545 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 01 16:28:42.788394 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.778801 2545 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 01 16:28:42.788394 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.778803 2545 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 01 16:28:42.788394 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.778808 2545 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 01 16:28:42.788394 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.778811 2545 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 01 16:28:42.788394 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.778823 2545 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 01 16:28:42.788394 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.778826 2545 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 01 16:28:42.788394 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.778829 2545 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 01 16:28:42.788394 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.778832 2545 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 01 16:28:42.788394 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.778834 2545 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 01 16:28:42.788394 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.778838 2545 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 01 16:28:42.789012 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.778841 2545 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 01 16:28:42.789012 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.778843 2545 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 01 16:28:42.789012 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.778846 2545 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 01 16:28:42.789012 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.778848 2545 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 01 16:28:42.789012 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.778851 2545 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 01 16:28:42.789012 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.778853 2545 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 01 16:28:42.789012 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.778856 2545 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 01 16:28:42.789012 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.778858 2545 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 01 16:28:42.789012 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.778861 2545 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 01 16:28:42.789012 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.778863 2545 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 01 16:28:42.789012 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.778866 2545 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 01 16:28:42.789012 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.778869 2545 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 01 16:28:42.789012 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.778871 2545 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 01 16:28:42.789012 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.778873 2545 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 01 16:28:42.789012 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:42.779531 2545 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 01 16:28:42.789595 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:42.786870 2545 server.go:529] "Kubelet version" kubeletVersion="v1.34.8" Jul 01 16:28:42.789595 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:42.786889 2545 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 01 16:28:42.789595 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:42.787662 2545 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 01 16:28:42.789595 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:42.787669 2545 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 01 16:28:42.789595 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.787761 2545 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 01 16:28:42.789595 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.787767 2545 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 01 16:28:42.789595 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.787771 2545 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 01 16:28:42.789595 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.787774 2545 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 01 16:28:42.789595 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.787776 2545 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 01 16:28:42.789595 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.787779 2545 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 01 16:28:42.789595 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.787782 2545 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 01 16:28:42.789595 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.787784 2545 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 01 16:28:42.789595 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.787787 2545 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 01 16:28:42.789595 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.787790 2545 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 01 16:28:42.789595 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.787793 2545 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 01 16:28:42.789595 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.787796 2545 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 01 16:28:42.789595 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.787798 2545 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 01 16:28:42.789595 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.787801 2545 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 01 16:28:42.789595 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.787803 2545 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 01 16:28:42.790087 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.787806 2545 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 01 16:28:42.790087 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.787808 2545 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 01 16:28:42.790087 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.787811 2545 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 01 16:28:42.790087 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.787814 2545 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 01 16:28:42.790087 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.787816 2545 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 01 16:28:42.790087 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.787818 2545 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 01 16:28:42.790087 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.787821 2545 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 01 16:28:42.790087 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.787823 2545 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 01 16:28:42.790087 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.787826 2545 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 01 16:28:42.790087 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.787829 2545 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 01 16:28:42.790087 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.787832 2545 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 01 16:28:42.790087 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.787835 2545 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 01 16:28:42.790087 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.787837 2545 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 01 16:28:42.790087 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.787840 2545 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 01 16:28:42.790087 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.787849 2545 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 01 16:28:42.790087 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.787852 2545 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 01 16:28:42.790087 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.787855 2545 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 01 16:28:42.790087 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.787857 2545 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 01 16:28:42.790087 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.787860 2545 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 01 16:28:42.790087 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.787863 2545 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 01 16:28:42.790575 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.787865 2545 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 01 16:28:42.790575 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.787868 2545 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 01 16:28:42.790575 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.787870 2545 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 01 16:28:42.790575 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.787873 2545 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 01 16:28:42.790575 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.787876 2545 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 01 16:28:42.790575 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.787878 2545 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 01 16:28:42.790575 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.787882 2545 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 01 16:28:42.790575 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.787885 2545 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 01 16:28:42.790575 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.787889 2545 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 01 16:28:42.790575 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.787892 2545 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 01 16:28:42.790575 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.787894 2545 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 01 16:28:42.790575 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.787897 2545 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 01 16:28:42.790575 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.787899 2545 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 01 16:28:42.790575 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.787902 2545 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 01 16:28:42.790575 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.787904 2545 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 01 16:28:42.790575 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.787907 2545 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 01 16:28:42.790575 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.787909 2545 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 01 16:28:42.790575 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.787912 2545 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 01 16:28:42.790575 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.787914 2545 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 01 16:28:42.790575 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.787917 2545 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 01 16:28:42.791113 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.787919 2545 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 01 16:28:42.791113 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.787921 2545 feature_gate.go:328] unrecognized feature gate: Example2 Jul 01 16:28:42.791113 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.787924 2545 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 01 16:28:42.791113 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.787928 2545 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 01 16:28:42.791113 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.787931 2545 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 01 16:28:42.791113 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.787934 2545 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 01 16:28:42.791113 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.787938 2545 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 01 16:28:42.791113 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.787947 2545 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 01 16:28:42.791113 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.787951 2545 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 01 16:28:42.791113 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.787954 2545 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 01 16:28:42.791113 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.787957 2545 feature_gate.go:328] unrecognized feature gate: Example Jul 01 16:28:42.791113 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.787960 2545 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 01 16:28:42.791113 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.787962 2545 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 01 16:28:42.791113 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.787965 2545 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 01 16:28:42.791113 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.787967 2545 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 01 16:28:42.791113 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.787970 2545 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 01 16:28:42.791113 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.787972 2545 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 01 16:28:42.791113 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.787975 2545 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 01 16:28:42.791113 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.787977 2545 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 01 16:28:42.791611 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.787980 2545 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 01 16:28:42.791611 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.787982 2545 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 01 16:28:42.791611 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.787985 2545 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 01 16:28:42.791611 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.787987 2545 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 01 16:28:42.791611 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.787990 2545 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 01 16:28:42.791611 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.787992 2545 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 01 16:28:42.791611 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.787995 2545 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 01 16:28:42.791611 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.787997 2545 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 01 16:28:42.791611 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.788000 2545 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 01 16:28:42.791611 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.788002 2545 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 01 16:28:42.791611 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.788005 2545 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 01 16:28:42.791611 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.788007 2545 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 01 16:28:42.791611 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.788010 2545 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 01 16:28:42.791611 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.788012 2545 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 01 16:28:42.791611 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.788014 2545 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 01 16:28:42.791611 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.788017 2545 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 01 16:28:42.791611 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.788019 2545 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 01 16:28:42.791611 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.788023 2545 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 01 16:28:42.791611 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.788027 2545 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 01 16:28:42.792137 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.788030 2545 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 01 16:28:42.792137 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.788032 2545 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 01 16:28:42.792137 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.788042 2545 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 01 16:28:42.792137 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.788044 2545 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 01 16:28:42.792137 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.788047 2545 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 01 16:28:42.792137 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.788050 2545 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 01 16:28:42.792137 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:42.788054 2545 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 01 16:28:42.792137 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.788215 2545 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 01 16:28:42.792137 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.788219 2545 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 01 16:28:42.792137 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.788222 2545 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 01 16:28:42.792137 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.788225 2545 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 01 16:28:42.792137 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.788229 2545 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 01 16:28:42.792137 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.788233 2545 feature_gate.go:328] unrecognized feature gate: Example Jul 01 16:28:42.792137 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.788235 2545 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 01 16:28:42.792137 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.788238 2545 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 01 16:28:42.792501 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.788241 2545 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 01 16:28:42.792501 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.788244 2545 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 01 16:28:42.792501 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.788246 2545 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 01 16:28:42.792501 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.788249 2545 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 01 16:28:42.792501 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.788251 2545 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 01 16:28:42.792501 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.788254 2545 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 01 16:28:42.792501 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.788256 2545 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 01 16:28:42.792501 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.788259 2545 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 01 16:28:42.792501 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.788262 2545 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 01 16:28:42.792501 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.788264 2545 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 01 16:28:42.792501 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.788266 2545 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 01 16:28:42.792501 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.788269 2545 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 01 16:28:42.792501 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.788271 2545 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 01 16:28:42.792501 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.788274 2545 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 01 16:28:42.792501 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.788276 2545 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 01 16:28:42.792501 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.788279 2545 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 01 16:28:42.792501 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.788282 2545 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 01 16:28:42.792501 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.788284 2545 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 01 16:28:42.792501 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.788287 2545 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 01 16:28:42.792501 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.788289 2545 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 01 16:28:42.792983 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.788297 2545 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 01 16:28:42.792983 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.788300 2545 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 01 16:28:42.792983 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.788302 2545 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 01 16:28:42.792983 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.788305 2545 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 01 16:28:42.792983 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.788307 2545 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 01 16:28:42.792983 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.788310 2545 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 01 16:28:42.792983 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.788312 2545 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 01 16:28:42.792983 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.788315 2545 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 01 16:28:42.792983 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.788317 2545 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 01 16:28:42.792983 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.788320 2545 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 01 16:28:42.792983 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.788322 2545 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 01 16:28:42.792983 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.788325 2545 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 01 16:28:42.792983 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.788327 2545 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 01 16:28:42.792983 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.788329 2545 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 01 16:28:42.792983 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.788332 2545 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 01 16:28:42.792983 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.788334 2545 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 01 16:28:42.792983 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.788337 2545 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 01 16:28:42.792983 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.788339 2545 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 01 16:28:42.792983 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.788341 2545 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 01 16:28:42.792983 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.788344 2545 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 01 16:28:42.793446 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.788346 2545 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 01 16:28:42.793446 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.788349 2545 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 01 16:28:42.793446 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.788351 2545 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 01 16:28:42.793446 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.788354 2545 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 01 16:28:42.793446 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.788356 2545 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 01 16:28:42.793446 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.788359 2545 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 01 16:28:42.793446 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.788361 2545 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 01 16:28:42.793446 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.788363 2545 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 01 16:28:42.793446 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.788366 2545 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 01 16:28:42.793446 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.788369 2545 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 01 16:28:42.793446 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.788371 2545 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 01 16:28:42.793446 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.788374 2545 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 01 16:28:42.793446 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.788377 2545 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 01 16:28:42.793446 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.788384 2545 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 01 16:28:42.793446 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.788387 2545 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 01 16:28:42.793446 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.788389 2545 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 01 16:28:42.793446 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.788392 2545 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 01 16:28:42.793446 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.788394 2545 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 01 16:28:42.793446 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.788397 2545 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 01 16:28:42.793446 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.788401 2545 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 01 16:28:42.793921 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.788404 2545 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 01 16:28:42.793921 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.788407 2545 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 01 16:28:42.793921 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.788410 2545 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 01 16:28:42.793921 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.788412 2545 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 01 16:28:42.793921 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.788414 2545 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 01 16:28:42.793921 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.788417 2545 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 01 16:28:42.793921 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.788419 2545 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 01 16:28:42.793921 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.788422 2545 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 01 16:28:42.793921 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.788424 2545 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 01 16:28:42.793921 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.788432 2545 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 01 16:28:42.793921 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.788435 2545 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 01 16:28:42.793921 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.788437 2545 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 01 16:28:42.793921 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.788440 2545 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 01 16:28:42.793921 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.788442 2545 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 01 16:28:42.793921 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.788445 2545 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 01 16:28:42.793921 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.788447 2545 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 01 16:28:42.793921 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.788450 2545 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 01 16:28:42.793921 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.788452 2545 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 01 16:28:42.793921 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.788454 2545 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 01 16:28:42.793921 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.788457 2545 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 01 16:28:42.794390 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.788460 2545 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 01 16:28:42.794390 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.788464 2545 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 01 16:28:42.794390 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.788467 2545 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 01 16:28:42.794390 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.788470 2545 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 01 16:28:42.794390 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.788473 2545 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 01 16:28:42.794390 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.788475 2545 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 01 16:28:42.794390 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.788484 2545 feature_gate.go:328] unrecognized feature gate: Example2 Jul 01 16:28:42.794390 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.788487 2545 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 01 16:28:42.794390 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.788489 2545 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 01 16:28:42.794390 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.788492 2545 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 01 16:28:42.794390 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:42.788494 2545 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 01 16:28:42.794390 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:42.788499 2545 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 01 16:28:42.794390 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:42.789977 2545 server.go:962] "Client rotation is on, will bootstrap in background" Jul 01 16:28:42.794390 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:42.792276 2545 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 01 16:28:42.794758 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:42.793738 2545 server.go:1019] "Starting client certificate rotation" Jul 01 16:28:42.794758 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:42.793837 2545 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 01 16:28:42.794758 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:42.794561 2545 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 01 16:28:42.796149 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:42.796136 2545 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 01 16:28:42.797337 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:42.797302 2545 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 01 16:28:42.809019 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:42.809000 2545 log.go:25] "Validated CRI v1 runtime API" Jul 01 16:28:42.811125 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:42.811108 2545 log.go:25] "Validated CRI v1 image API" Jul 01 16:28:42.811928 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:42.811912 2545 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 01 16:28:42.814157 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:42.814139 2545 fs.go:135] Filesystem UUIDs: map[310debcf-3a9a-4849-9e3e-9f5eb23c38d6:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 bfc18640-5173-4800-88eb-e995fd80e843:/dev/nvme0n1p3] Jul 01 16:28:42.814221 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:42.814156 2545 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 01 16:28:42.820041 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:42.820025 2545 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 01 16:28:42.820665 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:42.820554 2545 manager.go:216] Machine: {Timestamp:2026-07-01 16:28:42.818564159 +0000 UTC m=+1.399181653 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3099081 MemoryCapacity:33164496896 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec225a84ea6c64ff380955d838b0e06b SystemUUID:ec225a84-ea6c-64ff-3809-55d838b0e06b BootID:f241e63d-b9d8-4736-a911-dafee810b9be Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:c2:67:dc:7e:f9 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:c2:67:dc:7e:f9 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:ba:61:37:60:05:16 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164496896 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 01 16:28:42.820665 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:42.820663 2545 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 01 16:28:42.820783 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:42.820767 2545 manager.go:232] Version: {KernelVersion:5.14.0-570.124.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260623-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 01 16:28:42.825005 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:42.824830 2545 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 01 16:28:42.825147 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:42.825011 2545 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-130-227.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 01 16:28:42.825764 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:42.825753 2545 topology_manager.go:138] "Creating topology manager with none policy" Jul 01 16:28:42.825796 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:42.825767 2545 container_manager_linux.go:309] "Creating device plugin manager" Jul 01 16:28:42.825796 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:42.825773 2545 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 01 16:28:42.826549 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:42.826538 2545 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 01 16:28:42.827677 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:42.827667 2545 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 01 16:28:42.830898 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:42.830887 2545 state_mem.go:36] "Initialized new in-memory state store" Jul 01 16:28:42.831012 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:42.831003 2545 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 01 16:28:42.831045 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:42.831033 2545 kubelet.go:486] "Attempting to sync node with API server" Jul 01 16:28:42.831077 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:42.831059 2545 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 01 16:28:42.831077 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:42.831069 2545 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 01 16:28:42.831077 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:42.831078 2545 kubelet.go:398] "Adding apiserver pod source" Jul 01 16:28:42.831171 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:42.831086 2545 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 01 16:28:42.831478 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:42.831467 2545 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 01 16:28:42.831528 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:42.831488 2545 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 01 16:28:42.833356 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:42.833340 2545 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.8-3.rhaos4.21.git91b4b0b.el9" apiVersion="v1" Jul 01 16:28:42.835141 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:42.835128 2545 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 01 16:28:42.835230 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:42.835158 2545 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 01 16:28:42.835907 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:42.835896 2545 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 01 16:28:42.835948 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:42.835912 2545 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 01 16:28:42.835948 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:42.835919 2545 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 01 16:28:42.835948 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:42.835925 2545 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 01 16:28:42.835948 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:42.835935 2545 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 01 16:28:42.835948 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:42.835940 2545 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 01 16:28:42.835948 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:42.835948 2545 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 01 16:28:42.836100 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:42.835954 2545 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 01 16:28:42.836100 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:42.835960 2545 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 01 16:28:42.836100 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:42.835966 2545 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 01 16:28:42.836100 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:42.835974 2545 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 01 16:28:42.836100 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:42.835979 2545 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 01 16:28:42.837498 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:42.837485 2545 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 01 16:28:42.837498 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:42.837495 2545 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 01 16:28:42.838950 ip-10-0-130-227 kubenswrapper[2545]: E0701 16:28:42.838926 2545 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 01 16:28:42.839087 ip-10-0-130-227 kubenswrapper[2545]: E0701 16:28:42.839062 2545 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-130-227.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 01 16:28:42.839207 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:42.839192 2545 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-130-227.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 01 16:28:42.840516 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:42.840504 2545 server.go:1268] "Started kubelet" Jul 01 16:28:42.840598 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:42.840576 2545 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 01 16:28:42.840650 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:42.840584 2545 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 01 16:28:42.840650 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:42.840645 2545 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 01 16:28:42.841279 ip-10-0-130-227 systemd[1]: Started Kubernetes Kubelet. Jul 01 16:28:42.842069 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:42.842014 2545 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 01 16:28:42.842444 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:42.842411 2545 server.go:310] "Adding debug handlers to kubelet server" Jul 01 16:28:42.842698 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:42.842677 2545 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-r554m" Jul 01 16:28:42.845445 ip-10-0-130-227 kubenswrapper[2545]: E0701 16:28:42.844408 2545 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-130-227.ec2.internal.18be3750d7aa73fe default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-130-227.ec2.internal,UID:ip-10-0-130-227.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-130-227.ec2.internal,},FirstTimestamp:2026-07-01 16:28:42.840478718 +0000 UTC m=+1.421096212,LastTimestamp:2026-07-01 16:28:42.840478718 +0000 UTC m=+1.421096212,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-130-227.ec2.internal,}" Jul 01 16:28:42.846006 ip-10-0-130-227 kubenswrapper[2545]: E0701 16:28:42.845989 2545 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 01 16:28:42.849339 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:42.849321 2545 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 01 16:28:42.849458 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:42.849323 2545 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 01 16:28:42.849513 ip-10-0-130-227 kubenswrapper[2545]: E0701 16:28:42.849474 2545 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-227.ec2.internal\" not found" Jul 01 16:28:42.849875 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:42.849855 2545 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-r554m" Jul 01 16:28:42.850128 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:42.850112 2545 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 01 16:28:42.850202 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:42.850113 2545 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 01 16:28:42.850202 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:42.850151 2545 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 01 16:28:42.850202 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:42.850189 2545 factory.go:55] Registering systemd factory Jul 01 16:28:42.850202 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:42.850203 2545 factory.go:223] Registration of the systemd container factory successfully Jul 01 16:28:42.850388 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:42.850250 2545 reconstruct.go:97] "Volume reconstruction finished" Jul 01 16:28:42.850388 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:42.850273 2545 reconciler.go:29] "Reconciler: start to sync state" Jul 01 16:28:42.850491 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:42.850377 2545 factory.go:153] Registering CRI-O factory Jul 01 16:28:42.850524 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:42.850492 2545 factory.go:223] Registration of the crio container factory successfully Jul 01 16:28:42.850562 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:42.850551 2545 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 01 16:28:42.850617 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:42.850576 2545 factory.go:103] Registering Raw factory Jul 01 16:28:42.850617 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:42.850590 2545 manager.go:1196] Started watching for new ooms in manager Jul 01 16:28:42.851043 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:42.851022 2545 manager.go:349] Starting recovery of all containers Jul 01 16:28:42.860697 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:42.860676 2545 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 01 16:28:42.863139 ip-10-0-130-227 kubenswrapper[2545]: E0701 16:28:42.863114 2545 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-130-227.ec2.internal\" not found" node="ip-10-0-130-227.ec2.internal" Jul 01 16:28:42.864449 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:42.864432 2545 manager.go:354] Recovery completed Jul 01 16:28:42.868940 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:42.868928 2545 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 01 16:28:42.870765 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:42.870752 2545 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-227.ec2.internal" event="NodeHasSufficientMemory" Jul 01 16:28:42.870810 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:42.870776 2545 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-227.ec2.internal" event="NodeHasNoDiskPressure" Jul 01 16:28:42.870810 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:42.870786 2545 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-227.ec2.internal" event="NodeHasSufficientPID" Jul 01 16:28:42.871301 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:42.871285 2545 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 01 16:28:42.871301 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:42.871300 2545 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 01 16:28:42.871405 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:42.871317 2545 state_mem.go:36] "Initialized new in-memory state store" Jul 01 16:28:42.871405 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:42.871300 2545 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 01 16:28:42.872387 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:42.872375 2545 policy_none.go:49] "None policy: Start" Jul 01 16:28:42.872440 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:42.872390 2545 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 01 16:28:42.872440 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:42.872400 2545 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 01 16:28:42.872670 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:42.872656 2545 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-227.ec2.internal" event="NodeHasSufficientMemory" Jul 01 16:28:42.872737 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:42.872683 2545 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-227.ec2.internal" event="NodeHasNoDiskPressure" Jul 01 16:28:42.872737 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:42.872696 2545 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-227.ec2.internal" event="NodeHasSufficientPID" Jul 01 16:28:42.873330 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:42.873320 2545 policy_none.go:47] "Start" Jul 01 16:28:42.913365 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:42.913351 2545 manager.go:337] "Starting Device Plugin manager" Jul 01 16:28:42.913458 ip-10-0-130-227 kubenswrapper[2545]: E0701 16:28:42.913381 2545 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 01 16:28:42.913458 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:42.913390 2545 server.go:87] "Starting device plugin registration server" Jul 01 16:28:42.913638 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:42.913620 2545 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 01 16:28:42.913731 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:42.913639 2545 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 01 16:28:42.914075 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:42.913838 2545 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 01 16:28:42.914075 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:42.913922 2545 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 01 16:28:42.914075 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:42.913931 2545 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 01 16:28:42.914549 ip-10-0-130-227 kubenswrapper[2545]: E0701 16:28:42.914529 2545 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 01 16:28:42.914627 ip-10-0-130-227 kubenswrapper[2545]: E0701 16:28:42.914572 2545 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-130-227.ec2.internal\" not found" Jul 01 16:28:42.927999 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:42.927973 2545 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 01 16:28:42.932647 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:42.929158 2545 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 01 16:28:42.932647 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:42.929190 2545 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 01 16:28:42.932647 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:42.929208 2545 kubelet.go:2448] "Starting kubelet main sync loop" Jul 01 16:28:42.932647 ip-10-0-130-227 kubenswrapper[2545]: E0701 16:28:42.929238 2545 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 01 16:28:42.932647 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:42.930962 2545 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 01 16:28:43.014425 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:43.014398 2545 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 01 16:28:43.015303 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:43.015286 2545 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-227.ec2.internal" event="NodeHasSufficientMemory" Jul 01 16:28:43.015409 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:43.015320 2545 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-227.ec2.internal" event="NodeHasNoDiskPressure" Jul 01 16:28:43.015409 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:43.015335 2545 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-227.ec2.internal" event="NodeHasSufficientPID" Jul 01 16:28:43.015409 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:43.015362 2545 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-130-227.ec2.internal" Jul 01 16:28:43.020785 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:43.020769 2545 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-130-227.ec2.internal" Jul 01 16:28:43.020859 ip-10-0-130-227 kubenswrapper[2545]: E0701 16:28:43.020791 2545 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-130-227.ec2.internal\": node \"ip-10-0-130-227.ec2.internal\" not found" Jul 01 16:28:43.030335 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:43.030305 2545 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-227.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-130-227.ec2.internal"] Jul 01 16:28:43.030438 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:43.030375 2545 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 01 16:28:43.031040 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:43.031024 2545 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-227.ec2.internal" event="NodeHasSufficientMemory" Jul 01 16:28:43.031120 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:43.031054 2545 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-227.ec2.internal" event="NodeHasNoDiskPressure" Jul 01 16:28:43.031120 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:43.031068 2545 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-227.ec2.internal" event="NodeHasSufficientPID" Jul 01 16:28:43.032213 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:43.032198 2545 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 01 16:28:43.032316 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:43.032301 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-227.ec2.internal" Jul 01 16:28:43.032360 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:43.032337 2545 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 01 16:28:43.032848 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:43.032835 2545 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-227.ec2.internal" event="NodeHasSufficientMemory" Jul 01 16:28:43.032922 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:43.032856 2545 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-227.ec2.internal" event="NodeHasNoDiskPressure" Jul 01 16:28:43.032922 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:43.032865 2545 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-227.ec2.internal" event="NodeHasSufficientPID" Jul 01 16:28:43.032922 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:43.032895 2545 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-227.ec2.internal" event="NodeHasSufficientMemory" Jul 01 16:28:43.032922 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:43.032918 2545 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-227.ec2.internal" event="NodeHasNoDiskPressure" Jul 01 16:28:43.033057 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:43.032931 2545 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-227.ec2.internal" event="NodeHasSufficientPID" Jul 01 16:28:43.033057 ip-10-0-130-227 kubenswrapper[2545]: E0701 16:28:43.033035 2545 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-227.ec2.internal\" not found" Jul 01 16:28:43.033996 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:43.033967 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-227.ec2.internal" Jul 01 16:28:43.034049 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:43.033999 2545 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 01 16:28:43.034631 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:43.034616 2545 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-227.ec2.internal" event="NodeHasSufficientMemory" Jul 01 16:28:43.034724 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:43.034637 2545 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-227.ec2.internal" event="NodeHasNoDiskPressure" Jul 01 16:28:43.034724 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:43.034647 2545 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-227.ec2.internal" event="NodeHasSufficientPID" Jul 01 16:28:43.052125 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:43.052106 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/2611d4e32ebacb126b8b90d148d3f748-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-130-227.ec2.internal\" (UID: \"2611d4e32ebacb126b8b90d148d3f748\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-227.ec2.internal" Jul 01 16:28:43.052209 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:43.052129 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2611d4e32ebacb126b8b90d148d3f748-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-130-227.ec2.internal\" (UID: \"2611d4e32ebacb126b8b90d148d3f748\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-227.ec2.internal" Jul 01 16:28:43.052209 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:43.052148 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/7c07f7dba9c4bc09c77f869b83b22cdb-config\") pod \"kube-apiserver-proxy-ip-10-0-130-227.ec2.internal\" (UID: \"7c07f7dba9c4bc09c77f869b83b22cdb\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-130-227.ec2.internal" Jul 01 16:28:43.058238 ip-10-0-130-227 kubenswrapper[2545]: E0701 16:28:43.058222 2545 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-130-227.ec2.internal\" not found" node="ip-10-0-130-227.ec2.internal" Jul 01 16:28:43.062572 ip-10-0-130-227 kubenswrapper[2545]: E0701 16:28:43.062557 2545 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-130-227.ec2.internal\" not found" node="ip-10-0-130-227.ec2.internal" Jul 01 16:28:43.133358 ip-10-0-130-227 kubenswrapper[2545]: E0701 16:28:43.133331 2545 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-227.ec2.internal\" not found" Jul 01 16:28:43.153220 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:43.153202 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2611d4e32ebacb126b8b90d148d3f748-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-130-227.ec2.internal\" (UID: \"2611d4e32ebacb126b8b90d148d3f748\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-227.ec2.internal" Jul 01 16:28:43.153289 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:43.153232 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/7c07f7dba9c4bc09c77f869b83b22cdb-config\") pod \"kube-apiserver-proxy-ip-10-0-130-227.ec2.internal\" (UID: \"7c07f7dba9c4bc09c77f869b83b22cdb\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-130-227.ec2.internal" Jul 01 16:28:43.153289 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:43.153251 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/2611d4e32ebacb126b8b90d148d3f748-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-130-227.ec2.internal\" (UID: \"2611d4e32ebacb126b8b90d148d3f748\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-227.ec2.internal" Jul 01 16:28:43.153379 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:43.153278 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/2611d4e32ebacb126b8b90d148d3f748-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-130-227.ec2.internal\" (UID: \"2611d4e32ebacb126b8b90d148d3f748\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-227.ec2.internal" Jul 01 16:28:43.153379 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:43.153300 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2611d4e32ebacb126b8b90d148d3f748-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-130-227.ec2.internal\" (UID: \"2611d4e32ebacb126b8b90d148d3f748\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-227.ec2.internal" Jul 01 16:28:43.153379 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:43.153325 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/7c07f7dba9c4bc09c77f869b83b22cdb-config\") pod \"kube-apiserver-proxy-ip-10-0-130-227.ec2.internal\" (UID: \"7c07f7dba9c4bc09c77f869b83b22cdb\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-130-227.ec2.internal" Jul 01 16:28:43.234268 ip-10-0-130-227 kubenswrapper[2545]: E0701 16:28:43.234211 2545 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-227.ec2.internal\" not found" Jul 01 16:28:43.334987 ip-10-0-130-227 kubenswrapper[2545]: E0701 16:28:43.334962 2545 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-227.ec2.internal\" not found" Jul 01 16:28:43.362484 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:43.362455 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-227.ec2.internal" Jul 01 16:28:43.367462 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:43.367445 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-227.ec2.internal" Jul 01 16:28:43.436092 ip-10-0-130-227 kubenswrapper[2545]: E0701 16:28:43.435966 2545 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-227.ec2.internal\" not found" Jul 01 16:28:43.536544 ip-10-0-130-227 kubenswrapper[2545]: E0701 16:28:43.536485 2545 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-227.ec2.internal\" not found" Jul 01 16:28:43.637018 ip-10-0-130-227 kubenswrapper[2545]: E0701 16:28:43.636991 2545 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-227.ec2.internal\" not found" Jul 01 16:28:43.683821 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:43.683803 2545 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 01 16:28:43.737521 ip-10-0-130-227 kubenswrapper[2545]: E0701 16:28:43.737498 2545 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-227.ec2.internal\" not found" Jul 01 16:28:43.793068 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:43.793013 2545 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 01 16:28:43.793444 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:43.793122 2545 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 01 16:28:43.793444 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:43.793155 2545 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 01 16:28:43.793444 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:43.793158 2545 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 01 16:28:43.838366 ip-10-0-130-227 kubenswrapper[2545]: E0701 16:28:43.838344 2545 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-227.ec2.internal\" not found" Jul 01 16:28:43.855083 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:43.855044 2545 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-06-30 16:23:42 +0000 UTC" deadline="2028-03-24 16:18:53.006957436 +0000 UTC" Jul 01 16:28:43.855175 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:43.855096 2545 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15167h50m9.151882017s" Jul 01 16:28:43.939425 ip-10-0-130-227 kubenswrapper[2545]: E0701 16:28:43.939402 2545 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-227.ec2.internal\" not found" Jul 01 16:28:43.987506 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:43.987480 2545 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 01 16:28:44.050421 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:44.050366 2545 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-227.ec2.internal" Jul 01 16:28:44.060116 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:44.060090 2545 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 01 16:28:44.060978 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:44.060965 2545 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-227.ec2.internal" Jul 01 16:28:44.069112 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:44.069094 2545 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 01 16:28:44.152190 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:44.152161 2545 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7c07f7dba9c4bc09c77f869b83b22cdb.slice/crio-cd5ba20d7ca1cb6cfa038c03bfa7c75fb216a6a976ffb8029f91dd29ec62563c WatchSource:0}: Error finding container cd5ba20d7ca1cb6cfa038c03bfa7c75fb216a6a976ffb8029f91dd29ec62563c: Status 404 returned error can't find the container with id cd5ba20d7ca1cb6cfa038c03bfa7c75fb216a6a976ffb8029f91dd29ec62563c Jul 01 16:28:44.152468 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:44.152449 2545 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2611d4e32ebacb126b8b90d148d3f748.slice/crio-79aae67c2611005f51c4c5daf3427fead554ced6602def46f01af94498ad9c40 WatchSource:0}: Error finding container 79aae67c2611005f51c4c5daf3427fead554ced6602def46f01af94498ad9c40: Status 404 returned error can't find the container with id 79aae67c2611005f51c4c5daf3427fead554ced6602def46f01af94498ad9c40 Jul 01 16:28:44.156674 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:44.156658 2545 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 01 16:28:44.832534 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:44.832506 2545 apiserver.go:52] "Watching apiserver" Jul 01 16:28:44.840873 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:44.840855 2545 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 01 16:28:44.842005 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:44.841986 2545 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/node-resolver-t7nrc","openshift-image-registry/node-ca-wb2hh","openshift-multus/multus-additional-cni-plugins-gnsqt","openshift-network-operator/iptables-alerter-xwdht","openshift-ovn-kubernetes/ovnkube-node-5cr75","kube-system/global-pull-secret-syncer-npttx","kube-system/kube-apiserver-proxy-ip-10-0-130-227.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lp7sn","openshift-cluster-node-tuning-operator/tuned-nkd9r","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-227.ec2.internal","openshift-multus/multus-t7n99","openshift-multus/network-metrics-daemon-87ztj","openshift-network-diagnostics/network-check-target-kz46s","kube-system/konnectivity-agent-cprkr"] Jul 01 16:28:44.844175 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:44.844162 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kz46s" Jul 01 16:28:44.844238 ip-10-0-130-227 kubenswrapper[2545]: E0701 16:28:44.844220 2545 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kz46s" podUID="4e4b6e11-f86f-4955-aabc-3f94ca793252" Jul 01 16:28:44.845185 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:44.845173 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-wb2hh" Jul 01 16:28:44.846278 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:44.846258 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-gnsqt" Jul 01 16:28:44.847378 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:44.847354 2545 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 01 16:28:44.847480 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:44.847354 2545 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 01 16:28:44.847480 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:44.847453 2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-nznzd\"" Jul 01 16:28:44.847580 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:44.847549 2545 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 01 16:28:44.848342 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:44.848321 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-xwdht" Jul 01 16:28:44.848809 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:44.848787 2545 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 01 16:28:44.848902 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:44.848830 2545 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 01 16:28:44.848902 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:44.848879 2545 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 01 16:28:44.849122 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:44.849109 2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-5577m\"" Jul 01 16:28:44.849319 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:44.849304 2545 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 01 16:28:44.849402 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:44.849388 2545 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 01 16:28:44.849690 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:44.849675 2545 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 01 16:28:44.849768 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:44.849746 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-5cr75" Jul 01 16:28:44.850565 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:44.850539 2545 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 01 16:28:44.850664 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:44.850607 2545 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 01 16:28:44.850856 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:44.850842 2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-g26cp\"" Jul 01 16:28:44.851029 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:44.851009 2545 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 01 16:28:44.851782 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:44.851765 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-npttx" Jul 01 16:28:44.851846 ip-10-0-130-227 kubenswrapper[2545]: E0701 16:28:44.851827 2545 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-npttx" podUID="779fced8-cf7c-49a7-a236-c10a83bbd53c" Jul 01 16:28:44.852124 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:44.852107 2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-dwsmc\"" Jul 01 16:28:44.852769 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:44.852745 2545 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 01 16:28:44.852867 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:44.852782 2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 01 16:28:44.852867 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:44.852839 2545 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 01 16:28:44.854070 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:44.853197 2545 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 01 16:28:44.854070 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:44.853213 2545 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 01 16:28:44.854070 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:44.853834 2545 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 01 16:28:44.854278 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:44.854195 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-t7nrc" Jul 01 16:28:44.855962 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:44.855945 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lp7sn" Jul 01 16:28:44.856413 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:44.856398 2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-bm7jb\"" Jul 01 16:28:44.856560 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:44.856546 2545 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 01 16:28:44.856698 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:44.856687 2545 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 01 16:28:44.857298 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:44.857283 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-nkd9r" Jul 01 16:28:44.858275 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:44.858258 2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 01 16:28:44.858368 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:44.858284 2545 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 01 16:28:44.858411 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:44.858371 2545 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 01 16:28:44.858411 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:44.858386 2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-4nzcl\"" Jul 01 16:28:44.858643 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:44.858630 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-cprkr" Jul 01 16:28:44.859181 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:44.859165 2545 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 01 16:28:44.859314 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:44.859240 2545 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 01 16:28:44.859567 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:44.859549 2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-mlkdm\"" Jul 01 16:28:44.860146 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:44.860104 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-t7n99" Jul 01 16:28:44.861244 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:44.861224 2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 01 16:28:44.861244 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:44.861238 2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-vf7gp\"" Jul 01 16:28:44.861377 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:44.861224 2545 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 01 16:28:44.861377 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:44.861259 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/48cdc46f-d909-463d-942b-4d300d212062-os-release\") pod \"multus-additional-cni-plugins-gnsqt\" (UID: \"48cdc46f-d909-463d-942b-4d300d212062\") " pod="openshift-multus/multus-additional-cni-plugins-gnsqt" Jul 01 16:28:44.861377 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:44.861309 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/b43f95ee-7eec-446c-b1e2-8cf7a0c64864-iptables-alerter-script\") pod \"iptables-alerter-xwdht\" (UID: \"b43f95ee-7eec-446c-b1e2-8cf7a0c64864\") " pod="openshift-network-operator/iptables-alerter-xwdht" Jul 01 16:28:44.861377 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:44.861329 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/b43f95ee-7eec-446c-b1e2-8cf7a0c64864-host-slash\") pod \"iptables-alerter-xwdht\" (UID: \"b43f95ee-7eec-446c-b1e2-8cf7a0c64864\") " pod="openshift-network-operator/iptables-alerter-xwdht" Jul 01 16:28:44.861377 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:44.861357 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-87ztj" Jul 01 16:28:44.861609 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:44.861344 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-26qjf\" (UniqueName: \"kubernetes.io/projected/b43f95ee-7eec-446c-b1e2-8cf7a0c64864-kube-api-access-26qjf\") pod \"iptables-alerter-xwdht\" (UID: \"b43f95ee-7eec-446c-b1e2-8cf7a0c64864\") " pod="openshift-network-operator/iptables-alerter-xwdht" Jul 01 16:28:44.861609 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:44.861454 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/6c03a7ab-a7cc-44ed-86f7-62a750c49b71-node-log\") pod \"ovnkube-node-5cr75\" (UID: \"6c03a7ab-a7cc-44ed-86f7-62a750c49b71\") " pod="openshift-ovn-kubernetes/ovnkube-node-5cr75" Jul 01 16:28:44.861609 ip-10-0-130-227 kubenswrapper[2545]: E0701 16:28:44.861433 2545 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-87ztj" podUID="be2ddffa-466a-4a37-baba-5cb9ecf07e41" Jul 01 16:28:44.861609 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:44.861469 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ncjgt\" (UniqueName: \"kubernetes.io/projected/6c03a7ab-a7cc-44ed-86f7-62a750c49b71-kube-api-access-ncjgt\") pod \"ovnkube-node-5cr75\" (UID: \"6c03a7ab-a7cc-44ed-86f7-62a750c49b71\") " pod="openshift-ovn-kubernetes/ovnkube-node-5cr75" Jul 01 16:28:44.861609 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:44.861492 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/779fced8-cf7c-49a7-a236-c10a83bbd53c-dbus\") pod \"global-pull-secret-syncer-npttx\" (UID: \"779fced8-cf7c-49a7-a236-c10a83bbd53c\") " pod="kube-system/global-pull-secret-syncer-npttx" Jul 01 16:28:44.861609 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:44.861514 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/48cdc46f-d909-463d-942b-4d300d212062-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-gnsqt\" (UID: \"48cdc46f-d909-463d-942b-4d300d212062\") " pod="openshift-multus/multus-additional-cni-plugins-gnsqt" Jul 01 16:28:44.861609 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:44.861528 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/48cdc46f-d909-463d-942b-4d300d212062-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-gnsqt\" (UID: \"48cdc46f-d909-463d-942b-4d300d212062\") " pod="openshift-multus/multus-additional-cni-plugins-gnsqt" Jul 01 16:28:44.861609 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:44.861546 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/6c03a7ab-a7cc-44ed-86f7-62a750c49b71-systemd-units\") pod \"ovnkube-node-5cr75\" (UID: \"6c03a7ab-a7cc-44ed-86f7-62a750c49b71\") " pod="openshift-ovn-kubernetes/ovnkube-node-5cr75" Jul 01 16:28:44.861609 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:44.861586 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c5bsl\" (UniqueName: \"kubernetes.io/projected/4e4b6e11-f86f-4955-aabc-3f94ca793252-kube-api-access-c5bsl\") pod \"network-check-target-kz46s\" (UID: \"4e4b6e11-f86f-4955-aabc-3f94ca793252\") " pod="openshift-network-diagnostics/network-check-target-kz46s" Jul 01 16:28:44.861609 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:44.861605 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/6c03a7ab-a7cc-44ed-86f7-62a750c49b71-host-slash\") pod \"ovnkube-node-5cr75\" (UID: \"6c03a7ab-a7cc-44ed-86f7-62a750c49b71\") " pod="openshift-ovn-kubernetes/ovnkube-node-5cr75" Jul 01 16:28:44.862138 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:44.861621 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6dj59\" (UniqueName: \"kubernetes.io/projected/48cdc46f-d909-463d-942b-4d300d212062-kube-api-access-6dj59\") pod \"multus-additional-cni-plugins-gnsqt\" (UID: \"48cdc46f-d909-463d-942b-4d300d212062\") " pod="openshift-multus/multus-additional-cni-plugins-gnsqt" Jul 01 16:28:44.862138 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:44.861640 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/6c03a7ab-a7cc-44ed-86f7-62a750c49b71-log-socket\") pod \"ovnkube-node-5cr75\" (UID: \"6c03a7ab-a7cc-44ed-86f7-62a750c49b71\") " pod="openshift-ovn-kubernetes/ovnkube-node-5cr75" Jul 01 16:28:44.862138 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:44.861682 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6c03a7ab-a7cc-44ed-86f7-62a750c49b71-host-run-ovn-kubernetes\") pod \"ovnkube-node-5cr75\" (UID: \"6c03a7ab-a7cc-44ed-86f7-62a750c49b71\") " pod="openshift-ovn-kubernetes/ovnkube-node-5cr75" Jul 01 16:28:44.862138 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:44.861737 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/6c03a7ab-a7cc-44ed-86f7-62a750c49b71-env-overrides\") pod \"ovnkube-node-5cr75\" (UID: \"6c03a7ab-a7cc-44ed-86f7-62a750c49b71\") " pod="openshift-ovn-kubernetes/ovnkube-node-5cr75" Jul 01 16:28:44.862138 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:44.861756 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/48cdc46f-d909-463d-942b-4d300d212062-cni-binary-copy\") pod \"multus-additional-cni-plugins-gnsqt\" (UID: \"48cdc46f-d909-463d-942b-4d300d212062\") " pod="openshift-multus/multus-additional-cni-plugins-gnsqt" Jul 01 16:28:44.862138 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:44.861775 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/48cdc46f-d909-463d-942b-4d300d212062-tuning-conf-dir\") pod \"multus-additional-cni-plugins-gnsqt\" (UID: \"48cdc46f-d909-463d-942b-4d300d212062\") " pod="openshift-multus/multus-additional-cni-plugins-gnsqt" Jul 01 16:28:44.862138 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:44.861805 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/6c03a7ab-a7cc-44ed-86f7-62a750c49b71-host-kubelet\") pod \"ovnkube-node-5cr75\" (UID: \"6c03a7ab-a7cc-44ed-86f7-62a750c49b71\") " pod="openshift-ovn-kubernetes/ovnkube-node-5cr75" Jul 01 16:28:44.862138 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:44.861842 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/6c03a7ab-a7cc-44ed-86f7-62a750c49b71-run-ovn\") pod \"ovnkube-node-5cr75\" (UID: \"6c03a7ab-a7cc-44ed-86f7-62a750c49b71\") " pod="openshift-ovn-kubernetes/ovnkube-node-5cr75" Jul 01 16:28:44.862138 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:44.861898 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/6c03a7ab-a7cc-44ed-86f7-62a750c49b71-host-cni-netd\") pod \"ovnkube-node-5cr75\" (UID: \"6c03a7ab-a7cc-44ed-86f7-62a750c49b71\") " pod="openshift-ovn-kubernetes/ovnkube-node-5cr75" Jul 01 16:28:44.862138 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:44.861927 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6c03a7ab-a7cc-44ed-86f7-62a750c49b71-run-openvswitch\") pod \"ovnkube-node-5cr75\" (UID: \"6c03a7ab-a7cc-44ed-86f7-62a750c49b71\") " pod="openshift-ovn-kubernetes/ovnkube-node-5cr75" Jul 01 16:28:44.862138 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:44.861951 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/6c03a7ab-a7cc-44ed-86f7-62a750c49b71-host-cni-bin\") pod \"ovnkube-node-5cr75\" (UID: \"6c03a7ab-a7cc-44ed-86f7-62a750c49b71\") " pod="openshift-ovn-kubernetes/ovnkube-node-5cr75" Jul 01 16:28:44.862138 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:44.861974 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/6c03a7ab-a7cc-44ed-86f7-62a750c49b71-ovn-node-metrics-cert\") pod \"ovnkube-node-5cr75\" (UID: \"6c03a7ab-a7cc-44ed-86f7-62a750c49b71\") " pod="openshift-ovn-kubernetes/ovnkube-node-5cr75" Jul 01 16:28:44.862138 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:44.861996 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/6c03a7ab-a7cc-44ed-86f7-62a750c49b71-ovnkube-script-lib\") pod \"ovnkube-node-5cr75\" (UID: \"6c03a7ab-a7cc-44ed-86f7-62a750c49b71\") " pod="openshift-ovn-kubernetes/ovnkube-node-5cr75" Jul 01 16:28:44.862138 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:44.862019 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/779fced8-cf7c-49a7-a236-c10a83bbd53c-kubelet-config\") pod \"global-pull-secret-syncer-npttx\" (UID: \"779fced8-cf7c-49a7-a236-c10a83bbd53c\") " pod="kube-system/global-pull-secret-syncer-npttx" Jul 01 16:28:44.862138 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:44.862066 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v8tj7\" (UniqueName: \"kubernetes.io/projected/8242757c-9a33-4958-a860-0679d154b28c-kube-api-access-v8tj7\") pod \"node-ca-wb2hh\" (UID: \"8242757c-9a33-4958-a860-0679d154b28c\") " pod="openshift-image-registry/node-ca-wb2hh" Jul 01 16:28:44.862138 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:44.862085 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/48cdc46f-d909-463d-942b-4d300d212062-system-cni-dir\") pod \"multus-additional-cni-plugins-gnsqt\" (UID: \"48cdc46f-d909-463d-942b-4d300d212062\") " pod="openshift-multus/multus-additional-cni-plugins-gnsqt" Jul 01 16:28:44.862598 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:44.862124 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/6c03a7ab-a7cc-44ed-86f7-62a750c49b71-host-run-netns\") pod \"ovnkube-node-5cr75\" (UID: \"6c03a7ab-a7cc-44ed-86f7-62a750c49b71\") " pod="openshift-ovn-kubernetes/ovnkube-node-5cr75" Jul 01 16:28:44.862598 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:44.862156 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6c03a7ab-a7cc-44ed-86f7-62a750c49b71-var-lib-openvswitch\") pod \"ovnkube-node-5cr75\" (UID: \"6c03a7ab-a7cc-44ed-86f7-62a750c49b71\") " pod="openshift-ovn-kubernetes/ovnkube-node-5cr75" Jul 01 16:28:44.862598 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:44.862182 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6c03a7ab-a7cc-44ed-86f7-62a750c49b71-etc-openvswitch\") pod \"ovnkube-node-5cr75\" (UID: \"6c03a7ab-a7cc-44ed-86f7-62a750c49b71\") " pod="openshift-ovn-kubernetes/ovnkube-node-5cr75" Jul 01 16:28:44.862598 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:44.862207 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6c03a7ab-a7cc-44ed-86f7-62a750c49b71-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-5cr75\" (UID: \"6c03a7ab-a7cc-44ed-86f7-62a750c49b71\") " pod="openshift-ovn-kubernetes/ovnkube-node-5cr75" Jul 01 16:28:44.862598 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:44.862228 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/779fced8-cf7c-49a7-a236-c10a83bbd53c-original-pull-secret\") pod \"global-pull-secret-syncer-npttx\" (UID: \"779fced8-cf7c-49a7-a236-c10a83bbd53c\") " pod="kube-system/global-pull-secret-syncer-npttx" Jul 01 16:28:44.862598 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:44.862248 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/8242757c-9a33-4958-a860-0679d154b28c-host\") pod \"node-ca-wb2hh\" (UID: \"8242757c-9a33-4958-a860-0679d154b28c\") " pod="openshift-image-registry/node-ca-wb2hh" Jul 01 16:28:44.862598 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:44.862263 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/8242757c-9a33-4958-a860-0679d154b28c-serviceca\") pod \"node-ca-wb2hh\" (UID: \"8242757c-9a33-4958-a860-0679d154b28c\") " pod="openshift-image-registry/node-ca-wb2hh" Jul 01 16:28:44.862598 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:44.862283 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/6c03a7ab-a7cc-44ed-86f7-62a750c49b71-run-systemd\") pod \"ovnkube-node-5cr75\" (UID: \"6c03a7ab-a7cc-44ed-86f7-62a750c49b71\") " pod="openshift-ovn-kubernetes/ovnkube-node-5cr75" Jul 01 16:28:44.862598 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:44.862310 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/6c03a7ab-a7cc-44ed-86f7-62a750c49b71-ovnkube-config\") pod \"ovnkube-node-5cr75\" (UID: \"6c03a7ab-a7cc-44ed-86f7-62a750c49b71\") " pod="openshift-ovn-kubernetes/ovnkube-node-5cr75" Jul 01 16:28:44.862598 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:44.862318 2545 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 01 16:28:44.862598 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:44.862348 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/48cdc46f-d909-463d-942b-4d300d212062-cnibin\") pod \"multus-additional-cni-plugins-gnsqt\" (UID: \"48cdc46f-d909-463d-942b-4d300d212062\") " pod="openshift-multus/multus-additional-cni-plugins-gnsqt" Jul 01 16:28:44.862598 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:44.862420 2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-8wc7m\"" Jul 01 16:28:44.863602 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:44.863587 2545 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 01 16:28:44.880991 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:44.880974 2545 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-cw42p" Jul 01 16:28:44.888568 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:44.888553 2545 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-cw42p" Jul 01 16:28:44.933176 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:44.933139 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-227.ec2.internal" event={"ID":"2611d4e32ebacb126b8b90d148d3f748","Type":"ContainerStarted","Data":"79aae67c2611005f51c4c5daf3427fead554ced6602def46f01af94498ad9c40"} Jul 01 16:28:44.934129 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:44.934113 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-227.ec2.internal" event={"ID":"7c07f7dba9c4bc09c77f869b83b22cdb","Type":"ContainerStarted","Data":"cd5ba20d7ca1cb6cfa038c03bfa7c75fb216a6a976ffb8029f91dd29ec62563c"} Jul 01 16:28:44.950778 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:44.950760 2545 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 01 16:28:44.963105 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:44.963083 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kmjz8\" (UniqueName: \"kubernetes.io/projected/0b3759ce-7af8-4687-9a32-0578edfe706c-kube-api-access-kmjz8\") pod \"tuned-nkd9r\" (UID: \"0b3759ce-7af8-4687-9a32-0578edfe706c\") " pod="openshift-cluster-node-tuning-operator/tuned-nkd9r" Jul 01 16:28:44.963182 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:44.963111 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/6c03a7ab-a7cc-44ed-86f7-62a750c49b71-ovn-node-metrics-cert\") pod \"ovnkube-node-5cr75\" (UID: \"6c03a7ab-a7cc-44ed-86f7-62a750c49b71\") " pod="openshift-ovn-kubernetes/ovnkube-node-5cr75" Jul 01 16:28:44.963182 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:44.963128 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/6c03a7ab-a7cc-44ed-86f7-62a750c49b71-ovnkube-script-lib\") pod \"ovnkube-node-5cr75\" (UID: \"6c03a7ab-a7cc-44ed-86f7-62a750c49b71\") " pod="openshift-ovn-kubernetes/ovnkube-node-5cr75" Jul 01 16:28:44.963182 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:44.963142 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/76faf2eb-5dae-48e1-ba27-f1b94c85920d-host-var-lib-cni-multus\") pod \"multus-t7n99\" (UID: \"76faf2eb-5dae-48e1-ba27-f1b94c85920d\") " pod="openshift-multus/multus-t7n99" Jul 01 16:28:44.963182 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:44.963157 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qldwh\" (UniqueName: \"kubernetes.io/projected/76faf2eb-5dae-48e1-ba27-f1b94c85920d-kube-api-access-qldwh\") pod \"multus-t7n99\" (UID: \"76faf2eb-5dae-48e1-ba27-f1b94c85920d\") " pod="openshift-multus/multus-t7n99" Jul 01 16:28:44.963182 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:44.963175 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6c03a7ab-a7cc-44ed-86f7-62a750c49b71-var-lib-openvswitch\") pod \"ovnkube-node-5cr75\" (UID: \"6c03a7ab-a7cc-44ed-86f7-62a750c49b71\") " pod="openshift-ovn-kubernetes/ovnkube-node-5cr75" Jul 01 16:28:44.963359 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:44.963202 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6c03a7ab-a7cc-44ed-86f7-62a750c49b71-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-5cr75\" (UID: \"6c03a7ab-a7cc-44ed-86f7-62a750c49b71\") " pod="openshift-ovn-kubernetes/ovnkube-node-5cr75" Jul 01 16:28:44.963359 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:44.963224 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6c03a7ab-a7cc-44ed-86f7-62a750c49b71-var-lib-openvswitch\") pod \"ovnkube-node-5cr75\" (UID: \"6c03a7ab-a7cc-44ed-86f7-62a750c49b71\") " pod="openshift-ovn-kubernetes/ovnkube-node-5cr75" Jul 01 16:28:44.963359 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:44.963226 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/8242757c-9a33-4958-a860-0679d154b28c-host\") pod \"node-ca-wb2hh\" (UID: \"8242757c-9a33-4958-a860-0679d154b28c\") " pod="openshift-image-registry/node-ca-wb2hh" Jul 01 16:28:44.963359 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:44.963244 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6c03a7ab-a7cc-44ed-86f7-62a750c49b71-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-5cr75\" (UID: \"6c03a7ab-a7cc-44ed-86f7-62a750c49b71\") " pod="openshift-ovn-kubernetes/ovnkube-node-5cr75" Jul 01 16:28:44.963359 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:44.963262 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/8242757c-9a33-4958-a860-0679d154b28c-host\") pod \"node-ca-wb2hh\" (UID: \"8242757c-9a33-4958-a860-0679d154b28c\") " pod="openshift-image-registry/node-ca-wb2hh" Jul 01 16:28:44.963359 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:44.963275 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/8242757c-9a33-4958-a860-0679d154b28c-serviceca\") pod \"node-ca-wb2hh\" (UID: \"8242757c-9a33-4958-a860-0679d154b28c\") " pod="openshift-image-registry/node-ca-wb2hh" Jul 01 16:28:44.963359 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:44.963300 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/48cdc46f-d909-463d-942b-4d300d212062-os-release\") pod \"multus-additional-cni-plugins-gnsqt\" (UID: \"48cdc46f-d909-463d-942b-4d300d212062\") " pod="openshift-multus/multus-additional-cni-plugins-gnsqt" Jul 01 16:28:44.963359 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:44.963322 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/0b3759ce-7af8-4687-9a32-0578edfe706c-run\") pod \"tuned-nkd9r\" (UID: \"0b3759ce-7af8-4687-9a32-0578edfe706c\") " pod="openshift-cluster-node-tuning-operator/tuned-nkd9r" Jul 01 16:28:44.963740 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:44.963408 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/48cdc46f-d909-463d-942b-4d300d212062-os-release\") pod \"multus-additional-cni-plugins-gnsqt\" (UID: \"48cdc46f-d909-463d-942b-4d300d212062\") " pod="openshift-multus/multus-additional-cni-plugins-gnsqt" Jul 01 16:28:44.963740 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:44.963420 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/5be67da1-9165-4446-b3cb-2333db8c09cc-hosts-file\") pod \"node-resolver-t7nrc\" (UID: \"5be67da1-9165-4446-b3cb-2333db8c09cc\") " pod="openshift-dns/node-resolver-t7nrc" Jul 01 16:28:44.963740 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:44.963451 2545 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 01 16:28:44.963740 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:44.963456 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/1df6652f-89cc-41a1-89f7-0ed8568b6ee2-agent-certs\") pod \"konnectivity-agent-cprkr\" (UID: \"1df6652f-89cc-41a1-89f7-0ed8568b6ee2\") " pod="kube-system/konnectivity-agent-cprkr" Jul 01 16:28:44.963740 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:44.963504 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/6c03a7ab-a7cc-44ed-86f7-62a750c49b71-run-systemd\") pod \"ovnkube-node-5cr75\" (UID: \"6c03a7ab-a7cc-44ed-86f7-62a750c49b71\") " pod="openshift-ovn-kubernetes/ovnkube-node-5cr75" Jul 01 16:28:44.963740 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:44.963530 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/9ae0a7d9-2f7b-49ba-8a15-69acd2a6071c-kubelet-dir\") pod \"aws-ebs-csi-driver-node-lp7sn\" (UID: \"9ae0a7d9-2f7b-49ba-8a15-69acd2a6071c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lp7sn" Jul 01 16:28:44.963740 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:44.963553 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/9ae0a7d9-2f7b-49ba-8a15-69acd2a6071c-etc-selinux\") pod \"aws-ebs-csi-driver-node-lp7sn\" (UID: \"9ae0a7d9-2f7b-49ba-8a15-69acd2a6071c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lp7sn" Jul 01 16:28:44.963740 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:44.963581 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0b3759ce-7af8-4687-9a32-0578edfe706c-etc-kubernetes\") pod \"tuned-nkd9r\" (UID: \"0b3759ce-7af8-4687-9a32-0578edfe706c\") " pod="openshift-cluster-node-tuning-operator/tuned-nkd9r" Jul 01 16:28:44.963740 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:44.963603 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/6c03a7ab-a7cc-44ed-86f7-62a750c49b71-node-log\") pod \"ovnkube-node-5cr75\" (UID: \"6c03a7ab-a7cc-44ed-86f7-62a750c49b71\") " pod="openshift-ovn-kubernetes/ovnkube-node-5cr75" Jul 01 16:28:44.963740 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:44.963628 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/779fced8-cf7c-49a7-a236-c10a83bbd53c-dbus\") pod \"global-pull-secret-syncer-npttx\" (UID: \"779fced8-cf7c-49a7-a236-c10a83bbd53c\") " pod="kube-system/global-pull-secret-syncer-npttx" Jul 01 16:28:44.963740 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:44.963604 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/6c03a7ab-a7cc-44ed-86f7-62a750c49b71-run-systemd\") pod \"ovnkube-node-5cr75\" (UID: \"6c03a7ab-a7cc-44ed-86f7-62a750c49b71\") " pod="openshift-ovn-kubernetes/ovnkube-node-5cr75" Jul 01 16:28:44.963740 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:44.963650 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/6c03a7ab-a7cc-44ed-86f7-62a750c49b71-node-log\") pod \"ovnkube-node-5cr75\" (UID: \"6c03a7ab-a7cc-44ed-86f7-62a750c49b71\") " pod="openshift-ovn-kubernetes/ovnkube-node-5cr75" Jul 01 16:28:44.963740 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:44.963652 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/48cdc46f-d909-463d-942b-4d300d212062-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-gnsqt\" (UID: \"48cdc46f-d909-463d-942b-4d300d212062\") " pod="openshift-multus/multus-additional-cni-plugins-gnsqt" Jul 01 16:28:44.963740 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:44.963668 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/8242757c-9a33-4958-a860-0679d154b28c-serviceca\") pod \"node-ca-wb2hh\" (UID: \"8242757c-9a33-4958-a860-0679d154b28c\") " pod="openshift-image-registry/node-ca-wb2hh" Jul 01 16:28:44.963740 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:44.963691 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/76faf2eb-5dae-48e1-ba27-f1b94c85920d-system-cni-dir\") pod \"multus-t7n99\" (UID: \"76faf2eb-5dae-48e1-ba27-f1b94c85920d\") " pod="openshift-multus/multus-t7n99" Jul 01 16:28:44.963740 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:44.963732 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/76faf2eb-5dae-48e1-ba27-f1b94c85920d-host-run-k8s-cni-cncf-io\") pod \"multus-t7n99\" (UID: \"76faf2eb-5dae-48e1-ba27-f1b94c85920d\") " pod="openshift-multus/multus-t7n99" Jul 01 16:28:44.964455 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:44.963750 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/6c03a7ab-a7cc-44ed-86f7-62a750c49b71-ovnkube-script-lib\") pod \"ovnkube-node-5cr75\" (UID: \"6c03a7ab-a7cc-44ed-86f7-62a750c49b71\") " pod="openshift-ovn-kubernetes/ovnkube-node-5cr75" Jul 01 16:28:44.964455 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:44.963763 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/6c03a7ab-a7cc-44ed-86f7-62a750c49b71-systemd-units\") pod \"ovnkube-node-5cr75\" (UID: \"6c03a7ab-a7cc-44ed-86f7-62a750c49b71\") " pod="openshift-ovn-kubernetes/ovnkube-node-5cr75" Jul 01 16:28:44.964455 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:44.963799 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/6c03a7ab-a7cc-44ed-86f7-62a750c49b71-systemd-units\") pod \"ovnkube-node-5cr75\" (UID: \"6c03a7ab-a7cc-44ed-86f7-62a750c49b71\") " pod="openshift-ovn-kubernetes/ovnkube-node-5cr75" Jul 01 16:28:44.964455 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:44.963826 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/76faf2eb-5dae-48e1-ba27-f1b94c85920d-os-release\") pod \"multus-t7n99\" (UID: \"76faf2eb-5dae-48e1-ba27-f1b94c85920d\") " pod="openshift-multus/multus-t7n99" Jul 01 16:28:44.964455 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:44.963853 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/76faf2eb-5dae-48e1-ba27-f1b94c85920d-multus-socket-dir-parent\") pod \"multus-t7n99\" (UID: \"76faf2eb-5dae-48e1-ba27-f1b94c85920d\") " pod="openshift-multus/multus-t7n99" Jul 01 16:28:44.964455 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:44.963865 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/779fced8-cf7c-49a7-a236-c10a83bbd53c-dbus\") pod \"global-pull-secret-syncer-npttx\" (UID: \"779fced8-cf7c-49a7-a236-c10a83bbd53c\") " pod="kube-system/global-pull-secret-syncer-npttx" Jul 01 16:28:44.964455 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:44.963878 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/be2ddffa-466a-4a37-baba-5cb9ecf07e41-metrics-certs\") pod \"network-metrics-daemon-87ztj\" (UID: \"be2ddffa-466a-4a37-baba-5cb9ecf07e41\") " pod="openshift-multus/network-metrics-daemon-87ztj" Jul 01 16:28:44.964455 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:44.963903 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-55bw6\" (UniqueName: \"kubernetes.io/projected/be2ddffa-466a-4a37-baba-5cb9ecf07e41-kube-api-access-55bw6\") pod \"network-metrics-daemon-87ztj\" (UID: \"be2ddffa-466a-4a37-baba-5cb9ecf07e41\") " pod="openshift-multus/network-metrics-daemon-87ztj" Jul 01 16:28:44.964455 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:44.963925 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0b3759ce-7af8-4687-9a32-0578edfe706c-var-lib-kubelet\") pod \"tuned-nkd9r\" (UID: \"0b3759ce-7af8-4687-9a32-0578edfe706c\") " pod="openshift-cluster-node-tuning-operator/tuned-nkd9r" Jul 01 16:28:44.964455 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:44.963939 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dm56c\" (UniqueName: \"kubernetes.io/projected/9ae0a7d9-2f7b-49ba-8a15-69acd2a6071c-kube-api-access-dm56c\") pod \"aws-ebs-csi-driver-node-lp7sn\" (UID: \"9ae0a7d9-2f7b-49ba-8a15-69acd2a6071c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lp7sn" Jul 01 16:28:44.964455 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:44.963952 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/0b3759ce-7af8-4687-9a32-0578edfe706c-etc-sysctl-d\") pod \"tuned-nkd9r\" (UID: \"0b3759ce-7af8-4687-9a32-0578edfe706c\") " pod="openshift-cluster-node-tuning-operator/tuned-nkd9r" Jul 01 16:28:44.964455 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:44.963968 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kwmxh\" (UniqueName: \"kubernetes.io/projected/5be67da1-9165-4446-b3cb-2333db8c09cc-kube-api-access-kwmxh\") pod \"node-resolver-t7nrc\" (UID: \"5be67da1-9165-4446-b3cb-2333db8c09cc\") " pod="openshift-dns/node-resolver-t7nrc" Jul 01 16:28:44.964455 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:44.963990 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/6c03a7ab-a7cc-44ed-86f7-62a750c49b71-log-socket\") pod \"ovnkube-node-5cr75\" (UID: \"6c03a7ab-a7cc-44ed-86f7-62a750c49b71\") " pod="openshift-ovn-kubernetes/ovnkube-node-5cr75" Jul 01 16:28:44.964455 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:44.964009 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6c03a7ab-a7cc-44ed-86f7-62a750c49b71-host-run-ovn-kubernetes\") pod \"ovnkube-node-5cr75\" (UID: \"6c03a7ab-a7cc-44ed-86f7-62a750c49b71\") " pod="openshift-ovn-kubernetes/ovnkube-node-5cr75" Jul 01 16:28:44.964455 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:44.964032 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0b3759ce-7af8-4687-9a32-0578edfe706c-sys\") pod \"tuned-nkd9r\" (UID: \"0b3759ce-7af8-4687-9a32-0578edfe706c\") " pod="openshift-cluster-node-tuning-operator/tuned-nkd9r" Jul 01 16:28:44.964455 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:44.964056 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/0b3759ce-7af8-4687-9a32-0578edfe706c-etc-tuned\") pod \"tuned-nkd9r\" (UID: \"0b3759ce-7af8-4687-9a32-0578edfe706c\") " pod="openshift-cluster-node-tuning-operator/tuned-nkd9r" Jul 01 16:28:44.964455 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:44.964058 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6c03a7ab-a7cc-44ed-86f7-62a750c49b71-host-run-ovn-kubernetes\") pod \"ovnkube-node-5cr75\" (UID: \"6c03a7ab-a7cc-44ed-86f7-62a750c49b71\") " pod="openshift-ovn-kubernetes/ovnkube-node-5cr75" Jul 01 16:28:44.965211 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:44.964063 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/6c03a7ab-a7cc-44ed-86f7-62a750c49b71-log-socket\") pod \"ovnkube-node-5cr75\" (UID: \"6c03a7ab-a7cc-44ed-86f7-62a750c49b71\") " pod="openshift-ovn-kubernetes/ovnkube-node-5cr75" Jul 01 16:28:44.965211 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:44.964077 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/6c03a7ab-a7cc-44ed-86f7-62a750c49b71-host-cni-netd\") pod \"ovnkube-node-5cr75\" (UID: \"6c03a7ab-a7cc-44ed-86f7-62a750c49b71\") " pod="openshift-ovn-kubernetes/ovnkube-node-5cr75" Jul 01 16:28:44.965211 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:44.964105 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/76faf2eb-5dae-48e1-ba27-f1b94c85920d-cnibin\") pod \"multus-t7n99\" (UID: \"76faf2eb-5dae-48e1-ba27-f1b94c85920d\") " pod="openshift-multus/multus-t7n99" Jul 01 16:28:44.965211 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:44.964120 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/6c03a7ab-a7cc-44ed-86f7-62a750c49b71-host-cni-netd\") pod \"ovnkube-node-5cr75\" (UID: \"6c03a7ab-a7cc-44ed-86f7-62a750c49b71\") " pod="openshift-ovn-kubernetes/ovnkube-node-5cr75" Jul 01 16:28:44.965211 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:44.964130 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/9ae0a7d9-2f7b-49ba-8a15-69acd2a6071c-device-dir\") pod \"aws-ebs-csi-driver-node-lp7sn\" (UID: \"9ae0a7d9-2f7b-49ba-8a15-69acd2a6071c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lp7sn" Jul 01 16:28:44.965211 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:44.964157 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/76faf2eb-5dae-48e1-ba27-f1b94c85920d-host-run-multus-certs\") pod \"multus-t7n99\" (UID: \"76faf2eb-5dae-48e1-ba27-f1b94c85920d\") " pod="openshift-multus/multus-t7n99" Jul 01 16:28:44.965211 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:44.964167 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/48cdc46f-d909-463d-942b-4d300d212062-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-gnsqt\" (UID: \"48cdc46f-d909-463d-942b-4d300d212062\") " pod="openshift-multus/multus-additional-cni-plugins-gnsqt" Jul 01 16:28:44.965211 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:44.964185 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6c03a7ab-a7cc-44ed-86f7-62a750c49b71-run-openvswitch\") pod \"ovnkube-node-5cr75\" (UID: \"6c03a7ab-a7cc-44ed-86f7-62a750c49b71\") " pod="openshift-ovn-kubernetes/ovnkube-node-5cr75" Jul 01 16:28:44.965211 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:44.964209 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/6c03a7ab-a7cc-44ed-86f7-62a750c49b71-host-cni-bin\") pod \"ovnkube-node-5cr75\" (UID: \"6c03a7ab-a7cc-44ed-86f7-62a750c49b71\") " pod="openshift-ovn-kubernetes/ovnkube-node-5cr75" Jul 01 16:28:44.965211 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:44.964232 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/779fced8-cf7c-49a7-a236-c10a83bbd53c-kubelet-config\") pod \"global-pull-secret-syncer-npttx\" (UID: \"779fced8-cf7c-49a7-a236-c10a83bbd53c\") " pod="kube-system/global-pull-secret-syncer-npttx" Jul 01 16:28:44.965211 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:44.964233 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6c03a7ab-a7cc-44ed-86f7-62a750c49b71-run-openvswitch\") pod \"ovnkube-node-5cr75\" (UID: \"6c03a7ab-a7cc-44ed-86f7-62a750c49b71\") " pod="openshift-ovn-kubernetes/ovnkube-node-5cr75" Jul 01 16:28:44.965211 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:44.964250 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v8tj7\" (UniqueName: \"kubernetes.io/projected/8242757c-9a33-4958-a860-0679d154b28c-kube-api-access-v8tj7\") pod \"node-ca-wb2hh\" (UID: \"8242757c-9a33-4958-a860-0679d154b28c\") " pod="openshift-image-registry/node-ca-wb2hh" Jul 01 16:28:44.965211 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:44.964289 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/779fced8-cf7c-49a7-a236-c10a83bbd53c-kubelet-config\") pod \"global-pull-secret-syncer-npttx\" (UID: \"779fced8-cf7c-49a7-a236-c10a83bbd53c\") " pod="kube-system/global-pull-secret-syncer-npttx" Jul 01 16:28:44.965211 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:44.964298 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/48cdc46f-d909-463d-942b-4d300d212062-system-cni-dir\") pod \"multus-additional-cni-plugins-gnsqt\" (UID: \"48cdc46f-d909-463d-942b-4d300d212062\") " pod="openshift-multus/multus-additional-cni-plugins-gnsqt" Jul 01 16:28:44.965211 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:44.964317 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/6c03a7ab-a7cc-44ed-86f7-62a750c49b71-host-cni-bin\") pod \"ovnkube-node-5cr75\" (UID: \"6c03a7ab-a7cc-44ed-86f7-62a750c49b71\") " pod="openshift-ovn-kubernetes/ovnkube-node-5cr75" Jul 01 16:28:44.965211 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:44.964323 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/9ae0a7d9-2f7b-49ba-8a15-69acd2a6071c-registration-dir\") pod \"aws-ebs-csi-driver-node-lp7sn\" (UID: \"9ae0a7d9-2f7b-49ba-8a15-69acd2a6071c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lp7sn" Jul 01 16:28:44.965211 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:44.964357 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/48cdc46f-d909-463d-942b-4d300d212062-system-cni-dir\") pod \"multus-additional-cni-plugins-gnsqt\" (UID: \"48cdc46f-d909-463d-942b-4d300d212062\") " pod="openshift-multus/multus-additional-cni-plugins-gnsqt" Jul 01 16:28:44.965968 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:44.964366 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/6c03a7ab-a7cc-44ed-86f7-62a750c49b71-host-run-netns\") pod \"ovnkube-node-5cr75\" (UID: \"6c03a7ab-a7cc-44ed-86f7-62a750c49b71\") " pod="openshift-ovn-kubernetes/ovnkube-node-5cr75" Jul 01 16:28:44.965968 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:44.964391 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6c03a7ab-a7cc-44ed-86f7-62a750c49b71-etc-openvswitch\") pod \"ovnkube-node-5cr75\" (UID: \"6c03a7ab-a7cc-44ed-86f7-62a750c49b71\") " pod="openshift-ovn-kubernetes/ovnkube-node-5cr75" Jul 01 16:28:44.965968 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:44.964403 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/6c03a7ab-a7cc-44ed-86f7-62a750c49b71-host-run-netns\") pod \"ovnkube-node-5cr75\" (UID: \"6c03a7ab-a7cc-44ed-86f7-62a750c49b71\") " pod="openshift-ovn-kubernetes/ovnkube-node-5cr75" Jul 01 16:28:44.965968 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:44.964417 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/779fced8-cf7c-49a7-a236-c10a83bbd53c-original-pull-secret\") pod \"global-pull-secret-syncer-npttx\" (UID: \"779fced8-cf7c-49a7-a236-c10a83bbd53c\") " pod="kube-system/global-pull-secret-syncer-npttx" Jul 01 16:28:44.965968 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:44.964442 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/48cdc46f-d909-463d-942b-4d300d212062-cnibin\") pod \"multus-additional-cni-plugins-gnsqt\" (UID: \"48cdc46f-d909-463d-942b-4d300d212062\") " pod="openshift-multus/multus-additional-cni-plugins-gnsqt" Jul 01 16:28:44.965968 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:44.964442 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6c03a7ab-a7cc-44ed-86f7-62a750c49b71-etc-openvswitch\") pod \"ovnkube-node-5cr75\" (UID: \"6c03a7ab-a7cc-44ed-86f7-62a750c49b71\") " pod="openshift-ovn-kubernetes/ovnkube-node-5cr75" Jul 01 16:28:44.965968 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:44.964489 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/48cdc46f-d909-463d-942b-4d300d212062-cnibin\") pod \"multus-additional-cni-plugins-gnsqt\" (UID: \"48cdc46f-d909-463d-942b-4d300d212062\") " pod="openshift-multus/multus-additional-cni-plugins-gnsqt" Jul 01 16:28:44.965968 ip-10-0-130-227 kubenswrapper[2545]: E0701 16:28:44.964509 2545 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 01 16:28:44.965968 ip-10-0-130-227 kubenswrapper[2545]: E0701 16:28:44.964570 2545 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/779fced8-cf7c-49a7-a236-c10a83bbd53c-original-pull-secret podName:779fced8-cf7c-49a7-a236-c10a83bbd53c nodeName:}" failed. No retries permitted until 2026-07-01 16:28:45.464535836 +0000 UTC m=+4.045153322 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/779fced8-cf7c-49a7-a236-c10a83bbd53c-original-pull-secret") pod "global-pull-secret-syncer-npttx" (UID: "779fced8-cf7c-49a7-a236-c10a83bbd53c") : object "kube-system"/"original-pull-secret" not registered Jul 01 16:28:44.965968 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:44.964584 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/76faf2eb-5dae-48e1-ba27-f1b94c85920d-cni-binary-copy\") pod \"multus-t7n99\" (UID: \"76faf2eb-5dae-48e1-ba27-f1b94c85920d\") " pod="openshift-multus/multus-t7n99" Jul 01 16:28:44.965968 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:44.964604 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/76faf2eb-5dae-48e1-ba27-f1b94c85920d-host-var-lib-cni-bin\") pod \"multus-t7n99\" (UID: \"76faf2eb-5dae-48e1-ba27-f1b94c85920d\") " pod="openshift-multus/multus-t7n99" Jul 01 16:28:44.965968 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:44.964618 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/76faf2eb-5dae-48e1-ba27-f1b94c85920d-etc-kubernetes\") pod \"multus-t7n99\" (UID: \"76faf2eb-5dae-48e1-ba27-f1b94c85920d\") " pod="openshift-multus/multus-t7n99" Jul 01 16:28:44.965968 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:44.964645 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/76faf2eb-5dae-48e1-ba27-f1b94c85920d-host-var-lib-kubelet\") pod \"multus-t7n99\" (UID: \"76faf2eb-5dae-48e1-ba27-f1b94c85920d\") " pod="openshift-multus/multus-t7n99" Jul 01 16:28:44.965968 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:44.964677 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/76faf2eb-5dae-48e1-ba27-f1b94c85920d-multus-conf-dir\") pod \"multus-t7n99\" (UID: \"76faf2eb-5dae-48e1-ba27-f1b94c85920d\") " pod="openshift-multus/multus-t7n99" Jul 01 16:28:44.965968 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:44.964701 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/6c03a7ab-a7cc-44ed-86f7-62a750c49b71-ovnkube-config\") pod \"ovnkube-node-5cr75\" (UID: \"6c03a7ab-a7cc-44ed-86f7-62a750c49b71\") " pod="openshift-ovn-kubernetes/ovnkube-node-5cr75" Jul 01 16:28:44.965968 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:44.964740 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/b43f95ee-7eec-446c-b1e2-8cf7a0c64864-iptables-alerter-script\") pod \"iptables-alerter-xwdht\" (UID: \"b43f95ee-7eec-446c-b1e2-8cf7a0c64864\") " pod="openshift-network-operator/iptables-alerter-xwdht" Jul 01 16:28:44.965968 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:44.964763 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/b43f95ee-7eec-446c-b1e2-8cf7a0c64864-host-slash\") pod \"iptables-alerter-xwdht\" (UID: \"b43f95ee-7eec-446c-b1e2-8cf7a0c64864\") " pod="openshift-network-operator/iptables-alerter-xwdht" Jul 01 16:28:44.966439 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:44.964786 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-26qjf\" (UniqueName: \"kubernetes.io/projected/b43f95ee-7eec-446c-b1e2-8cf7a0c64864-kube-api-access-26qjf\") pod \"iptables-alerter-xwdht\" (UID: \"b43f95ee-7eec-446c-b1e2-8cf7a0c64864\") " pod="openshift-network-operator/iptables-alerter-xwdht" Jul 01 16:28:44.966439 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:44.964810 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/0b3759ce-7af8-4687-9a32-0578edfe706c-etc-modprobe-d\") pod \"tuned-nkd9r\" (UID: \"0b3759ce-7af8-4687-9a32-0578edfe706c\") " pod="openshift-cluster-node-tuning-operator/tuned-nkd9r" Jul 01 16:28:44.966439 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:44.964823 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/b43f95ee-7eec-446c-b1e2-8cf7a0c64864-host-slash\") pod \"iptables-alerter-xwdht\" (UID: \"b43f95ee-7eec-446c-b1e2-8cf7a0c64864\") " pod="openshift-network-operator/iptables-alerter-xwdht" Jul 01 16:28:44.966439 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:44.964834 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ncjgt\" (UniqueName: \"kubernetes.io/projected/6c03a7ab-a7cc-44ed-86f7-62a750c49b71-kube-api-access-ncjgt\") pod \"ovnkube-node-5cr75\" (UID: \"6c03a7ab-a7cc-44ed-86f7-62a750c49b71\") " pod="openshift-ovn-kubernetes/ovnkube-node-5cr75" Jul 01 16:28:44.966439 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:44.964860 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/48cdc46f-d909-463d-942b-4d300d212062-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-gnsqt\" (UID: \"48cdc46f-d909-463d-942b-4d300d212062\") " pod="openshift-multus/multus-additional-cni-plugins-gnsqt" Jul 01 16:28:44.966439 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:44.964909 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/76faf2eb-5dae-48e1-ba27-f1b94c85920d-multus-cni-dir\") pod \"multus-t7n99\" (UID: \"76faf2eb-5dae-48e1-ba27-f1b94c85920d\") " pod="openshift-multus/multus-t7n99" Jul 01 16:28:44.966439 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:44.964932 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/0b3759ce-7af8-4687-9a32-0578edfe706c-etc-sysctl-conf\") pod \"tuned-nkd9r\" (UID: \"0b3759ce-7af8-4687-9a32-0578edfe706c\") " pod="openshift-cluster-node-tuning-operator/tuned-nkd9r" Jul 01 16:28:44.966439 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:44.964955 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c5bsl\" (UniqueName: \"kubernetes.io/projected/4e4b6e11-f86f-4955-aabc-3f94ca793252-kube-api-access-c5bsl\") pod \"network-check-target-kz46s\" (UID: \"4e4b6e11-f86f-4955-aabc-3f94ca793252\") " pod="openshift-network-diagnostics/network-check-target-kz46s" Jul 01 16:28:44.966439 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:44.964979 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/76faf2eb-5dae-48e1-ba27-f1b94c85920d-multus-daemon-config\") pod \"multus-t7n99\" (UID: \"76faf2eb-5dae-48e1-ba27-f1b94c85920d\") " pod="openshift-multus/multus-t7n99" Jul 01 16:28:44.966439 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:44.965003 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/6c03a7ab-a7cc-44ed-86f7-62a750c49b71-host-slash\") pod \"ovnkube-node-5cr75\" (UID: \"6c03a7ab-a7cc-44ed-86f7-62a750c49b71\") " pod="openshift-ovn-kubernetes/ovnkube-node-5cr75" Jul 01 16:28:44.966439 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:44.965029 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6dj59\" (UniqueName: \"kubernetes.io/projected/48cdc46f-d909-463d-942b-4d300d212062-kube-api-access-6dj59\") pod \"multus-additional-cni-plugins-gnsqt\" (UID: \"48cdc46f-d909-463d-942b-4d300d212062\") " pod="openshift-multus/multus-additional-cni-plugins-gnsqt" Jul 01 16:28:44.966439 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:44.965052 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/9ae0a7d9-2f7b-49ba-8a15-69acd2a6071c-socket-dir\") pod \"aws-ebs-csi-driver-node-lp7sn\" (UID: \"9ae0a7d9-2f7b-49ba-8a15-69acd2a6071c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lp7sn" Jul 01 16:28:44.966439 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:44.965075 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/0b3759ce-7af8-4687-9a32-0578edfe706c-etc-sysconfig\") pod \"tuned-nkd9r\" (UID: \"0b3759ce-7af8-4687-9a32-0578edfe706c\") " pod="openshift-cluster-node-tuning-operator/tuned-nkd9r" Jul 01 16:28:44.966439 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:44.965080 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/6c03a7ab-a7cc-44ed-86f7-62a750c49b71-host-slash\") pod \"ovnkube-node-5cr75\" (UID: \"6c03a7ab-a7cc-44ed-86f7-62a750c49b71\") " pod="openshift-ovn-kubernetes/ovnkube-node-5cr75" Jul 01 16:28:44.966439 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:44.965096 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/0b3759ce-7af8-4687-9a32-0578edfe706c-etc-systemd\") pod \"tuned-nkd9r\" (UID: \"0b3759ce-7af8-4687-9a32-0578edfe706c\") " pod="openshift-cluster-node-tuning-operator/tuned-nkd9r" Jul 01 16:28:44.966439 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:44.965124 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/0b3759ce-7af8-4687-9a32-0578edfe706c-lib-modules\") pod \"tuned-nkd9r\" (UID: \"0b3759ce-7af8-4687-9a32-0578edfe706c\") " pod="openshift-cluster-node-tuning-operator/tuned-nkd9r" Jul 01 16:28:44.966439 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:44.965147 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5be67da1-9165-4446-b3cb-2333db8c09cc-tmp-dir\") pod \"node-resolver-t7nrc\" (UID: \"5be67da1-9165-4446-b3cb-2333db8c09cc\") " pod="openshift-dns/node-resolver-t7nrc" Jul 01 16:28:44.966956 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:44.965169 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/1df6652f-89cc-41a1-89f7-0ed8568b6ee2-konnectivity-ca\") pod \"konnectivity-agent-cprkr\" (UID: \"1df6652f-89cc-41a1-89f7-0ed8568b6ee2\") " pod="kube-system/konnectivity-agent-cprkr" Jul 01 16:28:44.966956 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:44.965192 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/6c03a7ab-a7cc-44ed-86f7-62a750c49b71-env-overrides\") pod \"ovnkube-node-5cr75\" (UID: \"6c03a7ab-a7cc-44ed-86f7-62a750c49b71\") " pod="openshift-ovn-kubernetes/ovnkube-node-5cr75" Jul 01 16:28:44.966956 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:44.965214 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/48cdc46f-d909-463d-942b-4d300d212062-cni-binary-copy\") pod \"multus-additional-cni-plugins-gnsqt\" (UID: \"48cdc46f-d909-463d-942b-4d300d212062\") " pod="openshift-multus/multus-additional-cni-plugins-gnsqt" Jul 01 16:28:44.966956 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:44.965235 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/6c03a7ab-a7cc-44ed-86f7-62a750c49b71-ovnkube-config\") pod \"ovnkube-node-5cr75\" (UID: \"6c03a7ab-a7cc-44ed-86f7-62a750c49b71\") " pod="openshift-ovn-kubernetes/ovnkube-node-5cr75" Jul 01 16:28:44.966956 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:44.965258 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/48cdc46f-d909-463d-942b-4d300d212062-tuning-conf-dir\") pod \"multus-additional-cni-plugins-gnsqt\" (UID: \"48cdc46f-d909-463d-942b-4d300d212062\") " pod="openshift-multus/multus-additional-cni-plugins-gnsqt" Jul 01 16:28:44.966956 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:44.965261 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/b43f95ee-7eec-446c-b1e2-8cf7a0c64864-iptables-alerter-script\") pod \"iptables-alerter-xwdht\" (UID: \"b43f95ee-7eec-446c-b1e2-8cf7a0c64864\") " pod="openshift-network-operator/iptables-alerter-xwdht" Jul 01 16:28:44.966956 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:44.965280 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/76faf2eb-5dae-48e1-ba27-f1b94c85920d-host-run-netns\") pod \"multus-t7n99\" (UID: \"76faf2eb-5dae-48e1-ba27-f1b94c85920d\") " pod="openshift-multus/multus-t7n99" Jul 01 16:28:44.966956 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:44.965328 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/76faf2eb-5dae-48e1-ba27-f1b94c85920d-hostroot\") pod \"multus-t7n99\" (UID: \"76faf2eb-5dae-48e1-ba27-f1b94c85920d\") " pod="openshift-multus/multus-t7n99" Jul 01 16:28:44.966956 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:44.965351 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/9ae0a7d9-2f7b-49ba-8a15-69acd2a6071c-sys-fs\") pod \"aws-ebs-csi-driver-node-lp7sn\" (UID: \"9ae0a7d9-2f7b-49ba-8a15-69acd2a6071c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lp7sn" Jul 01 16:28:44.966956 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:44.965403 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/6c03a7ab-a7cc-44ed-86f7-62a750c49b71-host-kubelet\") pod \"ovnkube-node-5cr75\" (UID: \"6c03a7ab-a7cc-44ed-86f7-62a750c49b71\") " pod="openshift-ovn-kubernetes/ovnkube-node-5cr75" Jul 01 16:28:44.966956 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:44.965430 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/48cdc46f-d909-463d-942b-4d300d212062-tuning-conf-dir\") pod \"multus-additional-cni-plugins-gnsqt\" (UID: \"48cdc46f-d909-463d-942b-4d300d212062\") " pod="openshift-multus/multus-additional-cni-plugins-gnsqt" Jul 01 16:28:44.966956 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:44.965454 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/48cdc46f-d909-463d-942b-4d300d212062-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-gnsqt\" (UID: \"48cdc46f-d909-463d-942b-4d300d212062\") " pod="openshift-multus/multus-additional-cni-plugins-gnsqt" Jul 01 16:28:44.966956 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:44.965459 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/6c03a7ab-a7cc-44ed-86f7-62a750c49b71-run-ovn\") pod \"ovnkube-node-5cr75\" (UID: \"6c03a7ab-a7cc-44ed-86f7-62a750c49b71\") " pod="openshift-ovn-kubernetes/ovnkube-node-5cr75" Jul 01 16:28:44.966956 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:44.965471 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/6c03a7ab-a7cc-44ed-86f7-62a750c49b71-host-kubelet\") pod \"ovnkube-node-5cr75\" (UID: \"6c03a7ab-a7cc-44ed-86f7-62a750c49b71\") " pod="openshift-ovn-kubernetes/ovnkube-node-5cr75" Jul 01 16:28:44.966956 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:44.965497 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/0b3759ce-7af8-4687-9a32-0578edfe706c-host\") pod \"tuned-nkd9r\" (UID: \"0b3759ce-7af8-4687-9a32-0578edfe706c\") " pod="openshift-cluster-node-tuning-operator/tuned-nkd9r" Jul 01 16:28:44.966956 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:44.965507 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/6c03a7ab-a7cc-44ed-86f7-62a750c49b71-run-ovn\") pod \"ovnkube-node-5cr75\" (UID: \"6c03a7ab-a7cc-44ed-86f7-62a750c49b71\") " pod="openshift-ovn-kubernetes/ovnkube-node-5cr75" Jul 01 16:28:44.966956 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:44.965517 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0b3759ce-7af8-4687-9a32-0578edfe706c-tmp\") pod \"tuned-nkd9r\" (UID: \"0b3759ce-7af8-4687-9a32-0578edfe706c\") " pod="openshift-cluster-node-tuning-operator/tuned-nkd9r" Jul 01 16:28:44.967400 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:44.965580 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/6c03a7ab-a7cc-44ed-86f7-62a750c49b71-env-overrides\") pod \"ovnkube-node-5cr75\" (UID: \"6c03a7ab-a7cc-44ed-86f7-62a750c49b71\") " pod="openshift-ovn-kubernetes/ovnkube-node-5cr75" Jul 01 16:28:44.967400 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:44.965700 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/48cdc46f-d909-463d-942b-4d300d212062-cni-binary-copy\") pod \"multus-additional-cni-plugins-gnsqt\" (UID: \"48cdc46f-d909-463d-942b-4d300d212062\") " pod="openshift-multus/multus-additional-cni-plugins-gnsqt" Jul 01 16:28:44.967400 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:44.966871 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/6c03a7ab-a7cc-44ed-86f7-62a750c49b71-ovn-node-metrics-cert\") pod \"ovnkube-node-5cr75\" (UID: \"6c03a7ab-a7cc-44ed-86f7-62a750c49b71\") " pod="openshift-ovn-kubernetes/ovnkube-node-5cr75" Jul 01 16:28:44.972982 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:44.972966 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v8tj7\" (UniqueName: \"kubernetes.io/projected/8242757c-9a33-4958-a860-0679d154b28c-kube-api-access-v8tj7\") pod \"node-ca-wb2hh\" (UID: \"8242757c-9a33-4958-a860-0679d154b28c\") " pod="openshift-image-registry/node-ca-wb2hh" Jul 01 16:28:44.974695 ip-10-0-130-227 kubenswrapper[2545]: E0701 16:28:44.974680 2545 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 01 16:28:44.974778 ip-10-0-130-227 kubenswrapper[2545]: E0701 16:28:44.974702 2545 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 01 16:28:44.974778 ip-10-0-130-227 kubenswrapper[2545]: E0701 16:28:44.974731 2545 projected.go:196] Error preparing data for projected volume kube-api-access-c5bsl for pod openshift-network-diagnostics/network-check-target-kz46s: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 01 16:28:44.974879 ip-10-0-130-227 kubenswrapper[2545]: E0701 16:28:44.974796 2545 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4e4b6e11-f86f-4955-aabc-3f94ca793252-kube-api-access-c5bsl podName:4e4b6e11-f86f-4955-aabc-3f94ca793252 nodeName:}" failed. No retries permitted until 2026-07-01 16:28:45.474779659 +0000 UTC m=+4.055397158 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-c5bsl" (UniqueName: "kubernetes.io/projected/4e4b6e11-f86f-4955-aabc-3f94ca793252-kube-api-access-c5bsl") pod "network-check-target-kz46s" (UID: "4e4b6e11-f86f-4955-aabc-3f94ca793252") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 01 16:28:44.976507 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:44.976480 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ncjgt\" (UniqueName: \"kubernetes.io/projected/6c03a7ab-a7cc-44ed-86f7-62a750c49b71-kube-api-access-ncjgt\") pod \"ovnkube-node-5cr75\" (UID: \"6c03a7ab-a7cc-44ed-86f7-62a750c49b71\") " pod="openshift-ovn-kubernetes/ovnkube-node-5cr75" Jul 01 16:28:44.976673 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:44.976657 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-26qjf\" (UniqueName: \"kubernetes.io/projected/b43f95ee-7eec-446c-b1e2-8cf7a0c64864-kube-api-access-26qjf\") pod \"iptables-alerter-xwdht\" (UID: \"b43f95ee-7eec-446c-b1e2-8cf7a0c64864\") " pod="openshift-network-operator/iptables-alerter-xwdht" Jul 01 16:28:44.976966 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:44.976948 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6dj59\" (UniqueName: \"kubernetes.io/projected/48cdc46f-d909-463d-942b-4d300d212062-kube-api-access-6dj59\") pod \"multus-additional-cni-plugins-gnsqt\" (UID: \"48cdc46f-d909-463d-942b-4d300d212062\") " pod="openshift-multus/multus-additional-cni-plugins-gnsqt" Jul 01 16:28:45.066406 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:45.066382 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/76faf2eb-5dae-48e1-ba27-f1b94c85920d-multus-socket-dir-parent\") pod \"multus-t7n99\" (UID: \"76faf2eb-5dae-48e1-ba27-f1b94c85920d\") " pod="openshift-multus/multus-t7n99" Jul 01 16:28:45.066544 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:45.066416 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/be2ddffa-466a-4a37-baba-5cb9ecf07e41-metrics-certs\") pod \"network-metrics-daemon-87ztj\" (UID: \"be2ddffa-466a-4a37-baba-5cb9ecf07e41\") " pod="openshift-multus/network-metrics-daemon-87ztj" Jul 01 16:28:45.066544 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:45.066433 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-55bw6\" (UniqueName: \"kubernetes.io/projected/be2ddffa-466a-4a37-baba-5cb9ecf07e41-kube-api-access-55bw6\") pod \"network-metrics-daemon-87ztj\" (UID: \"be2ddffa-466a-4a37-baba-5cb9ecf07e41\") " pod="openshift-multus/network-metrics-daemon-87ztj" Jul 01 16:28:45.066544 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:45.066498 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/76faf2eb-5dae-48e1-ba27-f1b94c85920d-multus-socket-dir-parent\") pod \"multus-t7n99\" (UID: \"76faf2eb-5dae-48e1-ba27-f1b94c85920d\") " pod="openshift-multus/multus-t7n99" Jul 01 16:28:45.066544 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:45.066501 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0b3759ce-7af8-4687-9a32-0578edfe706c-var-lib-kubelet\") pod \"tuned-nkd9r\" (UID: \"0b3759ce-7af8-4687-9a32-0578edfe706c\") " pod="openshift-cluster-node-tuning-operator/tuned-nkd9r" Jul 01 16:28:45.066736 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:45.066553 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dm56c\" (UniqueName: \"kubernetes.io/projected/9ae0a7d9-2f7b-49ba-8a15-69acd2a6071c-kube-api-access-dm56c\") pod \"aws-ebs-csi-driver-node-lp7sn\" (UID: \"9ae0a7d9-2f7b-49ba-8a15-69acd2a6071c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lp7sn" Jul 01 16:28:45.066736 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:45.066557 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0b3759ce-7af8-4687-9a32-0578edfe706c-var-lib-kubelet\") pod \"tuned-nkd9r\" (UID: \"0b3759ce-7af8-4687-9a32-0578edfe706c\") " pod="openshift-cluster-node-tuning-operator/tuned-nkd9r" Jul 01 16:28:45.066736 ip-10-0-130-227 kubenswrapper[2545]: E0701 16:28:45.066612 2545 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 01 16:28:45.066736 ip-10-0-130-227 kubenswrapper[2545]: E0701 16:28:45.066669 2545 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/be2ddffa-466a-4a37-baba-5cb9ecf07e41-metrics-certs podName:be2ddffa-466a-4a37-baba-5cb9ecf07e41 nodeName:}" failed. No retries permitted until 2026-07-01 16:28:45.56665387 +0000 UTC m=+4.147271383 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/be2ddffa-466a-4a37-baba-5cb9ecf07e41-metrics-certs") pod "network-metrics-daemon-87ztj" (UID: "be2ddffa-466a-4a37-baba-5cb9ecf07e41") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 01 16:28:45.066736 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:45.066686 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/0b3759ce-7af8-4687-9a32-0578edfe706c-etc-sysctl-d\") pod \"tuned-nkd9r\" (UID: \"0b3759ce-7af8-4687-9a32-0578edfe706c\") " pod="openshift-cluster-node-tuning-operator/tuned-nkd9r" Jul 01 16:28:45.066736 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:45.066705 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kwmxh\" (UniqueName: \"kubernetes.io/projected/5be67da1-9165-4446-b3cb-2333db8c09cc-kube-api-access-kwmxh\") pod \"node-resolver-t7nrc\" (UID: \"5be67da1-9165-4446-b3cb-2333db8c09cc\") " pod="openshift-dns/node-resolver-t7nrc" Jul 01 16:28:45.067017 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:45.066743 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0b3759ce-7af8-4687-9a32-0578edfe706c-sys\") pod \"tuned-nkd9r\" (UID: \"0b3759ce-7af8-4687-9a32-0578edfe706c\") " pod="openshift-cluster-node-tuning-operator/tuned-nkd9r" Jul 01 16:28:45.067017 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:45.066759 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/0b3759ce-7af8-4687-9a32-0578edfe706c-etc-tuned\") pod \"tuned-nkd9r\" (UID: \"0b3759ce-7af8-4687-9a32-0578edfe706c\") " pod="openshift-cluster-node-tuning-operator/tuned-nkd9r" Jul 01 16:28:45.067017 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:45.066785 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/76faf2eb-5dae-48e1-ba27-f1b94c85920d-cnibin\") pod \"multus-t7n99\" (UID: \"76faf2eb-5dae-48e1-ba27-f1b94c85920d\") " pod="openshift-multus/multus-t7n99" Jul 01 16:28:45.067017 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:45.066808 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/9ae0a7d9-2f7b-49ba-8a15-69acd2a6071c-device-dir\") pod \"aws-ebs-csi-driver-node-lp7sn\" (UID: \"9ae0a7d9-2f7b-49ba-8a15-69acd2a6071c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lp7sn" Jul 01 16:28:45.067017 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:45.066833 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/76faf2eb-5dae-48e1-ba27-f1b94c85920d-host-run-multus-certs\") pod \"multus-t7n99\" (UID: \"76faf2eb-5dae-48e1-ba27-f1b94c85920d\") " pod="openshift-multus/multus-t7n99" Jul 01 16:28:45.067017 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:45.066844 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/0b3759ce-7af8-4687-9a32-0578edfe706c-etc-sysctl-d\") pod \"tuned-nkd9r\" (UID: \"0b3759ce-7af8-4687-9a32-0578edfe706c\") " pod="openshift-cluster-node-tuning-operator/tuned-nkd9r" Jul 01 16:28:45.067017 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:45.066861 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/9ae0a7d9-2f7b-49ba-8a15-69acd2a6071c-registration-dir\") pod \"aws-ebs-csi-driver-node-lp7sn\" (UID: \"9ae0a7d9-2f7b-49ba-8a15-69acd2a6071c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lp7sn" Jul 01 16:28:45.067017 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:45.066876 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/76faf2eb-5dae-48e1-ba27-f1b94c85920d-cnibin\") pod \"multus-t7n99\" (UID: \"76faf2eb-5dae-48e1-ba27-f1b94c85920d\") " pod="openshift-multus/multus-t7n99" Jul 01 16:28:45.067017 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:45.066891 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/76faf2eb-5dae-48e1-ba27-f1b94c85920d-cni-binary-copy\") pod \"multus-t7n99\" (UID: \"76faf2eb-5dae-48e1-ba27-f1b94c85920d\") " pod="openshift-multus/multus-t7n99" Jul 01 16:28:45.067017 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:45.066844 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0b3759ce-7af8-4687-9a32-0578edfe706c-sys\") pod \"tuned-nkd9r\" (UID: \"0b3759ce-7af8-4687-9a32-0578edfe706c\") " pod="openshift-cluster-node-tuning-operator/tuned-nkd9r" Jul 01 16:28:45.067017 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:45.066913 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/76faf2eb-5dae-48e1-ba27-f1b94c85920d-host-run-multus-certs\") pod \"multus-t7n99\" (UID: \"76faf2eb-5dae-48e1-ba27-f1b94c85920d\") " pod="openshift-multus/multus-t7n99" Jul 01 16:28:45.067017 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:45.066920 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/9ae0a7d9-2f7b-49ba-8a15-69acd2a6071c-registration-dir\") pod \"aws-ebs-csi-driver-node-lp7sn\" (UID: \"9ae0a7d9-2f7b-49ba-8a15-69acd2a6071c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lp7sn" Jul 01 16:28:45.067017 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:45.066922 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/76faf2eb-5dae-48e1-ba27-f1b94c85920d-host-var-lib-cni-bin\") pod \"multus-t7n99\" (UID: \"76faf2eb-5dae-48e1-ba27-f1b94c85920d\") " pod="openshift-multus/multus-t7n99" Jul 01 16:28:45.067017 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:45.066956 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/76faf2eb-5dae-48e1-ba27-f1b94c85920d-etc-kubernetes\") pod \"multus-t7n99\" (UID: \"76faf2eb-5dae-48e1-ba27-f1b94c85920d\") " pod="openshift-multus/multus-t7n99" Jul 01 16:28:45.067017 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:45.066969 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/9ae0a7d9-2f7b-49ba-8a15-69acd2a6071c-device-dir\") pod \"aws-ebs-csi-driver-node-lp7sn\" (UID: \"9ae0a7d9-2f7b-49ba-8a15-69acd2a6071c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lp7sn" Jul 01 16:28:45.067017 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:45.066981 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/76faf2eb-5dae-48e1-ba27-f1b94c85920d-host-var-lib-kubelet\") pod \"multus-t7n99\" (UID: \"76faf2eb-5dae-48e1-ba27-f1b94c85920d\") " pod="openshift-multus/multus-t7n99" Jul 01 16:28:45.067017 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:45.067008 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/76faf2eb-5dae-48e1-ba27-f1b94c85920d-etc-kubernetes\") pod \"multus-t7n99\" (UID: \"76faf2eb-5dae-48e1-ba27-f1b94c85920d\") " pod="openshift-multus/multus-t7n99" Jul 01 16:28:45.067017 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:45.067016 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/76faf2eb-5dae-48e1-ba27-f1b94c85920d-multus-conf-dir\") pod \"multus-t7n99\" (UID: \"76faf2eb-5dae-48e1-ba27-f1b94c85920d\") " pod="openshift-multus/multus-t7n99" Jul 01 16:28:45.067784 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:45.067014 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/76faf2eb-5dae-48e1-ba27-f1b94c85920d-host-var-lib-cni-bin\") pod \"multus-t7n99\" (UID: \"76faf2eb-5dae-48e1-ba27-f1b94c85920d\") " pod="openshift-multus/multus-t7n99" Jul 01 16:28:45.067784 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:45.067028 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/76faf2eb-5dae-48e1-ba27-f1b94c85920d-host-var-lib-kubelet\") pod \"multus-t7n99\" (UID: \"76faf2eb-5dae-48e1-ba27-f1b94c85920d\") " pod="openshift-multus/multus-t7n99" Jul 01 16:28:45.067784 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:45.067060 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/0b3759ce-7af8-4687-9a32-0578edfe706c-etc-modprobe-d\") pod \"tuned-nkd9r\" (UID: \"0b3759ce-7af8-4687-9a32-0578edfe706c\") " pod="openshift-cluster-node-tuning-operator/tuned-nkd9r" Jul 01 16:28:45.067784 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:45.067099 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/76faf2eb-5dae-48e1-ba27-f1b94c85920d-multus-cni-dir\") pod \"multus-t7n99\" (UID: \"76faf2eb-5dae-48e1-ba27-f1b94c85920d\") " pod="openshift-multus/multus-t7n99" Jul 01 16:28:45.067784 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:45.067064 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/76faf2eb-5dae-48e1-ba27-f1b94c85920d-multus-conf-dir\") pod \"multus-t7n99\" (UID: \"76faf2eb-5dae-48e1-ba27-f1b94c85920d\") " pod="openshift-multus/multus-t7n99" Jul 01 16:28:45.067784 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:45.067121 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/0b3759ce-7af8-4687-9a32-0578edfe706c-etc-sysctl-conf\") pod \"tuned-nkd9r\" (UID: \"0b3759ce-7af8-4687-9a32-0578edfe706c\") " pod="openshift-cluster-node-tuning-operator/tuned-nkd9r" Jul 01 16:28:45.067784 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:45.067156 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/76faf2eb-5dae-48e1-ba27-f1b94c85920d-multus-daemon-config\") pod \"multus-t7n99\" (UID: \"76faf2eb-5dae-48e1-ba27-f1b94c85920d\") " pod="openshift-multus/multus-t7n99" Jul 01 16:28:45.067784 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:45.067169 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/0b3759ce-7af8-4687-9a32-0578edfe706c-etc-modprobe-d\") pod \"tuned-nkd9r\" (UID: \"0b3759ce-7af8-4687-9a32-0578edfe706c\") " pod="openshift-cluster-node-tuning-operator/tuned-nkd9r" Jul 01 16:28:45.067784 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:45.067170 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/76faf2eb-5dae-48e1-ba27-f1b94c85920d-multus-cni-dir\") pod \"multus-t7n99\" (UID: \"76faf2eb-5dae-48e1-ba27-f1b94c85920d\") " pod="openshift-multus/multus-t7n99" Jul 01 16:28:45.067784 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:45.067181 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/9ae0a7d9-2f7b-49ba-8a15-69acd2a6071c-socket-dir\") pod \"aws-ebs-csi-driver-node-lp7sn\" (UID: \"9ae0a7d9-2f7b-49ba-8a15-69acd2a6071c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lp7sn" Jul 01 16:28:45.067784 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:45.067229 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/0b3759ce-7af8-4687-9a32-0578edfe706c-etc-sysconfig\") pod \"tuned-nkd9r\" (UID: \"0b3759ce-7af8-4687-9a32-0578edfe706c\") " pod="openshift-cluster-node-tuning-operator/tuned-nkd9r" Jul 01 16:28:45.067784 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:45.067253 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/0b3759ce-7af8-4687-9a32-0578edfe706c-etc-systemd\") pod \"tuned-nkd9r\" (UID: \"0b3759ce-7af8-4687-9a32-0578edfe706c\") " pod="openshift-cluster-node-tuning-operator/tuned-nkd9r" Jul 01 16:28:45.067784 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:45.067273 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/0b3759ce-7af8-4687-9a32-0578edfe706c-etc-sysctl-conf\") pod \"tuned-nkd9r\" (UID: \"0b3759ce-7af8-4687-9a32-0578edfe706c\") " pod="openshift-cluster-node-tuning-operator/tuned-nkd9r" Jul 01 16:28:45.067784 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:45.067276 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/0b3759ce-7af8-4687-9a32-0578edfe706c-lib-modules\") pod \"tuned-nkd9r\" (UID: \"0b3759ce-7af8-4687-9a32-0578edfe706c\") " pod="openshift-cluster-node-tuning-operator/tuned-nkd9r" Jul 01 16:28:45.067784 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:45.067308 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5be67da1-9165-4446-b3cb-2333db8c09cc-tmp-dir\") pod \"node-resolver-t7nrc\" (UID: \"5be67da1-9165-4446-b3cb-2333db8c09cc\") " pod="openshift-dns/node-resolver-t7nrc" Jul 01 16:28:45.067784 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:45.067325 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/1df6652f-89cc-41a1-89f7-0ed8568b6ee2-konnectivity-ca\") pod \"konnectivity-agent-cprkr\" (UID: \"1df6652f-89cc-41a1-89f7-0ed8568b6ee2\") " pod="kube-system/konnectivity-agent-cprkr" Jul 01 16:28:45.067784 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:45.067326 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/0b3759ce-7af8-4687-9a32-0578edfe706c-etc-sysconfig\") pod \"tuned-nkd9r\" (UID: \"0b3759ce-7af8-4687-9a32-0578edfe706c\") " pod="openshift-cluster-node-tuning-operator/tuned-nkd9r" Jul 01 16:28:45.067784 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:45.067342 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/76faf2eb-5dae-48e1-ba27-f1b94c85920d-host-run-netns\") pod \"multus-t7n99\" (UID: \"76faf2eb-5dae-48e1-ba27-f1b94c85920d\") " pod="openshift-multus/multus-t7n99" Jul 01 16:28:45.068462 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:45.067361 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/76faf2eb-5dae-48e1-ba27-f1b94c85920d-hostroot\") pod \"multus-t7n99\" (UID: \"76faf2eb-5dae-48e1-ba27-f1b94c85920d\") " pod="openshift-multus/multus-t7n99" Jul 01 16:28:45.068462 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:45.067373 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/0b3759ce-7af8-4687-9a32-0578edfe706c-lib-modules\") pod \"tuned-nkd9r\" (UID: \"0b3759ce-7af8-4687-9a32-0578edfe706c\") " pod="openshift-cluster-node-tuning-operator/tuned-nkd9r" Jul 01 16:28:45.068462 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:45.067378 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/76faf2eb-5dae-48e1-ba27-f1b94c85920d-cni-binary-copy\") pod \"multus-t7n99\" (UID: \"76faf2eb-5dae-48e1-ba27-f1b94c85920d\") " pod="openshift-multus/multus-t7n99" Jul 01 16:28:45.068462 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:45.067400 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/76faf2eb-5dae-48e1-ba27-f1b94c85920d-host-run-netns\") pod \"multus-t7n99\" (UID: \"76faf2eb-5dae-48e1-ba27-f1b94c85920d\") " pod="openshift-multus/multus-t7n99" Jul 01 16:28:45.068462 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:45.067421 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/76faf2eb-5dae-48e1-ba27-f1b94c85920d-hostroot\") pod \"multus-t7n99\" (UID: \"76faf2eb-5dae-48e1-ba27-f1b94c85920d\") " pod="openshift-multus/multus-t7n99" Jul 01 16:28:45.068462 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:45.067425 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/0b3759ce-7af8-4687-9a32-0578edfe706c-etc-systemd\") pod \"tuned-nkd9r\" (UID: \"0b3759ce-7af8-4687-9a32-0578edfe706c\") " pod="openshift-cluster-node-tuning-operator/tuned-nkd9r" Jul 01 16:28:45.068462 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:45.067378 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/9ae0a7d9-2f7b-49ba-8a15-69acd2a6071c-sys-fs\") pod \"aws-ebs-csi-driver-node-lp7sn\" (UID: \"9ae0a7d9-2f7b-49ba-8a15-69acd2a6071c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lp7sn" Jul 01 16:28:45.068462 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:45.067440 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/9ae0a7d9-2f7b-49ba-8a15-69acd2a6071c-sys-fs\") pod \"aws-ebs-csi-driver-node-lp7sn\" (UID: \"9ae0a7d9-2f7b-49ba-8a15-69acd2a6071c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lp7sn" Jul 01 16:28:45.068462 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:45.067464 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/0b3759ce-7af8-4687-9a32-0578edfe706c-host\") pod \"tuned-nkd9r\" (UID: \"0b3759ce-7af8-4687-9a32-0578edfe706c\") " pod="openshift-cluster-node-tuning-operator/tuned-nkd9r" Jul 01 16:28:45.068462 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:45.067489 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0b3759ce-7af8-4687-9a32-0578edfe706c-tmp\") pod \"tuned-nkd9r\" (UID: \"0b3759ce-7af8-4687-9a32-0578edfe706c\") " pod="openshift-cluster-node-tuning-operator/tuned-nkd9r" Jul 01 16:28:45.068462 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:45.067535 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kmjz8\" (UniqueName: \"kubernetes.io/projected/0b3759ce-7af8-4687-9a32-0578edfe706c-kube-api-access-kmjz8\") pod \"tuned-nkd9r\" (UID: \"0b3759ce-7af8-4687-9a32-0578edfe706c\") " pod="openshift-cluster-node-tuning-operator/tuned-nkd9r" Jul 01 16:28:45.068462 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:45.067564 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/76faf2eb-5dae-48e1-ba27-f1b94c85920d-host-var-lib-cni-multus\") pod \"multus-t7n99\" (UID: \"76faf2eb-5dae-48e1-ba27-f1b94c85920d\") " pod="openshift-multus/multus-t7n99" Jul 01 16:28:45.068462 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:45.067587 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qldwh\" (UniqueName: \"kubernetes.io/projected/76faf2eb-5dae-48e1-ba27-f1b94c85920d-kube-api-access-qldwh\") pod \"multus-t7n99\" (UID: \"76faf2eb-5dae-48e1-ba27-f1b94c85920d\") " pod="openshift-multus/multus-t7n99" Jul 01 16:28:45.068462 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:45.067616 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/0b3759ce-7af8-4687-9a32-0578edfe706c-run\") pod \"tuned-nkd9r\" (UID: \"0b3759ce-7af8-4687-9a32-0578edfe706c\") " pod="openshift-cluster-node-tuning-operator/tuned-nkd9r" Jul 01 16:28:45.068462 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:45.067643 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/5be67da1-9165-4446-b3cb-2333db8c09cc-hosts-file\") pod \"node-resolver-t7nrc\" (UID: \"5be67da1-9165-4446-b3cb-2333db8c09cc\") " pod="openshift-dns/node-resolver-t7nrc" Jul 01 16:28:45.068462 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:45.067643 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/76faf2eb-5dae-48e1-ba27-f1b94c85920d-multus-daemon-config\") pod \"multus-t7n99\" (UID: \"76faf2eb-5dae-48e1-ba27-f1b94c85920d\") " pod="openshift-multus/multus-t7n99" Jul 01 16:28:45.068462 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:45.067651 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5be67da1-9165-4446-b3cb-2333db8c09cc-tmp-dir\") pod \"node-resolver-t7nrc\" (UID: \"5be67da1-9165-4446-b3cb-2333db8c09cc\") " pod="openshift-dns/node-resolver-t7nrc" Jul 01 16:28:45.068462 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:45.067666 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/1df6652f-89cc-41a1-89f7-0ed8568b6ee2-agent-certs\") pod \"konnectivity-agent-cprkr\" (UID: \"1df6652f-89cc-41a1-89f7-0ed8568b6ee2\") " pod="kube-system/konnectivity-agent-cprkr" Jul 01 16:28:45.068981 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:45.067539 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/0b3759ce-7af8-4687-9a32-0578edfe706c-host\") pod \"tuned-nkd9r\" (UID: \"0b3759ce-7af8-4687-9a32-0578edfe706c\") " pod="openshift-cluster-node-tuning-operator/tuned-nkd9r" Jul 01 16:28:45.068981 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:45.067692 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/9ae0a7d9-2f7b-49ba-8a15-69acd2a6071c-kubelet-dir\") pod \"aws-ebs-csi-driver-node-lp7sn\" (UID: \"9ae0a7d9-2f7b-49ba-8a15-69acd2a6071c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lp7sn" Jul 01 16:28:45.068981 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:45.067697 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/76faf2eb-5dae-48e1-ba27-f1b94c85920d-host-var-lib-cni-multus\") pod \"multus-t7n99\" (UID: \"76faf2eb-5dae-48e1-ba27-f1b94c85920d\") " pod="openshift-multus/multus-t7n99" Jul 01 16:28:45.068981 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:45.067509 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/9ae0a7d9-2f7b-49ba-8a15-69acd2a6071c-socket-dir\") pod \"aws-ebs-csi-driver-node-lp7sn\" (UID: \"9ae0a7d9-2f7b-49ba-8a15-69acd2a6071c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lp7sn" Jul 01 16:28:45.068981 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:45.067737 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/9ae0a7d9-2f7b-49ba-8a15-69acd2a6071c-etc-selinux\") pod \"aws-ebs-csi-driver-node-lp7sn\" (UID: \"9ae0a7d9-2f7b-49ba-8a15-69acd2a6071c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lp7sn" Jul 01 16:28:45.068981 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:45.067760 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0b3759ce-7af8-4687-9a32-0578edfe706c-etc-kubernetes\") pod \"tuned-nkd9r\" (UID: \"0b3759ce-7af8-4687-9a32-0578edfe706c\") " pod="openshift-cluster-node-tuning-operator/tuned-nkd9r" Jul 01 16:28:45.068981 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:45.067784 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/76faf2eb-5dae-48e1-ba27-f1b94c85920d-system-cni-dir\") pod \"multus-t7n99\" (UID: \"76faf2eb-5dae-48e1-ba27-f1b94c85920d\") " pod="openshift-multus/multus-t7n99" Jul 01 16:28:45.068981 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:45.067787 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/5be67da1-9165-4446-b3cb-2333db8c09cc-hosts-file\") pod \"node-resolver-t7nrc\" (UID: \"5be67da1-9165-4446-b3cb-2333db8c09cc\") " pod="openshift-dns/node-resolver-t7nrc" Jul 01 16:28:45.068981 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:45.067806 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/76faf2eb-5dae-48e1-ba27-f1b94c85920d-host-run-k8s-cni-cncf-io\") pod \"multus-t7n99\" (UID: \"76faf2eb-5dae-48e1-ba27-f1b94c85920d\") " pod="openshift-multus/multus-t7n99" Jul 01 16:28:45.068981 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:45.067850 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/76faf2eb-5dae-48e1-ba27-f1b94c85920d-os-release\") pod \"multus-t7n99\" (UID: \"76faf2eb-5dae-48e1-ba27-f1b94c85920d\") " pod="openshift-multus/multus-t7n99" Jul 01 16:28:45.068981 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:45.067875 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/9ae0a7d9-2f7b-49ba-8a15-69acd2a6071c-kubelet-dir\") pod \"aws-ebs-csi-driver-node-lp7sn\" (UID: \"9ae0a7d9-2f7b-49ba-8a15-69acd2a6071c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lp7sn" Jul 01 16:28:45.068981 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:45.067959 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/9ae0a7d9-2f7b-49ba-8a15-69acd2a6071c-etc-selinux\") pod \"aws-ebs-csi-driver-node-lp7sn\" (UID: \"9ae0a7d9-2f7b-49ba-8a15-69acd2a6071c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lp7sn" Jul 01 16:28:45.068981 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:45.067978 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/1df6652f-89cc-41a1-89f7-0ed8568b6ee2-konnectivity-ca\") pod \"konnectivity-agent-cprkr\" (UID: \"1df6652f-89cc-41a1-89f7-0ed8568b6ee2\") " pod="kube-system/konnectivity-agent-cprkr" Jul 01 16:28:45.068981 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:45.067991 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/76faf2eb-5dae-48e1-ba27-f1b94c85920d-os-release\") pod \"multus-t7n99\" (UID: \"76faf2eb-5dae-48e1-ba27-f1b94c85920d\") " pod="openshift-multus/multus-t7n99" Jul 01 16:28:45.068981 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:45.067994 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/0b3759ce-7af8-4687-9a32-0578edfe706c-run\") pod \"tuned-nkd9r\" (UID: \"0b3759ce-7af8-4687-9a32-0578edfe706c\") " pod="openshift-cluster-node-tuning-operator/tuned-nkd9r" Jul 01 16:28:45.068981 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:45.067994 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/76faf2eb-5dae-48e1-ba27-f1b94c85920d-host-run-k8s-cni-cncf-io\") pod \"multus-t7n99\" (UID: \"76faf2eb-5dae-48e1-ba27-f1b94c85920d\") " pod="openshift-multus/multus-t7n99" Jul 01 16:28:45.068981 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:45.068032 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/76faf2eb-5dae-48e1-ba27-f1b94c85920d-system-cni-dir\") pod \"multus-t7n99\" (UID: \"76faf2eb-5dae-48e1-ba27-f1b94c85920d\") " pod="openshift-multus/multus-t7n99" Jul 01 16:28:45.068981 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:45.068045 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0b3759ce-7af8-4687-9a32-0578edfe706c-etc-kubernetes\") pod \"tuned-nkd9r\" (UID: \"0b3759ce-7af8-4687-9a32-0578edfe706c\") " pod="openshift-cluster-node-tuning-operator/tuned-nkd9r" Jul 01 16:28:45.069586 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:45.069117 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/0b3759ce-7af8-4687-9a32-0578edfe706c-etc-tuned\") pod \"tuned-nkd9r\" (UID: \"0b3759ce-7af8-4687-9a32-0578edfe706c\") " pod="openshift-cluster-node-tuning-operator/tuned-nkd9r" Jul 01 16:28:45.069586 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:45.069511 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0b3759ce-7af8-4687-9a32-0578edfe706c-tmp\") pod \"tuned-nkd9r\" (UID: \"0b3759ce-7af8-4687-9a32-0578edfe706c\") " pod="openshift-cluster-node-tuning-operator/tuned-nkd9r" Jul 01 16:28:45.069762 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:45.069747 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/1df6652f-89cc-41a1-89f7-0ed8568b6ee2-agent-certs\") pod \"konnectivity-agent-cprkr\" (UID: \"1df6652f-89cc-41a1-89f7-0ed8568b6ee2\") " pod="kube-system/konnectivity-agent-cprkr" Jul 01 16:28:45.078865 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:45.078836 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dm56c\" (UniqueName: \"kubernetes.io/projected/9ae0a7d9-2f7b-49ba-8a15-69acd2a6071c-kube-api-access-dm56c\") pod \"aws-ebs-csi-driver-node-lp7sn\" (UID: \"9ae0a7d9-2f7b-49ba-8a15-69acd2a6071c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lp7sn" Jul 01 16:28:45.079081 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:45.079064 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-55bw6\" (UniqueName: \"kubernetes.io/projected/be2ddffa-466a-4a37-baba-5cb9ecf07e41-kube-api-access-55bw6\") pod \"network-metrics-daemon-87ztj\" (UID: \"be2ddffa-466a-4a37-baba-5cb9ecf07e41\") " pod="openshift-multus/network-metrics-daemon-87ztj" Jul 01 16:28:45.079148 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:45.079066 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kmjz8\" (UniqueName: \"kubernetes.io/projected/0b3759ce-7af8-4687-9a32-0578edfe706c-kube-api-access-kmjz8\") pod \"tuned-nkd9r\" (UID: \"0b3759ce-7af8-4687-9a32-0578edfe706c\") " pod="openshift-cluster-node-tuning-operator/tuned-nkd9r" Jul 01 16:28:45.079148 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:45.079064 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kwmxh\" (UniqueName: \"kubernetes.io/projected/5be67da1-9165-4446-b3cb-2333db8c09cc-kube-api-access-kwmxh\") pod \"node-resolver-t7nrc\" (UID: \"5be67da1-9165-4446-b3cb-2333db8c09cc\") " pod="openshift-dns/node-resolver-t7nrc" Jul 01 16:28:45.086286 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:45.086245 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qldwh\" (UniqueName: \"kubernetes.io/projected/76faf2eb-5dae-48e1-ba27-f1b94c85920d-kube-api-access-qldwh\") pod \"multus-t7n99\" (UID: \"76faf2eb-5dae-48e1-ba27-f1b94c85920d\") " pod="openshift-multus/multus-t7n99" Jul 01 16:28:45.157239 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:45.157218 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-wb2hh" Jul 01 16:28:45.163319 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:45.163297 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-gnsqt" Jul 01 16:28:45.164858 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:45.164827 2545 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8242757c_9a33_4958_a860_0679d154b28c.slice/crio-31c512806bbdce5db2515fa4d4e8597ed3a3ca88ba1a9bf6f6d55e7616aff0b6 WatchSource:0}: Error finding container 31c512806bbdce5db2515fa4d4e8597ed3a3ca88ba1a9bf6f6d55e7616aff0b6: Status 404 returned error can't find the container with id 31c512806bbdce5db2515fa4d4e8597ed3a3ca88ba1a9bf6f6d55e7616aff0b6 Jul 01 16:28:45.171770 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:45.171750 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-xwdht" Jul 01 16:28:45.173615 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:45.173593 2545 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod48cdc46f_d909_463d_942b_4d300d212062.slice/crio-d1d9dfd401785169a57ec01b57e6e8fdc604c2e0e022db4364881b54b9ae474c WatchSource:0}: Error finding container d1d9dfd401785169a57ec01b57e6e8fdc604c2e0e022db4364881b54b9ae474c: Status 404 returned error can't find the container with id d1d9dfd401785169a57ec01b57e6e8fdc604c2e0e022db4364881b54b9ae474c Jul 01 16:28:45.174873 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:45.174727 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-5cr75" Jul 01 16:28:45.179644 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:45.179624 2545 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb43f95ee_7eec_446c_b1e2_8cf7a0c64864.slice/crio-c95a777e23257413ee269c83af5883b123a06bea006f06cb4e1009238c986b0c WatchSource:0}: Error finding container c95a777e23257413ee269c83af5883b123a06bea006f06cb4e1009238c986b0c: Status 404 returned error can't find the container with id c95a777e23257413ee269c83af5883b123a06bea006f06cb4e1009238c986b0c Jul 01 16:28:45.180650 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:45.180506 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-t7nrc" Jul 01 16:28:45.185250 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:45.185225 2545 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6c03a7ab_a7cc_44ed_86f7_62a750c49b71.slice/crio-ed8bfe07f49a8fc65afb06d606600c6d6353d93b87a5f13cbe7ed6fe26a44398 WatchSource:0}: Error finding container ed8bfe07f49a8fc65afb06d606600c6d6353d93b87a5f13cbe7ed6fe26a44398: Status 404 returned error can't find the container with id ed8bfe07f49a8fc65afb06d606600c6d6353d93b87a5f13cbe7ed6fe26a44398 Jul 01 16:28:45.185346 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:45.185289 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lp7sn" Jul 01 16:28:45.191066 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:45.190918 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-nkd9r" Jul 01 16:28:45.195730 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:28:45.193693 2545 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5be67da1_9165_4446_b3cb_2333db8c09cc.slice/crio-eaaaa2d4445d9c0867c49b0565f5fed6dd7e04c510d3f28d0ab483f1f35b3498 WatchSource:0}: Error finding container eaaaa2d4445d9c0867c49b0565f5fed6dd7e04c510d3f28d0ab483f1f35b3498: Status 404 returned error can't find the container with id eaaaa2d4445d9c0867c49b0565f5fed6dd7e04c510d3f28d0ab483f1f35b3498 Jul 01 16:28:45.196598 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:45.196576 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-cprkr" Jul 01 16:28:45.200626 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:45.200611 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-t7n99" Jul 01 16:28:45.236067 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:45.235893 2545 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 01 16:28:45.322856 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:45.322829 2545 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 01 16:28:45.471546 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:45.471474 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/779fced8-cf7c-49a7-a236-c10a83bbd53c-original-pull-secret\") pod \"global-pull-secret-syncer-npttx\" (UID: \"779fced8-cf7c-49a7-a236-c10a83bbd53c\") " pod="kube-system/global-pull-secret-syncer-npttx" Jul 01 16:28:45.471704 ip-10-0-130-227 kubenswrapper[2545]: E0701 16:28:45.471613 2545 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 01 16:28:45.471704 ip-10-0-130-227 kubenswrapper[2545]: E0701 16:28:45.471676 2545 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/779fced8-cf7c-49a7-a236-c10a83bbd53c-original-pull-secret podName:779fced8-cf7c-49a7-a236-c10a83bbd53c nodeName:}" failed. No retries permitted until 2026-07-01 16:28:46.471658043 +0000 UTC m=+5.052275539 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/779fced8-cf7c-49a7-a236-c10a83bbd53c-original-pull-secret") pod "global-pull-secret-syncer-npttx" (UID: "779fced8-cf7c-49a7-a236-c10a83bbd53c") : object "kube-system"/"original-pull-secret" not registered Jul 01 16:28:45.572355 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:45.572322 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/be2ddffa-466a-4a37-baba-5cb9ecf07e41-metrics-certs\") pod \"network-metrics-daemon-87ztj\" (UID: \"be2ddffa-466a-4a37-baba-5cb9ecf07e41\") " pod="openshift-multus/network-metrics-daemon-87ztj" Jul 01 16:28:45.572522 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:45.572400 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c5bsl\" (UniqueName: \"kubernetes.io/projected/4e4b6e11-f86f-4955-aabc-3f94ca793252-kube-api-access-c5bsl\") pod \"network-check-target-kz46s\" (UID: \"4e4b6e11-f86f-4955-aabc-3f94ca793252\") " pod="openshift-network-diagnostics/network-check-target-kz46s" Jul 01 16:28:45.572522 ip-10-0-130-227 kubenswrapper[2545]: E0701 16:28:45.572478 2545 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 01 16:28:45.572642 ip-10-0-130-227 kubenswrapper[2545]: E0701 16:28:45.572525 2545 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 01 16:28:45.572642 ip-10-0-130-227 kubenswrapper[2545]: E0701 16:28:45.572542 2545 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 01 16:28:45.572642 ip-10-0-130-227 kubenswrapper[2545]: E0701 16:28:45.572554 2545 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/be2ddffa-466a-4a37-baba-5cb9ecf07e41-metrics-certs podName:be2ddffa-466a-4a37-baba-5cb9ecf07e41 nodeName:}" failed. No retries permitted until 2026-07-01 16:28:46.572531986 +0000 UTC m=+5.153149468 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/be2ddffa-466a-4a37-baba-5cb9ecf07e41-metrics-certs") pod "network-metrics-daemon-87ztj" (UID: "be2ddffa-466a-4a37-baba-5cb9ecf07e41") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 01 16:28:45.572642 ip-10-0-130-227 kubenswrapper[2545]: E0701 16:28:45.572556 2545 projected.go:196] Error preparing data for projected volume kube-api-access-c5bsl for pod openshift-network-diagnostics/network-check-target-kz46s: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 01 16:28:45.572642 ip-10-0-130-227 kubenswrapper[2545]: E0701 16:28:45.572599 2545 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4e4b6e11-f86f-4955-aabc-3f94ca793252-kube-api-access-c5bsl podName:4e4b6e11-f86f-4955-aabc-3f94ca793252 nodeName:}" failed. No retries permitted until 2026-07-01 16:28:46.572589411 +0000 UTC m=+5.153206897 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-c5bsl" (UniqueName: "kubernetes.io/projected/4e4b6e11-f86f-4955-aabc-3f94ca793252-kube-api-access-c5bsl") pod "network-check-target-kz46s" (UID: "4e4b6e11-f86f-4955-aabc-3f94ca793252") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 01 16:28:45.575471 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:45.575445 2545 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 01 16:28:45.890808 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:45.890673 2545 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-06-30 16:23:44 +0000 UTC" deadline="2028-01-06 15:44:37.240788808 +0000 UTC" Jul 01 16:28:45.890808 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:45.890751 2545 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13295h15m51.350041794s" Jul 01 16:28:45.937258 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:45.937227 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-cprkr" event={"ID":"1df6652f-89cc-41a1-89f7-0ed8568b6ee2","Type":"ContainerStarted","Data":"919ae76dff8065893198a7b22e76734d7706bc2a76657c5d80f666a73702027f"} Jul 01 16:28:45.938248 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:45.938220 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-nkd9r" event={"ID":"0b3759ce-7af8-4687-9a32-0578edfe706c","Type":"ContainerStarted","Data":"d07239a8e90a79fd5c7408a8c69e1f514d7b60ff43391a459aa0d06247d71f23"} Jul 01 16:28:45.939289 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:45.939264 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-xwdht" event={"ID":"b43f95ee-7eec-446c-b1e2-8cf7a0c64864","Type":"ContainerStarted","Data":"c95a777e23257413ee269c83af5883b123a06bea006f06cb4e1009238c986b0c"} Jul 01 16:28:45.940252 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:45.940228 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-gnsqt" event={"ID":"48cdc46f-d909-463d-942b-4d300d212062","Type":"ContainerStarted","Data":"d1d9dfd401785169a57ec01b57e6e8fdc604c2e0e022db4364881b54b9ae474c"} Jul 01 16:28:45.941243 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:45.941220 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-t7n99" event={"ID":"76faf2eb-5dae-48e1-ba27-f1b94c85920d","Type":"ContainerStarted","Data":"2969abb500ba3a840fa1cbfc8348b116d6f82d17ae4d7f3a14525ada159a2f7d"} Jul 01 16:28:45.942183 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:45.942162 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lp7sn" event={"ID":"9ae0a7d9-2f7b-49ba-8a15-69acd2a6071c","Type":"ContainerStarted","Data":"74886aea576780c363831fe45b0bc16450f46a365b97fdb92ee3d6d6705dcfc1"} Jul 01 16:28:45.943164 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:45.943144 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-t7nrc" event={"ID":"5be67da1-9165-4446-b3cb-2333db8c09cc","Type":"ContainerStarted","Data":"eaaaa2d4445d9c0867c49b0565f5fed6dd7e04c510d3f28d0ab483f1f35b3498"} Jul 01 16:28:45.945353 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:45.944332 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5cr75" event={"ID":"6c03a7ab-a7cc-44ed-86f7-62a750c49b71","Type":"ContainerStarted","Data":"ed8bfe07f49a8fc65afb06d606600c6d6353d93b87a5f13cbe7ed6fe26a44398"} Jul 01 16:28:45.945353 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:45.945341 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-wb2hh" event={"ID":"8242757c-9a33-4958-a860-0679d154b28c","Type":"ContainerStarted","Data":"31c512806bbdce5db2515fa4d4e8597ed3a3ca88ba1a9bf6f6d55e7616aff0b6"} Jul 01 16:28:46.479134 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:46.479102 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/779fced8-cf7c-49a7-a236-c10a83bbd53c-original-pull-secret\") pod \"global-pull-secret-syncer-npttx\" (UID: \"779fced8-cf7c-49a7-a236-c10a83bbd53c\") " pod="kube-system/global-pull-secret-syncer-npttx" Jul 01 16:28:46.479303 ip-10-0-130-227 kubenswrapper[2545]: E0701 16:28:46.479254 2545 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 01 16:28:46.479359 ip-10-0-130-227 kubenswrapper[2545]: E0701 16:28:46.479317 2545 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/779fced8-cf7c-49a7-a236-c10a83bbd53c-original-pull-secret podName:779fced8-cf7c-49a7-a236-c10a83bbd53c nodeName:}" failed. No retries permitted until 2026-07-01 16:28:48.479296457 +0000 UTC m=+7.059913962 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/779fced8-cf7c-49a7-a236-c10a83bbd53c-original-pull-secret") pod "global-pull-secret-syncer-npttx" (UID: "779fced8-cf7c-49a7-a236-c10a83bbd53c") : object "kube-system"/"original-pull-secret" not registered Jul 01 16:28:46.579834 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:46.579801 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/be2ddffa-466a-4a37-baba-5cb9ecf07e41-metrics-certs\") pod \"network-metrics-daemon-87ztj\" (UID: \"be2ddffa-466a-4a37-baba-5cb9ecf07e41\") " pod="openshift-multus/network-metrics-daemon-87ztj" Jul 01 16:28:46.579981 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:46.579875 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c5bsl\" (UniqueName: \"kubernetes.io/projected/4e4b6e11-f86f-4955-aabc-3f94ca793252-kube-api-access-c5bsl\") pod \"network-check-target-kz46s\" (UID: \"4e4b6e11-f86f-4955-aabc-3f94ca793252\") " pod="openshift-network-diagnostics/network-check-target-kz46s" Jul 01 16:28:46.579981 ip-10-0-130-227 kubenswrapper[2545]: E0701 16:28:46.579963 2545 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 01 16:28:46.580102 ip-10-0-130-227 kubenswrapper[2545]: E0701 16:28:46.580006 2545 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 01 16:28:46.580102 ip-10-0-130-227 kubenswrapper[2545]: E0701 16:28:46.580024 2545 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 01 16:28:46.580102 ip-10-0-130-227 kubenswrapper[2545]: E0701 16:28:46.580038 2545 projected.go:196] Error preparing data for projected volume kube-api-access-c5bsl for pod openshift-network-diagnostics/network-check-target-kz46s: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 01 16:28:46.580102 ip-10-0-130-227 kubenswrapper[2545]: E0701 16:28:46.580039 2545 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/be2ddffa-466a-4a37-baba-5cb9ecf07e41-metrics-certs podName:be2ddffa-466a-4a37-baba-5cb9ecf07e41 nodeName:}" failed. No retries permitted until 2026-07-01 16:28:48.580018535 +0000 UTC m=+7.160636019 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/be2ddffa-466a-4a37-baba-5cb9ecf07e41-metrics-certs") pod "network-metrics-daemon-87ztj" (UID: "be2ddffa-466a-4a37-baba-5cb9ecf07e41") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 01 16:28:46.580102 ip-10-0-130-227 kubenswrapper[2545]: E0701 16:28:46.580101 2545 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4e4b6e11-f86f-4955-aabc-3f94ca793252-kube-api-access-c5bsl podName:4e4b6e11-f86f-4955-aabc-3f94ca793252 nodeName:}" failed. No retries permitted until 2026-07-01 16:28:48.580085732 +0000 UTC m=+7.160703214 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-c5bsl" (UniqueName: "kubernetes.io/projected/4e4b6e11-f86f-4955-aabc-3f94ca793252-kube-api-access-c5bsl") pod "network-check-target-kz46s" (UID: "4e4b6e11-f86f-4955-aabc-3f94ca793252") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 01 16:28:46.891428 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:46.891344 2545 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-06-30 16:23:44 +0000 UTC" deadline="2028-01-26 02:32:43.900727677 +0000 UTC" Jul 01 16:28:46.891428 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:46.891381 2545 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13762h3m57.00934997s" Jul 01 16:28:46.932883 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:46.932854 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-87ztj" Jul 01 16:28:46.933052 ip-10-0-130-227 kubenswrapper[2545]: E0701 16:28:46.932998 2545 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-87ztj" podUID="be2ddffa-466a-4a37-baba-5cb9ecf07e41" Jul 01 16:28:46.933116 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:46.933084 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kz46s" Jul 01 16:28:46.933175 ip-10-0-130-227 kubenswrapper[2545]: E0701 16:28:46.933155 2545 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kz46s" podUID="4e4b6e11-f86f-4955-aabc-3f94ca793252" Jul 01 16:28:46.933226 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:46.933211 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-npttx" Jul 01 16:28:46.933371 ip-10-0-130-227 kubenswrapper[2545]: E0701 16:28:46.933279 2545 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-npttx" podUID="779fced8-cf7c-49a7-a236-c10a83bbd53c" Jul 01 16:28:47.969572 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:47.969529 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-227.ec2.internal" event={"ID":"7c07f7dba9c4bc09c77f869b83b22cdb","Type":"ContainerStarted","Data":"894c5087c2b14c40046da9afe63cc5936e72846c39c1466494660a4145479685"} Jul 01 16:28:48.498179 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:48.498054 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/779fced8-cf7c-49a7-a236-c10a83bbd53c-original-pull-secret\") pod \"global-pull-secret-syncer-npttx\" (UID: \"779fced8-cf7c-49a7-a236-c10a83bbd53c\") " pod="kube-system/global-pull-secret-syncer-npttx" Jul 01 16:28:48.498327 ip-10-0-130-227 kubenswrapper[2545]: E0701 16:28:48.498296 2545 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 01 16:28:48.498383 ip-10-0-130-227 kubenswrapper[2545]: E0701 16:28:48.498365 2545 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/779fced8-cf7c-49a7-a236-c10a83bbd53c-original-pull-secret podName:779fced8-cf7c-49a7-a236-c10a83bbd53c nodeName:}" failed. No retries permitted until 2026-07-01 16:28:52.498344078 +0000 UTC m=+11.078961566 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/779fced8-cf7c-49a7-a236-c10a83bbd53c-original-pull-secret") pod "global-pull-secret-syncer-npttx" (UID: "779fced8-cf7c-49a7-a236-c10a83bbd53c") : object "kube-system"/"original-pull-secret" not registered Jul 01 16:28:48.598541 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:48.598458 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/be2ddffa-466a-4a37-baba-5cb9ecf07e41-metrics-certs\") pod \"network-metrics-daemon-87ztj\" (UID: \"be2ddffa-466a-4a37-baba-5cb9ecf07e41\") " pod="openshift-multus/network-metrics-daemon-87ztj" Jul 01 16:28:48.598773 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:48.598544 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c5bsl\" (UniqueName: \"kubernetes.io/projected/4e4b6e11-f86f-4955-aabc-3f94ca793252-kube-api-access-c5bsl\") pod \"network-check-target-kz46s\" (UID: \"4e4b6e11-f86f-4955-aabc-3f94ca793252\") " pod="openshift-network-diagnostics/network-check-target-kz46s" Jul 01 16:28:48.598773 ip-10-0-130-227 kubenswrapper[2545]: E0701 16:28:48.598687 2545 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 01 16:28:48.598773 ip-10-0-130-227 kubenswrapper[2545]: E0701 16:28:48.598723 2545 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 01 16:28:48.598773 ip-10-0-130-227 kubenswrapper[2545]: E0701 16:28:48.598737 2545 projected.go:196] Error preparing data for projected volume kube-api-access-c5bsl for pod openshift-network-diagnostics/network-check-target-kz46s: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 01 16:28:48.598999 ip-10-0-130-227 kubenswrapper[2545]: E0701 16:28:48.598791 2545 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4e4b6e11-f86f-4955-aabc-3f94ca793252-kube-api-access-c5bsl podName:4e4b6e11-f86f-4955-aabc-3f94ca793252 nodeName:}" failed. No retries permitted until 2026-07-01 16:28:52.598773969 +0000 UTC m=+11.179391457 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-c5bsl" (UniqueName: "kubernetes.io/projected/4e4b6e11-f86f-4955-aabc-3f94ca793252-kube-api-access-c5bsl") pod "network-check-target-kz46s" (UID: "4e4b6e11-f86f-4955-aabc-3f94ca793252") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 01 16:28:48.599209 ip-10-0-130-227 kubenswrapper[2545]: E0701 16:28:48.599192 2545 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 01 16:28:48.599278 ip-10-0-130-227 kubenswrapper[2545]: E0701 16:28:48.599246 2545 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/be2ddffa-466a-4a37-baba-5cb9ecf07e41-metrics-certs podName:be2ddffa-466a-4a37-baba-5cb9ecf07e41 nodeName:}" failed. No retries permitted until 2026-07-01 16:28:52.599230231 +0000 UTC m=+11.179847717 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/be2ddffa-466a-4a37-baba-5cb9ecf07e41-metrics-certs") pod "network-metrics-daemon-87ztj" (UID: "be2ddffa-466a-4a37-baba-5cb9ecf07e41") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 01 16:28:48.931223 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:48.930892 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-87ztj" Jul 01 16:28:48.931223 ip-10-0-130-227 kubenswrapper[2545]: E0701 16:28:48.931033 2545 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-87ztj" podUID="be2ddffa-466a-4a37-baba-5cb9ecf07e41" Jul 01 16:28:48.931738 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:48.931475 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kz46s" Jul 01 16:28:48.931738 ip-10-0-130-227 kubenswrapper[2545]: E0701 16:28:48.931567 2545 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kz46s" podUID="4e4b6e11-f86f-4955-aabc-3f94ca793252" Jul 01 16:28:48.933506 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:48.932381 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-npttx" Jul 01 16:28:48.933506 ip-10-0-130-227 kubenswrapper[2545]: E0701 16:28:48.932474 2545 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-npttx" podUID="779fced8-cf7c-49a7-a236-c10a83bbd53c" Jul 01 16:28:48.983218 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:48.983187 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-cprkr" event={"ID":"1df6652f-89cc-41a1-89f7-0ed8568b6ee2","Type":"ContainerStarted","Data":"59c07972a0c971a36973f8cf71f4449800194edb1cae1a92b7585eda7b98ceb0"} Jul 01 16:28:48.992559 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:48.992526 2545 generic.go:356] "Generic (PLEG): container finished" podID="2611d4e32ebacb126b8b90d148d3f748" containerID="74e916386e1dff3c711f01f07b8885aee4e794b8847b0cd6df805dfa2c1125a5" exitCode=0 Jul 01 16:28:48.993435 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:48.993403 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-227.ec2.internal" event={"ID":"2611d4e32ebacb126b8b90d148d3f748","Type":"ContainerDied","Data":"74e916386e1dff3c711f01f07b8885aee4e794b8847b0cd6df805dfa2c1125a5"} Jul 01 16:28:48.998256 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:48.998208 2545 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-227.ec2.internal" podStartSLOduration=4.998192351 podStartE2EDuration="4.998192351s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-01 16:28:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 16:28:47.984879168 +0000 UTC m=+6.565496669" watchObservedRunningTime="2026-07-01 16:28:48.998192351 +0000 UTC m=+7.578809857" Jul 01 16:28:49.017555 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:49.016976 2545 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-cprkr" podStartSLOduration=3.398229259 podStartE2EDuration="6.016937654s" totalImagesPullingTime="2.618708395s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 16:28:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 16:28:48.999403499 +0000 UTC m=+7.580021003" watchObservedRunningTime="2026-07-01 16:28:49.016937654 +0000 UTC m=+7.597555160" Jul 01 16:28:50.932799 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:50.932005 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-87ztj" Jul 01 16:28:50.932799 ip-10-0-130-227 kubenswrapper[2545]: E0701 16:28:50.932140 2545 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-87ztj" podUID="be2ddffa-466a-4a37-baba-5cb9ecf07e41" Jul 01 16:28:50.932799 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:50.932503 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kz46s" Jul 01 16:28:50.932799 ip-10-0-130-227 kubenswrapper[2545]: E0701 16:28:50.932598 2545 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kz46s" podUID="4e4b6e11-f86f-4955-aabc-3f94ca793252" Jul 01 16:28:50.932799 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:50.932646 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-npttx" Jul 01 16:28:50.932799 ip-10-0-130-227 kubenswrapper[2545]: E0701 16:28:50.932737 2545 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-npttx" podUID="779fced8-cf7c-49a7-a236-c10a83bbd53c" Jul 01 16:28:52.530353 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:52.530306 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/779fced8-cf7c-49a7-a236-c10a83bbd53c-original-pull-secret\") pod \"global-pull-secret-syncer-npttx\" (UID: \"779fced8-cf7c-49a7-a236-c10a83bbd53c\") " pod="kube-system/global-pull-secret-syncer-npttx" Jul 01 16:28:52.530828 ip-10-0-130-227 kubenswrapper[2545]: E0701 16:28:52.530466 2545 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 01 16:28:52.530828 ip-10-0-130-227 kubenswrapper[2545]: E0701 16:28:52.530548 2545 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/779fced8-cf7c-49a7-a236-c10a83bbd53c-original-pull-secret podName:779fced8-cf7c-49a7-a236-c10a83bbd53c nodeName:}" failed. No retries permitted until 2026-07-01 16:29:00.530526345 +0000 UTC m=+19.111143841 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/779fced8-cf7c-49a7-a236-c10a83bbd53c-original-pull-secret") pod "global-pull-secret-syncer-npttx" (UID: "779fced8-cf7c-49a7-a236-c10a83bbd53c") : object "kube-system"/"original-pull-secret" not registered Jul 01 16:28:52.631212 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:52.631171 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/be2ddffa-466a-4a37-baba-5cb9ecf07e41-metrics-certs\") pod \"network-metrics-daemon-87ztj\" (UID: \"be2ddffa-466a-4a37-baba-5cb9ecf07e41\") " pod="openshift-multus/network-metrics-daemon-87ztj" Jul 01 16:28:52.631369 ip-10-0-130-227 kubenswrapper[2545]: E0701 16:28:52.631333 2545 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 01 16:28:52.631427 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:52.631358 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c5bsl\" (UniqueName: \"kubernetes.io/projected/4e4b6e11-f86f-4955-aabc-3f94ca793252-kube-api-access-c5bsl\") pod \"network-check-target-kz46s\" (UID: \"4e4b6e11-f86f-4955-aabc-3f94ca793252\") " pod="openshift-network-diagnostics/network-check-target-kz46s" Jul 01 16:28:52.631427 ip-10-0-130-227 kubenswrapper[2545]: E0701 16:28:52.631405 2545 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/be2ddffa-466a-4a37-baba-5cb9ecf07e41-metrics-certs podName:be2ddffa-466a-4a37-baba-5cb9ecf07e41 nodeName:}" failed. No retries permitted until 2026-07-01 16:29:00.631383549 +0000 UTC m=+19.212001055 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/be2ddffa-466a-4a37-baba-5cb9ecf07e41-metrics-certs") pod "network-metrics-daemon-87ztj" (UID: "be2ddffa-466a-4a37-baba-5cb9ecf07e41") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 01 16:28:52.631540 ip-10-0-130-227 kubenswrapper[2545]: E0701 16:28:52.631466 2545 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 01 16:28:52.631540 ip-10-0-130-227 kubenswrapper[2545]: E0701 16:28:52.631486 2545 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 01 16:28:52.631540 ip-10-0-130-227 kubenswrapper[2545]: E0701 16:28:52.631499 2545 projected.go:196] Error preparing data for projected volume kube-api-access-c5bsl for pod openshift-network-diagnostics/network-check-target-kz46s: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 01 16:28:52.631674 ip-10-0-130-227 kubenswrapper[2545]: E0701 16:28:52.631546 2545 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4e4b6e11-f86f-4955-aabc-3f94ca793252-kube-api-access-c5bsl podName:4e4b6e11-f86f-4955-aabc-3f94ca793252 nodeName:}" failed. No retries permitted until 2026-07-01 16:29:00.63153375 +0000 UTC m=+19.212151249 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-c5bsl" (UniqueName: "kubernetes.io/projected/4e4b6e11-f86f-4955-aabc-3f94ca793252-kube-api-access-c5bsl") pod "network-check-target-kz46s" (UID: "4e4b6e11-f86f-4955-aabc-3f94ca793252") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 01 16:28:52.931651 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:52.929620 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-npttx" Jul 01 16:28:52.931651 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:52.929621 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kz46s" Jul 01 16:28:52.931651 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:52.930583 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-87ztj" Jul 01 16:28:52.931651 ip-10-0-130-227 kubenswrapper[2545]: E0701 16:28:52.930725 2545 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-87ztj" podUID="be2ddffa-466a-4a37-baba-5cb9ecf07e41" Jul 01 16:28:52.931651 ip-10-0-130-227 kubenswrapper[2545]: E0701 16:28:52.930870 2545 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kz46s" podUID="4e4b6e11-f86f-4955-aabc-3f94ca793252" Jul 01 16:28:52.931651 ip-10-0-130-227 kubenswrapper[2545]: E0701 16:28:52.931262 2545 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-npttx" podUID="779fced8-cf7c-49a7-a236-c10a83bbd53c" Jul 01 16:28:53.926725 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:53.926259 2545 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-cprkr" Jul 01 16:28:53.927871 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:53.927849 2545 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-cprkr" Jul 01 16:28:54.929766 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:54.929733 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-npttx" Jul 01 16:28:54.930191 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:54.929773 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-87ztj" Jul 01 16:28:54.930191 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:54.929733 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kz46s" Jul 01 16:28:54.930191 ip-10-0-130-227 kubenswrapper[2545]: E0701 16:28:54.929862 2545 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-npttx" podUID="779fced8-cf7c-49a7-a236-c10a83bbd53c" Jul 01 16:28:54.930191 ip-10-0-130-227 kubenswrapper[2545]: E0701 16:28:54.929966 2545 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kz46s" podUID="4e4b6e11-f86f-4955-aabc-3f94ca793252" Jul 01 16:28:54.930191 ip-10-0-130-227 kubenswrapper[2545]: E0701 16:28:54.930061 2545 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-87ztj" podUID="be2ddffa-466a-4a37-baba-5cb9ecf07e41" Jul 01 16:28:56.930325 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:56.930249 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kz46s" Jul 01 16:28:56.930325 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:56.930268 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-87ztj" Jul 01 16:28:56.930898 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:56.930249 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-npttx" Jul 01 16:28:56.930898 ip-10-0-130-227 kubenswrapper[2545]: E0701 16:28:56.930367 2545 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kz46s" podUID="4e4b6e11-f86f-4955-aabc-3f94ca793252" Jul 01 16:28:56.930898 ip-10-0-130-227 kubenswrapper[2545]: E0701 16:28:56.930441 2545 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-npttx" podUID="779fced8-cf7c-49a7-a236-c10a83bbd53c" Jul 01 16:28:56.930898 ip-10-0-130-227 kubenswrapper[2545]: E0701 16:28:56.930537 2545 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-87ztj" podUID="be2ddffa-466a-4a37-baba-5cb9ecf07e41" Jul 01 16:28:58.930004 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:58.929975 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kz46s" Jul 01 16:28:58.930486 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:58.929978 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-npttx" Jul 01 16:28:58.930486 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:28:58.929984 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-87ztj" Jul 01 16:28:58.930486 ip-10-0-130-227 kubenswrapper[2545]: E0701 16:28:58.930100 2545 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kz46s" podUID="4e4b6e11-f86f-4955-aabc-3f94ca793252" Jul 01 16:28:58.930486 ip-10-0-130-227 kubenswrapper[2545]: E0701 16:28:58.930227 2545 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-87ztj" podUID="be2ddffa-466a-4a37-baba-5cb9ecf07e41" Jul 01 16:28:58.930486 ip-10-0-130-227 kubenswrapper[2545]: E0701 16:28:58.930333 2545 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-npttx" podUID="779fced8-cf7c-49a7-a236-c10a83bbd53c" Jul 01 16:29:00.600640 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:29:00.600561 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/779fced8-cf7c-49a7-a236-c10a83bbd53c-original-pull-secret\") pod \"global-pull-secret-syncer-npttx\" (UID: \"779fced8-cf7c-49a7-a236-c10a83bbd53c\") " pod="kube-system/global-pull-secret-syncer-npttx" Jul 01 16:29:00.601058 ip-10-0-130-227 kubenswrapper[2545]: E0701 16:29:00.600754 2545 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 01 16:29:00.601058 ip-10-0-130-227 kubenswrapper[2545]: E0701 16:29:00.600827 2545 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/779fced8-cf7c-49a7-a236-c10a83bbd53c-original-pull-secret podName:779fced8-cf7c-49a7-a236-c10a83bbd53c nodeName:}" failed. No retries permitted until 2026-07-01 16:29:16.600806615 +0000 UTC m=+35.181424105 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/779fced8-cf7c-49a7-a236-c10a83bbd53c-original-pull-secret") pod "global-pull-secret-syncer-npttx" (UID: "779fced8-cf7c-49a7-a236-c10a83bbd53c") : object "kube-system"/"original-pull-secret" not registered Jul 01 16:29:00.701665 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:29:00.701637 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/be2ddffa-466a-4a37-baba-5cb9ecf07e41-metrics-certs\") pod \"network-metrics-daemon-87ztj\" (UID: \"be2ddffa-466a-4a37-baba-5cb9ecf07e41\") " pod="openshift-multus/network-metrics-daemon-87ztj" Jul 01 16:29:00.701848 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:29:00.701689 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c5bsl\" (UniqueName: \"kubernetes.io/projected/4e4b6e11-f86f-4955-aabc-3f94ca793252-kube-api-access-c5bsl\") pod \"network-check-target-kz46s\" (UID: \"4e4b6e11-f86f-4955-aabc-3f94ca793252\") " pod="openshift-network-diagnostics/network-check-target-kz46s" Jul 01 16:29:00.701848 ip-10-0-130-227 kubenswrapper[2545]: E0701 16:29:00.701811 2545 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 01 16:29:00.701848 ip-10-0-130-227 kubenswrapper[2545]: E0701 16:29:00.701825 2545 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 01 16:29:00.701848 ip-10-0-130-227 kubenswrapper[2545]: E0701 16:29:00.701842 2545 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 01 16:29:00.701848 ip-10-0-130-227 kubenswrapper[2545]: E0701 16:29:00.701851 2545 projected.go:196] Error preparing data for projected volume kube-api-access-c5bsl for pod openshift-network-diagnostics/network-check-target-kz46s: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 01 16:29:00.702027 ip-10-0-130-227 kubenswrapper[2545]: E0701 16:29:00.701879 2545 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/be2ddffa-466a-4a37-baba-5cb9ecf07e41-metrics-certs podName:be2ddffa-466a-4a37-baba-5cb9ecf07e41 nodeName:}" failed. No retries permitted until 2026-07-01 16:29:16.701860224 +0000 UTC m=+35.282477711 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/be2ddffa-466a-4a37-baba-5cb9ecf07e41-metrics-certs") pod "network-metrics-daemon-87ztj" (UID: "be2ddffa-466a-4a37-baba-5cb9ecf07e41") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 01 16:29:00.702027 ip-10-0-130-227 kubenswrapper[2545]: E0701 16:29:00.701895 2545 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4e4b6e11-f86f-4955-aabc-3f94ca793252-kube-api-access-c5bsl podName:4e4b6e11-f86f-4955-aabc-3f94ca793252 nodeName:}" failed. No retries permitted until 2026-07-01 16:29:16.701888029 +0000 UTC m=+35.282505510 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-c5bsl" (UniqueName: "kubernetes.io/projected/4e4b6e11-f86f-4955-aabc-3f94ca793252-kube-api-access-c5bsl") pod "network-check-target-kz46s" (UID: "4e4b6e11-f86f-4955-aabc-3f94ca793252") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 01 16:29:00.929643 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:29:00.929565 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kz46s" Jul 01 16:29:00.929643 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:29:00.929565 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-npttx" Jul 01 16:29:00.929878 ip-10-0-130-227 kubenswrapper[2545]: E0701 16:29:00.929693 2545 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kz46s" podUID="4e4b6e11-f86f-4955-aabc-3f94ca793252" Jul 01 16:29:00.929878 ip-10-0-130-227 kubenswrapper[2545]: E0701 16:29:00.929858 2545 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-npttx" podUID="779fced8-cf7c-49a7-a236-c10a83bbd53c" Jul 01 16:29:00.929960 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:29:00.929882 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-87ztj" Jul 01 16:29:00.930029 ip-10-0-130-227 kubenswrapper[2545]: E0701 16:29:00.930008 2545 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-87ztj" podUID="be2ddffa-466a-4a37-baba-5cb9ecf07e41" Jul 01 16:29:02.930161 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:29:02.930092 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kz46s" Jul 01 16:29:02.930607 ip-10-0-130-227 kubenswrapper[2545]: E0701 16:29:02.930193 2545 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kz46s" podUID="4e4b6e11-f86f-4955-aabc-3f94ca793252" Jul 01 16:29:02.930607 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:29:02.930283 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-npttx" Jul 01 16:29:02.930607 ip-10-0-130-227 kubenswrapper[2545]: E0701 16:29:02.930404 2545 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-npttx" podUID="779fced8-cf7c-49a7-a236-c10a83bbd53c" Jul 01 16:29:02.930607 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:29:02.930454 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-87ztj" Jul 01 16:29:02.930607 ip-10-0-130-227 kubenswrapper[2545]: E0701 16:29:02.930533 2545 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-87ztj" podUID="be2ddffa-466a-4a37-baba-5cb9ecf07e41" Jul 01 16:29:04.930038 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:29:04.929865 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kz46s" Jul 01 16:29:04.930632 ip-10-0-130-227 kubenswrapper[2545]: E0701 16:29:04.930113 2545 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kz46s" podUID="4e4b6e11-f86f-4955-aabc-3f94ca793252" Jul 01 16:29:04.930632 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:29:04.930399 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-npttx" Jul 01 16:29:04.930632 ip-10-0-130-227 kubenswrapper[2545]: E0701 16:29:04.930474 2545 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-npttx" podUID="779fced8-cf7c-49a7-a236-c10a83bbd53c" Jul 01 16:29:04.930632 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:29:04.930539 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-87ztj" Jul 01 16:29:04.930632 ip-10-0-130-227 kubenswrapper[2545]: E0701 16:29:04.930613 2545 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-87ztj" podUID="be2ddffa-466a-4a37-baba-5cb9ecf07e41" Jul 01 16:29:05.027628 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:29:05.027591 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-nkd9r" event={"ID":"0b3759ce-7af8-4687-9a32-0578edfe706c","Type":"ContainerStarted","Data":"c3c9c4c059f5fb85f550a658d8f23a55ffbb650421b6505c46cc2d8459e8d5a7"} Jul 01 16:29:05.029143 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:29:05.029122 2545 generic.go:356] "Generic (PLEG): container finished" podID="48cdc46f-d909-463d-942b-4d300d212062" containerID="04eb750070e56030923e780f6cff8d2240d0bdd1c73dd386e8e1c9613165eed4" exitCode=0 Jul 01 16:29:05.029224 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:29:05.029186 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-gnsqt" event={"ID":"48cdc46f-d909-463d-942b-4d300d212062","Type":"ContainerDied","Data":"04eb750070e56030923e780f6cff8d2240d0bdd1c73dd386e8e1c9613165eed4"} Jul 01 16:29:05.031420 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:29:05.031380 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-227.ec2.internal" event={"ID":"2611d4e32ebacb126b8b90d148d3f748","Type":"ContainerStarted","Data":"ae037cca1eaccc46d5eb6622c3be8767c803eae70c3c8840282fc713c7ba0ed4"} Jul 01 16:29:05.033121 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:29:05.033096 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-t7n99" event={"ID":"76faf2eb-5dae-48e1-ba27-f1b94c85920d","Type":"ContainerStarted","Data":"f6559568100aa940317bb40c6ddb0a0fbf16e8c30e14587e5556a5e6875e00c7"} Jul 01 16:29:05.036824 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:29:05.036797 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lp7sn" event={"ID":"9ae0a7d9-2f7b-49ba-8a15-69acd2a6071c","Type":"ContainerStarted","Data":"b28f123b1815548a0832f1be5214555f706d9c01ac78ea2c286829237e2a72ab"} Jul 01 16:29:05.038928 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:29:05.038903 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-t7nrc" event={"ID":"5be67da1-9165-4446-b3cb-2333db8c09cc","Type":"ContainerStarted","Data":"d519c33309a217c8b758ecd28b5c5ff17ef4a2943f0be25005540b3b5912e9b4"} Jul 01 16:29:05.040613 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:29:05.040583 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5cr75" event={"ID":"6c03a7ab-a7cc-44ed-86f7-62a750c49b71","Type":"ContainerStarted","Data":"a39bb8d70c1314197407f7a8e9e9de2d02613b25cf82d94b33317f510335e5ff"} Jul 01 16:29:05.044364 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:29:05.044335 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-wb2hh" event={"ID":"8242757c-9a33-4958-a860-0679d154b28c","Type":"ContainerStarted","Data":"9fec65d775eb48b8858b0473024bca00680160951e155f0e381a6afbd757ccec"} Jul 01 16:29:05.048737 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:29:05.048673 2545 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-nkd9r" podStartSLOduration=2.604442252 podStartE2EDuration="22.048657058s" totalImagesPullingTime="19.444214806s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-01 16:28:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 16:29:05.048474592 +0000 UTC m=+23.629092096" watchObservedRunningTime="2026-07-01 16:29:05.048657058 +0000 UTC m=+23.629274570" Jul 01 16:29:05.064424 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:29:05.064386 2545 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-227.ec2.internal" podStartSLOduration=21.064374271 podStartE2EDuration="21.064374271s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-01 16:28:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 16:29:05.063937481 +0000 UTC m=+23.644554985" watchObservedRunningTime="2026-07-01 16:29:05.064374271 +0000 UTC m=+23.644991775" Jul 01 16:29:05.077768 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:29:05.077731 2545 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-wb2hh" podStartSLOduration=2.62134737 podStartE2EDuration="22.07769572s" totalImagesPullingTime="19.45634835s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-01 16:28:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 16:29:05.07724304 +0000 UTC m=+23.657860547" watchObservedRunningTime="2026-07-01 16:29:05.07769572 +0000 UTC m=+23.658313224" Jul 01 16:29:05.114359 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:29:05.114317 2545 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-t7n99" podStartSLOduration=2.668917127 podStartE2EDuration="22.114304146s" totalImagesPullingTime="19.445387019s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-01 16:28:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 16:29:05.11420863 +0000 UTC m=+23.694826133" watchObservedRunningTime="2026-07-01 16:29:05.114304146 +0000 UTC m=+23.694921646" Jul 01 16:29:05.128368 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:29:05.128329 2545 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-t7nrc" podStartSLOduration=2.701277865 podStartE2EDuration="22.12831695s" totalImagesPullingTime="19.427039085s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-01 16:28:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 16:29:05.127836951 +0000 UTC m=+23.708454455" watchObservedRunningTime="2026-07-01 16:29:05.12831695 +0000 UTC m=+23.708934454" Jul 01 16:29:06.049814 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:29:06.049794 2545 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5cr75_6c03a7ab-a7cc-44ed-86f7-62a750c49b71/ovn-acl-logging/0.log" Jul 01 16:29:06.050519 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:29:06.050146 2545 generic.go:356] "Generic (PLEG): container finished" podID="6c03a7ab-a7cc-44ed-86f7-62a750c49b71" containerID="ec92790df9a78afd0997e85221db90bd9134e02f2cb69c46e4e4292326dc5558" exitCode=1 Jul 01 16:29:06.050519 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:29:06.050242 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5cr75" event={"ID":"6c03a7ab-a7cc-44ed-86f7-62a750c49b71","Type":"ContainerStarted","Data":"419831ffab01c10c191da8396d8ac25b793568edad7ff3c6db68af5859df9666"} Jul 01 16:29:06.050519 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:29:06.050334 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5cr75" event={"ID":"6c03a7ab-a7cc-44ed-86f7-62a750c49b71","Type":"ContainerStarted","Data":"fcc7f67cc17f597a941ceb678f21b574b68f1684953be56f1f52c8bd98bf597d"} Jul 01 16:29:06.050519 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:29:06.050375 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5cr75" event={"ID":"6c03a7ab-a7cc-44ed-86f7-62a750c49b71","Type":"ContainerStarted","Data":"bbeabcb9792914099099db92c5fa1918d714dfc18e65e4a82ccd13415716f483"} Jul 01 16:29:06.050519 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:29:06.050420 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5cr75" event={"ID":"6c03a7ab-a7cc-44ed-86f7-62a750c49b71","Type":"ContainerStarted","Data":"607020d1f99c7c6948bad489cf65d6b780b5461715b39be307be5a5ecc7f3adf"} Jul 01 16:29:06.050519 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:29:06.050450 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5cr75" event={"ID":"6c03a7ab-a7cc-44ed-86f7-62a750c49b71","Type":"ContainerDied","Data":"ec92790df9a78afd0997e85221db90bd9134e02f2cb69c46e4e4292326dc5558"} Jul 01 16:29:06.145783 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:29:06.145737 2545 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 01 16:29:06.923985 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:29:06.923869 2545 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-01T16:29:06.145755587Z","UUID":"cb5e776e-d740-4283-afeb-b934b7236228","Handler":null,"Name":"","Endpoint":""} Jul 01 16:29:06.925533 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:29:06.925509 2545 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 01 16:29:06.925647 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:29:06.925553 2545 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 01 16:29:06.929615 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:29:06.929592 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-npttx" Jul 01 16:29:06.929792 ip-10-0-130-227 kubenswrapper[2545]: E0701 16:29:06.929767 2545 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-npttx" podUID="779fced8-cf7c-49a7-a236-c10a83bbd53c" Jul 01 16:29:06.929884 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:29:06.929820 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-87ztj" Jul 01 16:29:06.930073 ip-10-0-130-227 kubenswrapper[2545]: E0701 16:29:06.929939 2545 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-87ztj" podUID="be2ddffa-466a-4a37-baba-5cb9ecf07e41" Jul 01 16:29:06.930073 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:29:06.929976 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kz46s" Jul 01 16:29:06.930073 ip-10-0-130-227 kubenswrapper[2545]: E0701 16:29:06.930047 2545 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kz46s" podUID="4e4b6e11-f86f-4955-aabc-3f94ca793252" Jul 01 16:29:07.054568 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:29:07.054532 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lp7sn" event={"ID":"9ae0a7d9-2f7b-49ba-8a15-69acd2a6071c","Type":"ContainerStarted","Data":"bb76047ad8fb4829cf1fc2732510b3ecd4f56891b10ccec8a564e20ad36e9168"} Jul 01 16:29:07.056437 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:29:07.056407 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-xwdht" event={"ID":"b43f95ee-7eec-446c-b1e2-8cf7a0c64864","Type":"ContainerStarted","Data":"412415af634c8836c4fa6af026bb6ea77aafc47129b2d74669149a16a8ea50ee"} Jul 01 16:29:08.929977 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:29:08.929943 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-87ztj" Jul 01 16:29:08.930466 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:29:08.929949 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kz46s" Jul 01 16:29:08.930466 ip-10-0-130-227 kubenswrapper[2545]: E0701 16:29:08.930077 2545 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-87ztj" podUID="be2ddffa-466a-4a37-baba-5cb9ecf07e41" Jul 01 16:29:08.930466 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:29:08.929959 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-npttx" Jul 01 16:29:08.930466 ip-10-0-130-227 kubenswrapper[2545]: E0701 16:29:08.930131 2545 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kz46s" podUID="4e4b6e11-f86f-4955-aabc-3f94ca793252" Jul 01 16:29:08.930466 ip-10-0-130-227 kubenswrapper[2545]: E0701 16:29:08.930244 2545 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-npttx" podUID="779fced8-cf7c-49a7-a236-c10a83bbd53c" Jul 01 16:29:10.064679 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:29:10.064431 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lp7sn" event={"ID":"9ae0a7d9-2f7b-49ba-8a15-69acd2a6071c","Type":"ContainerStarted","Data":"2c9a53ecdcaa361e8a13c06628f35e6c95a627d2ff9f62fe9b45204a205cc15a"} Jul 01 16:29:10.067539 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:29:10.067517 2545 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5cr75_6c03a7ab-a7cc-44ed-86f7-62a750c49b71/ovn-acl-logging/0.log" Jul 01 16:29:10.067893 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:29:10.067874 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5cr75" event={"ID":"6c03a7ab-a7cc-44ed-86f7-62a750c49b71","Type":"ContainerStarted","Data":"ce53eaec0f96d037201dc8f4f769eb601155494c50f98d23afbb12620de60d44"} Jul 01 16:29:10.069577 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:29:10.069553 2545 generic.go:356] "Generic (PLEG): container finished" podID="48cdc46f-d909-463d-942b-4d300d212062" containerID="a1041b9ea30166ae696ba5799ad4d7b96d0fbf3b16f01fcdd97d2325eb4a247f" exitCode=0 Jul 01 16:29:10.069682 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:29:10.069594 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-gnsqt" event={"ID":"48cdc46f-d909-463d-942b-4d300d212062","Type":"ContainerDied","Data":"a1041b9ea30166ae696ba5799ad4d7b96d0fbf3b16f01fcdd97d2325eb4a247f"} Jul 01 16:29:10.081447 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:29:10.081411 2545 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-xwdht" podStartSLOduration=7.608907917 podStartE2EDuration="27.081396336s" totalImagesPullingTime="19.472488419s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-01 16:28:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 16:29:07.070365724 +0000 UTC m=+25.650983230" watchObservedRunningTime="2026-07-01 16:29:10.081396336 +0000 UTC m=+28.662013833" Jul 01 16:29:10.081700 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:29:10.081681 2545 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lp7sn" podStartSLOduration=3.383279552 podStartE2EDuration="27.081673949s" totalImagesPullingTime="23.698394397s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-01 16:28:43 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 16:29:10.081468831 +0000 UTC m=+28.662086335" watchObservedRunningTime="2026-07-01 16:29:10.081673949 +0000 UTC m=+28.662291453" Jul 01 16:29:10.884525 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:29:10.884494 2545 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-cprkr" Jul 01 16:29:10.884768 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:29:10.884624 2545 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Jul 01 16:29:10.885078 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:29:10.885057 2545 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-cprkr" Jul 01 16:29:10.929958 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:29:10.929937 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kz46s" Jul 01 16:29:10.929958 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:29:10.929954 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-87ztj" Jul 01 16:29:10.930141 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:29:10.929946 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-npttx" Jul 01 16:29:10.930141 ip-10-0-130-227 kubenswrapper[2545]: E0701 16:29:10.930041 2545 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kz46s" podUID="4e4b6e11-f86f-4955-aabc-3f94ca793252" Jul 01 16:29:10.930141 ip-10-0-130-227 kubenswrapper[2545]: E0701 16:29:10.930098 2545 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-npttx" podUID="779fced8-cf7c-49a7-a236-c10a83bbd53c" Jul 01 16:29:10.930284 ip-10-0-130-227 kubenswrapper[2545]: E0701 16:29:10.930175 2545 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-87ztj" podUID="be2ddffa-466a-4a37-baba-5cb9ecf07e41" Jul 01 16:29:12.075627 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:29:12.075431 2545 generic.go:356] "Generic (PLEG): container finished" podID="48cdc46f-d909-463d-942b-4d300d212062" containerID="6c81f0d72cd9960a2146e8e025336388df0232c65c8b9893a46402a1fbe9936b" exitCode=0 Jul 01 16:29:12.076321 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:29:12.075516 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-gnsqt" event={"ID":"48cdc46f-d909-463d-942b-4d300d212062","Type":"ContainerDied","Data":"6c81f0d72cd9960a2146e8e025336388df0232c65c8b9893a46402a1fbe9936b"} Jul 01 16:29:12.080969 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:29:12.080953 2545 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5cr75_6c03a7ab-a7cc-44ed-86f7-62a750c49b71/ovn-acl-logging/0.log" Jul 01 16:29:12.081323 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:29:12.081299 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5cr75" event={"ID":"6c03a7ab-a7cc-44ed-86f7-62a750c49b71","Type":"ContainerStarted","Data":"218e739c6bc89001098f9bfcbaceea6c0e416d80506decbd6f1bf7fcf0e0c46d"} Jul 01 16:29:12.081572 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:29:12.081557 2545 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-5cr75" Jul 01 16:29:12.081743 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:29:12.081726 2545 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-5cr75" Jul 01 16:29:12.081814 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:29:12.081733 2545 scope.go:117] "RemoveContainer" containerID="ec92790df9a78afd0997e85221db90bd9134e02f2cb69c46e4e4292326dc5558" Jul 01 16:29:12.098772 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:29:12.098752 2545 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-5cr75" Jul 01 16:29:12.100029 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:29:12.100011 2545 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-5cr75" Jul 01 16:29:12.930806 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:29:12.930776 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kz46s" Jul 01 16:29:12.930972 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:29:12.930901 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-87ztj" Jul 01 16:29:12.930972 ip-10-0-130-227 kubenswrapper[2545]: E0701 16:29:12.930909 2545 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kz46s" podUID="4e4b6e11-f86f-4955-aabc-3f94ca793252" Jul 01 16:29:12.931057 ip-10-0-130-227 kubenswrapper[2545]: E0701 16:29:12.930988 2545 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-87ztj" podUID="be2ddffa-466a-4a37-baba-5cb9ecf07e41" Jul 01 16:29:12.931057 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:29:12.931035 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-npttx" Jul 01 16:29:12.931129 ip-10-0-130-227 kubenswrapper[2545]: E0701 16:29:12.931082 2545 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-npttx" podUID="779fced8-cf7c-49a7-a236-c10a83bbd53c" Jul 01 16:29:12.964530 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:29:12.964507 2545 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-5cr75" Jul 01 16:29:13.087056 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:29:13.087030 2545 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5cr75_6c03a7ab-a7cc-44ed-86f7-62a750c49b71/ovn-acl-logging/0.log" Jul 01 16:29:13.087418 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:29:13.087316 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5cr75" event={"ID":"6c03a7ab-a7cc-44ed-86f7-62a750c49b71","Type":"ContainerStarted","Data":"6e01f419e777acb9c962e37064fed17385e8f0dcb76662c87cc2b8d83090af7c"} Jul 01 16:29:13.116496 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:29:13.116437 2545 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-5cr75" podStartSLOduration=10.415459045 podStartE2EDuration="30.116421001s" totalImagesPullingTime="19.700961956s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-01 16:28:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 16:29:13.115059206 +0000 UTC m=+31.695676710" watchObservedRunningTime="2026-07-01 16:29:13.116421001 +0000 UTC m=+31.697038505" Jul 01 16:29:13.633517 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:29:13.633484 2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-87ztj"] Jul 01 16:29:13.633695 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:29:13.633653 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-87ztj" Jul 01 16:29:13.633799 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:29:13.633779 2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-npttx"] Jul 01 16:29:13.633799 ip-10-0-130-227 kubenswrapper[2545]: E0701 16:29:13.633783 2545 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-87ztj" podUID="be2ddffa-466a-4a37-baba-5cb9ecf07e41" Jul 01 16:29:13.633936 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:29:13.633913 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-npttx" Jul 01 16:29:13.634064 ip-10-0-130-227 kubenswrapper[2545]: E0701 16:29:13.634040 2545 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-npttx" podUID="779fced8-cf7c-49a7-a236-c10a83bbd53c" Jul 01 16:29:13.636841 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:29:13.636816 2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-kz46s"] Jul 01 16:29:13.636965 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:29:13.636912 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kz46s" Jul 01 16:29:13.637015 ip-10-0-130-227 kubenswrapper[2545]: E0701 16:29:13.636996 2545 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kz46s" podUID="4e4b6e11-f86f-4955-aabc-3f94ca793252" Jul 01 16:29:14.929682 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:29:14.929502 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kz46s" Jul 01 16:29:14.930135 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:29:14.929508 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-npttx" Jul 01 16:29:14.930135 ip-10-0-130-227 kubenswrapper[2545]: E0701 16:29:14.929776 2545 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kz46s" podUID="4e4b6e11-f86f-4955-aabc-3f94ca793252" Jul 01 16:29:14.930135 ip-10-0-130-227 kubenswrapper[2545]: E0701 16:29:14.929818 2545 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-npttx" podUID="779fced8-cf7c-49a7-a236-c10a83bbd53c" Jul 01 16:29:15.093828 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:29:15.093799 2545 generic.go:356] "Generic (PLEG): container finished" podID="48cdc46f-d909-463d-942b-4d300d212062" containerID="a5d9bd9032d0e98566e0b138f24cc0129c2520d4982866f78ae08468f6fa2d64" exitCode=0 Jul 01 16:29:15.093990 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:29:15.093859 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-gnsqt" event={"ID":"48cdc46f-d909-463d-942b-4d300d212062","Type":"ContainerDied","Data":"a5d9bd9032d0e98566e0b138f24cc0129c2520d4982866f78ae08468f6fa2d64"} Jul 01 16:29:15.929796 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:29:15.929771 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-87ztj" Jul 01 16:29:15.930172 ip-10-0-130-227 kubenswrapper[2545]: E0701 16:29:15.929910 2545 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-87ztj" podUID="be2ddffa-466a-4a37-baba-5cb9ecf07e41" Jul 01 16:29:16.623764 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:29:16.623733 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/779fced8-cf7c-49a7-a236-c10a83bbd53c-original-pull-secret\") pod \"global-pull-secret-syncer-npttx\" (UID: \"779fced8-cf7c-49a7-a236-c10a83bbd53c\") " pod="kube-system/global-pull-secret-syncer-npttx" Jul 01 16:29:16.623936 ip-10-0-130-227 kubenswrapper[2545]: E0701 16:29:16.623877 2545 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 01 16:29:16.623997 ip-10-0-130-227 kubenswrapper[2545]: E0701 16:29:16.623942 2545 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/779fced8-cf7c-49a7-a236-c10a83bbd53c-original-pull-secret podName:779fced8-cf7c-49a7-a236-c10a83bbd53c nodeName:}" failed. No retries permitted until 2026-07-01 16:29:48.623924842 +0000 UTC m=+67.204542323 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/779fced8-cf7c-49a7-a236-c10a83bbd53c-original-pull-secret") pod "global-pull-secret-syncer-npttx" (UID: "779fced8-cf7c-49a7-a236-c10a83bbd53c") : object "kube-system"/"original-pull-secret" not registered Jul 01 16:29:16.724086 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:29:16.724042 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/be2ddffa-466a-4a37-baba-5cb9ecf07e41-metrics-certs\") pod \"network-metrics-daemon-87ztj\" (UID: \"be2ddffa-466a-4a37-baba-5cb9ecf07e41\") " pod="openshift-multus/network-metrics-daemon-87ztj" Jul 01 16:29:16.724228 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:29:16.724125 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c5bsl\" (UniqueName: \"kubernetes.io/projected/4e4b6e11-f86f-4955-aabc-3f94ca793252-kube-api-access-c5bsl\") pod \"network-check-target-kz46s\" (UID: \"4e4b6e11-f86f-4955-aabc-3f94ca793252\") " pod="openshift-network-diagnostics/network-check-target-kz46s" Jul 01 16:29:16.724228 ip-10-0-130-227 kubenswrapper[2545]: E0701 16:29:16.724184 2545 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 01 16:29:16.724312 ip-10-0-130-227 kubenswrapper[2545]: E0701 16:29:16.724249 2545 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/be2ddffa-466a-4a37-baba-5cb9ecf07e41-metrics-certs podName:be2ddffa-466a-4a37-baba-5cb9ecf07e41 nodeName:}" failed. No retries permitted until 2026-07-01 16:29:48.724229149 +0000 UTC m=+67.304846644 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/be2ddffa-466a-4a37-baba-5cb9ecf07e41-metrics-certs") pod "network-metrics-daemon-87ztj" (UID: "be2ddffa-466a-4a37-baba-5cb9ecf07e41") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 01 16:29:16.724312 ip-10-0-130-227 kubenswrapper[2545]: E0701 16:29:16.724269 2545 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 01 16:29:16.724312 ip-10-0-130-227 kubenswrapper[2545]: E0701 16:29:16.724286 2545 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 01 16:29:16.724312 ip-10-0-130-227 kubenswrapper[2545]: E0701 16:29:16.724299 2545 projected.go:196] Error preparing data for projected volume kube-api-access-c5bsl for pod openshift-network-diagnostics/network-check-target-kz46s: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 01 16:29:16.724455 ip-10-0-130-227 kubenswrapper[2545]: E0701 16:29:16.724345 2545 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4e4b6e11-f86f-4955-aabc-3f94ca793252-kube-api-access-c5bsl podName:4e4b6e11-f86f-4955-aabc-3f94ca793252 nodeName:}" failed. No retries permitted until 2026-07-01 16:29:48.724330546 +0000 UTC m=+67.304948032 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-c5bsl" (UniqueName: "kubernetes.io/projected/4e4b6e11-f86f-4955-aabc-3f94ca793252-kube-api-access-c5bsl") pod "network-check-target-kz46s" (UID: "4e4b6e11-f86f-4955-aabc-3f94ca793252") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 01 16:29:16.929870 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:29:16.929789 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kz46s" Jul 01 16:29:16.930261 ip-10-0-130-227 kubenswrapper[2545]: E0701 16:29:16.929910 2545 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-kz46s" podUID="4e4b6e11-f86f-4955-aabc-3f94ca793252" Jul 01 16:29:16.930261 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:29:16.929974 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-npttx" Jul 01 16:29:16.930261 ip-10-0-130-227 kubenswrapper[2545]: E0701 16:29:16.930087 2545 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-npttx" podUID="779fced8-cf7c-49a7-a236-c10a83bbd53c" Jul 01 16:29:17.739513 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:29:17.739488 2545 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-227.ec2.internal" event="NodeReady" Jul 01 16:29:17.739666 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:29:17.739624 2545 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 01 16:29:17.782867 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:29:17.782809 2545 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-55sd4"] Jul 01 16:29:17.785355 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:29:17.785336 2545 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-2fxwv"] Jul 01 16:29:17.785482 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:29:17.785464 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-55sd4" Jul 01 16:29:17.786948 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:29:17.786928 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-2fxwv" Jul 01 16:29:17.790028 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:29:17.790004 2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-zq8q2\"" Jul 01 16:29:17.790117 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:29:17.790039 2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 01 16:29:17.790117 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:29:17.790097 2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-k2lss\"" Jul 01 16:29:17.790305 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:29:17.790288 2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 01 16:29:17.790503 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:29:17.790484 2545 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 01 16:29:17.790611 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:29:17.790507 2545 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 01 16:29:17.790611 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:29:17.790484 2545 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 01 16:29:17.794906 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:29:17.794878 2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-2fxwv"] Jul 01 16:29:17.797613 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:29:17.797572 2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-55sd4"] Jul 01 16:29:17.929526 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:29:17.929496 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-87ztj" Jul 01 16:29:17.932420 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:29:17.932397 2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 01 16:29:17.932866 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:29:17.932423 2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-jf2dq\"" Jul 01 16:29:17.932866 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:29:17.932748 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fg4ps\" (UniqueName: \"kubernetes.io/projected/06e3403f-0530-476b-b689-ffc0a7e50132-kube-api-access-fg4ps\") pod \"ingress-canary-2fxwv\" (UID: \"06e3403f-0530-476b-b689-ffc0a7e50132\") " pod="openshift-ingress-canary/ingress-canary-2fxwv" Jul 01 16:29:17.932866 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:29:17.932781 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b4hhp\" (UniqueName: \"kubernetes.io/projected/c3cf9697-8b56-4162-b761-39c2a0557b3b-kube-api-access-b4hhp\") pod \"dns-default-55sd4\" (UID: \"c3cf9697-8b56-4162-b761-39c2a0557b3b\") " pod="openshift-dns/dns-default-55sd4" Jul 01 16:29:17.932866 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:29:17.932830 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/06e3403f-0530-476b-b689-ffc0a7e50132-cert\") pod \"ingress-canary-2fxwv\" (UID: \"06e3403f-0530-476b-b689-ffc0a7e50132\") " pod="openshift-ingress-canary/ingress-canary-2fxwv" Jul 01 16:29:17.932866 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:29:17.932859 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c3cf9697-8b56-4162-b761-39c2a0557b3b-tmp-dir\") pod \"dns-default-55sd4\" (UID: \"c3cf9697-8b56-4162-b761-39c2a0557b3b\") " pod="openshift-dns/dns-default-55sd4" Jul 01 16:29:17.933074 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:29:17.932908 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/c3cf9697-8b56-4162-b761-39c2a0557b3b-config-volume\") pod \"dns-default-55sd4\" (UID: \"c3cf9697-8b56-4162-b761-39c2a0557b3b\") " pod="openshift-dns/dns-default-55sd4" Jul 01 16:29:17.933074 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:29:17.932937 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c3cf9697-8b56-4162-b761-39c2a0557b3b-metrics-tls\") pod \"dns-default-55sd4\" (UID: \"c3cf9697-8b56-4162-b761-39c2a0557b3b\") " pod="openshift-dns/dns-default-55sd4" Jul 01 16:29:18.033623 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:29:18.033534 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c3cf9697-8b56-4162-b761-39c2a0557b3b-metrics-tls\") pod \"dns-default-55sd4\" (UID: \"c3cf9697-8b56-4162-b761-39c2a0557b3b\") " pod="openshift-dns/dns-default-55sd4" Jul 01 16:29:18.033822 ip-10-0-130-227 kubenswrapper[2545]: E0701 16:29:18.033726 2545 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 01 16:29:18.033822 ip-10-0-130-227 kubenswrapper[2545]: E0701 16:29:18.033812 2545 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c3cf9697-8b56-4162-b761-39c2a0557b3b-metrics-tls podName:c3cf9697-8b56-4162-b761-39c2a0557b3b nodeName:}" failed. No retries permitted until 2026-07-01 16:29:18.533785759 +0000 UTC m=+37.114403248 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/c3cf9697-8b56-4162-b761-39c2a0557b3b-metrics-tls") pod "dns-default-55sd4" (UID: "c3cf9697-8b56-4162-b761-39c2a0557b3b") : secret "dns-default-metrics-tls" not found Jul 01 16:29:18.033947 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:29:18.033732 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fg4ps\" (UniqueName: \"kubernetes.io/projected/06e3403f-0530-476b-b689-ffc0a7e50132-kube-api-access-fg4ps\") pod \"ingress-canary-2fxwv\" (UID: \"06e3403f-0530-476b-b689-ffc0a7e50132\") " pod="openshift-ingress-canary/ingress-canary-2fxwv" Jul 01 16:29:18.033991 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:29:18.033941 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b4hhp\" (UniqueName: \"kubernetes.io/projected/c3cf9697-8b56-4162-b761-39c2a0557b3b-kube-api-access-b4hhp\") pod \"dns-default-55sd4\" (UID: \"c3cf9697-8b56-4162-b761-39c2a0557b3b\") " pod="openshift-dns/dns-default-55sd4" Jul 01 16:29:18.034033 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:29:18.033998 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/06e3403f-0530-476b-b689-ffc0a7e50132-cert\") pod \"ingress-canary-2fxwv\" (UID: \"06e3403f-0530-476b-b689-ffc0a7e50132\") " pod="openshift-ingress-canary/ingress-canary-2fxwv" Jul 01 16:29:18.034080 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:29:18.034033 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c3cf9697-8b56-4162-b761-39c2a0557b3b-tmp-dir\") pod \"dns-default-55sd4\" (UID: \"c3cf9697-8b56-4162-b761-39c2a0557b3b\") " pod="openshift-dns/dns-default-55sd4" Jul 01 16:29:18.034150 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:29:18.034129 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/c3cf9697-8b56-4162-b761-39c2a0557b3b-config-volume\") pod \"dns-default-55sd4\" (UID: \"c3cf9697-8b56-4162-b761-39c2a0557b3b\") " pod="openshift-dns/dns-default-55sd4" Jul 01 16:29:18.034410 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:29:18.034393 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c3cf9697-8b56-4162-b761-39c2a0557b3b-tmp-dir\") pod \"dns-default-55sd4\" (UID: \"c3cf9697-8b56-4162-b761-39c2a0557b3b\") " pod="openshift-dns/dns-default-55sd4" Jul 01 16:29:18.034758 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:29:18.034737 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/c3cf9697-8b56-4162-b761-39c2a0557b3b-config-volume\") pod \"dns-default-55sd4\" (UID: \"c3cf9697-8b56-4162-b761-39c2a0557b3b\") " pod="openshift-dns/dns-default-55sd4" Jul 01 16:29:18.039054 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:29:18.039017 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/06e3403f-0530-476b-b689-ffc0a7e50132-cert\") pod \"ingress-canary-2fxwv\" (UID: \"06e3403f-0530-476b-b689-ffc0a7e50132\") " pod="openshift-ingress-canary/ingress-canary-2fxwv" Jul 01 16:29:18.042587 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:29:18.042556 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b4hhp\" (UniqueName: \"kubernetes.io/projected/c3cf9697-8b56-4162-b761-39c2a0557b3b-kube-api-access-b4hhp\") pod \"dns-default-55sd4\" (UID: \"c3cf9697-8b56-4162-b761-39c2a0557b3b\") " pod="openshift-dns/dns-default-55sd4" Jul 01 16:29:18.042758 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:29:18.042736 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fg4ps\" (UniqueName: \"kubernetes.io/projected/06e3403f-0530-476b-b689-ffc0a7e50132-kube-api-access-fg4ps\") pod \"ingress-canary-2fxwv\" (UID: \"06e3403f-0530-476b-b689-ffc0a7e50132\") " pod="openshift-ingress-canary/ingress-canary-2fxwv" Jul 01 16:29:18.104273 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:29:18.104248 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-2fxwv" Jul 01 16:29:18.538015 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:29:18.537979 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c3cf9697-8b56-4162-b761-39c2a0557b3b-metrics-tls\") pod \"dns-default-55sd4\" (UID: \"c3cf9697-8b56-4162-b761-39c2a0557b3b\") " pod="openshift-dns/dns-default-55sd4" Jul 01 16:29:18.538187 ip-10-0-130-227 kubenswrapper[2545]: E0701 16:29:18.538149 2545 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 01 16:29:18.538248 ip-10-0-130-227 kubenswrapper[2545]: E0701 16:29:18.538227 2545 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c3cf9697-8b56-4162-b761-39c2a0557b3b-metrics-tls podName:c3cf9697-8b56-4162-b761-39c2a0557b3b nodeName:}" failed. No retries permitted until 2026-07-01 16:29:19.538206241 +0000 UTC m=+38.118823722 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/c3cf9697-8b56-4162-b761-39c2a0557b3b-metrics-tls") pod "dns-default-55sd4" (UID: "c3cf9697-8b56-4162-b761-39c2a0557b3b") : secret "dns-default-metrics-tls" not found Jul 01 16:29:18.930509 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:29:18.930436 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kz46s" Jul 01 16:29:18.930509 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:29:18.930447 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-npttx" Jul 01 16:29:18.933993 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:29:18.933511 2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 01 16:29:18.936442 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:29:18.936168 2545 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 01 16:29:18.936442 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:29:18.936182 2545 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 01 16:29:18.936442 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:29:18.936291 2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-zg86h\"" Jul 01 16:29:18.985545 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:29:18.985398 2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-2fxwv"] Jul 01 16:29:18.988118 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:29:18.988096 2545 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod06e3403f_0530_476b_b689_ffc0a7e50132.slice/crio-61ea88eccb98483498c1f10d418c151466cfda16952fb2c150888eb01fa1fd3b WatchSource:0}: Error finding container 61ea88eccb98483498c1f10d418c151466cfda16952fb2c150888eb01fa1fd3b: Status 404 returned error can't find the container with id 61ea88eccb98483498c1f10d418c151466cfda16952fb2c150888eb01fa1fd3b Jul 01 16:29:19.105783 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:29:19.105728 2545 generic.go:356] "Generic (PLEG): container finished" podID="48cdc46f-d909-463d-942b-4d300d212062" containerID="ca21d9b372efedd985e985a80257ac1e0290972eb0821ac6a87dd2758611ab3a" exitCode=0 Jul 01 16:29:19.105871 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:29:19.105800 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-gnsqt" event={"ID":"48cdc46f-d909-463d-942b-4d300d212062","Type":"ContainerDied","Data":"ca21d9b372efedd985e985a80257ac1e0290972eb0821ac6a87dd2758611ab3a"} Jul 01 16:29:19.106923 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:29:19.106900 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-2fxwv" event={"ID":"06e3403f-0530-476b-b689-ffc0a7e50132","Type":"ContainerStarted","Data":"61ea88eccb98483498c1f10d418c151466cfda16952fb2c150888eb01fa1fd3b"} Jul 01 16:29:19.544010 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:29:19.543917 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c3cf9697-8b56-4162-b761-39c2a0557b3b-metrics-tls\") pod \"dns-default-55sd4\" (UID: \"c3cf9697-8b56-4162-b761-39c2a0557b3b\") " pod="openshift-dns/dns-default-55sd4" Jul 01 16:29:19.544144 ip-10-0-130-227 kubenswrapper[2545]: E0701 16:29:19.544073 2545 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 01 16:29:19.544144 ip-10-0-130-227 kubenswrapper[2545]: E0701 16:29:19.544140 2545 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c3cf9697-8b56-4162-b761-39c2a0557b3b-metrics-tls podName:c3cf9697-8b56-4162-b761-39c2a0557b3b nodeName:}" failed. No retries permitted until 2026-07-01 16:29:21.544121804 +0000 UTC m=+40.124739287 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/c3cf9697-8b56-4162-b761-39c2a0557b3b-metrics-tls") pod "dns-default-55sd4" (UID: "c3cf9697-8b56-4162-b761-39c2a0557b3b") : secret "dns-default-metrics-tls" not found Jul 01 16:29:20.113055 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:29:20.113020 2545 generic.go:356] "Generic (PLEG): container finished" podID="48cdc46f-d909-463d-942b-4d300d212062" containerID="ed174104e1ae23f587ebc18ecccb272ec659ffa1ac9333e38ae3b59f7dc9e490" exitCode=0 Jul 01 16:29:20.113485 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:29:20.113068 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-gnsqt" event={"ID":"48cdc46f-d909-463d-942b-4d300d212062","Type":"ContainerDied","Data":"ed174104e1ae23f587ebc18ecccb272ec659ffa1ac9333e38ae3b59f7dc9e490"} Jul 01 16:29:21.119575 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:29:21.119543 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-gnsqt" event={"ID":"48cdc46f-d909-463d-942b-4d300d212062","Type":"ContainerStarted","Data":"0a362f462eb468a8b2895d736200195f152739edcf3e35e25a13c1102d8a7ef9"} Jul 01 16:29:21.143789 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:29:21.143742 2545 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-gnsqt" podStartSLOduration=7.326570205 podStartE2EDuration="38.143698217s" totalImagesPullingTime="30.817128012s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-01 16:28:43 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 16:29:21.141835004 +0000 UTC m=+39.722452508" watchObservedRunningTime="2026-07-01 16:29:21.143698217 +0000 UTC m=+39.724315721" Jul 01 16:29:21.557224 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:29:21.557144 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c3cf9697-8b56-4162-b761-39c2a0557b3b-metrics-tls\") pod \"dns-default-55sd4\" (UID: \"c3cf9697-8b56-4162-b761-39c2a0557b3b\") " pod="openshift-dns/dns-default-55sd4" Jul 01 16:29:21.557393 ip-10-0-130-227 kubenswrapper[2545]: E0701 16:29:21.557258 2545 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 01 16:29:21.557393 ip-10-0-130-227 kubenswrapper[2545]: E0701 16:29:21.557313 2545 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c3cf9697-8b56-4162-b761-39c2a0557b3b-metrics-tls podName:c3cf9697-8b56-4162-b761-39c2a0557b3b nodeName:}" failed. No retries permitted until 2026-07-01 16:29:25.557299604 +0000 UTC m=+44.137917087 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/c3cf9697-8b56-4162-b761-39c2a0557b3b-metrics-tls") pod "dns-default-55sd4" (UID: "c3cf9697-8b56-4162-b761-39c2a0557b3b") : secret "dns-default-metrics-tls" not found Jul 01 16:29:23.128777 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:29:23.128742 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-2fxwv" event={"ID":"06e3403f-0530-476b-b689-ffc0a7e50132","Type":"ContainerStarted","Data":"a459e91f5e90a4dc4a9b063ce79f9e38de71df5a9602e3c6b1be825dd5057fad"} Jul 01 16:29:23.143532 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:29:23.143488 2545 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-2fxwv" podStartSLOduration=2.836862433 podStartE2EDuration="6.143475911s" totalImagesPullingTime="3.306613478s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 16:29:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 16:29:23.142604148 +0000 UTC m=+41.723221652" watchObservedRunningTime="2026-07-01 16:29:23.143475911 +0000 UTC m=+41.724093393" Jul 01 16:29:25.582519 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:29:25.582478 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c3cf9697-8b56-4162-b761-39c2a0557b3b-metrics-tls\") pod \"dns-default-55sd4\" (UID: \"c3cf9697-8b56-4162-b761-39c2a0557b3b\") " pod="openshift-dns/dns-default-55sd4" Jul 01 16:29:25.583025 ip-10-0-130-227 kubenswrapper[2545]: E0701 16:29:25.582624 2545 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 01 16:29:25.583025 ip-10-0-130-227 kubenswrapper[2545]: E0701 16:29:25.582700 2545 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c3cf9697-8b56-4162-b761-39c2a0557b3b-metrics-tls podName:c3cf9697-8b56-4162-b761-39c2a0557b3b nodeName:}" failed. No retries permitted until 2026-07-01 16:29:33.582680572 +0000 UTC m=+52.163298054 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/c3cf9697-8b56-4162-b761-39c2a0557b3b-metrics-tls") pod "dns-default-55sd4" (UID: "c3cf9697-8b56-4162-b761-39c2a0557b3b") : secret "dns-default-metrics-tls" not found Jul 01 16:29:27.499859 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:29:27.499828 2545 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6cbfc95fbb-r4fmh"] Jul 01 16:29:27.529761 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:29:27.529739 2545 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5bcdc4cb7d-hrl7g"] Jul 01 16:29:27.529887 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:29:27.529862 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6cbfc95fbb-r4fmh" Jul 01 16:29:27.532386 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:29:27.532365 2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Jul 01 16:29:27.532386 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:29:27.532374 2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-gcdlp\"" Jul 01 16:29:27.532565 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:29:27.532398 2545 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Jul 01 16:29:27.533664 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:29:27.533488 2545 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Jul 01 16:29:27.533664 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:29:27.533567 2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\"" Jul 01 16:29:27.545735 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:29:27.545647 2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6cbfc95fbb-r4fmh"] Jul 01 16:29:27.545735 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:29:27.545725 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5bcdc4cb7d-hrl7g" Jul 01 16:29:27.545844 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:29:27.545759 2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5bcdc4cb7d-hrl7g"] Jul 01 16:29:27.548062 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:29:27.548048 2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\"" Jul 01 16:29:27.548207 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:29:27.548188 2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\"" Jul 01 16:29:27.548260 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:29:27.548248 2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\"" Jul 01 16:29:27.548307 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:29:27.548209 2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\"" Jul 01 16:29:27.695757 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:29:27.695735 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/7330d150-1a30-4898-a108-c7e969c724dd-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-5bcdc4cb7d-hrl7g\" (UID: \"7330d150-1a30-4898-a108-c7e969c724dd\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5bcdc4cb7d-hrl7g" Jul 01 16:29:27.695892 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:29:27.695765 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gxzz6\" (UniqueName: \"kubernetes.io/projected/7330d150-1a30-4898-a108-c7e969c724dd-kube-api-access-gxzz6\") pod \"cluster-proxy-proxy-agent-5bcdc4cb7d-hrl7g\" (UID: \"7330d150-1a30-4898-a108-c7e969c724dd\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5bcdc4cb7d-hrl7g" Jul 01 16:29:27.695892 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:29:27.695790 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/2139ad3f-5edf-48ac-b3e1-8c066d528218-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-6cbfc95fbb-r4fmh\" (UID: \"2139ad3f-5edf-48ac-b3e1-8c066d528218\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6cbfc95fbb-r4fmh" Jul 01 16:29:27.695966 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:29:27.695885 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/7330d150-1a30-4898-a108-c7e969c724dd-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-5bcdc4cb7d-hrl7g\" (UID: \"7330d150-1a30-4898-a108-c7e969c724dd\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5bcdc4cb7d-hrl7g" Jul 01 16:29:27.695966 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:29:27.695910 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xc7fq\" (UniqueName: \"kubernetes.io/projected/2139ad3f-5edf-48ac-b3e1-8c066d528218-kube-api-access-xc7fq\") pod \"managed-serviceaccount-addon-agent-6cbfc95fbb-r4fmh\" (UID: \"2139ad3f-5edf-48ac-b3e1-8c066d528218\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6cbfc95fbb-r4fmh" Jul 01 16:29:27.695966 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:29:27.695941 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/7330d150-1a30-4898-a108-c7e969c724dd-additional-service-ca\") pod \"cluster-proxy-proxy-agent-5bcdc4cb7d-hrl7g\" (UID: \"7330d150-1a30-4898-a108-c7e969c724dd\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5bcdc4cb7d-hrl7g" Jul 01 16:29:27.696065 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:29:27.695964 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/7330d150-1a30-4898-a108-c7e969c724dd-ca\") pod \"cluster-proxy-proxy-agent-5bcdc4cb7d-hrl7g\" (UID: \"7330d150-1a30-4898-a108-c7e969c724dd\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5bcdc4cb7d-hrl7g" Jul 01 16:29:27.696065 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:29:27.696031 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/7330d150-1a30-4898-a108-c7e969c724dd-hub\") pod \"cluster-proxy-proxy-agent-5bcdc4cb7d-hrl7g\" (UID: \"7330d150-1a30-4898-a108-c7e969c724dd\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5bcdc4cb7d-hrl7g" Jul 01 16:29:27.797311 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:29:27.797247 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/7330d150-1a30-4898-a108-c7e969c724dd-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-5bcdc4cb7d-hrl7g\" (UID: \"7330d150-1a30-4898-a108-c7e969c724dd\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5bcdc4cb7d-hrl7g" Jul 01 16:29:27.797311 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:29:27.797277 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gxzz6\" (UniqueName: \"kubernetes.io/projected/7330d150-1a30-4898-a108-c7e969c724dd-kube-api-access-gxzz6\") pod \"cluster-proxy-proxy-agent-5bcdc4cb7d-hrl7g\" (UID: \"7330d150-1a30-4898-a108-c7e969c724dd\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5bcdc4cb7d-hrl7g" Jul 01 16:29:27.797311 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:29:27.797298 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/2139ad3f-5edf-48ac-b3e1-8c066d528218-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-6cbfc95fbb-r4fmh\" (UID: \"2139ad3f-5edf-48ac-b3e1-8c066d528218\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6cbfc95fbb-r4fmh" Jul 01 16:29:27.797555 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:29:27.797523 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/7330d150-1a30-4898-a108-c7e969c724dd-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-5bcdc4cb7d-hrl7g\" (UID: \"7330d150-1a30-4898-a108-c7e969c724dd\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5bcdc4cb7d-hrl7g" Jul 01 16:29:27.797614 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:29:27.797565 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xc7fq\" (UniqueName: \"kubernetes.io/projected/2139ad3f-5edf-48ac-b3e1-8c066d528218-kube-api-access-xc7fq\") pod \"managed-serviceaccount-addon-agent-6cbfc95fbb-r4fmh\" (UID: \"2139ad3f-5edf-48ac-b3e1-8c066d528218\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6cbfc95fbb-r4fmh" Jul 01 16:29:27.797614 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:29:27.797599 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/7330d150-1a30-4898-a108-c7e969c724dd-additional-service-ca\") pod \"cluster-proxy-proxy-agent-5bcdc4cb7d-hrl7g\" (UID: \"7330d150-1a30-4898-a108-c7e969c724dd\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5bcdc4cb7d-hrl7g" Jul 01 16:29:27.797753 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:29:27.797618 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/7330d150-1a30-4898-a108-c7e969c724dd-ca\") pod \"cluster-proxy-proxy-agent-5bcdc4cb7d-hrl7g\" (UID: \"7330d150-1a30-4898-a108-c7e969c724dd\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5bcdc4cb7d-hrl7g" Jul 01 16:29:27.797753 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:29:27.797686 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/7330d150-1a30-4898-a108-c7e969c724dd-hub\") pod \"cluster-proxy-proxy-agent-5bcdc4cb7d-hrl7g\" (UID: \"7330d150-1a30-4898-a108-c7e969c724dd\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5bcdc4cb7d-hrl7g" Jul 01 16:29:27.798409 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:29:27.798363 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/7330d150-1a30-4898-a108-c7e969c724dd-additional-service-ca\") pod \"cluster-proxy-proxy-agent-5bcdc4cb7d-hrl7g\" (UID: \"7330d150-1a30-4898-a108-c7e969c724dd\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5bcdc4cb7d-hrl7g" Jul 01 16:29:27.801246 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:29:27.801223 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/7330d150-1a30-4898-a108-c7e969c724dd-ca\") pod \"cluster-proxy-proxy-agent-5bcdc4cb7d-hrl7g\" (UID: \"7330d150-1a30-4898-a108-c7e969c724dd\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5bcdc4cb7d-hrl7g" Jul 01 16:29:27.801246 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:29:27.801241 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/7330d150-1a30-4898-a108-c7e969c724dd-hub\") pod \"cluster-proxy-proxy-agent-5bcdc4cb7d-hrl7g\" (UID: \"7330d150-1a30-4898-a108-c7e969c724dd\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5bcdc4cb7d-hrl7g" Jul 01 16:29:27.801390 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:29:27.801369 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/7330d150-1a30-4898-a108-c7e969c724dd-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-5bcdc4cb7d-hrl7g\" (UID: \"7330d150-1a30-4898-a108-c7e969c724dd\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5bcdc4cb7d-hrl7g" Jul 01 16:29:27.801428 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:29:27.801403 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/7330d150-1a30-4898-a108-c7e969c724dd-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-5bcdc4cb7d-hrl7g\" (UID: \"7330d150-1a30-4898-a108-c7e969c724dd\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5bcdc4cb7d-hrl7g" Jul 01 16:29:27.801428 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:29:27.801418 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/2139ad3f-5edf-48ac-b3e1-8c066d528218-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-6cbfc95fbb-r4fmh\" (UID: \"2139ad3f-5edf-48ac-b3e1-8c066d528218\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6cbfc95fbb-r4fmh" Jul 01 16:29:27.805055 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:29:27.805026 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xc7fq\" (UniqueName: \"kubernetes.io/projected/2139ad3f-5edf-48ac-b3e1-8c066d528218-kube-api-access-xc7fq\") pod \"managed-serviceaccount-addon-agent-6cbfc95fbb-r4fmh\" (UID: \"2139ad3f-5edf-48ac-b3e1-8c066d528218\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6cbfc95fbb-r4fmh" Jul 01 16:29:27.805211 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:29:27.805197 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gxzz6\" (UniqueName: \"kubernetes.io/projected/7330d150-1a30-4898-a108-c7e969c724dd-kube-api-access-gxzz6\") pod \"cluster-proxy-proxy-agent-5bcdc4cb7d-hrl7g\" (UID: \"7330d150-1a30-4898-a108-c7e969c724dd\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5bcdc4cb7d-hrl7g" Jul 01 16:29:27.851268 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:29:27.851245 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6cbfc95fbb-r4fmh" Jul 01 16:29:27.857415 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:29:27.857391 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5bcdc4cb7d-hrl7g" Jul 01 16:29:28.002173 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:29:28.002145 2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6cbfc95fbb-r4fmh"] Jul 01 16:29:28.004753 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:29:28.004700 2545 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2139ad3f_5edf_48ac_b3e1_8c066d528218.slice/crio-95b4fc0b95ab845abd3b47ad18973aff49053c4000197db962ff2e53d155b533 WatchSource:0}: Error finding container 95b4fc0b95ab845abd3b47ad18973aff49053c4000197db962ff2e53d155b533: Status 404 returned error can't find the container with id 95b4fc0b95ab845abd3b47ad18973aff49053c4000197db962ff2e53d155b533 Jul 01 16:29:28.025783 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:29:28.025750 2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5bcdc4cb7d-hrl7g"] Jul 01 16:29:28.029258 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:29:28.029229 2545 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7330d150_1a30_4898_a108_c7e969c724dd.slice/crio-454d3230f6e1737fb984a4a9a7c53091dfdf7a468907660e49fafd005bf2121d WatchSource:0}: Error finding container 454d3230f6e1737fb984a4a9a7c53091dfdf7a468907660e49fafd005bf2121d: Status 404 returned error can't find the container with id 454d3230f6e1737fb984a4a9a7c53091dfdf7a468907660e49fafd005bf2121d Jul 01 16:29:28.140580 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:29:28.140549 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6cbfc95fbb-r4fmh" event={"ID":"2139ad3f-5edf-48ac-b3e1-8c066d528218","Type":"ContainerStarted","Data":"95b4fc0b95ab845abd3b47ad18973aff49053c4000197db962ff2e53d155b533"} Jul 01 16:29:28.141543 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:29:28.141521 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5bcdc4cb7d-hrl7g" event={"ID":"7330d150-1a30-4898-a108-c7e969c724dd","Type":"ContainerStarted","Data":"454d3230f6e1737fb984a4a9a7c53091dfdf7a468907660e49fafd005bf2121d"} Jul 01 16:29:33.155961 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:29:33.155927 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5bcdc4cb7d-hrl7g" event={"ID":"7330d150-1a30-4898-a108-c7e969c724dd","Type":"ContainerStarted","Data":"b778e904639607edead2a777879983c9acd1833364e74d8f2cd22f0d15fad878"} Jul 01 16:29:33.155961 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:29:33.155967 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5bcdc4cb7d-hrl7g" event={"ID":"7330d150-1a30-4898-a108-c7e969c724dd","Type":"ContainerStarted","Data":"9f58265d195c3320732696e5244e57e7dfa93335c455474624ee0caa5ca33cdc"} Jul 01 16:29:33.156431 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:29:33.155980 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5bcdc4cb7d-hrl7g" event={"ID":"7330d150-1a30-4898-a108-c7e969c724dd","Type":"ContainerStarted","Data":"7dab13b10b2d5f04a4679772f2182ff342a463f553b47e8f8345f93586e09b54"} Jul 01 16:29:33.157347 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:29:33.157328 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6cbfc95fbb-r4fmh" event={"ID":"2139ad3f-5edf-48ac-b3e1-8c066d528218","Type":"ContainerStarted","Data":"e6000a603637b7c2a5f3e6d3f57972e2312e8dd5c4927dac52c5c6f467665139"} Jul 01 16:29:33.188159 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:29:33.188111 2545 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5bcdc4cb7d-hrl7g" podStartSLOduration=1.6452161520000002 podStartE2EDuration="6.188093784s" totalImagesPullingTime="4.542877632s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 16:29:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 16:29:33.174312084 +0000 UTC m=+51.754929588" watchObservedRunningTime="2026-07-01 16:29:33.188093784 +0000 UTC m=+51.768711290" Jul 01 16:29:33.645445 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:29:33.645416 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c3cf9697-8b56-4162-b761-39c2a0557b3b-metrics-tls\") pod \"dns-default-55sd4\" (UID: \"c3cf9697-8b56-4162-b761-39c2a0557b3b\") " pod="openshift-dns/dns-default-55sd4" Jul 01 16:29:33.645592 ip-10-0-130-227 kubenswrapper[2545]: E0701 16:29:33.645531 2545 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 01 16:29:33.645592 ip-10-0-130-227 kubenswrapper[2545]: E0701 16:29:33.645581 2545 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c3cf9697-8b56-4162-b761-39c2a0557b3b-metrics-tls podName:c3cf9697-8b56-4162-b761-39c2a0557b3b nodeName:}" failed. No retries permitted until 2026-07-01 16:29:49.645568401 +0000 UTC m=+68.226185883 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/c3cf9697-8b56-4162-b761-39c2a0557b3b-metrics-tls") pod "dns-default-55sd4" (UID: "c3cf9697-8b56-4162-b761-39c2a0557b3b") : secret "dns-default-metrics-tls" not found Jul 01 16:29:44.105024 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:29:44.104997 2545 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-5cr75" Jul 01 16:29:44.132027 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:29:44.131987 2545 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6cbfc95fbb-r4fmh" podStartSLOduration=12.58028508 podStartE2EDuration="17.131969994s" totalImagesPullingTime="4.551684914s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 16:29:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 16:29:33.188207769 +0000 UTC m=+51.768825275" watchObservedRunningTime="2026-07-01 16:29:44.131969994 +0000 UTC m=+62.712587499" Jul 01 16:29:48.650972 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:29:48.650936 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/779fced8-cf7c-49a7-a236-c10a83bbd53c-original-pull-secret\") pod \"global-pull-secret-syncer-npttx\" (UID: \"779fced8-cf7c-49a7-a236-c10a83bbd53c\") " pod="kube-system/global-pull-secret-syncer-npttx" Jul 01 16:29:48.653619 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:29:48.653593 2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 01 16:29:48.663696 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:29:48.663671 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/779fced8-cf7c-49a7-a236-c10a83bbd53c-original-pull-secret\") pod \"global-pull-secret-syncer-npttx\" (UID: \"779fced8-cf7c-49a7-a236-c10a83bbd53c\") " pod="kube-system/global-pull-secret-syncer-npttx" Jul 01 16:29:48.752192 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:29:48.752158 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c5bsl\" (UniqueName: \"kubernetes.io/projected/4e4b6e11-f86f-4955-aabc-3f94ca793252-kube-api-access-c5bsl\") pod \"network-check-target-kz46s\" (UID: \"4e4b6e11-f86f-4955-aabc-3f94ca793252\") " pod="openshift-network-diagnostics/network-check-target-kz46s" Jul 01 16:29:48.752348 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:29:48.752209 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/be2ddffa-466a-4a37-baba-5cb9ecf07e41-metrics-certs\") pod \"network-metrics-daemon-87ztj\" (UID: \"be2ddffa-466a-4a37-baba-5cb9ecf07e41\") " pod="openshift-multus/network-metrics-daemon-87ztj" Jul 01 16:29:48.754901 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:29:48.754885 2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 01 16:29:48.754959 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:29:48.754948 2545 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 01 16:29:48.763060 ip-10-0-130-227 kubenswrapper[2545]: E0701 16:29:48.763040 2545 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jul 01 16:29:48.763146 ip-10-0-130-227 kubenswrapper[2545]: E0701 16:29:48.763132 2545 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/be2ddffa-466a-4a37-baba-5cb9ecf07e41-metrics-certs podName:be2ddffa-466a-4a37-baba-5cb9ecf07e41 nodeName:}" failed. No retries permitted until 2026-07-01 16:30:52.763116605 +0000 UTC m=+131.343734088 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/be2ddffa-466a-4a37-baba-5cb9ecf07e41-metrics-certs") pod "network-metrics-daemon-87ztj" (UID: "be2ddffa-466a-4a37-baba-5cb9ecf07e41") : secret "metrics-daemon-secret" not found Jul 01 16:29:48.765165 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:29:48.765152 2545 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 01 16:29:48.775610 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:29:48.775586 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c5bsl\" (UniqueName: \"kubernetes.io/projected/4e4b6e11-f86f-4955-aabc-3f94ca793252-kube-api-access-c5bsl\") pod \"network-check-target-kz46s\" (UID: \"4e4b6e11-f86f-4955-aabc-3f94ca793252\") " pod="openshift-network-diagnostics/network-check-target-kz46s" Jul 01 16:29:48.949371 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:29:48.949294 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-npttx" Jul 01 16:29:48.958147 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:29:48.958128 2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-zg86h\"" Jul 01 16:29:48.966122 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:29:48.966099 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-kz46s" Jul 01 16:29:49.081791 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:29:49.081763 2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-npttx"] Jul 01 16:29:49.084351 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:29:49.084331 2545 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod779fced8_cf7c_49a7_a236_c10a83bbd53c.slice/crio-ac0c761649faa768e026758264f514c434b6a144c389992d0f04a1e4776dc310 WatchSource:0}: Error finding container ac0c761649faa768e026758264f514c434b6a144c389992d0f04a1e4776dc310: Status 404 returned error can't find the container with id ac0c761649faa768e026758264f514c434b6a144c389992d0f04a1e4776dc310 Jul 01 16:29:49.091684 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:29:49.091664 2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-kz46s"] Jul 01 16:29:49.094035 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:29:49.094016 2545 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4e4b6e11_f86f_4955_aabc_3f94ca793252.slice/crio-318e99e063b69097b12ccc2d841b7716ef441d506461d5f5780a0e3315f90602 WatchSource:0}: Error finding container 318e99e063b69097b12ccc2d841b7716ef441d506461d5f5780a0e3315f90602: Status 404 returned error can't find the container with id 318e99e063b69097b12ccc2d841b7716ef441d506461d5f5780a0e3315f90602 Jul 01 16:29:49.195213 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:29:49.195182 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-kz46s" event={"ID":"4e4b6e11-f86f-4955-aabc-3f94ca793252","Type":"ContainerStarted","Data":"318e99e063b69097b12ccc2d841b7716ef441d506461d5f5780a0e3315f90602"} Jul 01 16:29:49.196303 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:29:49.196284 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-npttx" event={"ID":"779fced8-cf7c-49a7-a236-c10a83bbd53c","Type":"ContainerStarted","Data":"ac0c761649faa768e026758264f514c434b6a144c389992d0f04a1e4776dc310"} Jul 01 16:29:49.658238 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:29:49.658207 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c3cf9697-8b56-4162-b761-39c2a0557b3b-metrics-tls\") pod \"dns-default-55sd4\" (UID: \"c3cf9697-8b56-4162-b761-39c2a0557b3b\") " pod="openshift-dns/dns-default-55sd4" Jul 01 16:29:49.658633 ip-10-0-130-227 kubenswrapper[2545]: E0701 16:29:49.658331 2545 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 01 16:29:49.658633 ip-10-0-130-227 kubenswrapper[2545]: E0701 16:29:49.658388 2545 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c3cf9697-8b56-4162-b761-39c2a0557b3b-metrics-tls podName:c3cf9697-8b56-4162-b761-39c2a0557b3b nodeName:}" failed. No retries permitted until 2026-07-01 16:30:21.658370682 +0000 UTC m=+100.238988165 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/c3cf9697-8b56-4162-b761-39c2a0557b3b-metrics-tls") pod "dns-default-55sd4" (UID: "c3cf9697-8b56-4162-b761-39c2a0557b3b") : secret "dns-default-metrics-tls" not found Jul 01 16:29:54.213339 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:29:54.213250 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-kz46s" event={"ID":"4e4b6e11-f86f-4955-aabc-3f94ca793252","Type":"ContainerStarted","Data":"590fc70e94c9bceda21d7058d8fc3d6294e3885c23c61d00abcd22677381a7ba"} Jul 01 16:29:54.213339 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:29:54.213339 2545 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-kz46s" Jul 01 16:29:54.214784 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:29:54.214757 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-npttx" event={"ID":"779fced8-cf7c-49a7-a236-c10a83bbd53c","Type":"ContainerStarted","Data":"c297c161b22e048eec606b3e85e6fbd468dfc61bd843546aa1fb25253835f3ae"} Jul 01 16:29:54.230031 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:29:54.229991 2545 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-kz46s" podStartSLOduration=66.397621533 podStartE2EDuration="1m11.22997737s" totalImagesPullingTime="4.832355837s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 16:28:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 16:29:54.229396541 +0000 UTC m=+72.810014046" watchObservedRunningTime="2026-07-01 16:29:54.22997737 +0000 UTC m=+72.810594873" Jul 01 16:29:54.243423 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:29:54.243384 2545 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-npttx" podStartSLOduration=66.398595003 podStartE2EDuration="1m11.243372155s" totalImagesPullingTime="4.844777152s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-01 16:28:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 16:29:54.242911592 +0000 UTC m=+72.823529097" watchObservedRunningTime="2026-07-01 16:29:54.243372155 +0000 UTC m=+72.823989660" Jul 01 16:30:21.663047 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:30:21.662934 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c3cf9697-8b56-4162-b761-39c2a0557b3b-metrics-tls\") pod \"dns-default-55sd4\" (UID: \"c3cf9697-8b56-4162-b761-39c2a0557b3b\") " pod="openshift-dns/dns-default-55sd4" Jul 01 16:30:21.663478 ip-10-0-130-227 kubenswrapper[2545]: E0701 16:30:21.663048 2545 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 01 16:30:21.663478 ip-10-0-130-227 kubenswrapper[2545]: E0701 16:30:21.663109 2545 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c3cf9697-8b56-4162-b761-39c2a0557b3b-metrics-tls podName:c3cf9697-8b56-4162-b761-39c2a0557b3b nodeName:}" failed. No retries permitted until 2026-07-01 16:31:25.663094933 +0000 UTC m=+164.243712414 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/c3cf9697-8b56-4162-b761-39c2a0557b3b-metrics-tls") pod "dns-default-55sd4" (UID: "c3cf9697-8b56-4162-b761-39c2a0557b3b") : secret "dns-default-metrics-tls" not found Jul 01 16:30:25.220237 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:30:25.220211 2545 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-kz46s" Jul 01 16:30:52.857873 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:30:52.857835 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/be2ddffa-466a-4a37-baba-5cb9ecf07e41-metrics-certs\") pod \"network-metrics-daemon-87ztj\" (UID: \"be2ddffa-466a-4a37-baba-5cb9ecf07e41\") " pod="openshift-multus/network-metrics-daemon-87ztj" Jul 01 16:30:52.858321 ip-10-0-130-227 kubenswrapper[2545]: E0701 16:30:52.857986 2545 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jul 01 16:30:52.858321 ip-10-0-130-227 kubenswrapper[2545]: E0701 16:30:52.858046 2545 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/be2ddffa-466a-4a37-baba-5cb9ecf07e41-metrics-certs podName:be2ddffa-466a-4a37-baba-5cb9ecf07e41 nodeName:}" failed. No retries permitted until 2026-07-01 16:32:54.858031499 +0000 UTC m=+253.438648984 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/be2ddffa-466a-4a37-baba-5cb9ecf07e41-metrics-certs") pod "network-metrics-daemon-87ztj" (UID: "be2ddffa-466a-4a37-baba-5cb9ecf07e41") : secret "metrics-daemon-secret" not found Jul 01 16:31:20.799327 ip-10-0-130-227 kubenswrapper[2545]: E0701 16:31:20.799278 2545 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-55sd4" podUID="c3cf9697-8b56-4162-b761-39c2a0557b3b" Jul 01 16:31:20.939416 ip-10-0-130-227 kubenswrapper[2545]: E0701 16:31:20.939384 2545 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-87ztj" podUID="be2ddffa-466a-4a37-baba-5cb9ecf07e41" Jul 01 16:31:21.454517 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:31:21.454487 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-55sd4" Jul 01 16:31:25.682015 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:31:25.681981 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c3cf9697-8b56-4162-b761-39c2a0557b3b-metrics-tls\") pod \"dns-default-55sd4\" (UID: \"c3cf9697-8b56-4162-b761-39c2a0557b3b\") " pod="openshift-dns/dns-default-55sd4" Jul 01 16:31:25.682444 ip-10-0-130-227 kubenswrapper[2545]: E0701 16:31:25.682094 2545 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 01 16:31:25.682444 ip-10-0-130-227 kubenswrapper[2545]: E0701 16:31:25.682149 2545 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c3cf9697-8b56-4162-b761-39c2a0557b3b-metrics-tls podName:c3cf9697-8b56-4162-b761-39c2a0557b3b nodeName:}" failed. No retries permitted until 2026-07-01 16:33:27.68213506 +0000 UTC m=+286.262752542 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/c3cf9697-8b56-4162-b761-39c2a0557b3b-metrics-tls") pod "dns-default-55sd4" (UID: "c3cf9697-8b56-4162-b761-39c2a0557b3b") : secret "dns-default-metrics-tls" not found Jul 01 16:31:31.929907 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:31:31.929875 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-87ztj" Jul 01 16:31:33.488170 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:31:33.488084 2545 generic.go:356] "Generic (PLEG): container finished" podID="2139ad3f-5edf-48ac-b3e1-8c066d528218" containerID="e6000a603637b7c2a5f3e6d3f57972e2312e8dd5c4927dac52c5c6f467665139" exitCode=255 Jul 01 16:31:33.488170 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:31:33.488138 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6cbfc95fbb-r4fmh" event={"ID":"2139ad3f-5edf-48ac-b3e1-8c066d528218","Type":"ContainerDied","Data":"e6000a603637b7c2a5f3e6d3f57972e2312e8dd5c4927dac52c5c6f467665139"} Jul 01 16:31:33.488657 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:31:33.488489 2545 scope.go:117] "RemoveContainer" containerID="e6000a603637b7c2a5f3e6d3f57972e2312e8dd5c4927dac52c5c6f467665139" Jul 01 16:31:34.492818 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:31:34.492785 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6cbfc95fbb-r4fmh" event={"ID":"2139ad3f-5edf-48ac-b3e1-8c066d528218","Type":"ContainerStarted","Data":"d83b56393615a413c2e8b23c9e42cbfe8e37b231c26d7661d4a2db2eb9cabc34"} Jul 01 16:32:26.752428 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:32:26.752392 2545 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-2bzw5"] Jul 01 16:32:26.754653 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:32:26.754632 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-2bzw5" Jul 01 16:32:26.757546 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:32:26.757525 2545 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 01 16:32:26.757892 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:32:26.757874 2545 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 01 16:32:26.758682 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:32:26.758665 2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-zs8wh\"" Jul 01 16:32:26.758798 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:32:26.758736 2545 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 01 16:32:26.758798 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:32:26.758749 2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 01 16:32:26.768807 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:32:26.768786 2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-2bzw5"] Jul 01 16:32:26.793684 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:32:26.793663 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rtxlw\" (UniqueName: \"kubernetes.io/projected/059f2f9d-a756-4692-9452-9a9b0ffb55ce-kube-api-access-rtxlw\") pod \"insights-runtime-extractor-2bzw5\" (UID: \"059f2f9d-a756-4692-9452-9a9b0ffb55ce\") " pod="openshift-insights/insights-runtime-extractor-2bzw5" Jul 01 16:32:26.793795 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:32:26.793690 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/059f2f9d-a756-4692-9452-9a9b0ffb55ce-crio-socket\") pod \"insights-runtime-extractor-2bzw5\" (UID: \"059f2f9d-a756-4692-9452-9a9b0ffb55ce\") " pod="openshift-insights/insights-runtime-extractor-2bzw5" Jul 01 16:32:26.793795 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:32:26.793743 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/059f2f9d-a756-4692-9452-9a9b0ffb55ce-data-volume\") pod \"insights-runtime-extractor-2bzw5\" (UID: \"059f2f9d-a756-4692-9452-9a9b0ffb55ce\") " pod="openshift-insights/insights-runtime-extractor-2bzw5" Jul 01 16:32:26.793795 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:32:26.793784 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/059f2f9d-a756-4692-9452-9a9b0ffb55ce-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-2bzw5\" (UID: \"059f2f9d-a756-4692-9452-9a9b0ffb55ce\") " pod="openshift-insights/insights-runtime-extractor-2bzw5" Jul 01 16:32:26.793894 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:32:26.793816 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/059f2f9d-a756-4692-9452-9a9b0ffb55ce-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2bzw5\" (UID: \"059f2f9d-a756-4692-9452-9a9b0ffb55ce\") " pod="openshift-insights/insights-runtime-extractor-2bzw5" Jul 01 16:32:26.895072 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:32:26.895048 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rtxlw\" (UniqueName: \"kubernetes.io/projected/059f2f9d-a756-4692-9452-9a9b0ffb55ce-kube-api-access-rtxlw\") pod \"insights-runtime-extractor-2bzw5\" (UID: \"059f2f9d-a756-4692-9452-9a9b0ffb55ce\") " pod="openshift-insights/insights-runtime-extractor-2bzw5" Jul 01 16:32:26.895178 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:32:26.895075 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/059f2f9d-a756-4692-9452-9a9b0ffb55ce-crio-socket\") pod \"insights-runtime-extractor-2bzw5\" (UID: \"059f2f9d-a756-4692-9452-9a9b0ffb55ce\") " pod="openshift-insights/insights-runtime-extractor-2bzw5" Jul 01 16:32:26.895178 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:32:26.895105 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/059f2f9d-a756-4692-9452-9a9b0ffb55ce-data-volume\") pod \"insights-runtime-extractor-2bzw5\" (UID: \"059f2f9d-a756-4692-9452-9a9b0ffb55ce\") " pod="openshift-insights/insights-runtime-extractor-2bzw5" Jul 01 16:32:26.895178 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:32:26.895127 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/059f2f9d-a756-4692-9452-9a9b0ffb55ce-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-2bzw5\" (UID: \"059f2f9d-a756-4692-9452-9a9b0ffb55ce\") " pod="openshift-insights/insights-runtime-extractor-2bzw5" Jul 01 16:32:26.895178 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:32:26.895151 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/059f2f9d-a756-4692-9452-9a9b0ffb55ce-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2bzw5\" (UID: \"059f2f9d-a756-4692-9452-9a9b0ffb55ce\") " pod="openshift-insights/insights-runtime-extractor-2bzw5" Jul 01 16:32:26.895334 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:32:26.895189 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/059f2f9d-a756-4692-9452-9a9b0ffb55ce-crio-socket\") pod \"insights-runtime-extractor-2bzw5\" (UID: \"059f2f9d-a756-4692-9452-9a9b0ffb55ce\") " pod="openshift-insights/insights-runtime-extractor-2bzw5" Jul 01 16:32:26.895334 ip-10-0-130-227 kubenswrapper[2545]: E0701 16:32:26.895204 2545 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 01 16:32:26.895334 ip-10-0-130-227 kubenswrapper[2545]: E0701 16:32:26.895260 2545 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 01 16:32:26.895334 ip-10-0-130-227 kubenswrapper[2545]: E0701 16:32:26.895274 2545 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/059f2f9d-a756-4692-9452-9a9b0ffb55ce-kube-rbac-proxy-cm podName:059f2f9d-a756-4692-9452-9a9b0ffb55ce nodeName:}" failed. No retries permitted until 2026-07-01 16:32:27.395257626 +0000 UTC m=+225.975875108 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/059f2f9d-a756-4692-9452-9a9b0ffb55ce-kube-rbac-proxy-cm") pod "insights-runtime-extractor-2bzw5" (UID: "059f2f9d-a756-4692-9452-9a9b0ffb55ce") : configmap "kube-rbac-proxy" not found Jul 01 16:32:26.895334 ip-10-0-130-227 kubenswrapper[2545]: E0701 16:32:26.895316 2545 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/059f2f9d-a756-4692-9452-9a9b0ffb55ce-insights-runtime-extractor-tls podName:059f2f9d-a756-4692-9452-9a9b0ffb55ce nodeName:}" failed. No retries permitted until 2026-07-01 16:32:27.395304868 +0000 UTC m=+225.975922351 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/059f2f9d-a756-4692-9452-9a9b0ffb55ce-insights-runtime-extractor-tls") pod "insights-runtime-extractor-2bzw5" (UID: "059f2f9d-a756-4692-9452-9a9b0ffb55ce") : secret "insights-runtime-extractor-tls" not found Jul 01 16:32:26.895500 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:32:26.895429 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/059f2f9d-a756-4692-9452-9a9b0ffb55ce-data-volume\") pod \"insights-runtime-extractor-2bzw5\" (UID: \"059f2f9d-a756-4692-9452-9a9b0ffb55ce\") " pod="openshift-insights/insights-runtime-extractor-2bzw5" Jul 01 16:32:26.905446 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:32:26.905423 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rtxlw\" (UniqueName: \"kubernetes.io/projected/059f2f9d-a756-4692-9452-9a9b0ffb55ce-kube-api-access-rtxlw\") pod \"insights-runtime-extractor-2bzw5\" (UID: \"059f2f9d-a756-4692-9452-9a9b0ffb55ce\") " pod="openshift-insights/insights-runtime-extractor-2bzw5" Jul 01 16:32:27.398015 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:32:27.397987 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/059f2f9d-a756-4692-9452-9a9b0ffb55ce-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-2bzw5\" (UID: \"059f2f9d-a756-4692-9452-9a9b0ffb55ce\") " pod="openshift-insights/insights-runtime-extractor-2bzw5" Jul 01 16:32:27.398015 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:32:27.398016 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/059f2f9d-a756-4692-9452-9a9b0ffb55ce-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2bzw5\" (UID: \"059f2f9d-a756-4692-9452-9a9b0ffb55ce\") " pod="openshift-insights/insights-runtime-extractor-2bzw5" Jul 01 16:32:27.398217 ip-10-0-130-227 kubenswrapper[2545]: E0701 16:32:27.398091 2545 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 01 16:32:27.398217 ip-10-0-130-227 kubenswrapper[2545]: E0701 16:32:27.398099 2545 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 01 16:32:27.398217 ip-10-0-130-227 kubenswrapper[2545]: E0701 16:32:27.398147 2545 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/059f2f9d-a756-4692-9452-9a9b0ffb55ce-kube-rbac-proxy-cm podName:059f2f9d-a756-4692-9452-9a9b0ffb55ce nodeName:}" failed. No retries permitted until 2026-07-01 16:32:28.398132036 +0000 UTC m=+226.978749522 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/059f2f9d-a756-4692-9452-9a9b0ffb55ce-kube-rbac-proxy-cm") pod "insights-runtime-extractor-2bzw5" (UID: "059f2f9d-a756-4692-9452-9a9b0ffb55ce") : configmap "kube-rbac-proxy" not found Jul 01 16:32:27.398217 ip-10-0-130-227 kubenswrapper[2545]: E0701 16:32:27.398161 2545 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/059f2f9d-a756-4692-9452-9a9b0ffb55ce-insights-runtime-extractor-tls podName:059f2f9d-a756-4692-9452-9a9b0ffb55ce nodeName:}" failed. No retries permitted until 2026-07-01 16:32:28.398153979 +0000 UTC m=+226.978771461 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/059f2f9d-a756-4692-9452-9a9b0ffb55ce-insights-runtime-extractor-tls") pod "insights-runtime-extractor-2bzw5" (UID: "059f2f9d-a756-4692-9452-9a9b0ffb55ce") : secret "insights-runtime-extractor-tls" not found Jul 01 16:32:28.404524 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:32:28.404485 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/059f2f9d-a756-4692-9452-9a9b0ffb55ce-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-2bzw5\" (UID: \"059f2f9d-a756-4692-9452-9a9b0ffb55ce\") " pod="openshift-insights/insights-runtime-extractor-2bzw5" Jul 01 16:32:28.404524 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:32:28.404523 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/059f2f9d-a756-4692-9452-9a9b0ffb55ce-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2bzw5\" (UID: \"059f2f9d-a756-4692-9452-9a9b0ffb55ce\") " pod="openshift-insights/insights-runtime-extractor-2bzw5" Jul 01 16:32:28.405005 ip-10-0-130-227 kubenswrapper[2545]: E0701 16:32:28.404622 2545 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 01 16:32:28.405005 ip-10-0-130-227 kubenswrapper[2545]: E0701 16:32:28.404621 2545 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 01 16:32:28.405005 ip-10-0-130-227 kubenswrapper[2545]: E0701 16:32:28.404672 2545 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/059f2f9d-a756-4692-9452-9a9b0ffb55ce-insights-runtime-extractor-tls podName:059f2f9d-a756-4692-9452-9a9b0ffb55ce nodeName:}" failed. No retries permitted until 2026-07-01 16:32:30.404658884 +0000 UTC m=+228.985276366 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/059f2f9d-a756-4692-9452-9a9b0ffb55ce-insights-runtime-extractor-tls") pod "insights-runtime-extractor-2bzw5" (UID: "059f2f9d-a756-4692-9452-9a9b0ffb55ce") : secret "insights-runtime-extractor-tls" not found Jul 01 16:32:28.405005 ip-10-0-130-227 kubenswrapper[2545]: E0701 16:32:28.404721 2545 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/059f2f9d-a756-4692-9452-9a9b0ffb55ce-kube-rbac-proxy-cm podName:059f2f9d-a756-4692-9452-9a9b0ffb55ce nodeName:}" failed. No retries permitted until 2026-07-01 16:32:30.404686103 +0000 UTC m=+228.985303599 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/059f2f9d-a756-4692-9452-9a9b0ffb55ce-kube-rbac-proxy-cm") pod "insights-runtime-extractor-2bzw5" (UID: "059f2f9d-a756-4692-9452-9a9b0ffb55ce") : configmap "kube-rbac-proxy" not found Jul 01 16:32:28.921551 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:32:28.921523 2545 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-t7nrc_5be67da1-9165-4446-b3cb-2333db8c09cc/dns-node-resolver/0.log" Jul 01 16:32:29.922242 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:32:29.922219 2545 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-wb2hh_8242757c-9a33-4958-a860-0679d154b28c/node-ca/0.log" Jul 01 16:32:30.322409 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:32:30.322374 2545 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-2fxwv_06e3403f-0530-476b-b689-ffc0a7e50132/serve-healthcheck-canary/0.log" Jul 01 16:32:30.417420 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:32:30.417391 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/059f2f9d-a756-4692-9452-9a9b0ffb55ce-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-2bzw5\" (UID: \"059f2f9d-a756-4692-9452-9a9b0ffb55ce\") " pod="openshift-insights/insights-runtime-extractor-2bzw5" Jul 01 16:32:30.417420 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:32:30.417421 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/059f2f9d-a756-4692-9452-9a9b0ffb55ce-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2bzw5\" (UID: \"059f2f9d-a756-4692-9452-9a9b0ffb55ce\") " pod="openshift-insights/insights-runtime-extractor-2bzw5" Jul 01 16:32:30.417572 ip-10-0-130-227 kubenswrapper[2545]: E0701 16:32:30.417512 2545 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 01 16:32:30.417572 ip-10-0-130-227 kubenswrapper[2545]: E0701 16:32:30.417566 2545 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 01 16:32:30.417638 ip-10-0-130-227 kubenswrapper[2545]: E0701 16:32:30.417574 2545 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/059f2f9d-a756-4692-9452-9a9b0ffb55ce-kube-rbac-proxy-cm podName:059f2f9d-a756-4692-9452-9a9b0ffb55ce nodeName:}" failed. No retries permitted until 2026-07-01 16:32:34.417557544 +0000 UTC m=+232.998175029 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/059f2f9d-a756-4692-9452-9a9b0ffb55ce-kube-rbac-proxy-cm") pod "insights-runtime-extractor-2bzw5" (UID: "059f2f9d-a756-4692-9452-9a9b0ffb55ce") : configmap "kube-rbac-proxy" not found Jul 01 16:32:30.417638 ip-10-0-130-227 kubenswrapper[2545]: E0701 16:32:30.417603 2545 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/059f2f9d-a756-4692-9452-9a9b0ffb55ce-insights-runtime-extractor-tls podName:059f2f9d-a756-4692-9452-9a9b0ffb55ce nodeName:}" failed. No retries permitted until 2026-07-01 16:32:34.417592929 +0000 UTC m=+232.998210412 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/059f2f9d-a756-4692-9452-9a9b0ffb55ce-insights-runtime-extractor-tls") pod "insights-runtime-extractor-2bzw5" (UID: "059f2f9d-a756-4692-9452-9a9b0ffb55ce") : secret "insights-runtime-extractor-tls" not found Jul 01 16:32:34.446035 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:32:34.446000 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/059f2f9d-a756-4692-9452-9a9b0ffb55ce-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-2bzw5\" (UID: \"059f2f9d-a756-4692-9452-9a9b0ffb55ce\") " pod="openshift-insights/insights-runtime-extractor-2bzw5" Jul 01 16:32:34.446035 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:32:34.446037 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/059f2f9d-a756-4692-9452-9a9b0ffb55ce-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2bzw5\" (UID: \"059f2f9d-a756-4692-9452-9a9b0ffb55ce\") " pod="openshift-insights/insights-runtime-extractor-2bzw5" Jul 01 16:32:34.446428 ip-10-0-130-227 kubenswrapper[2545]: E0701 16:32:34.446113 2545 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 01 16:32:34.446428 ip-10-0-130-227 kubenswrapper[2545]: E0701 16:32:34.446150 2545 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 01 16:32:34.446428 ip-10-0-130-227 kubenswrapper[2545]: E0701 16:32:34.446210 2545 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/059f2f9d-a756-4692-9452-9a9b0ffb55ce-insights-runtime-extractor-tls podName:059f2f9d-a756-4692-9452-9a9b0ffb55ce nodeName:}" failed. No retries permitted until 2026-07-01 16:32:42.446196846 +0000 UTC m=+241.026814328 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/059f2f9d-a756-4692-9452-9a9b0ffb55ce-insights-runtime-extractor-tls") pod "insights-runtime-extractor-2bzw5" (UID: "059f2f9d-a756-4692-9452-9a9b0ffb55ce") : secret "insights-runtime-extractor-tls" not found Jul 01 16:32:34.446428 ip-10-0-130-227 kubenswrapper[2545]: E0701 16:32:34.446224 2545 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/059f2f9d-a756-4692-9452-9a9b0ffb55ce-kube-rbac-proxy-cm podName:059f2f9d-a756-4692-9452-9a9b0ffb55ce nodeName:}" failed. No retries permitted until 2026-07-01 16:32:42.446217314 +0000 UTC m=+241.026834796 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/059f2f9d-a756-4692-9452-9a9b0ffb55ce-kube-rbac-proxy-cm") pod "insights-runtime-extractor-2bzw5" (UID: "059f2f9d-a756-4692-9452-9a9b0ffb55ce") : configmap "kube-rbac-proxy" not found Jul 01 16:32:42.503156 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:32:42.503119 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/059f2f9d-a756-4692-9452-9a9b0ffb55ce-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-2bzw5\" (UID: \"059f2f9d-a756-4692-9452-9a9b0ffb55ce\") " pod="openshift-insights/insights-runtime-extractor-2bzw5" Jul 01 16:32:42.503156 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:32:42.503157 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/059f2f9d-a756-4692-9452-9a9b0ffb55ce-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2bzw5\" (UID: \"059f2f9d-a756-4692-9452-9a9b0ffb55ce\") " pod="openshift-insights/insights-runtime-extractor-2bzw5" Jul 01 16:32:42.503688 ip-10-0-130-227 kubenswrapper[2545]: E0701 16:32:42.503248 2545 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 01 16:32:42.503688 ip-10-0-130-227 kubenswrapper[2545]: E0701 16:32:42.503299 2545 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/059f2f9d-a756-4692-9452-9a9b0ffb55ce-insights-runtime-extractor-tls podName:059f2f9d-a756-4692-9452-9a9b0ffb55ce nodeName:}" failed. No retries permitted until 2026-07-01 16:32:58.503287219 +0000 UTC m=+257.083904700 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/059f2f9d-a756-4692-9452-9a9b0ffb55ce-insights-runtime-extractor-tls") pod "insights-runtime-extractor-2bzw5" (UID: "059f2f9d-a756-4692-9452-9a9b0ffb55ce") : secret "insights-runtime-extractor-tls" not found Jul 01 16:32:42.503841 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:32:42.503789 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/059f2f9d-a756-4692-9452-9a9b0ffb55ce-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-2bzw5\" (UID: \"059f2f9d-a756-4692-9452-9a9b0ffb55ce\") " pod="openshift-insights/insights-runtime-extractor-2bzw5" Jul 01 16:32:48.860003 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:32:48.859969 2545 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-7bc8c8cb69-dt9sk"] Jul 01 16:32:48.861957 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:32:48.861943 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7bc8c8cb69-dt9sk" Jul 01 16:32:48.865220 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:32:48.865196 2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 01 16:32:48.865439 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:32:48.865422 2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-jvvfl\"" Jul 01 16:32:48.866669 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:32:48.866649 2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 01 16:32:48.866796 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:32:48.866740 2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 01 16:32:48.870489 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:32:48.870468 2545 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 01 16:32:48.875654 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:32:48.875625 2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7bc8c8cb69-dt9sk"] Jul 01 16:32:48.947692 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:32:48.947669 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/a97628e5-f93f-448e-8bae-83444e4fa34b-installation-pull-secrets\") pod \"image-registry-7bc8c8cb69-dt9sk\" (UID: \"a97628e5-f93f-448e-8bae-83444e4fa34b\") " pod="openshift-image-registry/image-registry-7bc8c8cb69-dt9sk" Jul 01 16:32:48.947815 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:32:48.947698 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/a97628e5-f93f-448e-8bae-83444e4fa34b-registry-certificates\") pod \"image-registry-7bc8c8cb69-dt9sk\" (UID: \"a97628e5-f93f-448e-8bae-83444e4fa34b\") " pod="openshift-image-registry/image-registry-7bc8c8cb69-dt9sk" Jul 01 16:32:48.947815 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:32:48.947748 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a97628e5-f93f-448e-8bae-83444e4fa34b-registry-tls\") pod \"image-registry-7bc8c8cb69-dt9sk\" (UID: \"a97628e5-f93f-448e-8bae-83444e4fa34b\") " pod="openshift-image-registry/image-registry-7bc8c8cb69-dt9sk" Jul 01 16:32:48.947815 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:32:48.947769 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/a97628e5-f93f-448e-8bae-83444e4fa34b-image-registry-private-configuration\") pod \"image-registry-7bc8c8cb69-dt9sk\" (UID: \"a97628e5-f93f-448e-8bae-83444e4fa34b\") " pod="openshift-image-registry/image-registry-7bc8c8cb69-dt9sk" Jul 01 16:32:48.947815 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:32:48.947792 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/a97628e5-f93f-448e-8bae-83444e4fa34b-ca-trust-extracted\") pod \"image-registry-7bc8c8cb69-dt9sk\" (UID: \"a97628e5-f93f-448e-8bae-83444e4fa34b\") " pod="openshift-image-registry/image-registry-7bc8c8cb69-dt9sk" Jul 01 16:32:48.947939 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:32:48.947816 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/a97628e5-f93f-448e-8bae-83444e4fa34b-trusted-ca\") pod \"image-registry-7bc8c8cb69-dt9sk\" (UID: \"a97628e5-f93f-448e-8bae-83444e4fa34b\") " pod="openshift-image-registry/image-registry-7bc8c8cb69-dt9sk" Jul 01 16:32:48.947939 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:32:48.947918 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a97628e5-f93f-448e-8bae-83444e4fa34b-bound-sa-token\") pod \"image-registry-7bc8c8cb69-dt9sk\" (UID: \"a97628e5-f93f-448e-8bae-83444e4fa34b\") " pod="openshift-image-registry/image-registry-7bc8c8cb69-dt9sk" Jul 01 16:32:48.948002 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:32:48.947950 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rqcvt\" (UniqueName: \"kubernetes.io/projected/a97628e5-f93f-448e-8bae-83444e4fa34b-kube-api-access-rqcvt\") pod \"image-registry-7bc8c8cb69-dt9sk\" (UID: \"a97628e5-f93f-448e-8bae-83444e4fa34b\") " pod="openshift-image-registry/image-registry-7bc8c8cb69-dt9sk" Jul 01 16:32:49.048410 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:32:49.048377 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a97628e5-f93f-448e-8bae-83444e4fa34b-registry-tls\") pod \"image-registry-7bc8c8cb69-dt9sk\" (UID: \"a97628e5-f93f-448e-8bae-83444e4fa34b\") " pod="openshift-image-registry/image-registry-7bc8c8cb69-dt9sk" Jul 01 16:32:49.048410 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:32:49.048415 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/a97628e5-f93f-448e-8bae-83444e4fa34b-image-registry-private-configuration\") pod \"image-registry-7bc8c8cb69-dt9sk\" (UID: \"a97628e5-f93f-448e-8bae-83444e4fa34b\") " pod="openshift-image-registry/image-registry-7bc8c8cb69-dt9sk" Jul 01 16:32:49.048646 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:32:49.048445 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/a97628e5-f93f-448e-8bae-83444e4fa34b-ca-trust-extracted\") pod \"image-registry-7bc8c8cb69-dt9sk\" (UID: \"a97628e5-f93f-448e-8bae-83444e4fa34b\") " pod="openshift-image-registry/image-registry-7bc8c8cb69-dt9sk" Jul 01 16:32:49.048646 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:32:49.048471 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/a97628e5-f93f-448e-8bae-83444e4fa34b-trusted-ca\") pod \"image-registry-7bc8c8cb69-dt9sk\" (UID: \"a97628e5-f93f-448e-8bae-83444e4fa34b\") " pod="openshift-image-registry/image-registry-7bc8c8cb69-dt9sk" Jul 01 16:32:49.048646 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:32:49.048509 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a97628e5-f93f-448e-8bae-83444e4fa34b-bound-sa-token\") pod \"image-registry-7bc8c8cb69-dt9sk\" (UID: \"a97628e5-f93f-448e-8bae-83444e4fa34b\") " pod="openshift-image-registry/image-registry-7bc8c8cb69-dt9sk" Jul 01 16:32:49.048646 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:32:49.048539 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rqcvt\" (UniqueName: \"kubernetes.io/projected/a97628e5-f93f-448e-8bae-83444e4fa34b-kube-api-access-rqcvt\") pod \"image-registry-7bc8c8cb69-dt9sk\" (UID: \"a97628e5-f93f-448e-8bae-83444e4fa34b\") " pod="openshift-image-registry/image-registry-7bc8c8cb69-dt9sk" Jul 01 16:32:49.048646 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:32:49.048586 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/a97628e5-f93f-448e-8bae-83444e4fa34b-installation-pull-secrets\") pod \"image-registry-7bc8c8cb69-dt9sk\" (UID: \"a97628e5-f93f-448e-8bae-83444e4fa34b\") " pod="openshift-image-registry/image-registry-7bc8c8cb69-dt9sk" Jul 01 16:32:49.048646 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:32:49.048616 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/a97628e5-f93f-448e-8bae-83444e4fa34b-registry-certificates\") pod \"image-registry-7bc8c8cb69-dt9sk\" (UID: \"a97628e5-f93f-448e-8bae-83444e4fa34b\") " pod="openshift-image-registry/image-registry-7bc8c8cb69-dt9sk" Jul 01 16:32:49.048967 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:32:49.048860 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/a97628e5-f93f-448e-8bae-83444e4fa34b-ca-trust-extracted\") pod \"image-registry-7bc8c8cb69-dt9sk\" (UID: \"a97628e5-f93f-448e-8bae-83444e4fa34b\") " pod="openshift-image-registry/image-registry-7bc8c8cb69-dt9sk" Jul 01 16:32:49.049405 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:32:49.049379 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/a97628e5-f93f-448e-8bae-83444e4fa34b-registry-certificates\") pod \"image-registry-7bc8c8cb69-dt9sk\" (UID: \"a97628e5-f93f-448e-8bae-83444e4fa34b\") " pod="openshift-image-registry/image-registry-7bc8c8cb69-dt9sk" Jul 01 16:32:49.049594 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:32:49.049576 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/a97628e5-f93f-448e-8bae-83444e4fa34b-trusted-ca\") pod \"image-registry-7bc8c8cb69-dt9sk\" (UID: \"a97628e5-f93f-448e-8bae-83444e4fa34b\") " pod="openshift-image-registry/image-registry-7bc8c8cb69-dt9sk" Jul 01 16:32:49.050880 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:32:49.050863 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/a97628e5-f93f-448e-8bae-83444e4fa34b-image-registry-private-configuration\") pod \"image-registry-7bc8c8cb69-dt9sk\" (UID: \"a97628e5-f93f-448e-8bae-83444e4fa34b\") " pod="openshift-image-registry/image-registry-7bc8c8cb69-dt9sk" Jul 01 16:32:49.051291 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:32:49.051276 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a97628e5-f93f-448e-8bae-83444e4fa34b-registry-tls\") pod \"image-registry-7bc8c8cb69-dt9sk\" (UID: \"a97628e5-f93f-448e-8bae-83444e4fa34b\") " pod="openshift-image-registry/image-registry-7bc8c8cb69-dt9sk" Jul 01 16:32:49.051411 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:32:49.051390 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/a97628e5-f93f-448e-8bae-83444e4fa34b-installation-pull-secrets\") pod \"image-registry-7bc8c8cb69-dt9sk\" (UID: \"a97628e5-f93f-448e-8bae-83444e4fa34b\") " pod="openshift-image-registry/image-registry-7bc8c8cb69-dt9sk" Jul 01 16:32:49.056615 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:32:49.056593 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a97628e5-f93f-448e-8bae-83444e4fa34b-bound-sa-token\") pod \"image-registry-7bc8c8cb69-dt9sk\" (UID: \"a97628e5-f93f-448e-8bae-83444e4fa34b\") " pod="openshift-image-registry/image-registry-7bc8c8cb69-dt9sk" Jul 01 16:32:49.057070 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:32:49.057050 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rqcvt\" (UniqueName: \"kubernetes.io/projected/a97628e5-f93f-448e-8bae-83444e4fa34b-kube-api-access-rqcvt\") pod \"image-registry-7bc8c8cb69-dt9sk\" (UID: \"a97628e5-f93f-448e-8bae-83444e4fa34b\") " pod="openshift-image-registry/image-registry-7bc8c8cb69-dt9sk" Jul 01 16:32:49.171797 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:32:49.171701 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7bc8c8cb69-dt9sk" Jul 01 16:32:49.291352 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:32:49.291327 2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7bc8c8cb69-dt9sk"] Jul 01 16:32:49.293652 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:32:49.293619 2545 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda97628e5_f93f_448e_8bae_83444e4fa34b.slice/crio-2b3b5bf4ca5c1545fe8bbd2676f0d01ed6cd9bb88f0919666d4ea227a2ef8993 WatchSource:0}: Error finding container 2b3b5bf4ca5c1545fe8bbd2676f0d01ed6cd9bb88f0919666d4ea227a2ef8993: Status 404 returned error can't find the container with id 2b3b5bf4ca5c1545fe8bbd2676f0d01ed6cd9bb88f0919666d4ea227a2ef8993 Jul 01 16:32:49.700538 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:32:49.700491 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7bc8c8cb69-dt9sk" event={"ID":"a97628e5-f93f-448e-8bae-83444e4fa34b","Type":"ContainerStarted","Data":"d0115d46239e6d02376a52dc24e6579cbdd82184932ba097f3aed412214b9559"} Jul 01 16:32:49.700703 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:32:49.700560 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7bc8c8cb69-dt9sk" event={"ID":"a97628e5-f93f-448e-8bae-83444e4fa34b","Type":"ContainerStarted","Data":"2b3b5bf4ca5c1545fe8bbd2676f0d01ed6cd9bb88f0919666d4ea227a2ef8993"} Jul 01 16:32:49.700703 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:32:49.700626 2545 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-7bc8c8cb69-dt9sk" Jul 01 16:32:49.718688 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:32:49.718646 2545 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-7bc8c8cb69-dt9sk" podStartSLOduration=1.71863309 podStartE2EDuration="1.71863309s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 16:32:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 16:32:49.717983972 +0000 UTC m=+248.298601475" watchObservedRunningTime="2026-07-01 16:32:49.71863309 +0000 UTC m=+248.299250593" Jul 01 16:32:54.888304 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:32:54.888270 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/be2ddffa-466a-4a37-baba-5cb9ecf07e41-metrics-certs\") pod \"network-metrics-daemon-87ztj\" (UID: \"be2ddffa-466a-4a37-baba-5cb9ecf07e41\") " pod="openshift-multus/network-metrics-daemon-87ztj" Jul 01 16:32:54.890432 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:32:54.890409 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/be2ddffa-466a-4a37-baba-5cb9ecf07e41-metrics-certs\") pod \"network-metrics-daemon-87ztj\" (UID: \"be2ddffa-466a-4a37-baba-5cb9ecf07e41\") " pod="openshift-multus/network-metrics-daemon-87ztj" Jul 01 16:32:55.033081 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:32:55.033050 2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-jf2dq\"" Jul 01 16:32:55.041073 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:32:55.041054 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-87ztj" Jul 01 16:32:55.158523 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:32:55.158449 2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-87ztj"] Jul 01 16:32:55.161936 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:32:55.161907 2545 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbe2ddffa_466a_4a37_baba_5cb9ecf07e41.slice/crio-07a85d5be42b38d449ff4eccb748341eac7c235355fbce77109a45630013ac29 WatchSource:0}: Error finding container 07a85d5be42b38d449ff4eccb748341eac7c235355fbce77109a45630013ac29: Status 404 returned error can't find the container with id 07a85d5be42b38d449ff4eccb748341eac7c235355fbce77109a45630013ac29 Jul 01 16:32:55.719509 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:32:55.719477 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-87ztj" event={"ID":"be2ddffa-466a-4a37-baba-5cb9ecf07e41","Type":"ContainerStarted","Data":"07a85d5be42b38d449ff4eccb748341eac7c235355fbce77109a45630013ac29"} Jul 01 16:32:56.724377 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:32:56.724334 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-87ztj" event={"ID":"be2ddffa-466a-4a37-baba-5cb9ecf07e41","Type":"ContainerStarted","Data":"ecd33e00edf70114ededdbcfec706931ae8c7fd1e400f11ac692f457d3c8521d"} Jul 01 16:32:56.724763 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:32:56.724387 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-87ztj" event={"ID":"be2ddffa-466a-4a37-baba-5cb9ecf07e41","Type":"ContainerStarted","Data":"47cd635f16ba0726e7f73e2364bdfcfed3a1e1d4ae7e38017792b3de46316737"} Jul 01 16:32:56.740424 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:32:56.740373 2545 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-87ztj" podStartSLOduration=252.717865699 podStartE2EDuration="4m13.740357439s" totalImagesPullingTime="1.02249174s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 16:28:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 16:32:56.738833839 +0000 UTC m=+255.319451344" watchObservedRunningTime="2026-07-01 16:32:56.740357439 +0000 UTC m=+255.320974944" Jul 01 16:32:58.514139 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:32:58.514099 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/059f2f9d-a756-4692-9452-9a9b0ffb55ce-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2bzw5\" (UID: \"059f2f9d-a756-4692-9452-9a9b0ffb55ce\") " pod="openshift-insights/insights-runtime-extractor-2bzw5" Jul 01 16:32:58.516423 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:32:58.516395 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/059f2f9d-a756-4692-9452-9a9b0ffb55ce-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2bzw5\" (UID: \"059f2f9d-a756-4692-9452-9a9b0ffb55ce\") " pod="openshift-insights/insights-runtime-extractor-2bzw5" Jul 01 16:32:58.566236 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:32:58.566211 2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-zs8wh\"" Jul 01 16:32:58.573391 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:32:58.573371 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-2bzw5" Jul 01 16:32:58.689625 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:32:58.689601 2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-2bzw5"] Jul 01 16:32:58.691969 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:32:58.691945 2545 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod059f2f9d_a756_4692_9452_9a9b0ffb55ce.slice/crio-7b4cb8564ca2f38694e016f5429687f0abd65e1286d55705a1ff73fdb7873473 WatchSource:0}: Error finding container 7b4cb8564ca2f38694e016f5429687f0abd65e1286d55705a1ff73fdb7873473: Status 404 returned error can't find the container with id 7b4cb8564ca2f38694e016f5429687f0abd65e1286d55705a1ff73fdb7873473 Jul 01 16:32:58.730935 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:32:58.730908 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-2bzw5" event={"ID":"059f2f9d-a756-4692-9452-9a9b0ffb55ce","Type":"ContainerStarted","Data":"7b4cb8564ca2f38694e016f5429687f0abd65e1286d55705a1ff73fdb7873473"} Jul 01 16:33:02.746786 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:33:02.746683 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-2bzw5" event={"ID":"059f2f9d-a756-4692-9452-9a9b0ffb55ce","Type":"ContainerStarted","Data":"d5a207c6e57f2b58137401e6bdbb6db3fe2ea8194cf7f137c27b5be77f823f8c"} Jul 01 16:33:03.752059 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:33:03.752016 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-2bzw5" event={"ID":"059f2f9d-a756-4692-9452-9a9b0ffb55ce","Type":"ContainerStarted","Data":"fe3b4f0d1aa807a6b2eb54ba9a66311b2068ee38f244af95ebf5795fd6f14259"} Jul 01 16:33:05.759822 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:33:05.759789 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-2bzw5" event={"ID":"059f2f9d-a756-4692-9452-9a9b0ffb55ce","Type":"ContainerStarted","Data":"b5bd79ad02fd6e5652a6198edcd93c1f360687599dd8ea3567fdd134bd9e3242"} Jul 01 16:33:05.776556 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:33:05.776504 2545 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-2bzw5" podStartSLOduration=33.042032879 podStartE2EDuration="39.776486588s" totalImagesPullingTime="6.734453709s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-01 16:32:26 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 16:33:05.77558178 +0000 UTC m=+264.356199285" watchObservedRunningTime="2026-07-01 16:33:05.776486588 +0000 UTC m=+264.357104092" Jul 01 16:33:07.858787 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:33:07.858740 2545 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5bcdc4cb7d-hrl7g" podUID="7330d150-1a30-4898-a108-c7e969c724dd" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500" Jul 01 16:33:08.517130 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:33:08.517100 2545 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-845b954455-hbgb8"] Jul 01 16:33:08.520323 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:33:08.520304 2545 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-rbnqk"] Jul 01 16:33:08.520436 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:33:08.520418 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-845b954455-hbgb8" Jul 01 16:33:08.523194 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:33:08.523175 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-rbnqk" Jul 01 16:33:08.524126 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:33:08.524108 2545 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 01 16:33:08.524126 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:33:08.524119 2545 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 01 16:33:08.524279 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:33:08.524108 2545 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 01 16:33:08.524345 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:33:08.524327 2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\"" Jul 01 16:33:08.524403 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:33:08.524337 2545 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\"" Jul 01 16:33:08.524403 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:33:08.524394 2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-xwncv\"" Jul 01 16:33:08.524478 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:33:08.524381 2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\"" Jul 01 16:33:08.525553 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:33:08.525536 2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-fx4fv\"" Jul 01 16:33:08.525660 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:33:08.525575 2545 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 01 16:33:08.525780 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:33:08.525765 2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 01 16:33:08.526296 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:33:08.526279 2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 01 16:33:08.533699 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:33:08.533680 2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-845b954455-hbgb8"] Jul 01 16:33:08.689799 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:33:08.689768 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/631adfb3-d231-4507-a3d5-d315e3f81d94-sys\") pod \"node-exporter-rbnqk\" (UID: \"631adfb3-d231-4507-a3d5-d315e3f81d94\") " pod="openshift-monitoring/node-exporter-rbnqk" Jul 01 16:33:08.689799 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:33:08.689798 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/631adfb3-d231-4507-a3d5-d315e3f81d94-node-exporter-textfile\") pod \"node-exporter-rbnqk\" (UID: \"631adfb3-d231-4507-a3d5-d315e3f81d94\") " pod="openshift-monitoring/node-exporter-rbnqk" Jul 01 16:33:08.689968 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:33:08.689825 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d3172b59-6654-4770-b297-db32819ec89b-metrics-client-ca\") pod \"kube-state-metrics-845b954455-hbgb8\" (UID: \"d3172b59-6654-4770-b297-db32819ec89b\") " pod="openshift-monitoring/kube-state-metrics-845b954455-hbgb8" Jul 01 16:33:08.689968 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:33:08.689843 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/631adfb3-d231-4507-a3d5-d315e3f81d94-node-exporter-wtmp\") pod \"node-exporter-rbnqk\" (UID: \"631adfb3-d231-4507-a3d5-d315e3f81d94\") " pod="openshift-monitoring/node-exporter-rbnqk" Jul 01 16:33:08.689968 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:33:08.689895 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/d3172b59-6654-4770-b297-db32819ec89b-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-845b954455-hbgb8\" (UID: \"d3172b59-6654-4770-b297-db32819ec89b\") " pod="openshift-monitoring/kube-state-metrics-845b954455-hbgb8" Jul 01 16:33:08.689968 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:33:08.689922 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/631adfb3-d231-4507-a3d5-d315e3f81d94-root\") pod \"node-exporter-rbnqk\" (UID: \"631adfb3-d231-4507-a3d5-d315e3f81d94\") " pod="openshift-monitoring/node-exporter-rbnqk" Jul 01 16:33:08.689968 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:33:08.689946 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d3172b59-6654-4770-b297-db32819ec89b-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-845b954455-hbgb8\" (UID: \"d3172b59-6654-4770-b297-db32819ec89b\") " pod="openshift-monitoring/kube-state-metrics-845b954455-hbgb8" Jul 01 16:33:08.689968 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:33:08.689965 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/d3172b59-6654-4770-b297-db32819ec89b-kube-state-metrics-tls\") pod \"kube-state-metrics-845b954455-hbgb8\" (UID: \"d3172b59-6654-4770-b297-db32819ec89b\") " pod="openshift-monitoring/kube-state-metrics-845b954455-hbgb8" Jul 01 16:33:08.690193 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:33:08.690010 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/d3172b59-6654-4770-b297-db32819ec89b-volume-directive-shadow\") pod \"kube-state-metrics-845b954455-hbgb8\" (UID: \"d3172b59-6654-4770-b297-db32819ec89b\") " pod="openshift-monitoring/kube-state-metrics-845b954455-hbgb8" Jul 01 16:33:08.690193 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:33:08.690030 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/631adfb3-d231-4507-a3d5-d315e3f81d94-node-exporter-tls\") pod \"node-exporter-rbnqk\" (UID: \"631adfb3-d231-4507-a3d5-d315e3f81d94\") " pod="openshift-monitoring/node-exporter-rbnqk" Jul 01 16:33:08.690193 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:33:08.690045 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/631adfb3-d231-4507-a3d5-d315e3f81d94-metrics-client-ca\") pod \"node-exporter-rbnqk\" (UID: \"631adfb3-d231-4507-a3d5-d315e3f81d94\") " pod="openshift-monitoring/node-exporter-rbnqk" Jul 01 16:33:08.690193 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:33:08.690070 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/631adfb3-d231-4507-a3d5-d315e3f81d94-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-rbnqk\" (UID: \"631adfb3-d231-4507-a3d5-d315e3f81d94\") " pod="openshift-monitoring/node-exporter-rbnqk" Jul 01 16:33:08.690193 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:33:08.690119 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nl55b\" (UniqueName: \"kubernetes.io/projected/631adfb3-d231-4507-a3d5-d315e3f81d94-kube-api-access-nl55b\") pod \"node-exporter-rbnqk\" (UID: \"631adfb3-d231-4507-a3d5-d315e3f81d94\") " pod="openshift-monitoring/node-exporter-rbnqk" Jul 01 16:33:08.690193 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:33:08.690145 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6h5g2\" (UniqueName: \"kubernetes.io/projected/d3172b59-6654-4770-b297-db32819ec89b-kube-api-access-6h5g2\") pod \"kube-state-metrics-845b954455-hbgb8\" (UID: \"d3172b59-6654-4770-b297-db32819ec89b\") " pod="openshift-monitoring/kube-state-metrics-845b954455-hbgb8" Jul 01 16:33:08.690193 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:33:08.690176 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/631adfb3-d231-4507-a3d5-d315e3f81d94-node-exporter-accelerators-collector-config\") pod \"node-exporter-rbnqk\" (UID: \"631adfb3-d231-4507-a3d5-d315e3f81d94\") " pod="openshift-monitoring/node-exporter-rbnqk" Jul 01 16:33:08.790783 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:33:08.790693 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d3172b59-6654-4770-b297-db32819ec89b-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-845b954455-hbgb8\" (UID: \"d3172b59-6654-4770-b297-db32819ec89b\") " pod="openshift-monitoring/kube-state-metrics-845b954455-hbgb8" Jul 01 16:33:08.790783 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:33:08.790744 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/d3172b59-6654-4770-b297-db32819ec89b-kube-state-metrics-tls\") pod \"kube-state-metrics-845b954455-hbgb8\" (UID: \"d3172b59-6654-4770-b297-db32819ec89b\") " pod="openshift-monitoring/kube-state-metrics-845b954455-hbgb8" Jul 01 16:33:08.790946 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:33:08.790784 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/d3172b59-6654-4770-b297-db32819ec89b-volume-directive-shadow\") pod \"kube-state-metrics-845b954455-hbgb8\" (UID: \"d3172b59-6654-4770-b297-db32819ec89b\") " pod="openshift-monitoring/kube-state-metrics-845b954455-hbgb8" Jul 01 16:33:08.790946 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:33:08.790803 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/631adfb3-d231-4507-a3d5-d315e3f81d94-node-exporter-tls\") pod \"node-exporter-rbnqk\" (UID: \"631adfb3-d231-4507-a3d5-d315e3f81d94\") " pod="openshift-monitoring/node-exporter-rbnqk" Jul 01 16:33:08.790946 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:33:08.790818 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/631adfb3-d231-4507-a3d5-d315e3f81d94-metrics-client-ca\") pod \"node-exporter-rbnqk\" (UID: \"631adfb3-d231-4507-a3d5-d315e3f81d94\") " pod="openshift-monitoring/node-exporter-rbnqk" Jul 01 16:33:08.790946 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:33:08.790841 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/631adfb3-d231-4507-a3d5-d315e3f81d94-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-rbnqk\" (UID: \"631adfb3-d231-4507-a3d5-d315e3f81d94\") " pod="openshift-monitoring/node-exporter-rbnqk" Jul 01 16:33:08.790946 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:33:08.790869 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nl55b\" (UniqueName: \"kubernetes.io/projected/631adfb3-d231-4507-a3d5-d315e3f81d94-kube-api-access-nl55b\") pod \"node-exporter-rbnqk\" (UID: \"631adfb3-d231-4507-a3d5-d315e3f81d94\") " pod="openshift-monitoring/node-exporter-rbnqk" Jul 01 16:33:08.790946 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:33:08.790893 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6h5g2\" (UniqueName: \"kubernetes.io/projected/d3172b59-6654-4770-b297-db32819ec89b-kube-api-access-6h5g2\") pod \"kube-state-metrics-845b954455-hbgb8\" (UID: \"d3172b59-6654-4770-b297-db32819ec89b\") " pod="openshift-monitoring/kube-state-metrics-845b954455-hbgb8" Jul 01 16:33:08.790946 ip-10-0-130-227 kubenswrapper[2545]: E0701 16:33:08.790920 2545 secret.go:189] Couldn't get secret openshift-monitoring/kube-state-metrics-tls: secret "kube-state-metrics-tls" not found Jul 01 16:33:08.790946 ip-10-0-130-227 kubenswrapper[2545]: E0701 16:33:08.790942 2545 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Jul 01 16:33:08.791328 ip-10-0-130-227 kubenswrapper[2545]: E0701 16:33:08.791000 2545 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d3172b59-6654-4770-b297-db32819ec89b-kube-state-metrics-tls podName:d3172b59-6654-4770-b297-db32819ec89b nodeName:}" failed. No retries permitted until 2026-07-01 16:33:09.290976988 +0000 UTC m=+267.871594487 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-state-metrics-tls" (UniqueName: "kubernetes.io/secret/d3172b59-6654-4770-b297-db32819ec89b-kube-state-metrics-tls") pod "kube-state-metrics-845b954455-hbgb8" (UID: "d3172b59-6654-4770-b297-db32819ec89b") : secret "kube-state-metrics-tls" not found Jul 01 16:33:08.791328 ip-10-0-130-227 kubenswrapper[2545]: E0701 16:33:08.791019 2545 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/631adfb3-d231-4507-a3d5-d315e3f81d94-node-exporter-tls podName:631adfb3-d231-4507-a3d5-d315e3f81d94 nodeName:}" failed. No retries permitted until 2026-07-01 16:33:09.291010087 +0000 UTC m=+267.871627584 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/631adfb3-d231-4507-a3d5-d315e3f81d94-node-exporter-tls") pod "node-exporter-rbnqk" (UID: "631adfb3-d231-4507-a3d5-d315e3f81d94") : secret "node-exporter-tls" not found Jul 01 16:33:08.791328 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:33:08.791194 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/631adfb3-d231-4507-a3d5-d315e3f81d94-node-exporter-accelerators-collector-config\") pod \"node-exporter-rbnqk\" (UID: \"631adfb3-d231-4507-a3d5-d315e3f81d94\") " pod="openshift-monitoring/node-exporter-rbnqk" Jul 01 16:33:08.791328 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:33:08.791245 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/631adfb3-d231-4507-a3d5-d315e3f81d94-sys\") pod \"node-exporter-rbnqk\" (UID: \"631adfb3-d231-4507-a3d5-d315e3f81d94\") " pod="openshift-monitoring/node-exporter-rbnqk" Jul 01 16:33:08.791328 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:33:08.791264 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/d3172b59-6654-4770-b297-db32819ec89b-volume-directive-shadow\") pod \"kube-state-metrics-845b954455-hbgb8\" (UID: \"d3172b59-6654-4770-b297-db32819ec89b\") " pod="openshift-monitoring/kube-state-metrics-845b954455-hbgb8" Jul 01 16:33:08.791328 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:33:08.791276 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/631adfb3-d231-4507-a3d5-d315e3f81d94-node-exporter-textfile\") pod \"node-exporter-rbnqk\" (UID: \"631adfb3-d231-4507-a3d5-d315e3f81d94\") " pod="openshift-monitoring/node-exporter-rbnqk" Jul 01 16:33:08.791328 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:33:08.791327 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d3172b59-6654-4770-b297-db32819ec89b-metrics-client-ca\") pod \"kube-state-metrics-845b954455-hbgb8\" (UID: \"d3172b59-6654-4770-b297-db32819ec89b\") " pod="openshift-monitoring/kube-state-metrics-845b954455-hbgb8" Jul 01 16:33:08.791673 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:33:08.791359 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/631adfb3-d231-4507-a3d5-d315e3f81d94-node-exporter-wtmp\") pod \"node-exporter-rbnqk\" (UID: \"631adfb3-d231-4507-a3d5-d315e3f81d94\") " pod="openshift-monitoring/node-exporter-rbnqk" Jul 01 16:33:08.791673 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:33:08.791399 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/d3172b59-6654-4770-b297-db32819ec89b-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-845b954455-hbgb8\" (UID: \"d3172b59-6654-4770-b297-db32819ec89b\") " pod="openshift-monitoring/kube-state-metrics-845b954455-hbgb8" Jul 01 16:33:08.791673 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:33:08.791402 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/631adfb3-d231-4507-a3d5-d315e3f81d94-sys\") pod \"node-exporter-rbnqk\" (UID: \"631adfb3-d231-4507-a3d5-d315e3f81d94\") " pod="openshift-monitoring/node-exporter-rbnqk" Jul 01 16:33:08.791673 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:33:08.791433 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/631adfb3-d231-4507-a3d5-d315e3f81d94-root\") pod \"node-exporter-rbnqk\" (UID: \"631adfb3-d231-4507-a3d5-d315e3f81d94\") " pod="openshift-monitoring/node-exporter-rbnqk" Jul 01 16:33:08.791673 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:33:08.791530 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/631adfb3-d231-4507-a3d5-d315e3f81d94-node-exporter-textfile\") pod \"node-exporter-rbnqk\" (UID: \"631adfb3-d231-4507-a3d5-d315e3f81d94\") " pod="openshift-monitoring/node-exporter-rbnqk" Jul 01 16:33:08.791673 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:33:08.791567 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/631adfb3-d231-4507-a3d5-d315e3f81d94-node-exporter-wtmp\") pod \"node-exporter-rbnqk\" (UID: \"631adfb3-d231-4507-a3d5-d315e3f81d94\") " pod="openshift-monitoring/node-exporter-rbnqk" Jul 01 16:33:08.791673 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:33:08.791591 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/631adfb3-d231-4507-a3d5-d315e3f81d94-root\") pod \"node-exporter-rbnqk\" (UID: \"631adfb3-d231-4507-a3d5-d315e3f81d94\") " pod="openshift-monitoring/node-exporter-rbnqk" Jul 01 16:33:08.792058 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:33:08.791750 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/631adfb3-d231-4507-a3d5-d315e3f81d94-node-exporter-accelerators-collector-config\") pod \"node-exporter-rbnqk\" (UID: \"631adfb3-d231-4507-a3d5-d315e3f81d94\") " pod="openshift-monitoring/node-exporter-rbnqk" Jul 01 16:33:08.792110 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:33:08.792084 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d3172b59-6654-4770-b297-db32819ec89b-metrics-client-ca\") pod \"kube-state-metrics-845b954455-hbgb8\" (UID: \"d3172b59-6654-4770-b297-db32819ec89b\") " pod="openshift-monitoring/kube-state-metrics-845b954455-hbgb8" Jul 01 16:33:08.792172 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:33:08.792121 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/d3172b59-6654-4770-b297-db32819ec89b-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-845b954455-hbgb8\" (UID: \"d3172b59-6654-4770-b297-db32819ec89b\") " pod="openshift-monitoring/kube-state-metrics-845b954455-hbgb8" Jul 01 16:33:08.792224 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:33:08.792195 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/631adfb3-d231-4507-a3d5-d315e3f81d94-metrics-client-ca\") pod \"node-exporter-rbnqk\" (UID: \"631adfb3-d231-4507-a3d5-d315e3f81d94\") " pod="openshift-monitoring/node-exporter-rbnqk" Jul 01 16:33:08.793315 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:33:08.793295 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/631adfb3-d231-4507-a3d5-d315e3f81d94-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-rbnqk\" (UID: \"631adfb3-d231-4507-a3d5-d315e3f81d94\") " pod="openshift-monitoring/node-exporter-rbnqk" Jul 01 16:33:08.793457 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:33:08.793422 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d3172b59-6654-4770-b297-db32819ec89b-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-845b954455-hbgb8\" (UID: \"d3172b59-6654-4770-b297-db32819ec89b\") " pod="openshift-monitoring/kube-state-metrics-845b954455-hbgb8" Jul 01 16:33:08.801177 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:33:08.801152 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nl55b\" (UniqueName: \"kubernetes.io/projected/631adfb3-d231-4507-a3d5-d315e3f81d94-kube-api-access-nl55b\") pod \"node-exporter-rbnqk\" (UID: \"631adfb3-d231-4507-a3d5-d315e3f81d94\") " pod="openshift-monitoring/node-exporter-rbnqk" Jul 01 16:33:08.801743 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:33:08.801725 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6h5g2\" (UniqueName: \"kubernetes.io/projected/d3172b59-6654-4770-b297-db32819ec89b-kube-api-access-6h5g2\") pod \"kube-state-metrics-845b954455-hbgb8\" (UID: \"d3172b59-6654-4770-b297-db32819ec89b\") " pod="openshift-monitoring/kube-state-metrics-845b954455-hbgb8" Jul 01 16:33:09.176076 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:33:09.176042 2545 patch_prober.go:28] interesting pod/image-registry-7bc8c8cb69-dt9sk container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jul 01 16:33:09.176442 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:33:09.176096 2545 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-7bc8c8cb69-dt9sk" podUID="a97628e5-f93f-448e-8bae-83444e4fa34b" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 01 16:33:09.294643 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:33:09.294613 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/631adfb3-d231-4507-a3d5-d315e3f81d94-node-exporter-tls\") pod \"node-exporter-rbnqk\" (UID: \"631adfb3-d231-4507-a3d5-d315e3f81d94\") " pod="openshift-monitoring/node-exporter-rbnqk" Jul 01 16:33:09.294820 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:33:09.294673 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/d3172b59-6654-4770-b297-db32819ec89b-kube-state-metrics-tls\") pod \"kube-state-metrics-845b954455-hbgb8\" (UID: \"d3172b59-6654-4770-b297-db32819ec89b\") " pod="openshift-monitoring/kube-state-metrics-845b954455-hbgb8" Jul 01 16:33:09.297020 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:33:09.296997 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/631adfb3-d231-4507-a3d5-d315e3f81d94-node-exporter-tls\") pod \"node-exporter-rbnqk\" (UID: \"631adfb3-d231-4507-a3d5-d315e3f81d94\") " pod="openshift-monitoring/node-exporter-rbnqk" Jul 01 16:33:09.297081 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:33:09.297066 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/d3172b59-6654-4770-b297-db32819ec89b-kube-state-metrics-tls\") pod \"kube-state-metrics-845b954455-hbgb8\" (UID: \"d3172b59-6654-4770-b297-db32819ec89b\") " pod="openshift-monitoring/kube-state-metrics-845b954455-hbgb8" Jul 01 16:33:09.431585 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:33:09.431507 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-845b954455-hbgb8" Jul 01 16:33:09.436882 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:33:09.436863 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-rbnqk" Jul 01 16:33:09.447071 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:33:09.447049 2545 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod631adfb3_d231_4507_a3d5_d315e3f81d94.slice/crio-e89a057e51991815c12283350ad8f84d2c408c3d9d85fe7684a995fd07a79c64 WatchSource:0}: Error finding container e89a057e51991815c12283350ad8f84d2c408c3d9d85fe7684a995fd07a79c64: Status 404 returned error can't find the container with id e89a057e51991815c12283350ad8f84d2c408c3d9d85fe7684a995fd07a79c64 Jul 01 16:33:09.558055 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:33:09.558028 2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-845b954455-hbgb8"] Jul 01 16:33:09.560099 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:33:09.560068 2545 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd3172b59_6654_4770_b297_db32819ec89b.slice/crio-31021d7fb487761e7d50f2c0b2ca8e5b2ffbf45cb2ada27054926cc21b95c066 WatchSource:0}: Error finding container 31021d7fb487761e7d50f2c0b2ca8e5b2ffbf45cb2ada27054926cc21b95c066: Status 404 returned error can't find the container with id 31021d7fb487761e7d50f2c0b2ca8e5b2ffbf45cb2ada27054926cc21b95c066 Jul 01 16:33:09.772789 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:33:09.772690 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-rbnqk" event={"ID":"631adfb3-d231-4507-a3d5-d315e3f81d94","Type":"ContainerStarted","Data":"e89a057e51991815c12283350ad8f84d2c408c3d9d85fe7684a995fd07a79c64"} Jul 01 16:33:09.773675 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:33:09.773655 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-845b954455-hbgb8" event={"ID":"d3172b59-6654-4770-b297-db32819ec89b","Type":"ContainerStarted","Data":"31021d7fb487761e7d50f2c0b2ca8e5b2ffbf45cb2ada27054926cc21b95c066"} Jul 01 16:33:10.708445 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:33:10.708408 2545 patch_prober.go:28] interesting pod/image-registry-7bc8c8cb69-dt9sk container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jul 01 16:33:10.708922 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:33:10.708467 2545 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-7bc8c8cb69-dt9sk" podUID="a97628e5-f93f-448e-8bae-83444e4fa34b" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 01 16:33:11.781249 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:33:11.781217 2545 generic.go:356] "Generic (PLEG): container finished" podID="631adfb3-d231-4507-a3d5-d315e3f81d94" containerID="ef40568dc3ae3a70dee568aeddf0b1810c6a178c69144a23c691d7390300b6a2" exitCode=0 Jul 01 16:33:11.781628 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:33:11.781309 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-rbnqk" event={"ID":"631adfb3-d231-4507-a3d5-d315e3f81d94","Type":"ContainerDied","Data":"ef40568dc3ae3a70dee568aeddf0b1810c6a178c69144a23c691d7390300b6a2"} Jul 01 16:33:11.783432 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:33:11.783409 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-845b954455-hbgb8" event={"ID":"d3172b59-6654-4770-b297-db32819ec89b","Type":"ContainerStarted","Data":"b080534b11139fb7305dcae1456f1836acd5aa4ca6ea321e710594c603d6f75f"} Jul 01 16:33:11.783515 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:33:11.783453 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-845b954455-hbgb8" event={"ID":"d3172b59-6654-4770-b297-db32819ec89b","Type":"ContainerStarted","Data":"6989fc2ad312eba662e54ec74299cf31f5665af8bec0f553579de0cd1bc7b549"} Jul 01 16:33:11.783515 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:33:11.783485 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-845b954455-hbgb8" event={"ID":"d3172b59-6654-4770-b297-db32819ec89b","Type":"ContainerStarted","Data":"b90c044ae14c51d48a2b874c0306719be4eadca4ed6a65d5d1d57e660af6d3ff"} Jul 01 16:33:11.818121 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:33:11.818082 2545 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-845b954455-hbgb8" podStartSLOduration=2.336681049 podStartE2EDuration="3.818068663s" totalImagesPullingTime="1.481387614s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 16:33:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 16:33:11.817611141 +0000 UTC m=+270.398228645" watchObservedRunningTime="2026-07-01 16:33:11.818068663 +0000 UTC m=+270.398686166" Jul 01 16:33:12.791445 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:33:12.791404 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-rbnqk" event={"ID":"631adfb3-d231-4507-a3d5-d315e3f81d94","Type":"ContainerStarted","Data":"069ac8c49ceb6a828b84e45d9938b66a1b9500f0e2aa01ffc07852de4b65462f"} Jul 01 16:33:12.791922 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:33:12.791473 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-rbnqk" event={"ID":"631adfb3-d231-4507-a3d5-d315e3f81d94","Type":"ContainerStarted","Data":"c5e00cd1ef1c31e6cb126288aefc7be696115e7a078bdd4340b512b8929eb5b3"} Jul 01 16:33:12.812828 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:33:12.812781 2545 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-rbnqk" podStartSLOduration=3.13689165 podStartE2EDuration="4.812766133s" totalImagesPullingTime="1.596374211s" totalInitContainerRuntime="79.500272ms" isStatefulPod=true podCreationTimestamp="2026-07-01 16:33:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 16:33:12.812474289 +0000 UTC m=+271.393091792" watchObservedRunningTime="2026-07-01 16:33:12.812766133 +0000 UTC m=+271.393383647" Jul 01 16:33:17.859056 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:33:17.859017 2545 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5bcdc4cb7d-hrl7g" podUID="7330d150-1a30-4898-a108-c7e969c724dd" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500" Jul 01 16:33:19.176462 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:33:19.176427 2545 patch_prober.go:28] interesting pod/image-registry-7bc8c8cb69-dt9sk container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jul 01 16:33:19.176829 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:33:19.176474 2545 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-7bc8c8cb69-dt9sk" podUID="a97628e5-f93f-448e-8bae-83444e4fa34b" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 01 16:33:20.707906 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:33:20.707876 2545 patch_prober.go:28] interesting pod/image-registry-7bc8c8cb69-dt9sk container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jul 01 16:33:20.708303 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:33:20.707924 2545 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-7bc8c8cb69-dt9sk" podUID="a97628e5-f93f-448e-8bae-83444e4fa34b" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 01 16:33:24.455796 ip-10-0-130-227 kubenswrapper[2545]: E0701 16:33:24.455748 2545 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-55sd4" podUID="c3cf9697-8b56-4162-b761-39c2a0557b3b" Jul 01 16:33:24.830819 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:33:24.830792 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-55sd4" Jul 01 16:33:27.721688 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:33:27.721656 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c3cf9697-8b56-4162-b761-39c2a0557b3b-metrics-tls\") pod \"dns-default-55sd4\" (UID: \"c3cf9697-8b56-4162-b761-39c2a0557b3b\") " pod="openshift-dns/dns-default-55sd4" Jul 01 16:33:27.723896 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:33:27.723878 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c3cf9697-8b56-4162-b761-39c2a0557b3b-metrics-tls\") pod \"dns-default-55sd4\" (UID: \"c3cf9697-8b56-4162-b761-39c2a0557b3b\") " pod="openshift-dns/dns-default-55sd4" Jul 01 16:33:27.834009 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:33:27.833979 2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-k2lss\"" Jul 01 16:33:27.841939 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:33:27.841920 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-55sd4" Jul 01 16:33:27.858295 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:33:27.858266 2545 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5bcdc4cb7d-hrl7g" podUID="7330d150-1a30-4898-a108-c7e969c724dd" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500" Jul 01 16:33:27.858387 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:33:27.858326 2545 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5bcdc4cb7d-hrl7g" Jul 01 16:33:27.858812 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:33:27.858774 2545 kuberuntime_manager.go:1192] "Message for Container of pod" containerName="service-proxy" containerStatusID={"Type":"cri-o","ID":"b778e904639607edead2a777879983c9acd1833364e74d8f2cd22f0d15fad878"} pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5bcdc4cb7d-hrl7g" containerMessage="Container service-proxy failed liveness probe, will be restarted" Jul 01 16:33:27.858870 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:33:27.858826 2545 kuberuntime_container.go:872] "Killing container with a grace period" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5bcdc4cb7d-hrl7g" podUID="7330d150-1a30-4898-a108-c7e969c724dd" containerName="service-proxy" containerID="cri-o://b778e904639607edead2a777879983c9acd1833364e74d8f2cd22f0d15fad878" gracePeriod=30 Jul 01 16:33:27.966364 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:33:27.966342 2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-55sd4"] Jul 01 16:33:27.968022 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:33:27.967995 2545 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc3cf9697_8b56_4162_b761_39c2a0557b3b.slice/crio-cb0c142511803e1d2caa4c919fb2c1f153ff22bcd103a9c4792ece10450432ff WatchSource:0}: Error finding container cb0c142511803e1d2caa4c919fb2c1f153ff22bcd103a9c4792ece10450432ff: Status 404 returned error can't find the container with id cb0c142511803e1d2caa4c919fb2c1f153ff22bcd103a9c4792ece10450432ff Jul 01 16:33:28.845614 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:33:28.845583 2545 generic.go:356] "Generic (PLEG): container finished" podID="7330d150-1a30-4898-a108-c7e969c724dd" containerID="b778e904639607edead2a777879983c9acd1833364e74d8f2cd22f0d15fad878" exitCode=2 Jul 01 16:33:28.846085 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:33:28.845667 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5bcdc4cb7d-hrl7g" event={"ID":"7330d150-1a30-4898-a108-c7e969c724dd","Type":"ContainerDied","Data":"b778e904639607edead2a777879983c9acd1833364e74d8f2cd22f0d15fad878"} Jul 01 16:33:28.846085 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:33:28.845733 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5bcdc4cb7d-hrl7g" event={"ID":"7330d150-1a30-4898-a108-c7e969c724dd","Type":"ContainerStarted","Data":"d0620c5c2a6117a5552efad4e017ce65f058bfaf2c29e76cfff2a0c6693e235d"} Jul 01 16:33:28.847092 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:33:28.847059 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-55sd4" event={"ID":"c3cf9697-8b56-4162-b761-39c2a0557b3b","Type":"ContainerStarted","Data":"cb0c142511803e1d2caa4c919fb2c1f153ff22bcd103a9c4792ece10450432ff"} Jul 01 16:33:29.177268 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:33:29.177189 2545 patch_prober.go:28] interesting pod/image-registry-7bc8c8cb69-dt9sk container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jul 01 16:33:29.177268 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:33:29.177246 2545 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-7bc8c8cb69-dt9sk" podUID="a97628e5-f93f-448e-8bae-83444e4fa34b" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 01 16:33:29.177461 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:33:29.177287 2545 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-image-registry/image-registry-7bc8c8cb69-dt9sk" Jul 01 16:33:29.177782 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:33:29.177758 2545 kuberuntime_manager.go:1192] "Message for Container of pod" containerName="registry" containerStatusID={"Type":"cri-o","ID":"d0115d46239e6d02376a52dc24e6579cbdd82184932ba097f3aed412214b9559"} pod="openshift-image-registry/image-registry-7bc8c8cb69-dt9sk" containerMessage="Container registry failed liveness probe, will be restarted" Jul 01 16:33:29.181882 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:33:29.181846 2545 patch_prober.go:28] interesting pod/image-registry-7bc8c8cb69-dt9sk container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jul 01 16:33:29.181987 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:33:29.181899 2545 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-7bc8c8cb69-dt9sk" podUID="a97628e5-f93f-448e-8bae-83444e4fa34b" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 01 16:33:30.855992 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:33:30.855958 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-55sd4" event={"ID":"c3cf9697-8b56-4162-b761-39c2a0557b3b","Type":"ContainerStarted","Data":"0f97ce17f4b699c8bfc1a3b3e55fce9e06639a3b78cb9efbedcebc8340113a9e"} Jul 01 16:33:30.856439 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:33:30.856005 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-55sd4" event={"ID":"c3cf9697-8b56-4162-b761-39c2a0557b3b","Type":"ContainerStarted","Data":"92a51c77a768b1210bfc5c0bbb4cba3829e938a350372654ec8badc7bee43f6b"} Jul 01 16:33:30.856439 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:33:30.856120 2545 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-55sd4" Jul 01 16:33:30.876352 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:33:30.876314 2545 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-55sd4" podStartSLOduration=252.00173359 podStartE2EDuration="4m13.876301785s" totalImagesPullingTime="1.874568195s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 16:29:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 16:33:30.874476683 +0000 UTC m=+289.455094186" watchObservedRunningTime="2026-07-01 16:33:30.876301785 +0000 UTC m=+289.456919289" Jul 01 16:33:33.867133 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:33:33.867052 2545 generic.go:356] "Generic (PLEG): container finished" podID="2139ad3f-5edf-48ac-b3e1-8c066d528218" containerID="d83b56393615a413c2e8b23c9e42cbfe8e37b231c26d7661d4a2db2eb9cabc34" exitCode=255 Jul 01 16:33:33.867133 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:33:33.867122 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6cbfc95fbb-r4fmh" event={"ID":"2139ad3f-5edf-48ac-b3e1-8c066d528218","Type":"ContainerDied","Data":"d83b56393615a413c2e8b23c9e42cbfe8e37b231c26d7661d4a2db2eb9cabc34"} Jul 01 16:33:33.867522 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:33:33.867168 2545 scope.go:117] "RemoveContainer" containerID="e6000a603637b7c2a5f3e6d3f57972e2312e8dd5c4927dac52c5c6f467665139" Jul 01 16:33:33.867522 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:33:33.867443 2545 scope.go:117] "RemoveContainer" containerID="d83b56393615a413c2e8b23c9e42cbfe8e37b231c26d7661d4a2db2eb9cabc34" Jul 01 16:33:33.867623 ip-10-0-130-227 kubenswrapper[2545]: E0701 16:33:33.867605 2545 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"addon-agent\" with CrashLoopBackOff: \"back-off 10s restarting failed container=addon-agent pod=managed-serviceaccount-addon-agent-6cbfc95fbb-r4fmh_open-cluster-management-agent-addon(2139ad3f-5edf-48ac-b3e1-8c066d528218)\"" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6cbfc95fbb-r4fmh" podUID="2139ad3f-5edf-48ac-b3e1-8c066d528218" Jul 01 16:33:37.852149 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:33:37.852116 2545 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6cbfc95fbb-r4fmh" Jul 01 16:33:37.852530 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:33:37.852417 2545 scope.go:117] "RemoveContainer" containerID="d83b56393615a413c2e8b23c9e42cbfe8e37b231c26d7661d4a2db2eb9cabc34" Jul 01 16:33:37.852570 ip-10-0-130-227 kubenswrapper[2545]: E0701 16:33:37.852554 2545 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"addon-agent\" with CrashLoopBackOff: \"back-off 10s restarting failed container=addon-agent pod=managed-serviceaccount-addon-agent-6cbfc95fbb-r4fmh_open-cluster-management-agent-addon(2139ad3f-5edf-48ac-b3e1-8c066d528218)\"" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6cbfc95fbb-r4fmh" podUID="2139ad3f-5edf-48ac-b3e1-8c066d528218" Jul 01 16:33:39.181757 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:33:39.181720 2545 patch_prober.go:28] interesting pod/image-registry-7bc8c8cb69-dt9sk container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jul 01 16:33:39.182117 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:33:39.181766 2545 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-7bc8c8cb69-dt9sk" podUID="a97628e5-f93f-448e-8bae-83444e4fa34b" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 01 16:33:40.861395 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:33:40.861370 2545 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-55sd4" Jul 01 16:33:42.862235 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:33:42.862210 2545 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5cr75_6c03a7ab-a7cc-44ed-86f7-62a750c49b71/ovn-acl-logging/0.log" Jul 01 16:33:42.866160 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:33:42.866143 2545 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5cr75_6c03a7ab-a7cc-44ed-86f7-62a750c49b71/ovn-acl-logging/0.log" Jul 01 16:33:42.867065 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:33:42.867047 2545 kubelet.go:1648] "Image garbage collection succeeded" Jul 01 16:33:48.930101 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:33:48.930071 2545 scope.go:117] "RemoveContainer" containerID="d83b56393615a413c2e8b23c9e42cbfe8e37b231c26d7661d4a2db2eb9cabc34" Jul 01 16:33:49.182191 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:33:49.182116 2545 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-7bc8c8cb69-dt9sk" Jul 01 16:33:49.924482 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:33:49.924444 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6cbfc95fbb-r4fmh" event={"ID":"2139ad3f-5edf-48ac-b3e1-8c066d528218","Type":"ContainerStarted","Data":"d80a4657e28119b518a00e9e31d6a82b71f017832db98139ff0b896960df12c8"} Jul 01 16:33:53.129160 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:33:53.129129 2545 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-55sd4_c3cf9697-8b56-4162-b761-39c2a0557b3b/dns/0.log" Jul 01 16:33:53.334490 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:33:53.334463 2545 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-55sd4_c3cf9697-8b56-4162-b761-39c2a0557b3b/kube-rbac-proxy/0.log" Jul 01 16:33:54.201021 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:33:54.200978 2545 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-image-registry/image-registry-7bc8c8cb69-dt9sk" podUID="a97628e5-f93f-448e-8bae-83444e4fa34b" containerName="registry" containerID="cri-o://d0115d46239e6d02376a52dc24e6579cbdd82184932ba097f3aed412214b9559" gracePeriod=30 Jul 01 16:33:54.729554 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:33:54.729527 2545 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-t7nrc_5be67da1-9165-4446-b3cb-2333db8c09cc/dns-node-resolver/0.log" Jul 01 16:33:55.129298 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:33:55.129275 2545 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-2fxwv_06e3403f-0530-476b-b689-ffc0a7e50132/serve-healthcheck-canary/0.log" Jul 01 16:33:55.946757 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:33:55.946695 2545 generic.go:356] "Generic (PLEG): container finished" podID="a97628e5-f93f-448e-8bae-83444e4fa34b" containerID="d0115d46239e6d02376a52dc24e6579cbdd82184932ba097f3aed412214b9559" exitCode=0 Jul 01 16:33:55.946757 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:33:55.946753 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7bc8c8cb69-dt9sk" event={"ID":"a97628e5-f93f-448e-8bae-83444e4fa34b","Type":"ContainerDied","Data":"d0115d46239e6d02376a52dc24e6579cbdd82184932ba097f3aed412214b9559"} Jul 01 16:33:55.947223 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:33:55.946798 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7bc8c8cb69-dt9sk" event={"ID":"a97628e5-f93f-448e-8bae-83444e4fa34b","Type":"ContainerStarted","Data":"0f9d21794d724ce76988fbeca9a549328bae471d86dd3581bac72de4ce50ede0"} Jul 01 16:33:55.947223 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:33:55.946819 2545 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-7bc8c8cb69-dt9sk" Jul 01 16:34:16.954236 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:34:16.954209 2545 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-7bc8c8cb69-dt9sk" Jul 01 16:37:33.530617 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:37:33.530539 2545 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-khfv4"] Jul 01 16:37:33.533608 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:37:33.533590 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-khfv4" Jul 01 16:37:33.536181 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:37:33.536160 2545 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 01 16:37:33.536181 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:37:33.536171 2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-4qsjl\"" Jul 01 16:37:33.536343 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:37:33.536198 2545 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 01 16:37:33.541625 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:37:33.541602 2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-khfv4"] Jul 01 16:37:33.631184 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:37:33.631154 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/24291043-3c8f-41d0-a01d-aabb3fcc0923-tmp\") pod \"openshift-lws-operator-fd8ccff4c-khfv4\" (UID: \"24291043-3c8f-41d0-a01d-aabb3fcc0923\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-khfv4" Jul 01 16:37:33.631309 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:37:33.631200 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f84m7\" (UniqueName: \"kubernetes.io/projected/24291043-3c8f-41d0-a01d-aabb3fcc0923-kube-api-access-f84m7\") pod \"openshift-lws-operator-fd8ccff4c-khfv4\" (UID: \"24291043-3c8f-41d0-a01d-aabb3fcc0923\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-khfv4" Jul 01 16:37:33.732270 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:37:33.732240 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/24291043-3c8f-41d0-a01d-aabb3fcc0923-tmp\") pod \"openshift-lws-operator-fd8ccff4c-khfv4\" (UID: \"24291043-3c8f-41d0-a01d-aabb3fcc0923\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-khfv4" Jul 01 16:37:33.732270 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:37:33.732273 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f84m7\" (UniqueName: \"kubernetes.io/projected/24291043-3c8f-41d0-a01d-aabb3fcc0923-kube-api-access-f84m7\") pod \"openshift-lws-operator-fd8ccff4c-khfv4\" (UID: \"24291043-3c8f-41d0-a01d-aabb3fcc0923\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-khfv4" Jul 01 16:37:33.732631 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:37:33.732614 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/24291043-3c8f-41d0-a01d-aabb3fcc0923-tmp\") pod \"openshift-lws-operator-fd8ccff4c-khfv4\" (UID: \"24291043-3c8f-41d0-a01d-aabb3fcc0923\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-khfv4" Jul 01 16:37:33.742787 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:37:33.742751 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-f84m7\" (UniqueName: \"kubernetes.io/projected/24291043-3c8f-41d0-a01d-aabb3fcc0923-kube-api-access-f84m7\") pod \"openshift-lws-operator-fd8ccff4c-khfv4\" (UID: \"24291043-3c8f-41d0-a01d-aabb3fcc0923\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-khfv4" Jul 01 16:37:33.842857 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:37:33.842815 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-khfv4" Jul 01 16:37:33.965545 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:37:33.965511 2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-khfv4"] Jul 01 16:37:33.968676 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:37:33.968652 2545 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod24291043_3c8f_41d0_a01d_aabb3fcc0923.slice/crio-e5a54b3cbbc16fadd71232d9216dbf2a9e90d4076d2038ade125d9852bd15a68 WatchSource:0}: Error finding container e5a54b3cbbc16fadd71232d9216dbf2a9e90d4076d2038ade125d9852bd15a68: Status 404 returned error can't find the container with id e5a54b3cbbc16fadd71232d9216dbf2a9e90d4076d2038ade125d9852bd15a68 Jul 01 16:37:33.970038 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:37:33.970021 2545 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 01 16:37:34.639107 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:37:34.639070 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-khfv4" event={"ID":"24291043-3c8f-41d0-a01d-aabb3fcc0923","Type":"ContainerStarted","Data":"e5a54b3cbbc16fadd71232d9216dbf2a9e90d4076d2038ade125d9852bd15a68"} Jul 01 16:37:39.658091 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:37:39.658059 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-khfv4" event={"ID":"24291043-3c8f-41d0-a01d-aabb3fcc0923","Type":"ContainerStarted","Data":"d58eb84d393df14f237c887b17142508ed92b4cd4938a1e3391d88876690df37"} Jul 01 16:37:39.674067 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:37:39.674019 2545 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-khfv4" podStartSLOduration=1.783899006 podStartE2EDuration="6.674004942s" totalImagesPullingTime="4.890105936s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 16:37:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 16:37:39.672955787 +0000 UTC m=+538.253573285" watchObservedRunningTime="2026-07-01 16:37:39.674004942 +0000 UTC m=+538.254622446" Jul 01 16:37:55.892075 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:37:55.892041 2545 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-9bbcfc888-7867m"] Jul 01 16:37:55.899779 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:37:55.899757 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-9bbcfc888-7867m" Jul 01 16:37:55.903127 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:37:55.903103 2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-7vdqb\"" Jul 01 16:37:55.903127 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:37:55.903121 2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 01 16:37:55.903314 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:37:55.903106 2545 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 01 16:37:55.903314 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:37:55.903106 2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 01 16:37:55.903314 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:37:55.903232 2545 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 01 16:37:55.917585 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:37:55.917564 2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-9bbcfc888-7867m"] Jul 01 16:37:55.988509 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:37:55.988480 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a93f7bc8-6c74-4635-a611-0bd42af7c080-tmp\") pod \"opendatahub-operator-controller-manager-9bbcfc888-7867m\" (UID: \"a93f7bc8-6c74-4635-a611-0bd42af7c080\") " pod="opendatahub/opendatahub-operator-controller-manager-9bbcfc888-7867m" Jul 01 16:37:55.988640 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:37:55.988512 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a93f7bc8-6c74-4635-a611-0bd42af7c080-webhook-cert\") pod \"opendatahub-operator-controller-manager-9bbcfc888-7867m\" (UID: \"a93f7bc8-6c74-4635-a611-0bd42af7c080\") " pod="opendatahub/opendatahub-operator-controller-manager-9bbcfc888-7867m" Jul 01 16:37:55.988640 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:37:55.988533 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/a93f7bc8-6c74-4635-a611-0bd42af7c080-apiservice-cert\") pod \"opendatahub-operator-controller-manager-9bbcfc888-7867m\" (UID: \"a93f7bc8-6c74-4635-a611-0bd42af7c080\") " pod="opendatahub/opendatahub-operator-controller-manager-9bbcfc888-7867m" Jul 01 16:37:55.988640 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:37:55.988623 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/a93f7bc8-6c74-4635-a611-0bd42af7c080-manifests\") pod \"opendatahub-operator-controller-manager-9bbcfc888-7867m\" (UID: \"a93f7bc8-6c74-4635-a611-0bd42af7c080\") " pod="opendatahub/opendatahub-operator-controller-manager-9bbcfc888-7867m" Jul 01 16:37:55.988789 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:37:55.988646 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hnvmk\" (UniqueName: \"kubernetes.io/projected/a93f7bc8-6c74-4635-a611-0bd42af7c080-kube-api-access-hnvmk\") pod \"opendatahub-operator-controller-manager-9bbcfc888-7867m\" (UID: \"a93f7bc8-6c74-4635-a611-0bd42af7c080\") " pod="opendatahub/opendatahub-operator-controller-manager-9bbcfc888-7867m" Jul 01 16:37:56.089436 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:37:56.089406 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a93f7bc8-6c74-4635-a611-0bd42af7c080-tmp\") pod \"opendatahub-operator-controller-manager-9bbcfc888-7867m\" (UID: \"a93f7bc8-6c74-4635-a611-0bd42af7c080\") " pod="opendatahub/opendatahub-operator-controller-manager-9bbcfc888-7867m" Jul 01 16:37:56.089572 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:37:56.089445 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a93f7bc8-6c74-4635-a611-0bd42af7c080-webhook-cert\") pod \"opendatahub-operator-controller-manager-9bbcfc888-7867m\" (UID: \"a93f7bc8-6c74-4635-a611-0bd42af7c080\") " pod="opendatahub/opendatahub-operator-controller-manager-9bbcfc888-7867m" Jul 01 16:37:56.089572 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:37:56.089472 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/a93f7bc8-6c74-4635-a611-0bd42af7c080-apiservice-cert\") pod \"opendatahub-operator-controller-manager-9bbcfc888-7867m\" (UID: \"a93f7bc8-6c74-4635-a611-0bd42af7c080\") " pod="opendatahub/opendatahub-operator-controller-manager-9bbcfc888-7867m" Jul 01 16:37:56.089572 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:37:56.089525 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/a93f7bc8-6c74-4635-a611-0bd42af7c080-manifests\") pod \"opendatahub-operator-controller-manager-9bbcfc888-7867m\" (UID: \"a93f7bc8-6c74-4635-a611-0bd42af7c080\") " pod="opendatahub/opendatahub-operator-controller-manager-9bbcfc888-7867m" Jul 01 16:37:56.089572 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:37:56.089548 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hnvmk\" (UniqueName: \"kubernetes.io/projected/a93f7bc8-6c74-4635-a611-0bd42af7c080-kube-api-access-hnvmk\") pod \"opendatahub-operator-controller-manager-9bbcfc888-7867m\" (UID: \"a93f7bc8-6c74-4635-a611-0bd42af7c080\") " pod="opendatahub/opendatahub-operator-controller-manager-9bbcfc888-7867m" Jul 01 16:37:56.089899 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:37:56.089864 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/a93f7bc8-6c74-4635-a611-0bd42af7c080-manifests\") pod \"opendatahub-operator-controller-manager-9bbcfc888-7867m\" (UID: \"a93f7bc8-6c74-4635-a611-0bd42af7c080\") " pod="opendatahub/opendatahub-operator-controller-manager-9bbcfc888-7867m" Jul 01 16:37:56.091741 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:37:56.091695 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a93f7bc8-6c74-4635-a611-0bd42af7c080-tmp\") pod \"opendatahub-operator-controller-manager-9bbcfc888-7867m\" (UID: \"a93f7bc8-6c74-4635-a611-0bd42af7c080\") " pod="opendatahub/opendatahub-operator-controller-manager-9bbcfc888-7867m" Jul 01 16:37:56.091897 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:37:56.091880 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/a93f7bc8-6c74-4635-a611-0bd42af7c080-apiservice-cert\") pod \"opendatahub-operator-controller-manager-9bbcfc888-7867m\" (UID: \"a93f7bc8-6c74-4635-a611-0bd42af7c080\") " pod="opendatahub/opendatahub-operator-controller-manager-9bbcfc888-7867m" Jul 01 16:37:56.091953 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:37:56.091942 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a93f7bc8-6c74-4635-a611-0bd42af7c080-webhook-cert\") pod \"opendatahub-operator-controller-manager-9bbcfc888-7867m\" (UID: \"a93f7bc8-6c74-4635-a611-0bd42af7c080\") " pod="opendatahub/opendatahub-operator-controller-manager-9bbcfc888-7867m" Jul 01 16:37:56.098451 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:37:56.098430 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hnvmk\" (UniqueName: \"kubernetes.io/projected/a93f7bc8-6c74-4635-a611-0bd42af7c080-kube-api-access-hnvmk\") pod \"opendatahub-operator-controller-manager-9bbcfc888-7867m\" (UID: \"a93f7bc8-6c74-4635-a611-0bd42af7c080\") " pod="opendatahub/opendatahub-operator-controller-manager-9bbcfc888-7867m" Jul 01 16:37:56.210033 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:37:56.209954 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-9bbcfc888-7867m" Jul 01 16:37:56.352341 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:37:56.352170 2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-9bbcfc888-7867m"] Jul 01 16:37:56.354998 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:37:56.354972 2545 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda93f7bc8_6c74_4635_a611_0bd42af7c080.slice/crio-69d1e95be57034716e1eefa5f0731f31745103f975f58d87043ce6261cab3c46 WatchSource:0}: Error finding container 69d1e95be57034716e1eefa5f0731f31745103f975f58d87043ce6261cab3c46: Status 404 returned error can't find the container with id 69d1e95be57034716e1eefa5f0731f31745103f975f58d87043ce6261cab3c46 Jul 01 16:37:56.714642 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:37:56.714604 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-9bbcfc888-7867m" event={"ID":"a93f7bc8-6c74-4635-a611-0bd42af7c080","Type":"ContainerStarted","Data":"69d1e95be57034716e1eefa5f0731f31745103f975f58d87043ce6261cab3c46"} Jul 01 16:37:59.727041 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:37:59.727017 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-9bbcfc888-7867m" event={"ID":"a93f7bc8-6c74-4635-a611-0bd42af7c080","Type":"ContainerStarted","Data":"76cf75ee68e2aa71003d5009cb40d6e16a91958757d98de49645b1cca93bf568"} Jul 01 16:38:00.731635 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:38:00.731599 2545 generic.go:356] "Generic (PLEG): container finished" podID="a93f7bc8-6c74-4635-a611-0bd42af7c080" containerID="76cf75ee68e2aa71003d5009cb40d6e16a91958757d98de49645b1cca93bf568" exitCode=0 Jul 01 16:38:00.731635 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:38:00.731633 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-9bbcfc888-7867m" event={"ID":"a93f7bc8-6c74-4635-a611-0bd42af7c080","Type":"ContainerDied","Data":"76cf75ee68e2aa71003d5009cb40d6e16a91958757d98de49645b1cca93bf568"} Jul 01 16:38:01.737189 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:38:01.737156 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-9bbcfc888-7867m" event={"ID":"a93f7bc8-6c74-4635-a611-0bd42af7c080","Type":"ContainerStarted","Data":"c687b20ebd180227c493565a950defef027f299f4b8fa3134f75bf8525b18e3c"} Jul 01 16:38:01.737551 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:38:01.737227 2545 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-9bbcfc888-7867m" Jul 01 16:38:01.759423 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:38:01.759383 2545 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-9bbcfc888-7867m" podStartSLOduration=2.043451108 podStartE2EDuration="6.759366741s" totalImagesPullingTime="3.732857748s" totalInitContainerRuntime="983.057885ms" isStatefulPod=false podCreationTimestamp="2026-07-01 16:37:55 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 16:38:01.75727287 +0000 UTC m=+560.337890376" watchObservedRunningTime="2026-07-01 16:38:01.759366741 +0000 UTC m=+560.339984246" Jul 01 16:38:12.743013 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:38:12.742983 2545 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-9bbcfc888-7867m" Jul 01 16:38:15.175121 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:38:15.175086 2545 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-75f5b69fd7-bqrqm"] Jul 01 16:38:15.177182 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:38:15.177152 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-75f5b69fd7-bqrqm" Jul 01 16:38:15.179766 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:38:15.179743 2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Jul 01 16:38:15.179878 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:38:15.179841 2545 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 01 16:38:15.180768 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:38:15.180750 2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-hcm9l\"" Jul 01 16:38:15.180912 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:38:15.180896 2545 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 01 16:38:15.181024 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:38:15.180980 2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Jul 01 16:38:15.195122 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:38:15.195103 2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-75f5b69fd7-bqrqm"] Jul 01 16:38:15.219816 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:38:15.219795 2545 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-6dc94f689f-566hl"] Jul 01 16:38:15.221590 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:38:15.221576 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-6dc94f689f-566hl" Jul 01 16:38:15.224065 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:38:15.224047 2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-w6th8\"" Jul 01 16:38:15.224154 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:38:15.224052 2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\"" Jul 01 16:38:15.232663 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:38:15.232643 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jd9xm\" (UniqueName: \"kubernetes.io/projected/e0ec6a34-fb56-4865-b460-d76ad8f8f3d7-kube-api-access-jd9xm\") pod \"kube-auth-proxy-75f5b69fd7-bqrqm\" (UID: \"e0ec6a34-fb56-4865-b460-d76ad8f8f3d7\") " pod="openshift-ingress/kube-auth-proxy-75f5b69fd7-bqrqm" Jul 01 16:38:15.232749 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:38:15.232679 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e0ec6a34-fb56-4865-b460-d76ad8f8f3d7-tls-certs\") pod \"kube-auth-proxy-75f5b69fd7-bqrqm\" (UID: \"e0ec6a34-fb56-4865-b460-d76ad8f8f3d7\") " pod="openshift-ingress/kube-auth-proxy-75f5b69fd7-bqrqm" Jul 01 16:38:15.232791 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:38:15.232744 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e0ec6a34-fb56-4865-b460-d76ad8f8f3d7-tmp\") pod \"kube-auth-proxy-75f5b69fd7-bqrqm\" (UID: \"e0ec6a34-fb56-4865-b460-d76ad8f8f3d7\") " pod="openshift-ingress/kube-auth-proxy-75f5b69fd7-bqrqm" Jul 01 16:38:15.236521 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:38:15.236502 2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-6dc94f689f-566hl"] Jul 01 16:38:15.333919 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:38:15.333894 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e0ec6a34-fb56-4865-b460-d76ad8f8f3d7-tmp\") pod \"kube-auth-proxy-75f5b69fd7-bqrqm\" (UID: \"e0ec6a34-fb56-4865-b460-d76ad8f8f3d7\") " pod="openshift-ingress/kube-auth-proxy-75f5b69fd7-bqrqm" Jul 01 16:38:15.334068 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:38:15.333932 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/67da1920-a7f6-4870-85f5-260d7988ce4c-cert\") pod \"odh-model-controller-6dc94f689f-566hl\" (UID: \"67da1920-a7f6-4870-85f5-260d7988ce4c\") " pod="opendatahub/odh-model-controller-6dc94f689f-566hl" Jul 01 16:38:15.334068 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:38:15.333964 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tql5f\" (UniqueName: \"kubernetes.io/projected/67da1920-a7f6-4870-85f5-260d7988ce4c-kube-api-access-tql5f\") pod \"odh-model-controller-6dc94f689f-566hl\" (UID: \"67da1920-a7f6-4870-85f5-260d7988ce4c\") " pod="opendatahub/odh-model-controller-6dc94f689f-566hl" Jul 01 16:38:15.334068 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:38:15.334035 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jd9xm\" (UniqueName: \"kubernetes.io/projected/e0ec6a34-fb56-4865-b460-d76ad8f8f3d7-kube-api-access-jd9xm\") pod \"kube-auth-proxy-75f5b69fd7-bqrqm\" (UID: \"e0ec6a34-fb56-4865-b460-d76ad8f8f3d7\") " pod="openshift-ingress/kube-auth-proxy-75f5b69fd7-bqrqm" Jul 01 16:38:15.334201 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:38:15.334074 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e0ec6a34-fb56-4865-b460-d76ad8f8f3d7-tls-certs\") pod \"kube-auth-proxy-75f5b69fd7-bqrqm\" (UID: \"e0ec6a34-fb56-4865-b460-d76ad8f8f3d7\") " pod="openshift-ingress/kube-auth-proxy-75f5b69fd7-bqrqm" Jul 01 16:38:15.336129 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:38:15.336105 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e0ec6a34-fb56-4865-b460-d76ad8f8f3d7-tmp\") pod \"kube-auth-proxy-75f5b69fd7-bqrqm\" (UID: \"e0ec6a34-fb56-4865-b460-d76ad8f8f3d7\") " pod="openshift-ingress/kube-auth-proxy-75f5b69fd7-bqrqm" Jul 01 16:38:15.336299 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:38:15.336278 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e0ec6a34-fb56-4865-b460-d76ad8f8f3d7-tls-certs\") pod \"kube-auth-proxy-75f5b69fd7-bqrqm\" (UID: \"e0ec6a34-fb56-4865-b460-d76ad8f8f3d7\") " pod="openshift-ingress/kube-auth-proxy-75f5b69fd7-bqrqm" Jul 01 16:38:15.342280 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:38:15.342260 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jd9xm\" (UniqueName: \"kubernetes.io/projected/e0ec6a34-fb56-4865-b460-d76ad8f8f3d7-kube-api-access-jd9xm\") pod \"kube-auth-proxy-75f5b69fd7-bqrqm\" (UID: \"e0ec6a34-fb56-4865-b460-d76ad8f8f3d7\") " pod="openshift-ingress/kube-auth-proxy-75f5b69fd7-bqrqm" Jul 01 16:38:15.435118 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:38:15.435063 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/67da1920-a7f6-4870-85f5-260d7988ce4c-cert\") pod \"odh-model-controller-6dc94f689f-566hl\" (UID: \"67da1920-a7f6-4870-85f5-260d7988ce4c\") " pod="opendatahub/odh-model-controller-6dc94f689f-566hl" Jul 01 16:38:15.435118 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:38:15.435100 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tql5f\" (UniqueName: \"kubernetes.io/projected/67da1920-a7f6-4870-85f5-260d7988ce4c-kube-api-access-tql5f\") pod \"odh-model-controller-6dc94f689f-566hl\" (UID: \"67da1920-a7f6-4870-85f5-260d7988ce4c\") " pod="opendatahub/odh-model-controller-6dc94f689f-566hl" Jul 01 16:38:15.435268 ip-10-0-130-227 kubenswrapper[2545]: E0701 16:38:15.435195 2545 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found Jul 01 16:38:15.435306 ip-10-0-130-227 kubenswrapper[2545]: E0701 16:38:15.435267 2545 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/67da1920-a7f6-4870-85f5-260d7988ce4c-cert podName:67da1920-a7f6-4870-85f5-260d7988ce4c nodeName:}" failed. No retries permitted until 2026-07-01 16:38:15.93525306 +0000 UTC m=+574.515870542 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/67da1920-a7f6-4870-85f5-260d7988ce4c-cert") pod "odh-model-controller-6dc94f689f-566hl" (UID: "67da1920-a7f6-4870-85f5-260d7988ce4c") : secret "odh-model-controller-webhook-cert" not found Jul 01 16:38:15.448741 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:38:15.448703 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tql5f\" (UniqueName: \"kubernetes.io/projected/67da1920-a7f6-4870-85f5-260d7988ce4c-kube-api-access-tql5f\") pod \"odh-model-controller-6dc94f689f-566hl\" (UID: \"67da1920-a7f6-4870-85f5-260d7988ce4c\") " pod="opendatahub/odh-model-controller-6dc94f689f-566hl" Jul 01 16:38:15.486589 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:38:15.486567 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-75f5b69fd7-bqrqm" Jul 01 16:38:15.607773 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:38:15.607747 2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-75f5b69fd7-bqrqm"] Jul 01 16:38:15.610181 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:38:15.610160 2545 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode0ec6a34_fb56_4865_b460_d76ad8f8f3d7.slice/crio-07d2298835a793e7c052791e0c57ba69a2065f9085ec4f295bc4f18abd1fd3b4 WatchSource:0}: Error finding container 07d2298835a793e7c052791e0c57ba69a2065f9085ec4f295bc4f18abd1fd3b4: Status 404 returned error can't find the container with id 07d2298835a793e7c052791e0c57ba69a2065f9085ec4f295bc4f18abd1fd3b4 Jul 01 16:38:15.786290 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:38:15.786216 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-75f5b69fd7-bqrqm" event={"ID":"e0ec6a34-fb56-4865-b460-d76ad8f8f3d7","Type":"ContainerStarted","Data":"07d2298835a793e7c052791e0c57ba69a2065f9085ec4f295bc4f18abd1fd3b4"} Jul 01 16:38:15.940011 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:38:15.939979 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/67da1920-a7f6-4870-85f5-260d7988ce4c-cert\") pod \"odh-model-controller-6dc94f689f-566hl\" (UID: \"67da1920-a7f6-4870-85f5-260d7988ce4c\") " pod="opendatahub/odh-model-controller-6dc94f689f-566hl" Jul 01 16:38:15.942280 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:38:15.942261 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/67da1920-a7f6-4870-85f5-260d7988ce4c-cert\") pod \"odh-model-controller-6dc94f689f-566hl\" (UID: \"67da1920-a7f6-4870-85f5-260d7988ce4c\") " pod="opendatahub/odh-model-controller-6dc94f689f-566hl" Jul 01 16:38:16.130927 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:38:16.130900 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-6dc94f689f-566hl" Jul 01 16:38:16.150456 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:38:16.150432 2545 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-5dd488fc48-5pjl6"] Jul 01 16:38:16.152695 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:38:16.152676 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-5dd488fc48-5pjl6" Jul 01 16:38:16.155286 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:38:16.155264 2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\"" Jul 01 16:38:16.155379 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:38:16.155283 2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-dockercfg-r6kk9\"" Jul 01 16:38:16.163372 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:38:16.163351 2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-5dd488fc48-5pjl6"] Jul 01 16:38:16.242295 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:38:16.242257 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-blc6p\" (UniqueName: \"kubernetes.io/projected/c9b15502-b5e0-4359-86a3-a58e6d4c20cd-kube-api-access-blc6p\") pod \"model-serving-api-5dd488fc48-5pjl6\" (UID: \"c9b15502-b5e0-4359-86a3-a58e6d4c20cd\") " pod="opendatahub/model-serving-api-5dd488fc48-5pjl6" Jul 01 16:38:16.242793 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:38:16.242374 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c9b15502-b5e0-4359-86a3-a58e6d4c20cd-tls-certs\") pod \"model-serving-api-5dd488fc48-5pjl6\" (UID: \"c9b15502-b5e0-4359-86a3-a58e6d4c20cd\") " pod="opendatahub/model-serving-api-5dd488fc48-5pjl6" Jul 01 16:38:16.262376 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:38:16.262351 2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-6dc94f689f-566hl"] Jul 01 16:38:16.264904 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:38:16.264881 2545 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod67da1920_a7f6_4870_85f5_260d7988ce4c.slice/crio-4a92a0fdb52dfa4d57874b764b1af8b76cf243a28bf372eecbd2e8be5118032d WatchSource:0}: Error finding container 4a92a0fdb52dfa4d57874b764b1af8b76cf243a28bf372eecbd2e8be5118032d: Status 404 returned error can't find the container with id 4a92a0fdb52dfa4d57874b764b1af8b76cf243a28bf372eecbd2e8be5118032d Jul 01 16:38:16.342851 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:38:16.342824 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-blc6p\" (UniqueName: \"kubernetes.io/projected/c9b15502-b5e0-4359-86a3-a58e6d4c20cd-kube-api-access-blc6p\") pod \"model-serving-api-5dd488fc48-5pjl6\" (UID: \"c9b15502-b5e0-4359-86a3-a58e6d4c20cd\") " pod="opendatahub/model-serving-api-5dd488fc48-5pjl6" Jul 01 16:38:16.342959 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:38:16.342896 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c9b15502-b5e0-4359-86a3-a58e6d4c20cd-tls-certs\") pod \"model-serving-api-5dd488fc48-5pjl6\" (UID: \"c9b15502-b5e0-4359-86a3-a58e6d4c20cd\") " pod="opendatahub/model-serving-api-5dd488fc48-5pjl6" Jul 01 16:38:16.345623 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:38:16.345601 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c9b15502-b5e0-4359-86a3-a58e6d4c20cd-tls-certs\") pod \"model-serving-api-5dd488fc48-5pjl6\" (UID: \"c9b15502-b5e0-4359-86a3-a58e6d4c20cd\") " pod="opendatahub/model-serving-api-5dd488fc48-5pjl6" Jul 01 16:38:16.380815 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:38:16.380793 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-blc6p\" (UniqueName: \"kubernetes.io/projected/c9b15502-b5e0-4359-86a3-a58e6d4c20cd-kube-api-access-blc6p\") pod \"model-serving-api-5dd488fc48-5pjl6\" (UID: \"c9b15502-b5e0-4359-86a3-a58e6d4c20cd\") " pod="opendatahub/model-serving-api-5dd488fc48-5pjl6" Jul 01 16:38:16.459031 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:38:16.458944 2545 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-4mb8s"] Jul 01 16:38:16.461285 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:38:16.461260 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-4mb8s" Jul 01 16:38:16.463816 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:38:16.463758 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-5dd488fc48-5pjl6" Jul 01 16:38:16.463816 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:38:16.463788 2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Jul 01 16:38:16.464014 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:38:16.463927 2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Jul 01 16:38:16.464076 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:38:16.464064 2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-v66gl\"" Jul 01 16:38:16.464134 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:38:16.464097 2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Jul 01 16:38:16.464187 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:38:16.464140 2545 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 01 16:38:16.474250 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:38:16.474224 2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-4mb8s"] Jul 01 16:38:16.544848 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:38:16.544821 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/44395f83-c8fd-494d-bcee-581293d99503-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-4mb8s\" (UID: \"44395f83-c8fd-494d-bcee-581293d99503\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-4mb8s" Jul 01 16:38:16.544985 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:38:16.544855 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/44395f83-c8fd-494d-bcee-581293d99503-cacerts\") pod \"istiod-openshift-gateway-7574459656-4mb8s\" (UID: \"44395f83-c8fd-494d-bcee-581293d99503\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-4mb8s" Jul 01 16:38:16.544985 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:38:16.544882 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/44395f83-c8fd-494d-bcee-581293d99503-local-certs\") pod \"istiod-openshift-gateway-7574459656-4mb8s\" (UID: \"44395f83-c8fd-494d-bcee-581293d99503\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-4mb8s" Jul 01 16:38:16.545116 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:38:16.545012 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-chl8z\" (UniqueName: \"kubernetes.io/projected/44395f83-c8fd-494d-bcee-581293d99503-kube-api-access-chl8z\") pod \"istiod-openshift-gateway-7574459656-4mb8s\" (UID: \"44395f83-c8fd-494d-bcee-581293d99503\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-4mb8s" Jul 01 16:38:16.545116 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:38:16.545058 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/44395f83-c8fd-494d-bcee-581293d99503-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-4mb8s\" (UID: \"44395f83-c8fd-494d-bcee-581293d99503\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-4mb8s" Jul 01 16:38:16.545116 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:38:16.545096 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/44395f83-c8fd-494d-bcee-581293d99503-istio-token\") pod \"istiod-openshift-gateway-7574459656-4mb8s\" (UID: \"44395f83-c8fd-494d-bcee-581293d99503\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-4mb8s" Jul 01 16:38:16.545239 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:38:16.545125 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/44395f83-c8fd-494d-bcee-581293d99503-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-4mb8s\" (UID: \"44395f83-c8fd-494d-bcee-581293d99503\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-4mb8s" Jul 01 16:38:16.610810 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:38:16.610665 2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-5dd488fc48-5pjl6"] Jul 01 16:38:16.613318 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:38:16.613287 2545 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc9b15502_b5e0_4359_86a3_a58e6d4c20cd.slice/crio-3a8a7e0d50eb2a59fbe23b2f5bf7799b488d16b2c86c3a7ffcf4397ff4094833 WatchSource:0}: Error finding container 3a8a7e0d50eb2a59fbe23b2f5bf7799b488d16b2c86c3a7ffcf4397ff4094833: Status 404 returned error can't find the container with id 3a8a7e0d50eb2a59fbe23b2f5bf7799b488d16b2c86c3a7ffcf4397ff4094833 Jul 01 16:38:16.645955 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:38:16.645932 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/44395f83-c8fd-494d-bcee-581293d99503-local-certs\") pod \"istiod-openshift-gateway-7574459656-4mb8s\" (UID: \"44395f83-c8fd-494d-bcee-581293d99503\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-4mb8s" Jul 01 16:38:16.646073 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:38:16.645993 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-chl8z\" (UniqueName: \"kubernetes.io/projected/44395f83-c8fd-494d-bcee-581293d99503-kube-api-access-chl8z\") pod \"istiod-openshift-gateway-7574459656-4mb8s\" (UID: \"44395f83-c8fd-494d-bcee-581293d99503\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-4mb8s" Jul 01 16:38:16.646073 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:38:16.646028 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/44395f83-c8fd-494d-bcee-581293d99503-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-4mb8s\" (UID: \"44395f83-c8fd-494d-bcee-581293d99503\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-4mb8s" Jul 01 16:38:16.646073 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:38:16.646053 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/44395f83-c8fd-494d-bcee-581293d99503-istio-token\") pod \"istiod-openshift-gateway-7574459656-4mb8s\" (UID: \"44395f83-c8fd-494d-bcee-581293d99503\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-4mb8s" Jul 01 16:38:16.646218 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:38:16.646082 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/44395f83-c8fd-494d-bcee-581293d99503-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-4mb8s\" (UID: \"44395f83-c8fd-494d-bcee-581293d99503\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-4mb8s" Jul 01 16:38:16.646218 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:38:16.646116 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/44395f83-c8fd-494d-bcee-581293d99503-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-4mb8s\" (UID: \"44395f83-c8fd-494d-bcee-581293d99503\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-4mb8s" Jul 01 16:38:16.646218 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:38:16.646140 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/44395f83-c8fd-494d-bcee-581293d99503-cacerts\") pod \"istiod-openshift-gateway-7574459656-4mb8s\" (UID: \"44395f83-c8fd-494d-bcee-581293d99503\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-4mb8s" Jul 01 16:38:16.646772 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:38:16.646748 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/44395f83-c8fd-494d-bcee-581293d99503-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-4mb8s\" (UID: \"44395f83-c8fd-494d-bcee-581293d99503\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-4mb8s" Jul 01 16:38:16.648472 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:38:16.648451 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/44395f83-c8fd-494d-bcee-581293d99503-local-certs\") pod \"istiod-openshift-gateway-7574459656-4mb8s\" (UID: \"44395f83-c8fd-494d-bcee-581293d99503\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-4mb8s" Jul 01 16:38:16.648767 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:38:16.648748 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/44395f83-c8fd-494d-bcee-581293d99503-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-4mb8s\" (UID: \"44395f83-c8fd-494d-bcee-581293d99503\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-4mb8s" Jul 01 16:38:16.648892 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:38:16.648870 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/44395f83-c8fd-494d-bcee-581293d99503-cacerts\") pod \"istiod-openshift-gateway-7574459656-4mb8s\" (UID: \"44395f83-c8fd-494d-bcee-581293d99503\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-4mb8s" Jul 01 16:38:16.649231 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:38:16.649212 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/44395f83-c8fd-494d-bcee-581293d99503-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-4mb8s\" (UID: \"44395f83-c8fd-494d-bcee-581293d99503\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-4mb8s" Jul 01 16:38:16.655450 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:38:16.655411 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/44395f83-c8fd-494d-bcee-581293d99503-istio-token\") pod \"istiod-openshift-gateway-7574459656-4mb8s\" (UID: \"44395f83-c8fd-494d-bcee-581293d99503\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-4mb8s" Jul 01 16:38:16.655927 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:38:16.655905 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-chl8z\" (UniqueName: \"kubernetes.io/projected/44395f83-c8fd-494d-bcee-581293d99503-kube-api-access-chl8z\") pod \"istiod-openshift-gateway-7574459656-4mb8s\" (UID: \"44395f83-c8fd-494d-bcee-581293d99503\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-4mb8s" Jul 01 16:38:16.788049 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:38:16.787975 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-4mb8s" Jul 01 16:38:16.797010 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:38:16.796971 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-5dd488fc48-5pjl6" event={"ID":"c9b15502-b5e0-4359-86a3-a58e6d4c20cd","Type":"ContainerStarted","Data":"3a8a7e0d50eb2a59fbe23b2f5bf7799b488d16b2c86c3a7ffcf4397ff4094833"} Jul 01 16:38:16.799204 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:38:16.799176 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-6dc94f689f-566hl" event={"ID":"67da1920-a7f6-4870-85f5-260d7988ce4c","Type":"ContainerStarted","Data":"4a92a0fdb52dfa4d57874b764b1af8b76cf243a28bf372eecbd2e8be5118032d"} Jul 01 16:38:16.954382 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:38:16.954315 2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-4mb8s"] Jul 01 16:38:16.961115 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:38:16.961078 2545 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod44395f83_c8fd_494d_bcee_581293d99503.slice/crio-cea668cd2f9b268a69fe8ab3f4fbf82fbba0438d7441d03cc05c9ad306b128b4 WatchSource:0}: Error finding container cea668cd2f9b268a69fe8ab3f4fbf82fbba0438d7441d03cc05c9ad306b128b4: Status 404 returned error can't find the container with id cea668cd2f9b268a69fe8ab3f4fbf82fbba0438d7441d03cc05c9ad306b128b4 Jul 01 16:38:17.810856 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:38:17.810820 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-4mb8s" event={"ID":"44395f83-c8fd-494d-bcee-581293d99503","Type":"ContainerStarted","Data":"cea668cd2f9b268a69fe8ab3f4fbf82fbba0438d7441d03cc05c9ad306b128b4"} Jul 01 16:38:21.786081 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:38:21.786027 2545 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 01 16:38:21.786461 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:38:21.786101 2545 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 01 16:38:22.835574 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:38:22.835536 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-75f5b69fd7-bqrqm" event={"ID":"e0ec6a34-fb56-4865-b460-d76ad8f8f3d7","Type":"ContainerStarted","Data":"da4811682f75af3568926c95a1564a37a2caf9ea6bde31efa023df46d4dfa276"} Jul 01 16:38:22.837096 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:38:22.837063 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-5dd488fc48-5pjl6" event={"ID":"c9b15502-b5e0-4359-86a3-a58e6d4c20cd","Type":"ContainerStarted","Data":"f9b0227822126bc4a555dfddf5c0708eed0b406fb6baa3ee671e3e7c64500e8c"} Jul 01 16:38:22.837241 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:38:22.837141 2545 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-5dd488fc48-5pjl6" Jul 01 16:38:22.838503 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:38:22.838475 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-6dc94f689f-566hl" event={"ID":"67da1920-a7f6-4870-85f5-260d7988ce4c","Type":"ContainerStarted","Data":"138061bd098ba495a7284ba0f5d76af87eb50dd1c4a136658aca1d2542b01ed6"} Jul 01 16:38:22.838601 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:38:22.838587 2545 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-6dc94f689f-566hl" Jul 01 16:38:22.840050 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:38:22.840022 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-4mb8s" event={"ID":"44395f83-c8fd-494d-bcee-581293d99503","Type":"ContainerStarted","Data":"931c6e3f5d78051a3ed69e465269e9426da611250c9899755dd5af93b377d501"} Jul 01 16:38:22.840158 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:38:22.840070 2545 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-4mb8s" Jul 01 16:38:22.841768 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:38:22.841745 2545 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7574459656-4mb8s container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 01 16:38:22.841892 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:38:22.841788 2545 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7574459656-4mb8s" podUID="44395f83-c8fd-494d-bcee-581293d99503" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 01 16:38:22.855141 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:38:22.855100 2545 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-75f5b69fd7-bqrqm" podStartSLOduration=1.724438035 podStartE2EDuration="7.855086421s" totalImagesPullingTime="6.130648386s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 16:38:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 16:38:22.853849264 +0000 UTC m=+581.434466766" watchObservedRunningTime="2026-07-01 16:38:22.855086421 +0000 UTC m=+581.435703925" Jul 01 16:38:22.875726 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:38:22.875668 2545 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7574459656-4mb8s" podStartSLOduration=2.054266788 podStartE2EDuration="6.875651258s" totalImagesPullingTime="4.82138447s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 16:38:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 16:38:22.875542772 +0000 UTC m=+581.456160308" watchObservedRunningTime="2026-07-01 16:38:22.875651258 +0000 UTC m=+581.456268764" Jul 01 16:38:22.893412 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:38:22.893371 2545 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-6dc94f689f-566hl" podStartSLOduration=2.386578779 podStartE2EDuration="7.893359149s" totalImagesPullingTime="5.50678037s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 16:38:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 16:38:22.891910011 +0000 UTC m=+581.472527516" watchObservedRunningTime="2026-07-01 16:38:22.893359149 +0000 UTC m=+581.473976653" Jul 01 16:38:22.914818 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:38:22.914786 2545 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-5dd488fc48-5pjl6" podStartSLOduration=1.745170933 podStartE2EDuration="6.914773297s" totalImagesPullingTime="5.169602364s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 16:38:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 16:38:22.914088521 +0000 UTC m=+581.494706035" watchObservedRunningTime="2026-07-01 16:38:22.914773297 +0000 UTC m=+581.495390800" Jul 01 16:38:23.257570 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:38:23.257480 2545 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4m8qbk"] Jul 01 16:38:23.259942 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:38:23.259916 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4m8qbk" Jul 01 16:38:23.262453 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:38:23.262433 2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-ch984\"" Jul 01 16:38:23.277554 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:38:23.277513 2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4m8qbk"] Jul 01 16:38:23.409612 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:38:23.409580 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0a98d33b-fc67-4515-8c76-fafdc010917e-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4m8qbk\" (UID: \"0a98d33b-fc67-4515-8c76-fafdc010917e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4m8qbk" Jul 01 16:38:23.409612 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:38:23.409611 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0a98d33b-fc67-4515-8c76-fafdc010917e-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4m8qbk\" (UID: \"0a98d33b-fc67-4515-8c76-fafdc010917e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4m8qbk" Jul 01 16:38:23.409826 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:38:23.409633 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0a98d33b-fc67-4515-8c76-fafdc010917e-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4m8qbk\" (UID: \"0a98d33b-fc67-4515-8c76-fafdc010917e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4m8qbk" Jul 01 16:38:23.409826 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:38:23.409687 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0a98d33b-fc67-4515-8c76-fafdc010917e-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4m8qbk\" (UID: \"0a98d33b-fc67-4515-8c76-fafdc010917e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4m8qbk" Jul 01 16:38:23.409826 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:38:23.409784 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p4vsk\" (UniqueName: \"kubernetes.io/projected/0a98d33b-fc67-4515-8c76-fafdc010917e-kube-api-access-p4vsk\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4m8qbk\" (UID: \"0a98d33b-fc67-4515-8c76-fafdc010917e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4m8qbk" Jul 01 16:38:23.409826 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:38:23.409824 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0a98d33b-fc67-4515-8c76-fafdc010917e-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4m8qbk\" (UID: \"0a98d33b-fc67-4515-8c76-fafdc010917e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4m8qbk" Jul 01 16:38:23.409956 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:38:23.409849 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0a98d33b-fc67-4515-8c76-fafdc010917e-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4m8qbk\" (UID: \"0a98d33b-fc67-4515-8c76-fafdc010917e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4m8qbk" Jul 01 16:38:23.409956 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:38:23.409867 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0a98d33b-fc67-4515-8c76-fafdc010917e-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4m8qbk\" (UID: \"0a98d33b-fc67-4515-8c76-fafdc010917e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4m8qbk" Jul 01 16:38:23.409956 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:38:23.409883 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0a98d33b-fc67-4515-8c76-fafdc010917e-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4m8qbk\" (UID: \"0a98d33b-fc67-4515-8c76-fafdc010917e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4m8qbk" Jul 01 16:38:23.510835 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:38:23.510758 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p4vsk\" (UniqueName: \"kubernetes.io/projected/0a98d33b-fc67-4515-8c76-fafdc010917e-kube-api-access-p4vsk\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4m8qbk\" (UID: \"0a98d33b-fc67-4515-8c76-fafdc010917e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4m8qbk" Jul 01 16:38:23.510835 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:38:23.510797 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0a98d33b-fc67-4515-8c76-fafdc010917e-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4m8qbk\" (UID: \"0a98d33b-fc67-4515-8c76-fafdc010917e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4m8qbk" Jul 01 16:38:23.510835 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:38:23.510821 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0a98d33b-fc67-4515-8c76-fafdc010917e-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4m8qbk\" (UID: \"0a98d33b-fc67-4515-8c76-fafdc010917e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4m8qbk" Jul 01 16:38:23.511092 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:38:23.510840 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0a98d33b-fc67-4515-8c76-fafdc010917e-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4m8qbk\" (UID: \"0a98d33b-fc67-4515-8c76-fafdc010917e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4m8qbk" Jul 01 16:38:23.511092 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:38:23.510865 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0a98d33b-fc67-4515-8c76-fafdc010917e-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4m8qbk\" (UID: \"0a98d33b-fc67-4515-8c76-fafdc010917e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4m8qbk" Jul 01 16:38:23.511092 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:38:23.510908 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0a98d33b-fc67-4515-8c76-fafdc010917e-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4m8qbk\" (UID: \"0a98d33b-fc67-4515-8c76-fafdc010917e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4m8qbk" Jul 01 16:38:23.511092 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:38:23.510930 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0a98d33b-fc67-4515-8c76-fafdc010917e-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4m8qbk\" (UID: \"0a98d33b-fc67-4515-8c76-fafdc010917e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4m8qbk" Jul 01 16:38:23.511092 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:38:23.510961 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0a98d33b-fc67-4515-8c76-fafdc010917e-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4m8qbk\" (UID: \"0a98d33b-fc67-4515-8c76-fafdc010917e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4m8qbk" Jul 01 16:38:23.511092 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:38:23.510997 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0a98d33b-fc67-4515-8c76-fafdc010917e-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4m8qbk\" (UID: \"0a98d33b-fc67-4515-8c76-fafdc010917e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4m8qbk" Jul 01 16:38:23.511407 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:38:23.511361 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0a98d33b-fc67-4515-8c76-fafdc010917e-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4m8qbk\" (UID: \"0a98d33b-fc67-4515-8c76-fafdc010917e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4m8qbk" Jul 01 16:38:23.511473 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:38:23.511427 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0a98d33b-fc67-4515-8c76-fafdc010917e-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4m8qbk\" (UID: \"0a98d33b-fc67-4515-8c76-fafdc010917e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4m8qbk" Jul 01 16:38:23.511529 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:38:23.511479 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0a98d33b-fc67-4515-8c76-fafdc010917e-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4m8qbk\" (UID: \"0a98d33b-fc67-4515-8c76-fafdc010917e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4m8qbk" Jul 01 16:38:23.511725 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:38:23.511684 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0a98d33b-fc67-4515-8c76-fafdc010917e-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4m8qbk\" (UID: \"0a98d33b-fc67-4515-8c76-fafdc010917e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4m8qbk" Jul 01 16:38:23.511867 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:38:23.511843 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0a98d33b-fc67-4515-8c76-fafdc010917e-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4m8qbk\" (UID: \"0a98d33b-fc67-4515-8c76-fafdc010917e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4m8qbk" Jul 01 16:38:23.513182 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:38:23.513163 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0a98d33b-fc67-4515-8c76-fafdc010917e-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4m8qbk\" (UID: \"0a98d33b-fc67-4515-8c76-fafdc010917e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4m8qbk" Jul 01 16:38:23.513401 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:38:23.513380 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0a98d33b-fc67-4515-8c76-fafdc010917e-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4m8qbk\" (UID: \"0a98d33b-fc67-4515-8c76-fafdc010917e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4m8qbk" Jul 01 16:38:23.523761 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:38:23.523741 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0a98d33b-fc67-4515-8c76-fafdc010917e-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4m8qbk\" (UID: \"0a98d33b-fc67-4515-8c76-fafdc010917e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4m8qbk" Jul 01 16:38:23.526840 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:38:23.526817 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p4vsk\" (UniqueName: \"kubernetes.io/projected/0a98d33b-fc67-4515-8c76-fafdc010917e-kube-api-access-p4vsk\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4m8qbk\" (UID: \"0a98d33b-fc67-4515-8c76-fafdc010917e\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4m8qbk" Jul 01 16:38:23.572845 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:38:23.572823 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4m8qbk" Jul 01 16:38:23.705518 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:38:23.705497 2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4m8qbk"] Jul 01 16:38:23.707640 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:38:23.707620 2545 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0a98d33b_fc67_4515_8c76_fafdc010917e.slice/crio-cba089d3f8479d81feec60c3f9823835144cb6e720a74b4db9f286a29e83efce WatchSource:0}: Error finding container cba089d3f8479d81feec60c3f9823835144cb6e720a74b4db9f286a29e83efce: Status 404 returned error can't find the container with id cba089d3f8479d81feec60c3f9823835144cb6e720a74b4db9f286a29e83efce Jul 01 16:38:23.845911 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:38:23.845883 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4m8qbk" event={"ID":"0a98d33b-fc67-4515-8c76-fafdc010917e","Type":"ContainerStarted","Data":"cba089d3f8479d81feec60c3f9823835144cb6e720a74b4db9f286a29e83efce"} Jul 01 16:38:23.846735 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:38:23.846692 2545 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-4mb8s" Jul 01 16:38:24.826867 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:38:24.826833 2545 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-69fc866d76-jptg4"] Jul 01 16:38:24.829128 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:38:24.829108 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-69fc866d76-jptg4" Jul 01 16:38:24.831769 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:38:24.831741 2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-j2psd\"" Jul 01 16:38:24.831769 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:38:24.831769 2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\"" Jul 01 16:38:24.846473 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:38:24.846451 2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-69fc866d76-jptg4"] Jul 01 16:38:24.860967 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:38:24.860941 2545 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-864bfd7bc9-knn5v"] Jul 01 16:38:24.863919 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:38:24.863857 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-864bfd7bc9-knn5v" Jul 01 16:38:24.866473 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:38:24.866443 2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\"" Jul 01 16:38:24.866678 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:38:24.866661 2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-bn8hn\"" Jul 01 16:38:24.886182 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:38:24.877726 2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-864bfd7bc9-knn5v"] Jul 01 16:38:24.932571 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:38:24.932542 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mgkzt\" (UniqueName: \"kubernetes.io/projected/4fcbbf9c-2b2b-4636-8c26-6b773c6c8ed1-kube-api-access-mgkzt\") pod \"kserve-controller-manager-69fc866d76-jptg4\" (UID: \"4fcbbf9c-2b2b-4636-8c26-6b773c6c8ed1\") " pod="opendatahub/kserve-controller-manager-69fc866d76-jptg4" Jul 01 16:38:24.932816 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:38:24.932603 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4fcbbf9c-2b2b-4636-8c26-6b773c6c8ed1-cert\") pod \"kserve-controller-manager-69fc866d76-jptg4\" (UID: \"4fcbbf9c-2b2b-4636-8c26-6b773c6c8ed1\") " pod="opendatahub/kserve-controller-manager-69fc866d76-jptg4" Jul 01 16:38:25.033388 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:38:25.033356 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4jtrz\" (UniqueName: \"kubernetes.io/projected/9eae99c9-6daf-4a90-aada-126f22386acb-kube-api-access-4jtrz\") pod \"llmisvc-controller-manager-864bfd7bc9-knn5v\" (UID: \"9eae99c9-6daf-4a90-aada-126f22386acb\") " pod="opendatahub/llmisvc-controller-manager-864bfd7bc9-knn5v" Jul 01 16:38:25.033578 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:38:25.033523 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9eae99c9-6daf-4a90-aada-126f22386acb-cert\") pod \"llmisvc-controller-manager-864bfd7bc9-knn5v\" (UID: \"9eae99c9-6daf-4a90-aada-126f22386acb\") " pod="opendatahub/llmisvc-controller-manager-864bfd7bc9-knn5v" Jul 01 16:38:25.033578 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:38:25.033560 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mgkzt\" (UniqueName: \"kubernetes.io/projected/4fcbbf9c-2b2b-4636-8c26-6b773c6c8ed1-kube-api-access-mgkzt\") pod \"kserve-controller-manager-69fc866d76-jptg4\" (UID: \"4fcbbf9c-2b2b-4636-8c26-6b773c6c8ed1\") " pod="opendatahub/kserve-controller-manager-69fc866d76-jptg4" Jul 01 16:38:25.033694 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:38:25.033630 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4fcbbf9c-2b2b-4636-8c26-6b773c6c8ed1-cert\") pod \"kserve-controller-manager-69fc866d76-jptg4\" (UID: \"4fcbbf9c-2b2b-4636-8c26-6b773c6c8ed1\") " pod="opendatahub/kserve-controller-manager-69fc866d76-jptg4" Jul 01 16:38:25.036105 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:38:25.036083 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4fcbbf9c-2b2b-4636-8c26-6b773c6c8ed1-cert\") pod \"kserve-controller-manager-69fc866d76-jptg4\" (UID: \"4fcbbf9c-2b2b-4636-8c26-6b773c6c8ed1\") " pod="opendatahub/kserve-controller-manager-69fc866d76-jptg4" Jul 01 16:38:25.051414 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:38:25.051392 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mgkzt\" (UniqueName: \"kubernetes.io/projected/4fcbbf9c-2b2b-4636-8c26-6b773c6c8ed1-kube-api-access-mgkzt\") pod \"kserve-controller-manager-69fc866d76-jptg4\" (UID: \"4fcbbf9c-2b2b-4636-8c26-6b773c6c8ed1\") " pod="opendatahub/kserve-controller-manager-69fc866d76-jptg4" Jul 01 16:38:25.134491 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:38:25.134416 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4jtrz\" (UniqueName: \"kubernetes.io/projected/9eae99c9-6daf-4a90-aada-126f22386acb-kube-api-access-4jtrz\") pod \"llmisvc-controller-manager-864bfd7bc9-knn5v\" (UID: \"9eae99c9-6daf-4a90-aada-126f22386acb\") " pod="opendatahub/llmisvc-controller-manager-864bfd7bc9-knn5v" Jul 01 16:38:25.134491 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:38:25.134484 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9eae99c9-6daf-4a90-aada-126f22386acb-cert\") pod \"llmisvc-controller-manager-864bfd7bc9-knn5v\" (UID: \"9eae99c9-6daf-4a90-aada-126f22386acb\") " pod="opendatahub/llmisvc-controller-manager-864bfd7bc9-knn5v" Jul 01 16:38:25.134675 ip-10-0-130-227 kubenswrapper[2545]: E0701 16:38:25.134585 2545 secret.go:189] Couldn't get secret opendatahub/llmisvc-webhook-server-cert: secret "llmisvc-webhook-server-cert" not found Jul 01 16:38:25.134675 ip-10-0-130-227 kubenswrapper[2545]: E0701 16:38:25.134640 2545 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9eae99c9-6daf-4a90-aada-126f22386acb-cert podName:9eae99c9-6daf-4a90-aada-126f22386acb nodeName:}" failed. No retries permitted until 2026-07-01 16:38:25.634623596 +0000 UTC m=+584.215241077 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/9eae99c9-6daf-4a90-aada-126f22386acb-cert") pod "llmisvc-controller-manager-864bfd7bc9-knn5v" (UID: "9eae99c9-6daf-4a90-aada-126f22386acb") : secret "llmisvc-webhook-server-cert" not found Jul 01 16:38:25.145701 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:38:25.145673 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-69fc866d76-jptg4" Jul 01 16:38:25.148471 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:38:25.148430 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4jtrz\" (UniqueName: \"kubernetes.io/projected/9eae99c9-6daf-4a90-aada-126f22386acb-kube-api-access-4jtrz\") pod \"llmisvc-controller-manager-864bfd7bc9-knn5v\" (UID: \"9eae99c9-6daf-4a90-aada-126f22386acb\") " pod="opendatahub/llmisvc-controller-manager-864bfd7bc9-knn5v" Jul 01 16:38:25.298624 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:38:25.298494 2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-69fc866d76-jptg4"] Jul 01 16:38:25.638791 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:38:25.638759 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9eae99c9-6daf-4a90-aada-126f22386acb-cert\") pod \"llmisvc-controller-manager-864bfd7bc9-knn5v\" (UID: \"9eae99c9-6daf-4a90-aada-126f22386acb\") " pod="opendatahub/llmisvc-controller-manager-864bfd7bc9-knn5v" Jul 01 16:38:25.641372 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:38:25.641347 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9eae99c9-6daf-4a90-aada-126f22386acb-cert\") pod \"llmisvc-controller-manager-864bfd7bc9-knn5v\" (UID: \"9eae99c9-6daf-4a90-aada-126f22386acb\") " pod="opendatahub/llmisvc-controller-manager-864bfd7bc9-knn5v" Jul 01 16:38:25.789640 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:38:25.789609 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-864bfd7bc9-knn5v" Jul 01 16:38:26.154975 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:38:26.154929 2545 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4fcbbf9c_2b2b_4636_8c26_6b773c6c8ed1.slice/crio-d2cf19ba6c27e66a5f03fb8acf16410f35cb2c066866af85406afaa6feac81e3 WatchSource:0}: Error finding container d2cf19ba6c27e66a5f03fb8acf16410f35cb2c066866af85406afaa6feac81e3: Status 404 returned error can't find the container with id d2cf19ba6c27e66a5f03fb8acf16410f35cb2c066866af85406afaa6feac81e3 Jul 01 16:38:26.204290 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:38:26.204254 2545 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 01 16:38:26.204406 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:38:26.204328 2545 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 01 16:38:26.204406 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:38:26.204360 2545 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 01 16:38:26.317948 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:38:26.317858 2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-864bfd7bc9-knn5v"] Jul 01 16:38:26.320981 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:38:26.320952 2545 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod9eae99c9_6daf_4a90_aada_126f22386acb.slice/crio-44a43605d2c4736307d5f0d7af2848018d6828358004551a4aa1813823497210 WatchSource:0}: Error finding container 44a43605d2c4736307d5f0d7af2848018d6828358004551a4aa1813823497210: Status 404 returned error can't find the container with id 44a43605d2c4736307d5f0d7af2848018d6828358004551a4aa1813823497210 Jul 01 16:38:26.869561 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:38:26.869525 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4m8qbk" event={"ID":"0a98d33b-fc67-4515-8c76-fafdc010917e","Type":"ContainerStarted","Data":"748a9881c730ac67b0645161b4a8933a183e6e50f103581a83f71f047e394d49"} Jul 01 16:38:26.870913 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:38:26.870884 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-864bfd7bc9-knn5v" event={"ID":"9eae99c9-6daf-4a90-aada-126f22386acb","Type":"ContainerStarted","Data":"44a43605d2c4736307d5f0d7af2848018d6828358004551a4aa1813823497210"} Jul 01 16:38:26.872090 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:38:26.872068 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-69fc866d76-jptg4" event={"ID":"4fcbbf9c-2b2b-4636-8c26-6b773c6c8ed1","Type":"ContainerStarted","Data":"d2cf19ba6c27e66a5f03fb8acf16410f35cb2c066866af85406afaa6feac81e3"} Jul 01 16:38:26.893161 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:38:26.893101 2545 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4m8qbk" podStartSLOduration=1.39898343 podStartE2EDuration="3.893086226s" totalImagesPullingTime="2.494102796s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 16:38:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 16:38:26.891449998 +0000 UTC m=+585.472067502" watchObservedRunningTime="2026-07-01 16:38:26.893086226 +0000 UTC m=+585.473703791" Jul 01 16:38:27.299851 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:38:27.299744 2545 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/dashboard-redirect-9497df747-zjqqj"] Jul 01 16:38:27.302306 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:38:27.302283 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/dashboard-redirect-9497df747-zjqqj" Jul 01 16:38:27.305774 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:38:27.305734 2545 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"dashboard-redirect-config\"" Jul 01 16:38:27.306117 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:38:27.306093 2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-7brp2\"" Jul 01 16:38:27.320261 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:38:27.320228 2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/dashboard-redirect-9497df747-zjqqj"] Jul 01 16:38:27.456349 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:38:27.456314 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"redirect-config\" (UniqueName: \"kubernetes.io/configmap/676197d5-3145-4bfc-84f9-b2e2adc79d05-redirect-config\") pod \"dashboard-redirect-9497df747-zjqqj\" (UID: \"676197d5-3145-4bfc-84f9-b2e2adc79d05\") " pod="opendatahub/dashboard-redirect-9497df747-zjqqj" Jul 01 16:38:27.456499 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:38:27.456382 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j64hv\" (UniqueName: \"kubernetes.io/projected/676197d5-3145-4bfc-84f9-b2e2adc79d05-kube-api-access-j64hv\") pod \"dashboard-redirect-9497df747-zjqqj\" (UID: \"676197d5-3145-4bfc-84f9-b2e2adc79d05\") " pod="opendatahub/dashboard-redirect-9497df747-zjqqj" Jul 01 16:38:27.557047 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:38:27.557020 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"redirect-config\" (UniqueName: \"kubernetes.io/configmap/676197d5-3145-4bfc-84f9-b2e2adc79d05-redirect-config\") pod \"dashboard-redirect-9497df747-zjqqj\" (UID: \"676197d5-3145-4bfc-84f9-b2e2adc79d05\") " pod="opendatahub/dashboard-redirect-9497df747-zjqqj" Jul 01 16:38:27.557210 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:38:27.557080 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j64hv\" (UniqueName: \"kubernetes.io/projected/676197d5-3145-4bfc-84f9-b2e2adc79d05-kube-api-access-j64hv\") pod \"dashboard-redirect-9497df747-zjqqj\" (UID: \"676197d5-3145-4bfc-84f9-b2e2adc79d05\") " pod="opendatahub/dashboard-redirect-9497df747-zjqqj" Jul 01 16:38:27.557916 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:38:27.557880 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"redirect-config\" (UniqueName: \"kubernetes.io/configmap/676197d5-3145-4bfc-84f9-b2e2adc79d05-redirect-config\") pod \"dashboard-redirect-9497df747-zjqqj\" (UID: \"676197d5-3145-4bfc-84f9-b2e2adc79d05\") " pod="opendatahub/dashboard-redirect-9497df747-zjqqj" Jul 01 16:38:27.565730 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:38:27.565692 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j64hv\" (UniqueName: \"kubernetes.io/projected/676197d5-3145-4bfc-84f9-b2e2adc79d05-kube-api-access-j64hv\") pod \"dashboard-redirect-9497df747-zjqqj\" (UID: \"676197d5-3145-4bfc-84f9-b2e2adc79d05\") " pod="opendatahub/dashboard-redirect-9497df747-zjqqj" Jul 01 16:38:27.573652 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:38:27.573542 2545 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4m8qbk" Jul 01 16:38:27.579448 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:38:27.579428 2545 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4m8qbk" Jul 01 16:38:27.615626 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:38:27.615597 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/dashboard-redirect-9497df747-zjqqj" Jul 01 16:38:27.767298 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:38:27.767221 2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/dashboard-redirect-9497df747-zjqqj"] Jul 01 16:38:27.770207 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:38:27.770175 2545 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod676197d5_3145_4bfc_84f9_b2e2adc79d05.slice/crio-6406f3afd2c97f2e6991e50f27f7189565cdfeca1c786996ce674c1c4dfc317e WatchSource:0}: Error finding container 6406f3afd2c97f2e6991e50f27f7189565cdfeca1c786996ce674c1c4dfc317e: Status 404 returned error can't find the container with id 6406f3afd2c97f2e6991e50f27f7189565cdfeca1c786996ce674c1c4dfc317e Jul 01 16:38:27.878119 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:38:27.878036 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/dashboard-redirect-9497df747-zjqqj" event={"ID":"676197d5-3145-4bfc-84f9-b2e2adc79d05","Type":"ContainerStarted","Data":"6406f3afd2c97f2e6991e50f27f7189565cdfeca1c786996ce674c1c4dfc317e"} Jul 01 16:38:27.878273 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:38:27.878230 2545 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4m8qbk" Jul 01 16:38:27.879307 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:38:27.879287 2545 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4m8qbk" Jul 01 16:38:28.062345 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:38:28.062315 2545 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-4mb8s"] Jul 01 16:38:28.062524 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:38:28.062496 2545 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7574459656-4mb8s" podUID="44395f83-c8fd-494d-bcee-581293d99503" containerName="discovery" containerID="cri-o://931c6e3f5d78051a3ed69e465269e9426da611250c9899755dd5af93b377d501" gracePeriod=30 Jul 01 16:38:28.884381 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:38:28.884351 2545 generic.go:356] "Generic (PLEG): container finished" podID="44395f83-c8fd-494d-bcee-581293d99503" containerID="931c6e3f5d78051a3ed69e465269e9426da611250c9899755dd5af93b377d501" exitCode=0 Jul 01 16:38:28.884381 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:38:28.884374 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-4mb8s" event={"ID":"44395f83-c8fd-494d-bcee-581293d99503","Type":"ContainerDied","Data":"931c6e3f5d78051a3ed69e465269e9426da611250c9899755dd5af93b377d501"} Jul 01 16:38:28.886160 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:38:28.886136 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-69fc866d76-jptg4" event={"ID":"4fcbbf9c-2b2b-4636-8c26-6b773c6c8ed1","Type":"ContainerStarted","Data":"2ffb5986432bd1cddd3912a2d4998b6a8085b26663187dcfd8e2eba9560ff93d"} Jul 01 16:38:28.886305 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:38:28.886288 2545 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-69fc866d76-jptg4" Jul 01 16:38:28.904796 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:38:28.904751 2545 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-69fc866d76-jptg4" podStartSLOduration=2.354825183 podStartE2EDuration="4.904736357s" totalImagesPullingTime="2.549911174s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 16:38:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 16:38:28.90285782 +0000 UTC m=+587.483475325" watchObservedRunningTime="2026-07-01 16:38:28.904736357 +0000 UTC m=+587.485353871" Jul 01 16:38:29.320537 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:38:29.320512 2545 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-4mb8s" Jul 01 16:38:29.473810 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:38:29.473782 2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/44395f83-c8fd-494d-bcee-581293d99503-istio-kubeconfig\") pod \"44395f83-c8fd-494d-bcee-581293d99503\" (UID: \"44395f83-c8fd-494d-bcee-581293d99503\") " Jul 01 16:38:29.473944 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:38:29.473840 2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/44395f83-c8fd-494d-bcee-581293d99503-local-certs\") pod \"44395f83-c8fd-494d-bcee-581293d99503\" (UID: \"44395f83-c8fd-494d-bcee-581293d99503\") " Jul 01 16:38:29.473944 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:38:29.473896 2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/44395f83-c8fd-494d-bcee-581293d99503-istio-token\") pod \"44395f83-c8fd-494d-bcee-581293d99503\" (UID: \"44395f83-c8fd-494d-bcee-581293d99503\") " Jul 01 16:38:29.473944 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:38:29.473920 2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/44395f83-c8fd-494d-bcee-581293d99503-istio-csr-ca-configmap\") pod \"44395f83-c8fd-494d-bcee-581293d99503\" (UID: \"44395f83-c8fd-494d-bcee-581293d99503\") " Jul 01 16:38:29.474102 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:38:29.473972 2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/44395f83-c8fd-494d-bcee-581293d99503-cacerts\") pod \"44395f83-c8fd-494d-bcee-581293d99503\" (UID: \"44395f83-c8fd-494d-bcee-581293d99503\") " Jul 01 16:38:29.474102 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:38:29.474011 2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-chl8z\" (UniqueName: \"kubernetes.io/projected/44395f83-c8fd-494d-bcee-581293d99503-kube-api-access-chl8z\") pod \"44395f83-c8fd-494d-bcee-581293d99503\" (UID: \"44395f83-c8fd-494d-bcee-581293d99503\") " Jul 01 16:38:29.474102 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:38:29.474064 2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/44395f83-c8fd-494d-bcee-581293d99503-istio-csr-dns-cert\") pod \"44395f83-c8fd-494d-bcee-581293d99503\" (UID: \"44395f83-c8fd-494d-bcee-581293d99503\") " Jul 01 16:38:29.474488 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:38:29.474458 2545 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/44395f83-c8fd-494d-bcee-581293d99503-istio-csr-ca-configmap" (OuterVolumeSpecName: "istio-csr-ca-configmap") pod "44395f83-c8fd-494d-bcee-581293d99503" (UID: "44395f83-c8fd-494d-bcee-581293d99503"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 01 16:38:29.477233 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:38:29.477187 2545 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/44395f83-c8fd-494d-bcee-581293d99503-istio-csr-dns-cert" (OuterVolumeSpecName: "istio-csr-dns-cert") pod "44395f83-c8fd-494d-bcee-581293d99503" (UID: "44395f83-c8fd-494d-bcee-581293d99503"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 01 16:38:29.477421 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:38:29.477387 2545 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/44395f83-c8fd-494d-bcee-581293d99503-istio-token" (OuterVolumeSpecName: "istio-token") pod "44395f83-c8fd-494d-bcee-581293d99503" (UID: "44395f83-c8fd-494d-bcee-581293d99503"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 01 16:38:29.477493 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:38:29.477392 2545 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/44395f83-c8fd-494d-bcee-581293d99503-local-certs" (OuterVolumeSpecName: "local-certs") pod "44395f83-c8fd-494d-bcee-581293d99503" (UID: "44395f83-c8fd-494d-bcee-581293d99503"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 01 16:38:29.477909 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:38:29.477881 2545 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/44395f83-c8fd-494d-bcee-581293d99503-cacerts" (OuterVolumeSpecName: "cacerts") pod "44395f83-c8fd-494d-bcee-581293d99503" (UID: "44395f83-c8fd-494d-bcee-581293d99503"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 01 16:38:29.478439 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:38:29.478410 2545 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/44395f83-c8fd-494d-bcee-581293d99503-istio-kubeconfig" (OuterVolumeSpecName: "istio-kubeconfig") pod "44395f83-c8fd-494d-bcee-581293d99503" (UID: "44395f83-c8fd-494d-bcee-581293d99503"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 01 16:38:29.478544 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:38:29.478495 2545 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/44395f83-c8fd-494d-bcee-581293d99503-kube-api-access-chl8z" (OuterVolumeSpecName: "kube-api-access-chl8z") pod "44395f83-c8fd-494d-bcee-581293d99503" (UID: "44395f83-c8fd-494d-bcee-581293d99503"). InnerVolumeSpecName "kube-api-access-chl8z". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 01 16:38:29.575500 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:38:29.575427 2545 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/44395f83-c8fd-494d-bcee-581293d99503-istio-token\") on node \"ip-10-0-130-227.ec2.internal\" DevicePath \"\"" Jul 01 16:38:29.575500 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:38:29.575463 2545 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/44395f83-c8fd-494d-bcee-581293d99503-istio-csr-ca-configmap\") on node \"ip-10-0-130-227.ec2.internal\" DevicePath \"\"" Jul 01 16:38:29.575500 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:38:29.575481 2545 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/44395f83-c8fd-494d-bcee-581293d99503-cacerts\") on node \"ip-10-0-130-227.ec2.internal\" DevicePath \"\"" Jul 01 16:38:29.575500 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:38:29.575496 2545 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-chl8z\" (UniqueName: \"kubernetes.io/projected/44395f83-c8fd-494d-bcee-581293d99503-kube-api-access-chl8z\") on node \"ip-10-0-130-227.ec2.internal\" DevicePath \"\"" Jul 01 16:38:29.575801 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:38:29.575510 2545 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/44395f83-c8fd-494d-bcee-581293d99503-istio-csr-dns-cert\") on node \"ip-10-0-130-227.ec2.internal\" DevicePath \"\"" Jul 01 16:38:29.575801 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:38:29.575523 2545 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/44395f83-c8fd-494d-bcee-581293d99503-istio-kubeconfig\") on node \"ip-10-0-130-227.ec2.internal\" DevicePath \"\"" Jul 01 16:38:29.575801 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:38:29.575536 2545 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/44395f83-c8fd-494d-bcee-581293d99503-local-certs\") on node \"ip-10-0-130-227.ec2.internal\" DevicePath \"\"" Jul 01 16:38:29.893384 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:38:29.893300 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-4mb8s" event={"ID":"44395f83-c8fd-494d-bcee-581293d99503","Type":"ContainerDied","Data":"cea668cd2f9b268a69fe8ab3f4fbf82fbba0438d7441d03cc05c9ad306b128b4"} Jul 01 16:38:29.893384 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:38:29.893370 2545 scope.go:117] "RemoveContainer" containerID="931c6e3f5d78051a3ed69e465269e9426da611250c9899755dd5af93b377d501" Jul 01 16:38:29.893384 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:38:29.893372 2545 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-4mb8s" Jul 01 16:38:29.896772 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:38:29.896133 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-864bfd7bc9-knn5v" event={"ID":"9eae99c9-6daf-4a90-aada-126f22386acb","Type":"ContainerStarted","Data":"db093cad7ae54765c26d9c6f1e07f5c07f8648a0917b58bbc5243c47dcf3278e"} Jul 01 16:38:29.896772 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:38:29.896181 2545 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-864bfd7bc9-knn5v" Jul 01 16:38:29.916209 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:38:29.916167 2545 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-864bfd7bc9-knn5v" podStartSLOduration=2.902152029 podStartE2EDuration="5.916150261s" totalImagesPullingTime="3.013998232s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 16:38:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 16:38:29.914845162 +0000 UTC m=+588.495462666" watchObservedRunningTime="2026-07-01 16:38:29.916150261 +0000 UTC m=+588.496767771" Jul 01 16:38:29.930430 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:38:29.930406 2545 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-4mb8s"] Jul 01 16:38:29.936723 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:38:29.936692 2545 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-4mb8s"] Jul 01 16:38:30.937316 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:38:30.937281 2545 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="44395f83-c8fd-494d-bcee-581293d99503" path="/var/lib/kubelet/pods/44395f83-c8fd-494d-bcee-581293d99503/volumes" Jul 01 16:38:33.848308 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:38:33.848284 2545 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-6dc94f689f-566hl" Jul 01 16:38:33.850487 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:38:33.850471 2545 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-5dd488fc48-5pjl6" Jul 01 16:38:33.915780 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:38:33.915736 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/dashboard-redirect-9497df747-zjqqj" event={"ID":"676197d5-3145-4bfc-84f9-b2e2adc79d05","Type":"ContainerStarted","Data":"b35a1be1dd6bc773b4f1ef6a0aa35570d304256c63ed2be368bc33e888f19584"} Jul 01 16:38:33.930386 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:38:33.930342 2545 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/dashboard-redirect-9497df747-zjqqj" podStartSLOduration=1.221751387 podStartE2EDuration="6.930326409s" totalImagesPullingTime="5.708575022s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 16:38:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 16:38:33.9293665 +0000 UTC m=+592.509984001" watchObservedRunningTime="2026-07-01 16:38:33.930326409 +0000 UTC m=+592.510943913" Jul 01 16:38:42.892637 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:38:42.892612 2545 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5cr75_6c03a7ab-a7cc-44ed-86f7-62a750c49b71/ovn-acl-logging/0.log" Jul 01 16:38:42.897385 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:38:42.897357 2545 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5cr75_6c03a7ab-a7cc-44ed-86f7-62a750c49b71/ovn-acl-logging/0.log" Jul 01 16:38:59.902670 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:38:59.902637 2545 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-69fc866d76-jptg4" Jul 01 16:39:00.902546 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:39:00.902519 2545 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-864bfd7bc9-knn5v" Jul 01 16:39:16.210104 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:39:16.210022 2545 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-5ghx4"] Jul 01 16:39:16.210645 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:39:16.210484 2545 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="44395f83-c8fd-494d-bcee-581293d99503" containerName="discovery" Jul 01 16:39:16.210645 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:39:16.210502 2545 state_mem.go:107] "Deleted CPUSet assignment" podUID="44395f83-c8fd-494d-bcee-581293d99503" containerName="discovery" Jul 01 16:39:16.210645 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:39:16.210572 2545 memory_manager.go:368] "RemoveStaleState removing state" podUID="44395f83-c8fd-494d-bcee-581293d99503" containerName="discovery" Jul 01 16:39:16.213360 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:39:16.213340 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-5ghx4" Jul 01 16:39:16.215855 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:39:16.215833 2545 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 01 16:39:16.215964 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:39:16.215878 2545 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 01 16:39:16.217054 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:39:16.217031 2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-catalog-dockercfg-22fp5\"" Jul 01 16:39:16.221798 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:39:16.221772 2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-5ghx4"] Jul 01 16:39:16.340309 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:39:16.340286 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fgtnk\" (UniqueName: \"kubernetes.io/projected/da637008-319b-4b8f-9ff9-ddc518eea72c-kube-api-access-fgtnk\") pod \"kuadrant-operator-catalog-5ghx4\" (UID: \"da637008-319b-4b8f-9ff9-ddc518eea72c\") " pod="kuadrant-system/kuadrant-operator-catalog-5ghx4" Jul 01 16:39:16.441735 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:39:16.441682 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fgtnk\" (UniqueName: \"kubernetes.io/projected/da637008-319b-4b8f-9ff9-ddc518eea72c-kube-api-access-fgtnk\") pod \"kuadrant-operator-catalog-5ghx4\" (UID: \"da637008-319b-4b8f-9ff9-ddc518eea72c\") " pod="kuadrant-system/kuadrant-operator-catalog-5ghx4" Jul 01 16:39:16.450560 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:39:16.450528 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fgtnk\" (UniqueName: \"kubernetes.io/projected/da637008-319b-4b8f-9ff9-ddc518eea72c-kube-api-access-fgtnk\") pod \"kuadrant-operator-catalog-5ghx4\" (UID: \"da637008-319b-4b8f-9ff9-ddc518eea72c\") " pod="kuadrant-system/kuadrant-operator-catalog-5ghx4" Jul 01 16:39:16.525921 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:39:16.525866 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-5ghx4" Jul 01 16:39:16.579785 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:39:16.579753 2545 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-5ghx4"] Jul 01 16:39:16.665095 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:39:16.665069 2545 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-5ghx4"] Jul 01 16:39:16.667235 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:39:16.667212 2545 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podda637008_319b_4b8f_9ff9_ddc518eea72c.slice/crio-03ea7b284f3cd38f92afac13a3ed7d77a62e464cefb828dd86f3ad40eca4823b WatchSource:0}: Error finding container 03ea7b284f3cd38f92afac13a3ed7d77a62e464cefb828dd86f3ad40eca4823b: Status 404 returned error can't find the container with id 03ea7b284f3cd38f92afac13a3ed7d77a62e464cefb828dd86f3ad40eca4823b Jul 01 16:39:16.788430 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:39:16.788354 2545 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-4xqln"] Jul 01 16:39:16.792579 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:39:16.792564 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-4xqln" Jul 01 16:39:16.799840 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:39:16.799639 2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-4xqln"] Jul 01 16:39:16.946910 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:39:16.946882 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r57gq\" (UniqueName: \"kubernetes.io/projected/ab6090cc-d5cf-42a7-b98a-4088b3a0d6e8-kube-api-access-r57gq\") pod \"kuadrant-operator-catalog-4xqln\" (UID: \"ab6090cc-d5cf-42a7-b98a-4088b3a0d6e8\") " pod="kuadrant-system/kuadrant-operator-catalog-4xqln" Jul 01 16:39:17.047791 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:39:17.047699 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r57gq\" (UniqueName: \"kubernetes.io/projected/ab6090cc-d5cf-42a7-b98a-4088b3a0d6e8-kube-api-access-r57gq\") pod \"kuadrant-operator-catalog-4xqln\" (UID: \"ab6090cc-d5cf-42a7-b98a-4088b3a0d6e8\") " pod="kuadrant-system/kuadrant-operator-catalog-4xqln" Jul 01 16:39:17.055489 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:39:17.055461 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r57gq\" (UniqueName: \"kubernetes.io/projected/ab6090cc-d5cf-42a7-b98a-4088b3a0d6e8-kube-api-access-r57gq\") pod \"kuadrant-operator-catalog-4xqln\" (UID: \"ab6090cc-d5cf-42a7-b98a-4088b3a0d6e8\") " pod="kuadrant-system/kuadrant-operator-catalog-4xqln" Jul 01 16:39:17.090778 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:39:17.090749 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-5ghx4" event={"ID":"da637008-319b-4b8f-9ff9-ddc518eea72c","Type":"ContainerStarted","Data":"03ea7b284f3cd38f92afac13a3ed7d77a62e464cefb828dd86f3ad40eca4823b"} Jul 01 16:39:17.102958 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:39:17.102938 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-4xqln" Jul 01 16:39:17.227724 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:39:17.227685 2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-4xqln"] Jul 01 16:39:17.229996 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:39:17.229978 2545 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podab6090cc_d5cf_42a7_b98a_4088b3a0d6e8.slice/crio-fc13fdda9e7b991ecd4d4d4dbeccd5a8fb0cc628b04d0873f6f484c2cbf61438 WatchSource:0}: Error finding container fc13fdda9e7b991ecd4d4d4dbeccd5a8fb0cc628b04d0873f6f484c2cbf61438: Status 404 returned error can't find the container with id fc13fdda9e7b991ecd4d4d4dbeccd5a8fb0cc628b04d0873f6f484c2cbf61438 Jul 01 16:39:18.097500 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:39:18.097462 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-4xqln" event={"ID":"ab6090cc-d5cf-42a7-b98a-4088b3a0d6e8","Type":"ContainerStarted","Data":"fc13fdda9e7b991ecd4d4d4dbeccd5a8fb0cc628b04d0873f6f484c2cbf61438"} Jul 01 16:39:19.102665 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:39:19.102561 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-4xqln" event={"ID":"ab6090cc-d5cf-42a7-b98a-4088b3a0d6e8","Type":"ContainerStarted","Data":"5ad6d3787428d875ec31e88aaa671a7f9593e9b0f13e80d188944bee9231ffbc"} Jul 01 16:39:19.103947 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:39:19.103927 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-5ghx4" event={"ID":"da637008-319b-4b8f-9ff9-ddc518eea72c","Type":"ContainerStarted","Data":"1d7332a9fee77c31815849dbcfafc44db6697730d43e5b89260c5c8f057eb5b8"} Jul 01 16:39:19.104038 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:39:19.104026 2545 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-catalog-5ghx4" podUID="da637008-319b-4b8f-9ff9-ddc518eea72c" containerName="registry-server" containerID="cri-o://1d7332a9fee77c31815849dbcfafc44db6697730d43e5b89260c5c8f057eb5b8" gracePeriod=2 Jul 01 16:39:19.120144 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:39:19.120106 2545 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-4xqln" podStartSLOduration=1.5013593360000002 podStartE2EDuration="3.120091403s" totalImagesPullingTime="1.618732067s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 16:39:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 16:39:19.117992338 +0000 UTC m=+637.698609842" watchObservedRunningTime="2026-07-01 16:39:19.120091403 +0000 UTC m=+637.700708907" Jul 01 16:39:19.135858 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:39:19.135818 2545 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-5ghx4" podStartSLOduration=0.957600265 podStartE2EDuration="3.135804335s" totalImagesPullingTime="2.17820407s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 16:39:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 16:39:19.134282876 +0000 UTC m=+637.714900380" watchObservedRunningTime="2026-07-01 16:39:19.135804335 +0000 UTC m=+637.716421839" Jul 01 16:39:19.336082 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:39:19.336059 2545 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-5ghx4" Jul 01 16:39:19.471891 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:39:19.471819 2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fgtnk\" (UniqueName: \"kubernetes.io/projected/da637008-319b-4b8f-9ff9-ddc518eea72c-kube-api-access-fgtnk\") pod \"da637008-319b-4b8f-9ff9-ddc518eea72c\" (UID: \"da637008-319b-4b8f-9ff9-ddc518eea72c\") " Jul 01 16:39:19.473917 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:39:19.473896 2545 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/da637008-319b-4b8f-9ff9-ddc518eea72c-kube-api-access-fgtnk" (OuterVolumeSpecName: "kube-api-access-fgtnk") pod "da637008-319b-4b8f-9ff9-ddc518eea72c" (UID: "da637008-319b-4b8f-9ff9-ddc518eea72c"). InnerVolumeSpecName "kube-api-access-fgtnk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 01 16:39:19.572741 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:39:19.572696 2545 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fgtnk\" (UniqueName: \"kubernetes.io/projected/da637008-319b-4b8f-9ff9-ddc518eea72c-kube-api-access-fgtnk\") on node \"ip-10-0-130-227.ec2.internal\" DevicePath \"\"" Jul 01 16:39:20.109074 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:39:20.109043 2545 generic.go:356] "Generic (PLEG): container finished" podID="da637008-319b-4b8f-9ff9-ddc518eea72c" containerID="1d7332a9fee77c31815849dbcfafc44db6697730d43e5b89260c5c8f057eb5b8" exitCode=0 Jul 01 16:39:20.109499 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:39:20.109084 2545 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-5ghx4" Jul 01 16:39:20.109499 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:39:20.109136 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-5ghx4" event={"ID":"da637008-319b-4b8f-9ff9-ddc518eea72c","Type":"ContainerDied","Data":"1d7332a9fee77c31815849dbcfafc44db6697730d43e5b89260c5c8f057eb5b8"} Jul 01 16:39:20.109499 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:39:20.109183 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-5ghx4" event={"ID":"da637008-319b-4b8f-9ff9-ddc518eea72c","Type":"ContainerDied","Data":"03ea7b284f3cd38f92afac13a3ed7d77a62e464cefb828dd86f3ad40eca4823b"} Jul 01 16:39:20.109499 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:39:20.109208 2545 scope.go:117] "RemoveContainer" containerID="1d7332a9fee77c31815849dbcfafc44db6697730d43e5b89260c5c8f057eb5b8" Jul 01 16:39:20.117645 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:39:20.117629 2545 scope.go:117] "RemoveContainer" containerID="1d7332a9fee77c31815849dbcfafc44db6697730d43e5b89260c5c8f057eb5b8" Jul 01 16:39:20.117940 ip-10-0-130-227 kubenswrapper[2545]: E0701 16:39:20.117921 2545 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1d7332a9fee77c31815849dbcfafc44db6697730d43e5b89260c5c8f057eb5b8\": container with ID starting with 1d7332a9fee77c31815849dbcfafc44db6697730d43e5b89260c5c8f057eb5b8 not found: ID does not exist" containerID="1d7332a9fee77c31815849dbcfafc44db6697730d43e5b89260c5c8f057eb5b8" Jul 01 16:39:20.117999 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:39:20.117949 2545 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1d7332a9fee77c31815849dbcfafc44db6697730d43e5b89260c5c8f057eb5b8"} err="failed to get container status \"1d7332a9fee77c31815849dbcfafc44db6697730d43e5b89260c5c8f057eb5b8\": rpc error: code = NotFound desc = could not find container \"1d7332a9fee77c31815849dbcfafc44db6697730d43e5b89260c5c8f057eb5b8\": container with ID starting with 1d7332a9fee77c31815849dbcfafc44db6697730d43e5b89260c5c8f057eb5b8 not found: ID does not exist" Jul 01 16:39:20.129072 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:39:20.129048 2545 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-5ghx4"] Jul 01 16:39:20.132988 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:39:20.132959 2545 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-5ghx4"] Jul 01 16:39:20.934807 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:39:20.934777 2545 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="da637008-319b-4b8f-9ff9-ddc518eea72c" path="/var/lib/kubelet/pods/da637008-319b-4b8f-9ff9-ddc518eea72c/volumes" Jul 01 16:39:27.103661 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:39:27.103617 2545 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kuadrant-system/kuadrant-operator-catalog-4xqln" Jul 01 16:39:27.104235 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:39:27.103684 2545 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-catalog-4xqln" Jul 01 16:39:27.125584 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:39:27.125560 2545 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kuadrant-system/kuadrant-operator-catalog-4xqln" Jul 01 16:39:27.164088 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:39:27.164059 2545 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-catalog-4xqln" Jul 01 16:39:49.335510 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:39:49.335475 2545 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-v9ql8"] Jul 01 16:39:49.335915 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:39:49.335781 2545 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="da637008-319b-4b8f-9ff9-ddc518eea72c" containerName="registry-server" Jul 01 16:39:49.335915 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:39:49.335793 2545 state_mem.go:107] "Deleted CPUSet assignment" podUID="da637008-319b-4b8f-9ff9-ddc518eea72c" containerName="registry-server" Jul 01 16:39:49.335915 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:39:49.335856 2545 memory_manager.go:368] "RemoveStaleState removing state" podUID="da637008-319b-4b8f-9ff9-ddc518eea72c" containerName="registry-server" Jul 01 16:39:49.338446 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:39:49.338430 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-v9ql8" Jul 01 16:39:49.341123 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:39:49.341100 2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-stkrn\"" Jul 01 16:39:49.350794 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:39:49.350766 2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-v9ql8"] Jul 01 16:39:49.398315 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:39:49.398291 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cq5bq\" (UniqueName: \"kubernetes.io/projected/0ea1fe60-777b-4358-b06e-543850d67128-kube-api-access-cq5bq\") pod \"authorino-operator-75cc8d5cb7-v9ql8\" (UID: \"0ea1fe60-777b-4358-b06e-543850d67128\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-v9ql8" Jul 01 16:39:49.499350 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:39:49.499320 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cq5bq\" (UniqueName: \"kubernetes.io/projected/0ea1fe60-777b-4358-b06e-543850d67128-kube-api-access-cq5bq\") pod \"authorino-operator-75cc8d5cb7-v9ql8\" (UID: \"0ea1fe60-777b-4358-b06e-543850d67128\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-v9ql8" Jul 01 16:39:49.513583 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:39:49.513555 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cq5bq\" (UniqueName: \"kubernetes.io/projected/0ea1fe60-777b-4358-b06e-543850d67128-kube-api-access-cq5bq\") pod \"authorino-operator-75cc8d5cb7-v9ql8\" (UID: \"0ea1fe60-777b-4358-b06e-543850d67128\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-v9ql8" Jul 01 16:39:49.648353 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:39:49.648260 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-v9ql8" Jul 01 16:39:49.781172 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:39:49.781142 2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-v9ql8"] Jul 01 16:39:49.784934 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:39:49.784900 2545 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0ea1fe60_777b_4358_b06e_543850d67128.slice/crio-ac2209b0ac938802c28cc608215c0a4cb7010430347b1bde7da28cdacc6b00b9 WatchSource:0}: Error finding container ac2209b0ac938802c28cc608215c0a4cb7010430347b1bde7da28cdacc6b00b9: Status 404 returned error can't find the container with id ac2209b0ac938802c28cc608215c0a4cb7010430347b1bde7da28cdacc6b00b9 Jul 01 16:39:50.235895 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:39:50.235861 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-v9ql8" event={"ID":"0ea1fe60-777b-4358-b06e-543850d67128","Type":"ContainerStarted","Data":"ac2209b0ac938802c28cc608215c0a4cb7010430347b1bde7da28cdacc6b00b9"} Jul 01 16:39:53.252160 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:39:53.252116 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-v9ql8" event={"ID":"0ea1fe60-777b-4358-b06e-543850d67128","Type":"ContainerStarted","Data":"3bdefc128b2d0297eab89052037acc82c5e7d0d2200b6bd61d50621bfc09bd85"} Jul 01 16:39:53.252603 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:39:53.252217 2545 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-v9ql8" Jul 01 16:39:53.269538 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:39:53.269492 2545 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-75cc8d5cb7-v9ql8" podStartSLOduration=1.696206332 podStartE2EDuration="4.269479071s" totalImagesPullingTime="2.573272739s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 16:39:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 16:39:53.267917639 +0000 UTC m=+671.848535144" watchObservedRunningTime="2026-07-01 16:39:53.269479071 +0000 UTC m=+671.850096574" Jul 01 16:39:54.226319 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:39:54.226287 2545 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-w9px9"] Jul 01 16:39:54.230957 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:39:54.230934 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-w9px9" Jul 01 16:39:54.233544 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:39:54.233519 2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-bsgwz\"" Jul 01 16:39:54.234427 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:39:54.234405 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/21714ab0-af31-49fc-9cde-3519b4da6b42-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-w9px9\" (UID: \"21714ab0-af31-49fc-9cde-3519b4da6b42\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-w9px9" Jul 01 16:39:54.234510 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:39:54.234442 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9j9bm\" (UniqueName: \"kubernetes.io/projected/21714ab0-af31-49fc-9cde-3519b4da6b42-kube-api-access-9j9bm\") pod \"kuadrant-operator-controller-manager-8684c9ff49-w9px9\" (UID: \"21714ab0-af31-49fc-9cde-3519b4da6b42\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-w9px9" Jul 01 16:39:54.241218 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:39:54.241197 2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-w9px9"] Jul 01 16:39:54.335219 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:39:54.335181 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/21714ab0-af31-49fc-9cde-3519b4da6b42-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-w9px9\" (UID: \"21714ab0-af31-49fc-9cde-3519b4da6b42\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-w9px9" Jul 01 16:39:54.335667 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:39:54.335436 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9j9bm\" (UniqueName: \"kubernetes.io/projected/21714ab0-af31-49fc-9cde-3519b4da6b42-kube-api-access-9j9bm\") pod \"kuadrant-operator-controller-manager-8684c9ff49-w9px9\" (UID: \"21714ab0-af31-49fc-9cde-3519b4da6b42\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-w9px9" Jul 01 16:39:54.335667 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:39:54.335612 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/21714ab0-af31-49fc-9cde-3519b4da6b42-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-w9px9\" (UID: \"21714ab0-af31-49fc-9cde-3519b4da6b42\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-w9px9" Jul 01 16:39:54.345840 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:39:54.345823 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9j9bm\" (UniqueName: \"kubernetes.io/projected/21714ab0-af31-49fc-9cde-3519b4da6b42-kube-api-access-9j9bm\") pod \"kuadrant-operator-controller-manager-8684c9ff49-w9px9\" (UID: \"21714ab0-af31-49fc-9cde-3519b4da6b42\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-w9px9" Jul 01 16:39:54.541202 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:39:54.541128 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-w9px9" Jul 01 16:39:54.674070 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:39:54.673984 2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-w9px9"] Jul 01 16:39:54.676329 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:39:54.676290 2545 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod21714ab0_af31_49fc_9cde_3519b4da6b42.slice/crio-d6c7649ceeff81d0325c9b675955b6cb4391f65ea7ea3501ee4a87e2d170cea3 WatchSource:0}: Error finding container d6c7649ceeff81d0325c9b675955b6cb4391f65ea7ea3501ee4a87e2d170cea3: Status 404 returned error can't find the container with id d6c7649ceeff81d0325c9b675955b6cb4391f65ea7ea3501ee4a87e2d170cea3 Jul 01 16:39:55.262595 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:39:55.262556 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-w9px9" event={"ID":"21714ab0-af31-49fc-9cde-3519b4da6b42","Type":"ContainerStarted","Data":"d6c7649ceeff81d0325c9b675955b6cb4391f65ea7ea3501ee4a87e2d170cea3"} Jul 01 16:39:57.399009 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:39:57.398971 2545 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4n5c7"] Jul 01 16:39:57.402266 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:39:57.402243 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4n5c7" Jul 01 16:39:57.404979 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:39:57.404956 2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-r9wqr\"" Jul 01 16:39:57.415483 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:39:57.415458 2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4n5c7"] Jul 01 16:39:57.456985 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:39:57.456953 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d7nsr\" (UniqueName: \"kubernetes.io/projected/b7f334e9-da90-42ce-90d7-a46a19ea4095-kube-api-access-d7nsr\") pod \"limitador-operator-controller-manager-859d7b78bd-4n5c7\" (UID: \"b7f334e9-da90-42ce-90d7-a46a19ea4095\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4n5c7" Jul 01 16:39:57.558301 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:39:57.558269 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d7nsr\" (UniqueName: \"kubernetes.io/projected/b7f334e9-da90-42ce-90d7-a46a19ea4095-kube-api-access-d7nsr\") pod \"limitador-operator-controller-manager-859d7b78bd-4n5c7\" (UID: \"b7f334e9-da90-42ce-90d7-a46a19ea4095\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4n5c7" Jul 01 16:39:57.567108 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:39:57.567081 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d7nsr\" (UniqueName: \"kubernetes.io/projected/b7f334e9-da90-42ce-90d7-a46a19ea4095-kube-api-access-d7nsr\") pod \"limitador-operator-controller-manager-859d7b78bd-4n5c7\" (UID: \"b7f334e9-da90-42ce-90d7-a46a19ea4095\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4n5c7" Jul 01 16:39:57.714957 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:39:57.714868 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4n5c7" Jul 01 16:39:57.866140 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:39:57.866118 2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4n5c7"] Jul 01 16:39:57.867900 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:39:57.867868 2545 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb7f334e9_da90_42ce_90d7_a46a19ea4095.slice/crio-0b081eb786623066f3d50213df1fe1882f491191aefdbaaacaa219930f904a22 WatchSource:0}: Error finding container 0b081eb786623066f3d50213df1fe1882f491191aefdbaaacaa219930f904a22: Status 404 returned error can't find the container with id 0b081eb786623066f3d50213df1fe1882f491191aefdbaaacaa219930f904a22 Jul 01 16:39:58.276834 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:39:58.276803 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4n5c7" event={"ID":"b7f334e9-da90-42ce-90d7-a46a19ea4095","Type":"ContainerStarted","Data":"0b081eb786623066f3d50213df1fe1882f491191aefdbaaacaa219930f904a22"} Jul 01 16:40:00.290080 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:40:00.290045 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4n5c7" event={"ID":"b7f334e9-da90-42ce-90d7-a46a19ea4095","Type":"ContainerStarted","Data":"bbfb81f504c441e156da87a695241552ca04e94e075d5dbd498b1518e85a4aa5"} Jul 01 16:40:00.290461 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:40:00.290192 2545 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4n5c7" Jul 01 16:40:00.308305 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:40:00.308261 2545 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4n5c7" podStartSLOduration=1.032328308 podStartE2EDuration="3.308247778s" totalImagesPullingTime="2.27591947s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 16:39:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 16:40:00.306877585 +0000 UTC m=+678.887495090" watchObservedRunningTime="2026-07-01 16:40:00.308247778 +0000 UTC m=+678.888865282" Jul 01 16:40:03.308894 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:40:03.308861 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-w9px9" event={"ID":"21714ab0-af31-49fc-9cde-3519b4da6b42","Type":"ContainerStarted","Data":"fc0319812744be0234e0c279798bd0e1c8e0a3aa6992dafca43d2817c6311f1d"} Jul 01 16:40:03.309313 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:40:03.309102 2545 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-w9px9" Jul 01 16:40:03.329517 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:40:03.329469 2545 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-w9px9" podStartSLOduration=1.7583096889999998 podStartE2EDuration="9.329456005s" totalImagesPullingTime="7.571146316s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 16:39:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 16:40:03.326946298 +0000 UTC m=+681.907563811" watchObservedRunningTime="2026-07-01 16:40:03.329456005 +0000 UTC m=+681.910073509" Jul 01 16:40:04.259053 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:40:04.259025 2545 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-v9ql8" Jul 01 16:40:11.300670 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:40:11.300643 2545 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4n5c7" Jul 01 16:40:14.315851 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:40:14.315819 2545 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-w9px9" Jul 01 16:40:16.186486 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:40:16.186456 2545 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-w9px9"] Jul 01 16:40:16.186941 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:40:16.186627 2545 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-w9px9" podUID="21714ab0-af31-49fc-9cde-3519b4da6b42" containerName="manager" containerID="cri-o://fc0319812744be0234e0c279798bd0e1c8e0a3aa6992dafca43d2817c6311f1d" gracePeriod=2 Jul 01 16:40:16.196196 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:40:16.196114 2545 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-w9px9"] Jul 01 16:40:16.196518 ip-10-0-130-227 kubenswrapper[2545]: E0701 16:40:16.196486 2545 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"21714ab0-af31-49fc-9cde-3519b4da6b42\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Initialized\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Ready\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"observedGeneration\\\":2}}\" for pod \"kuadrant-system\"/\"kuadrant-operator-controller-manager-8684c9ff49-w9px9\": pods \"kuadrant-operator-controller-manager-8684c9ff49-w9px9\" not found" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-w9px9" Jul 01 16:40:16.213389 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:40:16.213361 2545 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-sq7js"] Jul 01 16:40:16.213885 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:40:16.213864 2545 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="21714ab0-af31-49fc-9cde-3519b4da6b42" containerName="manager" Jul 01 16:40:16.214029 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:40:16.214005 2545 state_mem.go:107] "Deleted CPUSet assignment" podUID="21714ab0-af31-49fc-9cde-3519b4da6b42" containerName="manager" Jul 01 16:40:16.214237 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:40:16.214223 2545 memory_manager.go:368] "RemoveStaleState removing state" podUID="21714ab0-af31-49fc-9cde-3519b4da6b42" containerName="manager" Jul 01 16:40:16.218256 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:40:16.218237 2545 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4n5c7"] Jul 01 16:40:16.218565 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:40:16.218443 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-sq7js" Jul 01 16:40:16.219194 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:40:16.219171 2545 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4n5c7" podUID="b7f334e9-da90-42ce-90d7-a46a19ea4095" containerName="manager" containerID="cri-o://bbfb81f504c441e156da87a695241552ca04e94e075d5dbd498b1518e85a4aa5" gracePeriod=2 Jul 01 16:40:16.222603 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:40:16.222328 2545 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4n5c7"] Jul 01 16:40:16.222603 ip-10-0-130-227 kubenswrapper[2545]: E0701 16:40:16.222484 2545 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-w9px9\" is forbidden: User \"system:node:ip-10-0-130-227.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-227.ec2.internal' and this object" podUID="21714ab0-af31-49fc-9cde-3519b4da6b42" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-w9px9" Jul 01 16:40:16.232703 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:40:16.232678 2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-sq7js"] Jul 01 16:40:16.248140 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:40:16.248024 2545 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-8zqvn"] Jul 01 16:40:16.248387 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:40:16.248373 2545 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b7f334e9-da90-42ce-90d7-a46a19ea4095" containerName="manager" Jul 01 16:40:16.248437 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:40:16.248389 2545 state_mem.go:107] "Deleted CPUSet assignment" podUID="b7f334e9-da90-42ce-90d7-a46a19ea4095" containerName="manager" Jul 01 16:40:16.248480 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:40:16.248437 2545 memory_manager.go:368] "RemoveStaleState removing state" podUID="b7f334e9-da90-42ce-90d7-a46a19ea4095" containerName="manager" Jul 01 16:40:16.251585 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:40:16.251568 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-8zqvn" Jul 01 16:40:16.252582 ip-10-0-130-227 kubenswrapper[2545]: E0701 16:40:16.252560 2545 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-4n5c7\" is forbidden: User \"system:node:ip-10-0-130-227.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-227.ec2.internal' and this object" podUID="b7f334e9-da90-42ce-90d7-a46a19ea4095" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4n5c7" Jul 01 16:40:16.254466 ip-10-0-130-227 kubenswrapper[2545]: E0701 16:40:16.254443 2545 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-w9px9\" is forbidden: User \"system:node:ip-10-0-130-227.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-227.ec2.internal' and this object" podUID="21714ab0-af31-49fc-9cde-3519b4da6b42" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-w9px9" Jul 01 16:40:16.256466 ip-10-0-130-227 kubenswrapper[2545]: E0701 16:40:16.256443 2545 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-w9px9\" is forbidden: User \"system:node:ip-10-0-130-227.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-227.ec2.internal' and this object" podUID="21714ab0-af31-49fc-9cde-3519b4da6b42" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-w9px9" Jul 01 16:40:16.259006 ip-10-0-130-227 kubenswrapper[2545]: E0701 16:40:16.258848 2545 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-4n5c7\" is forbidden: User \"system:node:ip-10-0-130-227.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-227.ec2.internal' and this object" podUID="b7f334e9-da90-42ce-90d7-a46a19ea4095" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4n5c7" Jul 01 16:40:16.260875 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:40:16.260854 2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-8zqvn"] Jul 01 16:40:16.308453 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:40:16.308428 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d8ba7bbf-67db-4f4e-b514-9ebba8aadd01-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-sq7js\" (UID: \"d8ba7bbf-67db-4f4e-b514-9ebba8aadd01\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-sq7js" Jul 01 16:40:16.308561 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:40:16.308492 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lbnx2\" (UniqueName: \"kubernetes.io/projected/6fbd1636-ade8-443e-8c81-e86ac53520f9-kube-api-access-lbnx2\") pod \"limitador-operator-controller-manager-859d7b78bd-8zqvn\" (UID: \"6fbd1636-ade8-443e-8c81-e86ac53520f9\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-8zqvn" Jul 01 16:40:16.308561 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:40:16.308531 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zv8j9\" (UniqueName: \"kubernetes.io/projected/d8ba7bbf-67db-4f4e-b514-9ebba8aadd01-kube-api-access-zv8j9\") pod \"kuadrant-operator-controller-manager-8684c9ff49-sq7js\" (UID: \"d8ba7bbf-67db-4f4e-b514-9ebba8aadd01\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-sq7js" Jul 01 16:40:16.365354 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:40:16.365316 2545 generic.go:356] "Generic (PLEG): container finished" podID="b7f334e9-da90-42ce-90d7-a46a19ea4095" containerID="bbfb81f504c441e156da87a695241552ca04e94e075d5dbd498b1518e85a4aa5" exitCode=0 Jul 01 16:40:16.367357 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:40:16.367326 2545 generic.go:356] "Generic (PLEG): container finished" podID="21714ab0-af31-49fc-9cde-3519b4da6b42" containerID="fc0319812744be0234e0c279798bd0e1c8e0a3aa6992dafca43d2817c6311f1d" exitCode=0 Jul 01 16:40:16.409283 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:40:16.409259 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d8ba7bbf-67db-4f4e-b514-9ebba8aadd01-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-sq7js\" (UID: \"d8ba7bbf-67db-4f4e-b514-9ebba8aadd01\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-sq7js" Jul 01 16:40:16.409396 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:40:16.409315 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lbnx2\" (UniqueName: \"kubernetes.io/projected/6fbd1636-ade8-443e-8c81-e86ac53520f9-kube-api-access-lbnx2\") pod \"limitador-operator-controller-manager-859d7b78bd-8zqvn\" (UID: \"6fbd1636-ade8-443e-8c81-e86ac53520f9\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-8zqvn" Jul 01 16:40:16.409396 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:40:16.409351 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zv8j9\" (UniqueName: \"kubernetes.io/projected/d8ba7bbf-67db-4f4e-b514-9ebba8aadd01-kube-api-access-zv8j9\") pod \"kuadrant-operator-controller-manager-8684c9ff49-sq7js\" (UID: \"d8ba7bbf-67db-4f4e-b514-9ebba8aadd01\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-sq7js" Jul 01 16:40:16.409758 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:40:16.409730 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d8ba7bbf-67db-4f4e-b514-9ebba8aadd01-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-sq7js\" (UID: \"d8ba7bbf-67db-4f4e-b514-9ebba8aadd01\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-sq7js" Jul 01 16:40:16.420432 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:40:16.420367 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zv8j9\" (UniqueName: \"kubernetes.io/projected/d8ba7bbf-67db-4f4e-b514-9ebba8aadd01-kube-api-access-zv8j9\") pod \"kuadrant-operator-controller-manager-8684c9ff49-sq7js\" (UID: \"d8ba7bbf-67db-4f4e-b514-9ebba8aadd01\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-sq7js" Jul 01 16:40:16.423591 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:40:16.423569 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lbnx2\" (UniqueName: \"kubernetes.io/projected/6fbd1636-ade8-443e-8c81-e86ac53520f9-kube-api-access-lbnx2\") pod \"limitador-operator-controller-manager-859d7b78bd-8zqvn\" (UID: \"6fbd1636-ade8-443e-8c81-e86ac53520f9\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-8zqvn" Jul 01 16:40:16.465644 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:40:16.465621 2545 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4n5c7" Jul 01 16:40:16.468092 ip-10-0-130-227 kubenswrapper[2545]: E0701 16:40:16.468066 2545 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-4n5c7\" is forbidden: User \"system:node:ip-10-0-130-227.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-227.ec2.internal' and this object" podUID="b7f334e9-da90-42ce-90d7-a46a19ea4095" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4n5c7" Jul 01 16:40:16.468326 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:40:16.468311 2545 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-w9px9" Jul 01 16:40:16.470043 ip-10-0-130-227 kubenswrapper[2545]: E0701 16:40:16.470021 2545 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-w9px9\" is forbidden: User \"system:node:ip-10-0-130-227.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-227.ec2.internal' and this object" podUID="21714ab0-af31-49fc-9cde-3519b4da6b42" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-w9px9" Jul 01 16:40:16.471892 ip-10-0-130-227 kubenswrapper[2545]: E0701 16:40:16.471870 2545 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-4n5c7\" is forbidden: User \"system:node:ip-10-0-130-227.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-227.ec2.internal' and this object" podUID="b7f334e9-da90-42ce-90d7-a46a19ea4095" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4n5c7" Jul 01 16:40:16.473786 ip-10-0-130-227 kubenswrapper[2545]: E0701 16:40:16.473753 2545 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-w9px9\" is forbidden: User \"system:node:ip-10-0-130-227.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-227.ec2.internal' and this object" podUID="21714ab0-af31-49fc-9cde-3519b4da6b42" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-w9px9" Jul 01 16:40:16.510177 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:40:16.510156 2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/21714ab0-af31-49fc-9cde-3519b4da6b42-extensions-socket-volume\") pod \"21714ab0-af31-49fc-9cde-3519b4da6b42\" (UID: \"21714ab0-af31-49fc-9cde-3519b4da6b42\") " Jul 01 16:40:16.510269 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:40:16.510187 2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d7nsr\" (UniqueName: \"kubernetes.io/projected/b7f334e9-da90-42ce-90d7-a46a19ea4095-kube-api-access-d7nsr\") pod \"b7f334e9-da90-42ce-90d7-a46a19ea4095\" (UID: \"b7f334e9-da90-42ce-90d7-a46a19ea4095\") " Jul 01 16:40:16.510269 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:40:16.510223 2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9j9bm\" (UniqueName: \"kubernetes.io/projected/21714ab0-af31-49fc-9cde-3519b4da6b42-kube-api-access-9j9bm\") pod \"21714ab0-af31-49fc-9cde-3519b4da6b42\" (UID: \"21714ab0-af31-49fc-9cde-3519b4da6b42\") " Jul 01 16:40:16.510615 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:40:16.510596 2545 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/21714ab0-af31-49fc-9cde-3519b4da6b42-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "21714ab0-af31-49fc-9cde-3519b4da6b42" (UID: "21714ab0-af31-49fc-9cde-3519b4da6b42"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 01 16:40:16.512128 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:40:16.512109 2545 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/21714ab0-af31-49fc-9cde-3519b4da6b42-kube-api-access-9j9bm" (OuterVolumeSpecName: "kube-api-access-9j9bm") pod "21714ab0-af31-49fc-9cde-3519b4da6b42" (UID: "21714ab0-af31-49fc-9cde-3519b4da6b42"). InnerVolumeSpecName "kube-api-access-9j9bm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 01 16:40:16.512199 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:40:16.512144 2545 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b7f334e9-da90-42ce-90d7-a46a19ea4095-kube-api-access-d7nsr" (OuterVolumeSpecName: "kube-api-access-d7nsr") pod "b7f334e9-da90-42ce-90d7-a46a19ea4095" (UID: "b7f334e9-da90-42ce-90d7-a46a19ea4095"). InnerVolumeSpecName "kube-api-access-d7nsr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 01 16:40:16.611372 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:40:16.611344 2545 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/21714ab0-af31-49fc-9cde-3519b4da6b42-extensions-socket-volume\") on node \"ip-10-0-130-227.ec2.internal\" DevicePath \"\"" Jul 01 16:40:16.611372 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:40:16.611367 2545 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d7nsr\" (UniqueName: \"kubernetes.io/projected/b7f334e9-da90-42ce-90d7-a46a19ea4095-kube-api-access-d7nsr\") on node \"ip-10-0-130-227.ec2.internal\" DevicePath \"\"" Jul 01 16:40:16.611372 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:40:16.611377 2545 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9j9bm\" (UniqueName: \"kubernetes.io/projected/21714ab0-af31-49fc-9cde-3519b4da6b42-kube-api-access-9j9bm\") on node \"ip-10-0-130-227.ec2.internal\" DevicePath \"\"" Jul 01 16:40:16.618298 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:40:16.618280 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-sq7js" Jul 01 16:40:16.624105 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:40:16.624089 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-8zqvn" Jul 01 16:40:16.765308 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:40:16.765284 2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-sq7js"] Jul 01 16:40:16.767345 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:40:16.767317 2545 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd8ba7bbf_67db_4f4e_b514_9ebba8aadd01.slice/crio-36d061292d8126bef61ac68f83230122e6894cbeeb7bd2a7fe403866837ecd7a WatchSource:0}: Error finding container 36d061292d8126bef61ac68f83230122e6894cbeeb7bd2a7fe403866837ecd7a: Status 404 returned error can't find the container with id 36d061292d8126bef61ac68f83230122e6894cbeeb7bd2a7fe403866837ecd7a Jul 01 16:40:16.782837 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:40:16.782637 2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-8zqvn"] Jul 01 16:40:16.784739 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:40:16.784704 2545 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6fbd1636_ade8_443e_8c81_e86ac53520f9.slice/crio-a5d90539f2c6fd8b78043a791cd0520342c6b90b20ea78c6b8c7d37bfdf61eb1 WatchSource:0}: Error finding container a5d90539f2c6fd8b78043a791cd0520342c6b90b20ea78c6b8c7d37bfdf61eb1: Status 404 returned error can't find the container with id a5d90539f2c6fd8b78043a791cd0520342c6b90b20ea78c6b8c7d37bfdf61eb1 Jul 01 16:40:16.936568 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:40:16.936525 2545 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="21714ab0-af31-49fc-9cde-3519b4da6b42" path="/var/lib/kubelet/pods/21714ab0-af31-49fc-9cde-3519b4da6b42/volumes" Jul 01 16:40:16.936900 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:40:16.936883 2545 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b7f334e9-da90-42ce-90d7-a46a19ea4095" path="/var/lib/kubelet/pods/b7f334e9-da90-42ce-90d7-a46a19ea4095/volumes" Jul 01 16:40:17.373594 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:40:17.373563 2545 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4n5c7" Jul 01 16:40:17.373594 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:40:17.373593 2545 scope.go:117] "RemoveContainer" containerID="bbfb81f504c441e156da87a695241552ca04e94e075d5dbd498b1518e85a4aa5" Jul 01 16:40:17.375694 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:40:17.375427 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-sq7js" event={"ID":"d8ba7bbf-67db-4f4e-b514-9ebba8aadd01","Type":"ContainerStarted","Data":"024a2b9f50e5ac3548d738c68ca128a200b9dcbc6bdac84c679937d593548d04"} Jul 01 16:40:17.375694 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:40:17.375488 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-sq7js" event={"ID":"d8ba7bbf-67db-4f4e-b514-9ebba8aadd01","Type":"ContainerStarted","Data":"36d061292d8126bef61ac68f83230122e6894cbeeb7bd2a7fe403866837ecd7a"} Jul 01 16:40:17.375694 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:40:17.375554 2545 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-sq7js" Jul 01 16:40:17.377203 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:40:17.377186 2545 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-w9px9" Jul 01 16:40:17.378723 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:40:17.378681 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-8zqvn" event={"ID":"6fbd1636-ade8-443e-8c81-e86ac53520f9","Type":"ContainerStarted","Data":"8241d1a1e07c093be64d6b43aa4538dbbdec7e1cc88d06ea2f7897f049ba8a07"} Jul 01 16:40:17.378810 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:40:17.378756 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-8zqvn" event={"ID":"6fbd1636-ade8-443e-8c81-e86ac53520f9","Type":"ContainerStarted","Data":"a5d90539f2c6fd8b78043a791cd0520342c6b90b20ea78c6b8c7d37bfdf61eb1"} Jul 01 16:40:17.378870 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:40:17.378827 2545 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-8zqvn" Jul 01 16:40:17.383193 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:40:17.383177 2545 scope.go:117] "RemoveContainer" containerID="fc0319812744be0234e0c279798bd0e1c8e0a3aa6992dafca43d2817c6311f1d" Jul 01 16:40:17.408113 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:40:17.408066 2545 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-sq7js" podStartSLOduration=1.408051544 podStartE2EDuration="1.408051544s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 16:40:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 16:40:17.407944644 +0000 UTC m=+695.988562149" watchObservedRunningTime="2026-07-01 16:40:17.408051544 +0000 UTC m=+695.988669049" Jul 01 16:40:17.428655 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:40:17.428622 2545 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-8zqvn" podStartSLOduration=1.428610791 podStartE2EDuration="1.428610791s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 16:40:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 16:40:17.427856401 +0000 UTC m=+696.008473906" watchObservedRunningTime="2026-07-01 16:40:17.428610791 +0000 UTC m=+696.009228295" Jul 01 16:40:28.386577 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:40:28.386547 2545 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-8zqvn" Jul 01 16:40:28.386997 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:40:28.386621 2545 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-sq7js" Jul 01 16:40:33.556853 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:40:33.556565 2545 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-sq7js"] Jul 01 16:40:33.556853 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:40:33.556804 2545 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-sq7js" podUID="d8ba7bbf-67db-4f4e-b514-9ebba8aadd01" containerName="manager" containerID="cri-o://024a2b9f50e5ac3548d738c68ca128a200b9dcbc6bdac84c679937d593548d04" gracePeriod=10 Jul 01 16:40:33.775436 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:40:33.775406 2545 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-w4gt8"] Jul 01 16:40:33.777669 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:40:33.777653 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-w4gt8" Jul 01 16:40:33.793265 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:40:33.793241 2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-w4gt8"] Jul 01 16:40:33.816942 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:40:33.816894 2545 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-sq7js" Jul 01 16:40:33.842644 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:40:33.842618 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4lp6t\" (UniqueName: \"kubernetes.io/projected/de017bac-8d83-4337-8d42-6692d57f07c2-kube-api-access-4lp6t\") pod \"kuadrant-operator-controller-manager-86878c769f-w4gt8\" (UID: \"de017bac-8d83-4337-8d42-6692d57f07c2\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-w4gt8" Jul 01 16:40:33.842786 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:40:33.842748 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/de017bac-8d83-4337-8d42-6692d57f07c2-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-w4gt8\" (UID: \"de017bac-8d83-4337-8d42-6692d57f07c2\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-w4gt8" Jul 01 16:40:33.943753 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:40:33.943702 2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d8ba7bbf-67db-4f4e-b514-9ebba8aadd01-extensions-socket-volume\") pod \"d8ba7bbf-67db-4f4e-b514-9ebba8aadd01\" (UID: \"d8ba7bbf-67db-4f4e-b514-9ebba8aadd01\") " Jul 01 16:40:33.943753 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:40:33.943767 2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zv8j9\" (UniqueName: \"kubernetes.io/projected/d8ba7bbf-67db-4f4e-b514-9ebba8aadd01-kube-api-access-zv8j9\") pod \"d8ba7bbf-67db-4f4e-b514-9ebba8aadd01\" (UID: \"d8ba7bbf-67db-4f4e-b514-9ebba8aadd01\") " Jul 01 16:40:33.943990 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:40:33.943906 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/de017bac-8d83-4337-8d42-6692d57f07c2-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-w4gt8\" (UID: \"de017bac-8d83-4337-8d42-6692d57f07c2\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-w4gt8" Jul 01 16:40:33.943990 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:40:33.943942 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4lp6t\" (UniqueName: \"kubernetes.io/projected/de017bac-8d83-4337-8d42-6692d57f07c2-kube-api-access-4lp6t\") pod \"kuadrant-operator-controller-manager-86878c769f-w4gt8\" (UID: \"de017bac-8d83-4337-8d42-6692d57f07c2\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-w4gt8" Jul 01 16:40:33.944144 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:40:33.944117 2545 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d8ba7bbf-67db-4f4e-b514-9ebba8aadd01-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "d8ba7bbf-67db-4f4e-b514-9ebba8aadd01" (UID: "d8ba7bbf-67db-4f4e-b514-9ebba8aadd01"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 01 16:40:33.944331 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:40:33.944301 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/de017bac-8d83-4337-8d42-6692d57f07c2-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-w4gt8\" (UID: \"de017bac-8d83-4337-8d42-6692d57f07c2\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-w4gt8" Jul 01 16:40:33.945877 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:40:33.945849 2545 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d8ba7bbf-67db-4f4e-b514-9ebba8aadd01-kube-api-access-zv8j9" (OuterVolumeSpecName: "kube-api-access-zv8j9") pod "d8ba7bbf-67db-4f4e-b514-9ebba8aadd01" (UID: "d8ba7bbf-67db-4f4e-b514-9ebba8aadd01"). InnerVolumeSpecName "kube-api-access-zv8j9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 01 16:40:33.952162 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:40:33.952142 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4lp6t\" (UniqueName: \"kubernetes.io/projected/de017bac-8d83-4337-8d42-6692d57f07c2-kube-api-access-4lp6t\") pod \"kuadrant-operator-controller-manager-86878c769f-w4gt8\" (UID: \"de017bac-8d83-4337-8d42-6692d57f07c2\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-w4gt8" Jul 01 16:40:34.045276 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:40:34.045245 2545 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d8ba7bbf-67db-4f4e-b514-9ebba8aadd01-extensions-socket-volume\") on node \"ip-10-0-130-227.ec2.internal\" DevicePath \"\"" Jul 01 16:40:34.045276 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:40:34.045273 2545 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zv8j9\" (UniqueName: \"kubernetes.io/projected/d8ba7bbf-67db-4f4e-b514-9ebba8aadd01-kube-api-access-zv8j9\") on node \"ip-10-0-130-227.ec2.internal\" DevicePath \"\"" Jul 01 16:40:34.088397 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:40:34.088329 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-w4gt8" Jul 01 16:40:34.228744 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:40:34.228722 2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-w4gt8"] Jul 01 16:40:34.230724 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:40:34.230678 2545 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podde017bac_8d83_4337_8d42_6692d57f07c2.slice/crio-2ef0bfe80ad74400b1df30a0f7e250e2ef3d75833b12f8b880003918c364f32e WatchSource:0}: Error finding container 2ef0bfe80ad74400b1df30a0f7e250e2ef3d75833b12f8b880003918c364f32e: Status 404 returned error can't find the container with id 2ef0bfe80ad74400b1df30a0f7e250e2ef3d75833b12f8b880003918c364f32e Jul 01 16:40:34.457720 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:40:34.457605 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-w4gt8" event={"ID":"de017bac-8d83-4337-8d42-6692d57f07c2","Type":"ContainerStarted","Data":"ab74361df31d3762a72707b1495b709c6eaa3f0b1ce0b05b7545e0521eb2db45"} Jul 01 16:40:34.457720 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:40:34.457665 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-w4gt8" event={"ID":"de017bac-8d83-4337-8d42-6692d57f07c2","Type":"ContainerStarted","Data":"2ef0bfe80ad74400b1df30a0f7e250e2ef3d75833b12f8b880003918c364f32e"} Jul 01 16:40:34.457720 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:40:34.457694 2545 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-w4gt8" Jul 01 16:40:34.458931 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:40:34.458905 2545 generic.go:356] "Generic (PLEG): container finished" podID="d8ba7bbf-67db-4f4e-b514-9ebba8aadd01" containerID="024a2b9f50e5ac3548d738c68ca128a200b9dcbc6bdac84c679937d593548d04" exitCode=0 Jul 01 16:40:34.459018 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:40:34.458952 2545 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-sq7js" Jul 01 16:40:34.459018 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:40:34.458965 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-sq7js" event={"ID":"d8ba7bbf-67db-4f4e-b514-9ebba8aadd01","Type":"ContainerDied","Data":"024a2b9f50e5ac3548d738c68ca128a200b9dcbc6bdac84c679937d593548d04"} Jul 01 16:40:34.459103 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:40:34.459021 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-sq7js" event={"ID":"d8ba7bbf-67db-4f4e-b514-9ebba8aadd01","Type":"ContainerDied","Data":"36d061292d8126bef61ac68f83230122e6894cbeeb7bd2a7fe403866837ecd7a"} Jul 01 16:40:34.459103 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:40:34.459054 2545 scope.go:117] "RemoveContainer" containerID="024a2b9f50e5ac3548d738c68ca128a200b9dcbc6bdac84c679937d593548d04" Jul 01 16:40:34.467338 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:40:34.467323 2545 scope.go:117] "RemoveContainer" containerID="024a2b9f50e5ac3548d738c68ca128a200b9dcbc6bdac84c679937d593548d04" Jul 01 16:40:34.467575 ip-10-0-130-227 kubenswrapper[2545]: E0701 16:40:34.467556 2545 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"024a2b9f50e5ac3548d738c68ca128a200b9dcbc6bdac84c679937d593548d04\": container with ID starting with 024a2b9f50e5ac3548d738c68ca128a200b9dcbc6bdac84c679937d593548d04 not found: ID does not exist" containerID="024a2b9f50e5ac3548d738c68ca128a200b9dcbc6bdac84c679937d593548d04" Jul 01 16:40:34.467636 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:40:34.467587 2545 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"024a2b9f50e5ac3548d738c68ca128a200b9dcbc6bdac84c679937d593548d04"} err="failed to get container status \"024a2b9f50e5ac3548d738c68ca128a200b9dcbc6bdac84c679937d593548d04\": rpc error: code = NotFound desc = could not find container \"024a2b9f50e5ac3548d738c68ca128a200b9dcbc6bdac84c679937d593548d04\": container with ID starting with 024a2b9f50e5ac3548d738c68ca128a200b9dcbc6bdac84c679937d593548d04 not found: ID does not exist" Jul 01 16:40:34.476855 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:40:34.476816 2545 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-w4gt8" podStartSLOduration=1.476803324 podStartE2EDuration="1.476803324s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 16:40:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 16:40:34.475318462 +0000 UTC m=+713.055935972" watchObservedRunningTime="2026-07-01 16:40:34.476803324 +0000 UTC m=+713.057420831" Jul 01 16:40:34.491901 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:40:34.491870 2545 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-sq7js"] Jul 01 16:40:34.502497 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:40:34.502479 2545 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-sq7js"] Jul 01 16:40:34.936413 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:40:34.936384 2545 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d8ba7bbf-67db-4f4e-b514-9ebba8aadd01" path="/var/lib/kubelet/pods/d8ba7bbf-67db-4f4e-b514-9ebba8aadd01/volumes" Jul 01 16:40:45.466597 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:40:45.466568 2545 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-w4gt8" Jul 01 16:40:49.852671 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:40:49.852641 2545 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-g8wlh"] Jul 01 16:40:49.853038 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:40:49.853006 2545 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d8ba7bbf-67db-4f4e-b514-9ebba8aadd01" containerName="manager" Jul 01 16:40:49.853038 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:40:49.853017 2545 state_mem.go:107] "Deleted CPUSet assignment" podUID="d8ba7bbf-67db-4f4e-b514-9ebba8aadd01" containerName="manager" Jul 01 16:40:49.853112 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:40:49.853070 2545 memory_manager.go:368] "RemoveStaleState removing state" podUID="d8ba7bbf-67db-4f4e-b514-9ebba8aadd01" containerName="manager" Jul 01 16:40:49.856065 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:40:49.856047 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-g8wlh" Jul 01 16:40:49.858792 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:40:49.858774 2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-5xpvh\"" Jul 01 16:40:49.865425 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:40:49.865402 2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-g8wlh"] Jul 01 16:40:49.875426 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:40:49.875403 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b18e0e25-6471-4e9d-bbe6-7f41f9390dc8-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-g8wlh\" (UID: \"b18e0e25-6471-4e9d-bbe6-7f41f9390dc8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-g8wlh" Jul 01 16:40:49.875533 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:40:49.875435 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b18e0e25-6471-4e9d-bbe6-7f41f9390dc8-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-g8wlh\" (UID: \"b18e0e25-6471-4e9d-bbe6-7f41f9390dc8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-g8wlh" Jul 01 16:40:49.875533 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:40:49.875489 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b18e0e25-6471-4e9d-bbe6-7f41f9390dc8-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-g8wlh\" (UID: \"b18e0e25-6471-4e9d-bbe6-7f41f9390dc8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-g8wlh" Jul 01 16:40:49.875533 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:40:49.875524 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b18e0e25-6471-4e9d-bbe6-7f41f9390dc8-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-g8wlh\" (UID: \"b18e0e25-6471-4e9d-bbe6-7f41f9390dc8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-g8wlh" Jul 01 16:40:49.875731 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:40:49.875571 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b18e0e25-6471-4e9d-bbe6-7f41f9390dc8-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-g8wlh\" (UID: \"b18e0e25-6471-4e9d-bbe6-7f41f9390dc8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-g8wlh" Jul 01 16:40:49.875731 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:40:49.875606 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b18e0e25-6471-4e9d-bbe6-7f41f9390dc8-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-g8wlh\" (UID: \"b18e0e25-6471-4e9d-bbe6-7f41f9390dc8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-g8wlh" Jul 01 16:40:49.875731 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:40:49.875659 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b18e0e25-6471-4e9d-bbe6-7f41f9390dc8-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-g8wlh\" (UID: \"b18e0e25-6471-4e9d-bbe6-7f41f9390dc8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-g8wlh" Jul 01 16:40:49.875731 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:40:49.875698 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dhcg2\" (UniqueName: \"kubernetes.io/projected/b18e0e25-6471-4e9d-bbe6-7f41f9390dc8-kube-api-access-dhcg2\") pod \"maas-default-gateway-openshift-default-56654dcf7-g8wlh\" (UID: \"b18e0e25-6471-4e9d-bbe6-7f41f9390dc8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-g8wlh" Jul 01 16:40:49.876026 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:40:49.875792 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b18e0e25-6471-4e9d-bbe6-7f41f9390dc8-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-g8wlh\" (UID: \"b18e0e25-6471-4e9d-bbe6-7f41f9390dc8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-g8wlh" Jul 01 16:40:49.976659 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:40:49.976619 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b18e0e25-6471-4e9d-bbe6-7f41f9390dc8-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-g8wlh\" (UID: \"b18e0e25-6471-4e9d-bbe6-7f41f9390dc8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-g8wlh" Jul 01 16:40:49.976849 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:40:49.976691 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b18e0e25-6471-4e9d-bbe6-7f41f9390dc8-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-g8wlh\" (UID: \"b18e0e25-6471-4e9d-bbe6-7f41f9390dc8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-g8wlh" Jul 01 16:40:49.976849 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:40:49.976751 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b18e0e25-6471-4e9d-bbe6-7f41f9390dc8-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-g8wlh\" (UID: \"b18e0e25-6471-4e9d-bbe6-7f41f9390dc8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-g8wlh" Jul 01 16:40:49.976971 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:40:49.976923 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b18e0e25-6471-4e9d-bbe6-7f41f9390dc8-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-g8wlh\" (UID: \"b18e0e25-6471-4e9d-bbe6-7f41f9390dc8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-g8wlh" Jul 01 16:40:49.977028 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:40:49.976968 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b18e0e25-6471-4e9d-bbe6-7f41f9390dc8-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-g8wlh\" (UID: \"b18e0e25-6471-4e9d-bbe6-7f41f9390dc8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-g8wlh" Jul 01 16:40:49.977028 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:40:49.976995 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dhcg2\" (UniqueName: \"kubernetes.io/projected/b18e0e25-6471-4e9d-bbe6-7f41f9390dc8-kube-api-access-dhcg2\") pod \"maas-default-gateway-openshift-default-56654dcf7-g8wlh\" (UID: \"b18e0e25-6471-4e9d-bbe6-7f41f9390dc8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-g8wlh" Jul 01 16:40:49.977131 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:40:49.977027 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b18e0e25-6471-4e9d-bbe6-7f41f9390dc8-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-g8wlh\" (UID: \"b18e0e25-6471-4e9d-bbe6-7f41f9390dc8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-g8wlh" Jul 01 16:40:49.977131 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:40:49.977098 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b18e0e25-6471-4e9d-bbe6-7f41f9390dc8-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-g8wlh\" (UID: \"b18e0e25-6471-4e9d-bbe6-7f41f9390dc8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-g8wlh" Jul 01 16:40:49.977232 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:40:49.977135 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b18e0e25-6471-4e9d-bbe6-7f41f9390dc8-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-g8wlh\" (UID: \"b18e0e25-6471-4e9d-bbe6-7f41f9390dc8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-g8wlh" Jul 01 16:40:49.977288 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:40:49.977246 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b18e0e25-6471-4e9d-bbe6-7f41f9390dc8-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-g8wlh\" (UID: \"b18e0e25-6471-4e9d-bbe6-7f41f9390dc8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-g8wlh" Jul 01 16:40:49.977368 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:40:49.977342 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b18e0e25-6471-4e9d-bbe6-7f41f9390dc8-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-g8wlh\" (UID: \"b18e0e25-6471-4e9d-bbe6-7f41f9390dc8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-g8wlh" Jul 01 16:40:49.977434 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:40:49.977413 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b18e0e25-6471-4e9d-bbe6-7f41f9390dc8-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-g8wlh\" (UID: \"b18e0e25-6471-4e9d-bbe6-7f41f9390dc8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-g8wlh" Jul 01 16:40:49.977490 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:40:49.977471 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b18e0e25-6471-4e9d-bbe6-7f41f9390dc8-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-g8wlh\" (UID: \"b18e0e25-6471-4e9d-bbe6-7f41f9390dc8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-g8wlh" Jul 01 16:40:49.978014 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:40:49.977991 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b18e0e25-6471-4e9d-bbe6-7f41f9390dc8-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-g8wlh\" (UID: \"b18e0e25-6471-4e9d-bbe6-7f41f9390dc8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-g8wlh" Jul 01 16:40:49.979156 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:40:49.979135 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b18e0e25-6471-4e9d-bbe6-7f41f9390dc8-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-g8wlh\" (UID: \"b18e0e25-6471-4e9d-bbe6-7f41f9390dc8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-g8wlh" Jul 01 16:40:49.979565 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:40:49.979543 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b18e0e25-6471-4e9d-bbe6-7f41f9390dc8-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-g8wlh\" (UID: \"b18e0e25-6471-4e9d-bbe6-7f41f9390dc8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-g8wlh" Jul 01 16:40:49.984793 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:40:49.984764 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dhcg2\" (UniqueName: \"kubernetes.io/projected/b18e0e25-6471-4e9d-bbe6-7f41f9390dc8-kube-api-access-dhcg2\") pod \"maas-default-gateway-openshift-default-56654dcf7-g8wlh\" (UID: \"b18e0e25-6471-4e9d-bbe6-7f41f9390dc8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-g8wlh" Jul 01 16:40:49.984991 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:40:49.984971 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b18e0e25-6471-4e9d-bbe6-7f41f9390dc8-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-g8wlh\" (UID: \"b18e0e25-6471-4e9d-bbe6-7f41f9390dc8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-g8wlh" Jul 01 16:40:50.166252 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:40:50.166175 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-g8wlh" Jul 01 16:40:50.297155 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:40:50.297130 2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-g8wlh"] Jul 01 16:40:50.298747 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:40:50.298704 2545 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb18e0e25_6471_4e9d_bbe6_7f41f9390dc8.slice/crio-806cac86493e81daff0e9c7e4ed4a16bc6c11919bf54053f1420be0a9d8f6e2b WatchSource:0}: Error finding container 806cac86493e81daff0e9c7e4ed4a16bc6c11919bf54053f1420be0a9d8f6e2b: Status 404 returned error can't find the container with id 806cac86493e81daff0e9c7e4ed4a16bc6c11919bf54053f1420be0a9d8f6e2b Jul 01 16:40:50.300854 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:40:50.300822 2545 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 01 16:40:50.300911 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:40:50.300892 2545 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 01 16:40:50.300950 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:40:50.300922 2545 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 01 16:40:50.543806 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:40:50.543705 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-g8wlh" event={"ID":"b18e0e25-6471-4e9d-bbe6-7f41f9390dc8","Type":"ContainerStarted","Data":"e0120bdb858f67368486f5b125bafa45701dd06a20580fa622966263aaba0469"} Jul 01 16:40:50.543806 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:40:50.543791 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-g8wlh" event={"ID":"b18e0e25-6471-4e9d-bbe6-7f41f9390dc8","Type":"ContainerStarted","Data":"806cac86493e81daff0e9c7e4ed4a16bc6c11919bf54053f1420be0a9d8f6e2b"} Jul 01 16:40:50.562086 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:40:50.562033 2545 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-g8wlh" podStartSLOduration=1.562011614 podStartE2EDuration="1.562011614s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 16:40:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 16:40:50.560636716 +0000 UTC m=+729.141254230" watchObservedRunningTime="2026-07-01 16:40:50.562011614 +0000 UTC m=+729.142629120" Jul 01 16:40:51.166454 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:40:51.166417 2545 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-g8wlh" Jul 01 16:40:51.171193 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:40:51.171173 2545 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-g8wlh" Jul 01 16:40:51.548110 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:40:51.548025 2545 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-g8wlh" Jul 01 16:40:51.548953 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:40:51.548933 2545 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-g8wlh" Jul 01 16:41:05.168204 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:41:05.168173 2545 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-58cd955675-w95rl"] Jul 01 16:41:05.171888 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:41:05.171870 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-w95rl" Jul 01 16:41:05.174536 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:41:05.174516 2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-9cgp4\"" Jul 01 16:41:05.179099 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:41:05.179080 2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-w95rl"] Jul 01 16:41:05.192870 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:41:05.192847 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7g759\" (UniqueName: \"kubernetes.io/projected/64880e0c-58d3-4112-92b1-e8661affce18-kube-api-access-7g759\") pod \"authorino-58cd955675-w95rl\" (UID: \"64880e0c-58d3-4112-92b1-e8661affce18\") " pod="kuadrant-system/authorino-58cd955675-w95rl" Jul 01 16:41:05.294019 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:41:05.293988 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7g759\" (UniqueName: \"kubernetes.io/projected/64880e0c-58d3-4112-92b1-e8661affce18-kube-api-access-7g759\") pod \"authorino-58cd955675-w95rl\" (UID: \"64880e0c-58d3-4112-92b1-e8661affce18\") " pod="kuadrant-system/authorino-58cd955675-w95rl" Jul 01 16:41:05.301544 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:41:05.301524 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7g759\" (UniqueName: \"kubernetes.io/projected/64880e0c-58d3-4112-92b1-e8661affce18-kube-api-access-7g759\") pod \"authorino-58cd955675-w95rl\" (UID: \"64880e0c-58d3-4112-92b1-e8661affce18\") " pod="kuadrant-system/authorino-58cd955675-w95rl" Jul 01 16:41:05.482353 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:41:05.482293 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-w95rl" Jul 01 16:41:05.605393 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:41:05.605364 2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-w95rl"] Jul 01 16:41:05.607369 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:41:05.607345 2545 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod64880e0c_58d3_4112_92b1_e8661affce18.slice/crio-c56c0bab95c5ba105d2b051b96bafcc69d0d652c4b1af20a4ba5fc3434107649 WatchSource:0}: Error finding container c56c0bab95c5ba105d2b051b96bafcc69d0d652c4b1af20a4ba5fc3434107649: Status 404 returned error can't find the container with id c56c0bab95c5ba105d2b051b96bafcc69d0d652c4b1af20a4ba5fc3434107649 Jul 01 16:41:05.617231 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:41:05.617206 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-w95rl" event={"ID":"64880e0c-58d3-4112-92b1-e8661affce18","Type":"ContainerStarted","Data":"c56c0bab95c5ba105d2b051b96bafcc69d0d652c4b1af20a4ba5fc3434107649"} Jul 01 16:41:09.640889 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:41:09.640854 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-w95rl" event={"ID":"64880e0c-58d3-4112-92b1-e8661affce18","Type":"ContainerStarted","Data":"01b73596910aeec32b9b5acafa2d6c36f6e2bc0c51a1cce3628e67fc70faf13a"} Jul 01 16:41:09.656000 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:41:09.655954 2545 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-58cd955675-w95rl" podStartSLOduration=1.328548075 podStartE2EDuration="4.655939827s" totalImagesPullingTime="3.327391752s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 16:41:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 16:41:09.654150998 +0000 UTC m=+748.234768503" watchObservedRunningTime="2026-07-01 16:41:09.655939827 +0000 UTC m=+748.236557331" Jul 01 16:41:11.047312 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:41:11.047282 2545 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-w95rl"] Jul 01 16:41:11.650526 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:41:11.650484 2545 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-58cd955675-w95rl" podUID="64880e0c-58d3-4112-92b1-e8661affce18" containerName="authorino" containerID="cri-o://01b73596910aeec32b9b5acafa2d6c36f6e2bc0c51a1cce3628e67fc70faf13a" gracePeriod=30 Jul 01 16:41:11.899627 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:41:11.899604 2545 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-w95rl" Jul 01 16:41:11.951477 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:41:11.951400 2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7g759\" (UniqueName: \"kubernetes.io/projected/64880e0c-58d3-4112-92b1-e8661affce18-kube-api-access-7g759\") pod \"64880e0c-58d3-4112-92b1-e8661affce18\" (UID: \"64880e0c-58d3-4112-92b1-e8661affce18\") " Jul 01 16:41:11.953368 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:41:11.953344 2545 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/64880e0c-58d3-4112-92b1-e8661affce18-kube-api-access-7g759" (OuterVolumeSpecName: "kube-api-access-7g759") pod "64880e0c-58d3-4112-92b1-e8661affce18" (UID: "64880e0c-58d3-4112-92b1-e8661affce18"). InnerVolumeSpecName "kube-api-access-7g759". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 01 16:41:12.052962 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:41:12.052919 2545 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7g759\" (UniqueName: \"kubernetes.io/projected/64880e0c-58d3-4112-92b1-e8661affce18-kube-api-access-7g759\") on node \"ip-10-0-130-227.ec2.internal\" DevicePath \"\"" Jul 01 16:41:12.656315 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:41:12.656282 2545 generic.go:356] "Generic (PLEG): container finished" podID="64880e0c-58d3-4112-92b1-e8661affce18" containerID="01b73596910aeec32b9b5acafa2d6c36f6e2bc0c51a1cce3628e67fc70faf13a" exitCode=0 Jul 01 16:41:12.656487 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:41:12.656325 2545 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-w95rl" Jul 01 16:41:12.656487 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:41:12.656375 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-w95rl" event={"ID":"64880e0c-58d3-4112-92b1-e8661affce18","Type":"ContainerDied","Data":"01b73596910aeec32b9b5acafa2d6c36f6e2bc0c51a1cce3628e67fc70faf13a"} Jul 01 16:41:12.656487 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:41:12.656421 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-w95rl" event={"ID":"64880e0c-58d3-4112-92b1-e8661affce18","Type":"ContainerDied","Data":"c56c0bab95c5ba105d2b051b96bafcc69d0d652c4b1af20a4ba5fc3434107649"} Jul 01 16:41:12.656487 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:41:12.656443 2545 scope.go:117] "RemoveContainer" containerID="01b73596910aeec32b9b5acafa2d6c36f6e2bc0c51a1cce3628e67fc70faf13a" Jul 01 16:41:12.664740 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:41:12.664723 2545 scope.go:117] "RemoveContainer" containerID="01b73596910aeec32b9b5acafa2d6c36f6e2bc0c51a1cce3628e67fc70faf13a" Jul 01 16:41:12.664972 ip-10-0-130-227 kubenswrapper[2545]: E0701 16:41:12.664955 2545 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"01b73596910aeec32b9b5acafa2d6c36f6e2bc0c51a1cce3628e67fc70faf13a\": container with ID starting with 01b73596910aeec32b9b5acafa2d6c36f6e2bc0c51a1cce3628e67fc70faf13a not found: ID does not exist" containerID="01b73596910aeec32b9b5acafa2d6c36f6e2bc0c51a1cce3628e67fc70faf13a" Jul 01 16:41:12.665017 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:41:12.664980 2545 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"01b73596910aeec32b9b5acafa2d6c36f6e2bc0c51a1cce3628e67fc70faf13a"} err="failed to get container status \"01b73596910aeec32b9b5acafa2d6c36f6e2bc0c51a1cce3628e67fc70faf13a\": rpc error: code = NotFound desc = could not find container \"01b73596910aeec32b9b5acafa2d6c36f6e2bc0c51a1cce3628e67fc70faf13a\": container with ID starting with 01b73596910aeec32b9b5acafa2d6c36f6e2bc0c51a1cce3628e67fc70faf13a not found: ID does not exist" Jul 01 16:41:12.675466 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:41:12.675446 2545 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-w95rl"] Jul 01 16:41:12.681441 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:41:12.681421 2545 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-58cd955675-w95rl"] Jul 01 16:41:12.936697 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:41:12.936625 2545 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="64880e0c-58d3-4112-92b1-e8661affce18" path="/var/lib/kubelet/pods/64880e0c-58d3-4112-92b1-e8661affce18/volumes" Jul 01 16:42:02.019537 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:42:02.019503 2545 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["keycloak-system/maas-keycloak-0"] Jul 01 16:42:02.019949 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:42:02.019863 2545 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="64880e0c-58d3-4112-92b1-e8661affce18" containerName="authorino" Jul 01 16:42:02.019949 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:42:02.019875 2545 state_mem.go:107] "Deleted CPUSet assignment" podUID="64880e0c-58d3-4112-92b1-e8661affce18" containerName="authorino" Jul 01 16:42:02.019949 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:42:02.019948 2545 memory_manager.go:368] "RemoveStaleState removing state" podUID="64880e0c-58d3-4112-92b1-e8661affce18" containerName="authorino" Jul 01 16:42:02.025790 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:42:02.025771 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="keycloak-system/maas-keycloak-0" Jul 01 16:42:02.028633 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:42:02.028606 2545 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"keycloak-system\"/\"kube-root-ca.crt\"" Jul 01 16:42:02.028755 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:42:02.028691 2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"keycloak-system\"/\"default-dockercfg-r9khd\"" Jul 01 16:42:02.029521 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:42:02.029497 2545 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"keycloak-system\"/\"openshift-service-ca.crt\"" Jul 01 16:42:02.029521 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:42:02.029518 2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"keycloak-system\"/\"maas-keycloak-initial-admin\"" Jul 01 16:42:02.031591 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:42:02.031566 2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["keycloak-system/maas-keycloak-0"] Jul 01 16:42:02.052730 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:42:02.052684 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j9vnj\" (UniqueName: \"kubernetes.io/projected/585fa80e-cdea-49cc-9fbf-86aa93a1fe94-kube-api-access-j9vnj\") pod \"maas-keycloak-0\" (UID: \"585fa80e-cdea-49cc-9fbf-86aa93a1fe94\") " pod="keycloak-system/maas-keycloak-0" Jul 01 16:42:02.153977 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:42:02.153948 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j9vnj\" (UniqueName: \"kubernetes.io/projected/585fa80e-cdea-49cc-9fbf-86aa93a1fe94-kube-api-access-j9vnj\") pod \"maas-keycloak-0\" (UID: \"585fa80e-cdea-49cc-9fbf-86aa93a1fe94\") " pod="keycloak-system/maas-keycloak-0" Jul 01 16:42:02.161148 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:42:02.161130 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j9vnj\" (UniqueName: \"kubernetes.io/projected/585fa80e-cdea-49cc-9fbf-86aa93a1fe94-kube-api-access-j9vnj\") pod \"maas-keycloak-0\" (UID: \"585fa80e-cdea-49cc-9fbf-86aa93a1fe94\") " pod="keycloak-system/maas-keycloak-0" Jul 01 16:42:02.335653 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:42:02.335624 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="keycloak-system/maas-keycloak-0" Jul 01 16:42:02.460124 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:42:02.460102 2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["keycloak-system/maas-keycloak-0"] Jul 01 16:42:02.462303 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:42:02.462275 2545 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod585fa80e_cdea_49cc_9fbf_86aa93a1fe94.slice/crio-4440d403e6b0644c8433c2ebcce0ee1b6e8ddb71cb384a52869bebf7ab4c86c2 WatchSource:0}: Error finding container 4440d403e6b0644c8433c2ebcce0ee1b6e8ddb71cb384a52869bebf7ab4c86c2: Status 404 returned error can't find the container with id 4440d403e6b0644c8433c2ebcce0ee1b6e8ddb71cb384a52869bebf7ab4c86c2 Jul 01 16:42:02.880621 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:42:02.880583 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="keycloak-system/maas-keycloak-0" event={"ID":"585fa80e-cdea-49cc-9fbf-86aa93a1fe94","Type":"ContainerStarted","Data":"4440d403e6b0644c8433c2ebcce0ee1b6e8ddb71cb384a52869bebf7ab4c86c2"} Jul 01 16:42:07.908972 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:42:07.908918 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="keycloak-system/maas-keycloak-0" event={"ID":"585fa80e-cdea-49cc-9fbf-86aa93a1fe94","Type":"ContainerStarted","Data":"218d67afb9bf3be10b0d53dcb5eb911ac177314a0c39b242bb39b82da30467fb"} Jul 01 16:42:07.925799 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:42:07.925737 2545 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="keycloak-system/maas-keycloak-0" podStartSLOduration=0.924794893 podStartE2EDuration="5.925686114s" totalImagesPullingTime="5.000891221s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 16:42:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 16:42:07.924628707 +0000 UTC m=+806.505246219" watchObservedRunningTime="2026-07-01 16:42:07.925686114 +0000 UTC m=+806.506303619" Jul 01 16:42:08.336367 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:42:08.336324 2545 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="keycloak-system/maas-keycloak-0" Jul 01 16:42:08.337908 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:42:08.337873 2545 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="585fa80e-cdea-49cc-9fbf-86aa93a1fe94" containerName="keycloak" probeResult="failure" output="Get \"http://10.134.0.33:9000/health/started\": dial tcp 10.134.0.33:9000: connect: connection refused" Jul 01 16:42:09.336791 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:42:09.336741 2545 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="585fa80e-cdea-49cc-9fbf-86aa93a1fe94" containerName="keycloak" probeResult="failure" output="Get \"http://10.134.0.33:9000/health/started\": dial tcp 10.134.0.33:9000: connect: connection refused" Jul 01 16:42:10.336675 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:42:10.336627 2545 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="585fa80e-cdea-49cc-9fbf-86aa93a1fe94" containerName="keycloak" probeResult="failure" output="Get \"http://10.134.0.33:9000/health/started\": dial tcp 10.134.0.33:9000: connect: connection refused" Jul 01 16:42:11.336361 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:42:11.336311 2545 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="585fa80e-cdea-49cc-9fbf-86aa93a1fe94" containerName="keycloak" probeResult="failure" output="Get \"http://10.134.0.33:9000/health/started\": dial tcp 10.134.0.33:9000: connect: connection refused" Jul 01 16:42:12.336557 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:42:12.336513 2545 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="keycloak-system/maas-keycloak-0" Jul 01 16:42:12.337070 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:42:12.336840 2545 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="585fa80e-cdea-49cc-9fbf-86aa93a1fe94" containerName="keycloak" probeResult="failure" output="Get \"http://10.134.0.33:9000/health/started\": dial tcp 10.134.0.33:9000: connect: connection refused" Jul 01 16:42:13.336399 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:42:13.336352 2545 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="585fa80e-cdea-49cc-9fbf-86aa93a1fe94" containerName="keycloak" probeResult="failure" output="Get \"http://10.134.0.33:9000/health/started\": dial tcp 10.134.0.33:9000: connect: connection refused" Jul 01 16:42:14.336647 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:42:14.336596 2545 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="585fa80e-cdea-49cc-9fbf-86aa93a1fe94" containerName="keycloak" probeResult="failure" output="Get \"http://10.134.0.33:9000/health/started\": dial tcp 10.134.0.33:9000: connect: connection refused" Jul 01 16:42:15.336571 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:42:15.336524 2545 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="585fa80e-cdea-49cc-9fbf-86aa93a1fe94" containerName="keycloak" probeResult="failure" output="Get \"http://10.134.0.33:9000/health/started\": dial tcp 10.134.0.33:9000: connect: connection refused" Jul 01 16:42:16.336538 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:42:16.336487 2545 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="585fa80e-cdea-49cc-9fbf-86aa93a1fe94" containerName="keycloak" probeResult="failure" output="Get \"http://10.134.0.33:9000/health/started\": dial tcp 10.134.0.33:9000: connect: connection refused" Jul 01 16:42:17.336962 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:42:17.336907 2545 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="585fa80e-cdea-49cc-9fbf-86aa93a1fe94" containerName="keycloak" probeResult="failure" output="Get \"http://10.134.0.33:9000/health/started\": dial tcp 10.134.0.33:9000: connect: connection refused" Jul 01 16:42:18.336551 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:42:18.336501 2545 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="585fa80e-cdea-49cc-9fbf-86aa93a1fe94" containerName="keycloak" probeResult="failure" output="Get \"http://10.134.0.33:9000/health/started\": dial tcp 10.134.0.33:9000: connect: connection refused" Jul 01 16:42:19.336303 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:42:19.336253 2545 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="585fa80e-cdea-49cc-9fbf-86aa93a1fe94" containerName="keycloak" probeResult="failure" output="Get \"http://10.134.0.33:9000/health/started\": dial tcp 10.134.0.33:9000: connect: connection refused" Jul 01 16:42:20.443619 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:42:20.443569 2545 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="keycloak-system/maas-keycloak-0" Jul 01 16:42:20.456890 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:42:20.456850 2545 prober.go:120] "Probe failed" probeType="Readiness" pod="keycloak-system/maas-keycloak-0" podUID="585fa80e-cdea-49cc-9fbf-86aa93a1fe94" containerName="keycloak" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 01 16:42:30.449632 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:42:30.449600 2545 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="keycloak-system/maas-keycloak-0" Jul 01 16:42:37.685728 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:42:37.685682 2545 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-6d8648fc6f-hg2j4"] Jul 01 16:42:37.688267 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:42:37.688244 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6d8648fc6f-hg2j4" Jul 01 16:42:37.691031 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:42:37.691008 2545 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jul 01 16:42:37.691031 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:42:37.691022 2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jul 01 16:42:37.691188 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:42:37.691088 2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-wkfgm\"" Jul 01 16:42:37.695413 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:42:37.695388 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ldj2m\" (UniqueName: \"kubernetes.io/projected/8c17827d-c585-4c20-91e6-caccb6398c6f-kube-api-access-ldj2m\") pod \"maas-controller-6d8648fc6f-hg2j4\" (UID: \"8c17827d-c585-4c20-91e6-caccb6398c6f\") " pod="opendatahub/maas-controller-6d8648fc6f-hg2j4" Jul 01 16:42:37.695512 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:42:37.695453 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8c17827d-c585-4c20-91e6-caccb6398c6f-webhook-cert\") pod \"maas-controller-6d8648fc6f-hg2j4\" (UID: \"8c17827d-c585-4c20-91e6-caccb6398c6f\") " pod="opendatahub/maas-controller-6d8648fc6f-hg2j4" Jul 01 16:42:37.699376 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:42:37.699357 2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6d8648fc6f-hg2j4"] Jul 01 16:42:37.796343 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:42:37.796310 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ldj2m\" (UniqueName: \"kubernetes.io/projected/8c17827d-c585-4c20-91e6-caccb6398c6f-kube-api-access-ldj2m\") pod \"maas-controller-6d8648fc6f-hg2j4\" (UID: \"8c17827d-c585-4c20-91e6-caccb6398c6f\") " pod="opendatahub/maas-controller-6d8648fc6f-hg2j4" Jul 01 16:42:37.796499 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:42:37.796375 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8c17827d-c585-4c20-91e6-caccb6398c6f-webhook-cert\") pod \"maas-controller-6d8648fc6f-hg2j4\" (UID: \"8c17827d-c585-4c20-91e6-caccb6398c6f\") " pod="opendatahub/maas-controller-6d8648fc6f-hg2j4" Jul 01 16:42:37.796499 ip-10-0-130-227 kubenswrapper[2545]: E0701 16:42:37.796493 2545 secret.go:189] Couldn't get secret opendatahub/maas-controller-webhook-cert: secret "maas-controller-webhook-cert" not found Jul 01 16:42:37.796586 ip-10-0-130-227 kubenswrapper[2545]: E0701 16:42:37.796576 2545 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8c17827d-c585-4c20-91e6-caccb6398c6f-webhook-cert podName:8c17827d-c585-4c20-91e6-caccb6398c6f nodeName:}" failed. No retries permitted until 2026-07-01 16:42:38.29655392 +0000 UTC m=+836.877171402 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "webhook-cert" (UniqueName: "kubernetes.io/secret/8c17827d-c585-4c20-91e6-caccb6398c6f-webhook-cert") pod "maas-controller-6d8648fc6f-hg2j4" (UID: "8c17827d-c585-4c20-91e6-caccb6398c6f") : secret "maas-controller-webhook-cert" not found Jul 01 16:42:37.804453 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:42:37.804424 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ldj2m\" (UniqueName: \"kubernetes.io/projected/8c17827d-c585-4c20-91e6-caccb6398c6f-kube-api-access-ldj2m\") pod \"maas-controller-6d8648fc6f-hg2j4\" (UID: \"8c17827d-c585-4c20-91e6-caccb6398c6f\") " pod="opendatahub/maas-controller-6d8648fc6f-hg2j4" Jul 01 16:42:37.853102 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:42:37.853077 2545 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-5476db56c7-8vzlk"] Jul 01 16:42:37.855779 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:42:37.855761 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5476db56c7-8vzlk" Jul 01 16:42:37.866065 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:42:37.866042 2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5476db56c7-8vzlk"] Jul 01 16:42:37.896893 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:42:37.896867 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/893752c2-7088-4970-8f1e-cc860de1e571-webhook-cert\") pod \"maas-controller-5476db56c7-8vzlk\" (UID: \"893752c2-7088-4970-8f1e-cc860de1e571\") " pod="opendatahub/maas-controller-5476db56c7-8vzlk" Jul 01 16:42:37.896994 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:42:37.896896 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q5wjv\" (UniqueName: \"kubernetes.io/projected/893752c2-7088-4970-8f1e-cc860de1e571-kube-api-access-q5wjv\") pod \"maas-controller-5476db56c7-8vzlk\" (UID: \"893752c2-7088-4970-8f1e-cc860de1e571\") " pod="opendatahub/maas-controller-5476db56c7-8vzlk" Jul 01 16:42:37.998296 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:42:37.998217 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/893752c2-7088-4970-8f1e-cc860de1e571-webhook-cert\") pod \"maas-controller-5476db56c7-8vzlk\" (UID: \"893752c2-7088-4970-8f1e-cc860de1e571\") " pod="opendatahub/maas-controller-5476db56c7-8vzlk" Jul 01 16:42:37.998296 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:42:37.998254 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q5wjv\" (UniqueName: \"kubernetes.io/projected/893752c2-7088-4970-8f1e-cc860de1e571-kube-api-access-q5wjv\") pod \"maas-controller-5476db56c7-8vzlk\" (UID: \"893752c2-7088-4970-8f1e-cc860de1e571\") " pod="opendatahub/maas-controller-5476db56c7-8vzlk" Jul 01 16:42:38.000615 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:42:38.000593 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/893752c2-7088-4970-8f1e-cc860de1e571-webhook-cert\") pod \"maas-controller-5476db56c7-8vzlk\" (UID: \"893752c2-7088-4970-8f1e-cc860de1e571\") " pod="opendatahub/maas-controller-5476db56c7-8vzlk" Jul 01 16:42:38.007475 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:42:38.007450 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q5wjv\" (UniqueName: \"kubernetes.io/projected/893752c2-7088-4970-8f1e-cc860de1e571-kube-api-access-q5wjv\") pod \"maas-controller-5476db56c7-8vzlk\" (UID: \"893752c2-7088-4970-8f1e-cc860de1e571\") " pod="opendatahub/maas-controller-5476db56c7-8vzlk" Jul 01 16:42:38.167317 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:42:38.167270 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5476db56c7-8vzlk" Jul 01 16:42:38.291394 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:42:38.291324 2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5476db56c7-8vzlk"] Jul 01 16:42:38.294180 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:42:38.294158 2545 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod893752c2_7088_4970_8f1e_cc860de1e571.slice/crio-c2455a7199aed3a5e69443bdddae9ae7a1f798d7bf4e519c2fa964528112316d WatchSource:0}: Error finding container c2455a7199aed3a5e69443bdddae9ae7a1f798d7bf4e519c2fa964528112316d: Status 404 returned error can't find the container with id c2455a7199aed3a5e69443bdddae9ae7a1f798d7bf4e519c2fa964528112316d Jul 01 16:42:38.295471 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:42:38.295449 2545 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 01 16:42:38.302391 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:42:38.302371 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8c17827d-c585-4c20-91e6-caccb6398c6f-webhook-cert\") pod \"maas-controller-6d8648fc6f-hg2j4\" (UID: \"8c17827d-c585-4c20-91e6-caccb6398c6f\") " pod="opendatahub/maas-controller-6d8648fc6f-hg2j4" Jul 01 16:42:38.304515 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:42:38.304498 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8c17827d-c585-4c20-91e6-caccb6398c6f-webhook-cert\") pod \"maas-controller-6d8648fc6f-hg2j4\" (UID: \"8c17827d-c585-4c20-91e6-caccb6398c6f\") " pod="opendatahub/maas-controller-6d8648fc6f-hg2j4" Jul 01 16:42:38.599307 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:42:38.599280 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6d8648fc6f-hg2j4" Jul 01 16:42:38.725077 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:42:38.725052 2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6d8648fc6f-hg2j4"] Jul 01 16:42:38.727473 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:42:38.727446 2545 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8c17827d_c585_4c20_91e6_caccb6398c6f.slice/crio-830b5d8466266930299f6043bd17672e4bda6c8681041aa56c9849b7066f3064 WatchSource:0}: Error finding container 830b5d8466266930299f6043bd17672e4bda6c8681041aa56c9849b7066f3064: Status 404 returned error can't find the container with id 830b5d8466266930299f6043bd17672e4bda6c8681041aa56c9849b7066f3064 Jul 01 16:42:39.087983 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:42:39.087949 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6d8648fc6f-hg2j4" event={"ID":"8c17827d-c585-4c20-91e6-caccb6398c6f","Type":"ContainerStarted","Data":"830b5d8466266930299f6043bd17672e4bda6c8681041aa56c9849b7066f3064"} Jul 01 16:42:39.088991 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:42:39.088973 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5476db56c7-8vzlk" event={"ID":"893752c2-7088-4970-8f1e-cc860de1e571","Type":"ContainerStarted","Data":"c2455a7199aed3a5e69443bdddae9ae7a1f798d7bf4e519c2fa964528112316d"} Jul 01 16:42:41.104343 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:42:41.104309 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6d8648fc6f-hg2j4" event={"ID":"8c17827d-c585-4c20-91e6-caccb6398c6f","Type":"ContainerStarted","Data":"a0bf7d42c2e1a14b204603c6184271a51e1cefd0690662013f3be16bdf8ce431"} Jul 01 16:42:41.104766 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:42:41.104421 2545 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-6d8648fc6f-hg2j4" Jul 01 16:42:41.106275 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:42:41.106239 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5476db56c7-8vzlk" event={"ID":"893752c2-7088-4970-8f1e-cc860de1e571","Type":"ContainerStarted","Data":"5470f26db38425c89bbd1a5c14274477f79f00dc0e76fc8fe2de7d70bc24f677"} Jul 01 16:42:41.106403 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:42:41.106373 2545 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-5476db56c7-8vzlk" Jul 01 16:42:41.121120 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:42:41.121076 2545 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-6d8648fc6f-hg2j4" podStartSLOduration=1.83354858 podStartE2EDuration="4.121063661s" totalImagesPullingTime="2.287515081s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 16:42:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 16:42:41.119414693 +0000 UTC m=+839.700032220" watchObservedRunningTime="2026-07-01 16:42:41.121063661 +0000 UTC m=+839.701681165" Jul 01 16:42:41.135820 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:42:41.135776 2545 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-5476db56c7-8vzlk" podStartSLOduration=1.4144781 podStartE2EDuration="4.135757679s" totalImagesPullingTime="2.721279579s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 16:42:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 16:42:41.13319626 +0000 UTC m=+839.713813766" watchObservedRunningTime="2026-07-01 16:42:41.135757679 +0000 UTC m=+839.716375184" Jul 01 16:42:52.115493 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:42:52.115464 2545 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-5476db56c7-8vzlk" Jul 01 16:42:52.115917 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:42:52.115534 2545 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-6d8648fc6f-hg2j4" Jul 01 16:42:52.167805 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:42:52.167772 2545 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-6d8648fc6f-hg2j4"] Jul 01 16:42:52.168125 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:42:52.167967 2545 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-controller-6d8648fc6f-hg2j4" podUID="8c17827d-c585-4c20-91e6-caccb6398c6f" containerName="manager" containerID="cri-o://a0bf7d42c2e1a14b204603c6184271a51e1cefd0690662013f3be16bdf8ce431" gracePeriod=30 Jul 01 16:42:52.414608 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:42:52.414584 2545 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6d8648fc6f-hg2j4" Jul 01 16:42:52.531867 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:42:52.531824 2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ldj2m\" (UniqueName: \"kubernetes.io/projected/8c17827d-c585-4c20-91e6-caccb6398c6f-kube-api-access-ldj2m\") pod \"8c17827d-c585-4c20-91e6-caccb6398c6f\" (UID: \"8c17827d-c585-4c20-91e6-caccb6398c6f\") " Jul 01 16:42:52.532058 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:42:52.531888 2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8c17827d-c585-4c20-91e6-caccb6398c6f-webhook-cert\") pod \"8c17827d-c585-4c20-91e6-caccb6398c6f\" (UID: \"8c17827d-c585-4c20-91e6-caccb6398c6f\") " Jul 01 16:42:52.534126 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:42:52.534095 2545 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8c17827d-c585-4c20-91e6-caccb6398c6f-kube-api-access-ldj2m" (OuterVolumeSpecName: "kube-api-access-ldj2m") pod "8c17827d-c585-4c20-91e6-caccb6398c6f" (UID: "8c17827d-c585-4c20-91e6-caccb6398c6f"). InnerVolumeSpecName "kube-api-access-ldj2m". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 01 16:42:52.534234 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:42:52.534158 2545 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8c17827d-c585-4c20-91e6-caccb6398c6f-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "8c17827d-c585-4c20-91e6-caccb6398c6f" (UID: "8c17827d-c585-4c20-91e6-caccb6398c6f"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 01 16:42:52.632882 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:42:52.632854 2545 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ldj2m\" (UniqueName: \"kubernetes.io/projected/8c17827d-c585-4c20-91e6-caccb6398c6f-kube-api-access-ldj2m\") on node \"ip-10-0-130-227.ec2.internal\" DevicePath \"\"" Jul 01 16:42:52.633001 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:42:52.632893 2545 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8c17827d-c585-4c20-91e6-caccb6398c6f-webhook-cert\") on node \"ip-10-0-130-227.ec2.internal\" DevicePath \"\"" Jul 01 16:42:53.159460 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:42:53.159425 2545 generic.go:356] "Generic (PLEG): container finished" podID="8c17827d-c585-4c20-91e6-caccb6398c6f" containerID="a0bf7d42c2e1a14b204603c6184271a51e1cefd0690662013f3be16bdf8ce431" exitCode=0 Jul 01 16:42:53.159876 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:42:53.159484 2545 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6d8648fc6f-hg2j4" Jul 01 16:42:53.159876 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:42:53.159509 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6d8648fc6f-hg2j4" event={"ID":"8c17827d-c585-4c20-91e6-caccb6398c6f","Type":"ContainerDied","Data":"a0bf7d42c2e1a14b204603c6184271a51e1cefd0690662013f3be16bdf8ce431"} Jul 01 16:42:53.159876 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:42:53.159582 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6d8648fc6f-hg2j4" event={"ID":"8c17827d-c585-4c20-91e6-caccb6398c6f","Type":"ContainerDied","Data":"830b5d8466266930299f6043bd17672e4bda6c8681041aa56c9849b7066f3064"} Jul 01 16:42:53.159876 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:42:53.159603 2545 scope.go:117] "RemoveContainer" containerID="a0bf7d42c2e1a14b204603c6184271a51e1cefd0690662013f3be16bdf8ce431" Jul 01 16:42:53.168650 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:42:53.168634 2545 scope.go:117] "RemoveContainer" containerID="a0bf7d42c2e1a14b204603c6184271a51e1cefd0690662013f3be16bdf8ce431" Jul 01 16:42:53.168945 ip-10-0-130-227 kubenswrapper[2545]: E0701 16:42:53.168927 2545 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a0bf7d42c2e1a14b204603c6184271a51e1cefd0690662013f3be16bdf8ce431\": container with ID starting with a0bf7d42c2e1a14b204603c6184271a51e1cefd0690662013f3be16bdf8ce431 not found: ID does not exist" containerID="a0bf7d42c2e1a14b204603c6184271a51e1cefd0690662013f3be16bdf8ce431" Jul 01 16:42:53.168998 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:42:53.168954 2545 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a0bf7d42c2e1a14b204603c6184271a51e1cefd0690662013f3be16bdf8ce431"} err="failed to get container status \"a0bf7d42c2e1a14b204603c6184271a51e1cefd0690662013f3be16bdf8ce431\": rpc error: code = NotFound desc = could not find container \"a0bf7d42c2e1a14b204603c6184271a51e1cefd0690662013f3be16bdf8ce431\": container with ID starting with a0bf7d42c2e1a14b204603c6184271a51e1cefd0690662013f3be16bdf8ce431 not found: ID does not exist" Jul 01 16:42:53.174957 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:42:53.174936 2545 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-6d8648fc6f-hg2j4"] Jul 01 16:42:53.178100 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:42:53.178078 2545 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-6d8648fc6f-hg2j4"] Jul 01 16:42:54.935438 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:42:54.935398 2545 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8c17827d-c585-4c20-91e6-caccb6398c6f" path="/var/lib/kubelet/pods/8c17827d-c585-4c20-91e6-caccb6398c6f/volumes" Jul 01 16:43:14.188240 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:43:14.188206 2545 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-99968fc6d-v2jkl"] Jul 01 16:43:14.188829 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:43:14.188728 2545 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8c17827d-c585-4c20-91e6-caccb6398c6f" containerName="manager" Jul 01 16:43:14.188829 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:43:14.188748 2545 state_mem.go:107] "Deleted CPUSet assignment" podUID="8c17827d-c585-4c20-91e6-caccb6398c6f" containerName="manager" Jul 01 16:43:14.188954 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:43:14.188857 2545 memory_manager.go:368] "RemoveStaleState removing state" podUID="8c17827d-c585-4c20-91e6-caccb6398c6f" containerName="manager" Jul 01 16:43:14.193677 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:43:14.193653 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-99968fc6d-v2jkl" Jul 01 16:43:14.195992 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:43:14.195967 2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-serving-cert\"" Jul 01 16:43:14.196099 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:43:14.195988 2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-klmlm\"" Jul 01 16:43:14.199910 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:43:14.199888 2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-99968fc6d-v2jkl"] Jul 01 16:43:14.318591 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:43:14.318551 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/60c3923d-5ea5-49e5-87e4-62f60638bb5b-maas-api-tls\") pod \"maas-api-99968fc6d-v2jkl\" (UID: \"60c3923d-5ea5-49e5-87e4-62f60638bb5b\") " pod="opendatahub/maas-api-99968fc6d-v2jkl" Jul 01 16:43:14.318591 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:43:14.318589 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/60c3923d-5ea5-49e5-87e4-62f60638bb5b-openshift-service-ca\") pod \"maas-api-99968fc6d-v2jkl\" (UID: \"60c3923d-5ea5-49e5-87e4-62f60638bb5b\") " pod="opendatahub/maas-api-99968fc6d-v2jkl" Jul 01 16:43:14.318855 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:43:14.318705 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ghtlq\" (UniqueName: \"kubernetes.io/projected/60c3923d-5ea5-49e5-87e4-62f60638bb5b-kube-api-access-ghtlq\") pod \"maas-api-99968fc6d-v2jkl\" (UID: \"60c3923d-5ea5-49e5-87e4-62f60638bb5b\") " pod="opendatahub/maas-api-99968fc6d-v2jkl" Jul 01 16:43:14.419170 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:43:14.419137 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/60c3923d-5ea5-49e5-87e4-62f60638bb5b-maas-api-tls\") pod \"maas-api-99968fc6d-v2jkl\" (UID: \"60c3923d-5ea5-49e5-87e4-62f60638bb5b\") " pod="opendatahub/maas-api-99968fc6d-v2jkl" Jul 01 16:43:14.419338 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:43:14.419183 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/60c3923d-5ea5-49e5-87e4-62f60638bb5b-openshift-service-ca\") pod \"maas-api-99968fc6d-v2jkl\" (UID: \"60c3923d-5ea5-49e5-87e4-62f60638bb5b\") " pod="opendatahub/maas-api-99968fc6d-v2jkl" Jul 01 16:43:14.419338 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:43:14.419271 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ghtlq\" (UniqueName: \"kubernetes.io/projected/60c3923d-5ea5-49e5-87e4-62f60638bb5b-kube-api-access-ghtlq\") pod \"maas-api-99968fc6d-v2jkl\" (UID: \"60c3923d-5ea5-49e5-87e4-62f60638bb5b\") " pod="opendatahub/maas-api-99968fc6d-v2jkl" Jul 01 16:43:14.419822 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:43:14.419797 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/60c3923d-5ea5-49e5-87e4-62f60638bb5b-openshift-service-ca\") pod \"maas-api-99968fc6d-v2jkl\" (UID: \"60c3923d-5ea5-49e5-87e4-62f60638bb5b\") " pod="opendatahub/maas-api-99968fc6d-v2jkl" Jul 01 16:43:14.421560 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:43:14.421539 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/60c3923d-5ea5-49e5-87e4-62f60638bb5b-maas-api-tls\") pod \"maas-api-99968fc6d-v2jkl\" (UID: \"60c3923d-5ea5-49e5-87e4-62f60638bb5b\") " pod="opendatahub/maas-api-99968fc6d-v2jkl" Jul 01 16:43:14.427344 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:43:14.427316 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ghtlq\" (UniqueName: \"kubernetes.io/projected/60c3923d-5ea5-49e5-87e4-62f60638bb5b-kube-api-access-ghtlq\") pod \"maas-api-99968fc6d-v2jkl\" (UID: \"60c3923d-5ea5-49e5-87e4-62f60638bb5b\") " pod="opendatahub/maas-api-99968fc6d-v2jkl" Jul 01 16:43:14.505968 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:43:14.505880 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-99968fc6d-v2jkl" Jul 01 16:43:14.652856 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:43:14.652824 2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-99968fc6d-v2jkl"] Jul 01 16:43:14.656519 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:43:14.656486 2545 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod60c3923d_5ea5_49e5_87e4_62f60638bb5b.slice/crio-4c84e92f8149011f4597670f844f3298d8403d4f6814b3a4f3162571f4a3ebd5 WatchSource:0}: Error finding container 4c84e92f8149011f4597670f844f3298d8403d4f6814b3a4f3162571f4a3ebd5: Status 404 returned error can't find the container with id 4c84e92f8149011f4597670f844f3298d8403d4f6814b3a4f3162571f4a3ebd5 Jul 01 16:43:14.658603 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:43:14.658583 2545 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-59f49fd4f6-l52k7"] Jul 01 16:43:14.676024 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:43:14.676000 2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-59f49fd4f6-l52k7"] Jul 01 16:43:14.676136 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:43:14.676081 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-59f49fd4f6-l52k7" Jul 01 16:43:14.678372 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:43:14.678350 2545 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\"" Jul 01 16:43:14.823470 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:43:14.823432 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9lnh2\" (UniqueName: \"kubernetes.io/projected/ad581954-9eef-49a2-b40d-c6892641bb77-kube-api-access-9lnh2\") pod \"payload-pre-processing-59f49fd4f6-l52k7\" (UID: \"ad581954-9eef-49a2-b40d-c6892641bb77\") " pod="openshift-ingress/payload-pre-processing-59f49fd4f6-l52k7" Jul 01 16:43:14.823634 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:43:14.823578 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ad581954-9eef-49a2-b40d-c6892641bb77-plugins-config-volume\") pod \"payload-pre-processing-59f49fd4f6-l52k7\" (UID: \"ad581954-9eef-49a2-b40d-c6892641bb77\") " pod="openshift-ingress/payload-pre-processing-59f49fd4f6-l52k7" Jul 01 16:43:14.924819 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:43:14.924789 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ad581954-9eef-49a2-b40d-c6892641bb77-plugins-config-volume\") pod \"payload-pre-processing-59f49fd4f6-l52k7\" (UID: \"ad581954-9eef-49a2-b40d-c6892641bb77\") " pod="openshift-ingress/payload-pre-processing-59f49fd4f6-l52k7" Jul 01 16:43:14.924819 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:43:14.924828 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9lnh2\" (UniqueName: \"kubernetes.io/projected/ad581954-9eef-49a2-b40d-c6892641bb77-kube-api-access-9lnh2\") pod \"payload-pre-processing-59f49fd4f6-l52k7\" (UID: \"ad581954-9eef-49a2-b40d-c6892641bb77\") " pod="openshift-ingress/payload-pre-processing-59f49fd4f6-l52k7" Jul 01 16:43:14.925539 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:43:14.925518 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ad581954-9eef-49a2-b40d-c6892641bb77-plugins-config-volume\") pod \"payload-pre-processing-59f49fd4f6-l52k7\" (UID: \"ad581954-9eef-49a2-b40d-c6892641bb77\") " pod="openshift-ingress/payload-pre-processing-59f49fd4f6-l52k7" Jul 01 16:43:14.932175 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:43:14.932154 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9lnh2\" (UniqueName: \"kubernetes.io/projected/ad581954-9eef-49a2-b40d-c6892641bb77-kube-api-access-9lnh2\") pod \"payload-pre-processing-59f49fd4f6-l52k7\" (UID: \"ad581954-9eef-49a2-b40d-c6892641bb77\") " pod="openshift-ingress/payload-pre-processing-59f49fd4f6-l52k7" Jul 01 16:43:14.986394 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:43:14.986362 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-59f49fd4f6-l52k7" Jul 01 16:43:15.123735 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:43:15.123690 2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-59f49fd4f6-l52k7"] Jul 01 16:43:15.125736 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:43:15.125688 2545 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podad581954_9eef_49a2_b40d_c6892641bb77.slice/crio-781fcfdf6388517015f91c8bc56578a5cbb925cf6ec173cc026526709fc9a5e6 WatchSource:0}: Error finding container 781fcfdf6388517015f91c8bc56578a5cbb925cf6ec173cc026526709fc9a5e6: Status 404 returned error can't find the container with id 781fcfdf6388517015f91c8bc56578a5cbb925cf6ec173cc026526709fc9a5e6 Jul 01 16:43:15.266272 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:43:15.266242 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-59f49fd4f6-l52k7" event={"ID":"ad581954-9eef-49a2-b40d-c6892641bb77","Type":"ContainerStarted","Data":"781fcfdf6388517015f91c8bc56578a5cbb925cf6ec173cc026526709fc9a5e6"} Jul 01 16:43:15.267698 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:43:15.267672 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-99968fc6d-v2jkl" event={"ID":"60c3923d-5ea5-49e5-87e4-62f60638bb5b","Type":"ContainerStarted","Data":"4c84e92f8149011f4597670f844f3298d8403d4f6814b3a4f3162571f4a3ebd5"} Jul 01 16:43:19.295053 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:43:19.295016 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-59f49fd4f6-l52k7" event={"ID":"ad581954-9eef-49a2-b40d-c6892641bb77","Type":"ContainerStarted","Data":"ca8cb2694a42a18d2b0b40b42a82e82e478e01e741b05c07f54e1276fe476fdc"} Jul 01 16:43:19.295468 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:43:19.295127 2545 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-59f49fd4f6-l52k7" Jul 01 16:43:19.302203 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:43:19.302171 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-99968fc6d-v2jkl" event={"ID":"60c3923d-5ea5-49e5-87e4-62f60638bb5b","Type":"ContainerStarted","Data":"511ac0b74b900832a1e2c62d33e8715893c38924bbf1446cb1dfbb1962becb38"} Jul 01 16:43:19.302340 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:43:19.302247 2545 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-99968fc6d-v2jkl" Jul 01 16:43:19.318111 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:43:19.318071 2545 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-59f49fd4f6-l52k7" podStartSLOduration=2.037987952 podStartE2EDuration="5.318058318s" totalImagesPullingTime="3.280070366s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 16:43:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 16:43:19.316168991 +0000 UTC m=+877.896786495" watchObservedRunningTime="2026-07-01 16:43:19.318058318 +0000 UTC m=+877.898675855" Jul 01 16:43:22.445773 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:43:22.445736 2545 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-g8wlh container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.31:15021/healthz/ready\": read tcp 10.134.0.2:58770->10.134.0.31:15021: read: connection reset by peer" start-of-body= Jul 01 16:43:22.446154 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:43:22.445806 2545 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-g8wlh" podUID="b18e0e25-6471-4e9d-bbe6-7f41f9390dc8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.31:15021/healthz/ready\": read tcp 10.134.0.2:58770->10.134.0.31:15021: read: connection reset by peer" Jul 01 16:43:22.460352 ip-10-0-130-227 kubenswrapper[2545]: E0701 16:43:22.460329 2545 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/system.slice/system-systemd\\\\x2dcoredump.slice/systemd-coredump@0-14699-0.service\": RecentStats: unable to find data in memory cache]" Jul 01 16:43:23.324097 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:43:23.324064 2545 generic.go:356] "Generic (PLEG): container finished" podID="b18e0e25-6471-4e9d-bbe6-7f41f9390dc8" containerID="e0120bdb858f67368486f5b125bafa45701dd06a20580fa622966263aaba0469" exitCode=0 Jul 01 16:43:23.324097 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:43:23.324104 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-g8wlh" event={"ID":"b18e0e25-6471-4e9d-bbe6-7f41f9390dc8","Type":"ContainerDied","Data":"e0120bdb858f67368486f5b125bafa45701dd06a20580fa622966263aaba0469"} Jul 01 16:43:23.324513 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:43:23.324496 2545 scope.go:117] "RemoveContainer" containerID="e0120bdb858f67368486f5b125bafa45701dd06a20580fa622966263aaba0469" Jul 01 16:43:23.325507 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:43:23.325477 2545 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 01 16:43:23.325610 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:43:23.325564 2545 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 01 16:43:23.325670 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:43:23.325613 2545 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 01 16:43:23.344757 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:43:23.344700 2545 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-99968fc6d-v2jkl" podStartSLOduration=5.595649933 podStartE2EDuration="9.344684222s" totalImagesPullingTime="3.749034289s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 16:43:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 16:43:19.338915634 +0000 UTC m=+877.919533142" watchObservedRunningTime="2026-07-01 16:43:23.344684222 +0000 UTC m=+881.925301743" Jul 01 16:43:24.330796 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:43:24.330753 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-g8wlh" event={"ID":"b18e0e25-6471-4e9d-bbe6-7f41f9390dc8","Type":"ContainerStarted","Data":"7369f1b52799814b78d515de8ca38dca1cdac1849a2ee854770f447816a91ba4"} Jul 01 16:43:25.167176 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:43:25.167133 2545 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-g8wlh" Jul 01 16:43:25.167357 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:43:25.167309 2545 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-g8wlh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.31:15021/healthz/ready\": dial tcp 10.134.0.31:15021: connect: connection refused" start-of-body= Jul 01 16:43:25.167408 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:43:25.167350 2545 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-g8wlh" podUID="b18e0e25-6471-4e9d-bbe6-7f41f9390dc8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.31:15021/healthz/ready\": dial tcp 10.134.0.31:15021: connect: connection refused" Jul 01 16:43:25.313551 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:43:25.313520 2545 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-99968fc6d-v2jkl" Jul 01 16:43:26.166846 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:43:26.166813 2545 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-g8wlh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.31:15021/healthz/ready\": dial tcp 10.134.0.31:15021: connect: connection refused" start-of-body= Jul 01 16:43:26.167226 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:43:26.166878 2545 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-g8wlh" podUID="b18e0e25-6471-4e9d-bbe6-7f41f9390dc8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.31:15021/healthz/ready\": dial tcp 10.134.0.31:15021: connect: connection refused" Jul 01 16:43:26.705101 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:43:26.705060 2545 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["keycloak-system/maas-keycloak-0"] Jul 01 16:43:26.705313 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:43:26.705285 2545 kuberuntime_container.go:872] "Killing container with a grace period" pod="keycloak-system/maas-keycloak-0" podUID="585fa80e-cdea-49cc-9fbf-86aa93a1fe94" containerName="keycloak" containerID="cri-o://218d67afb9bf3be10b0d53dcb5eb911ac177314a0c39b242bb39b82da30467fb" gracePeriod=30 Jul 01 16:43:27.166526 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:43:27.166491 2545 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-g8wlh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.31:15021/healthz/ready\": dial tcp 10.134.0.31:15021: connect: connection refused" start-of-body= Jul 01 16:43:27.166689 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:43:27.166544 2545 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-g8wlh" podUID="b18e0e25-6471-4e9d-bbe6-7f41f9390dc8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.31:15021/healthz/ready\": dial tcp 10.134.0.31:15021: connect: connection refused" Jul 01 16:43:28.166742 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:43:28.166690 2545 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-g8wlh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.31:15021/healthz/ready\": dial tcp 10.134.0.31:15021: connect: connection refused" start-of-body= Jul 01 16:43:28.167073 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:43:28.166769 2545 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-g8wlh" podUID="b18e0e25-6471-4e9d-bbe6-7f41f9390dc8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.31:15021/healthz/ready\": dial tcp 10.134.0.31:15021: connect: connection refused" Jul 01 16:43:28.357819 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:43:28.357792 2545 generic.go:356] "Generic (PLEG): container finished" podID="585fa80e-cdea-49cc-9fbf-86aa93a1fe94" containerID="218d67afb9bf3be10b0d53dcb5eb911ac177314a0c39b242bb39b82da30467fb" exitCode=143 Jul 01 16:43:28.357935 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:43:28.357861 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="keycloak-system/maas-keycloak-0" event={"ID":"585fa80e-cdea-49cc-9fbf-86aa93a1fe94","Type":"ContainerDied","Data":"218d67afb9bf3be10b0d53dcb5eb911ac177314a0c39b242bb39b82da30467fb"} Jul 01 16:43:28.357935 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:43:28.357931 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="keycloak-system/maas-keycloak-0" event={"ID":"585fa80e-cdea-49cc-9fbf-86aa93a1fe94","Type":"ContainerDied","Data":"4440d403e6b0644c8433c2ebcce0ee1b6e8ddb71cb384a52869bebf7ab4c86c2"} Jul 01 16:43:28.358003 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:43:28.357946 2545 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4440d403e6b0644c8433c2ebcce0ee1b6e8ddb71cb384a52869bebf7ab4c86c2" Jul 01 16:43:28.358399 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:43:28.358378 2545 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="keycloak-system/maas-keycloak-0" Jul 01 16:43:28.470200 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:43:28.470116 2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-j9vnj\" (UniqueName: \"kubernetes.io/projected/585fa80e-cdea-49cc-9fbf-86aa93a1fe94-kube-api-access-j9vnj\") pod \"585fa80e-cdea-49cc-9fbf-86aa93a1fe94\" (UID: \"585fa80e-cdea-49cc-9fbf-86aa93a1fe94\") " Jul 01 16:43:28.472261 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:43:28.472240 2545 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/585fa80e-cdea-49cc-9fbf-86aa93a1fe94-kube-api-access-j9vnj" (OuterVolumeSpecName: "kube-api-access-j9vnj") pod "585fa80e-cdea-49cc-9fbf-86aa93a1fe94" (UID: "585fa80e-cdea-49cc-9fbf-86aa93a1fe94"). InnerVolumeSpecName "kube-api-access-j9vnj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 01 16:43:28.570815 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:43:28.570785 2545 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j9vnj\" (UniqueName: \"kubernetes.io/projected/585fa80e-cdea-49cc-9fbf-86aa93a1fe94-kube-api-access-j9vnj\") on node \"ip-10-0-130-227.ec2.internal\" DevicePath \"\"" Jul 01 16:43:29.171169 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:43:29.171143 2545 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-g8wlh" Jul 01 16:43:29.171519 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:43:29.171500 2545 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-g8wlh" Jul 01 16:43:29.172129 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:43:29.172111 2545 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-g8wlh" Jul 01 16:43:29.362431 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:43:29.362401 2545 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="keycloak-system/maas-keycloak-0" Jul 01 16:43:29.378321 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:43:29.378294 2545 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["keycloak-system/maas-keycloak-0"] Jul 01 16:43:29.381657 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:43:29.381633 2545 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["keycloak-system/maas-keycloak-0"] Jul 01 16:43:29.402601 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:43:29.402569 2545 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["keycloak-system/maas-keycloak-0"] Jul 01 16:43:29.402949 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:43:29.402934 2545 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="585fa80e-cdea-49cc-9fbf-86aa93a1fe94" containerName="keycloak" Jul 01 16:43:29.402949 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:43:29.402949 2545 state_mem.go:107] "Deleted CPUSet assignment" podUID="585fa80e-cdea-49cc-9fbf-86aa93a1fe94" containerName="keycloak" Jul 01 16:43:29.403044 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:43:29.403032 2545 memory_manager.go:368] "RemoveStaleState removing state" podUID="585fa80e-cdea-49cc-9fbf-86aa93a1fe94" containerName="keycloak" Jul 01 16:43:29.405161 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:43:29.405145 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="keycloak-system/maas-keycloak-0" Jul 01 16:43:29.407476 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:43:29.407442 2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"keycloak-system\"/\"maas-keycloak-initial-admin\"" Jul 01 16:43:29.407476 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:43:29.407464 2545 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"keycloak-system\"/\"kube-root-ca.crt\"" Jul 01 16:43:29.407476 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:43:29.407447 2545 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"keycloak-system\"/\"openshift-service-ca.crt\"" Jul 01 16:43:29.407640 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:43:29.407519 2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"keycloak-system\"/\"default-dockercfg-r9khd\"" Jul 01 16:43:29.407640 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:43:29.407475 2545 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"keycloak-system\"/\"keycloak-test-realms\"" Jul 01 16:43:29.417251 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:43:29.417232 2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["keycloak-system/maas-keycloak-0"] Jul 01 16:43:29.478959 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:43:29.478930 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"test-realms\" (UniqueName: \"kubernetes.io/configmap/c0d2c528-43f4-4c43-a27a-be36d47c36b2-test-realms\") pod \"maas-keycloak-0\" (UID: \"c0d2c528-43f4-4c43-a27a-be36d47c36b2\") " pod="keycloak-system/maas-keycloak-0" Jul 01 16:43:29.479110 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:43:29.478992 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-njfl4\" (UniqueName: \"kubernetes.io/projected/c0d2c528-43f4-4c43-a27a-be36d47c36b2-kube-api-access-njfl4\") pod \"maas-keycloak-0\" (UID: \"c0d2c528-43f4-4c43-a27a-be36d47c36b2\") " pod="keycloak-system/maas-keycloak-0" Jul 01 16:43:29.580032 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:43:29.580004 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"test-realms\" (UniqueName: \"kubernetes.io/configmap/c0d2c528-43f4-4c43-a27a-be36d47c36b2-test-realms\") pod \"maas-keycloak-0\" (UID: \"c0d2c528-43f4-4c43-a27a-be36d47c36b2\") " pod="keycloak-system/maas-keycloak-0" Jul 01 16:43:29.580181 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:43:29.580052 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-njfl4\" (UniqueName: \"kubernetes.io/projected/c0d2c528-43f4-4c43-a27a-be36d47c36b2-kube-api-access-njfl4\") pod \"maas-keycloak-0\" (UID: \"c0d2c528-43f4-4c43-a27a-be36d47c36b2\") " pod="keycloak-system/maas-keycloak-0" Jul 01 16:43:29.580642 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:43:29.580624 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"test-realms\" (UniqueName: \"kubernetes.io/configmap/c0d2c528-43f4-4c43-a27a-be36d47c36b2-test-realms\") pod \"maas-keycloak-0\" (UID: \"c0d2c528-43f4-4c43-a27a-be36d47c36b2\") " pod="keycloak-system/maas-keycloak-0" Jul 01 16:43:29.588611 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:43:29.588589 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-njfl4\" (UniqueName: \"kubernetes.io/projected/c0d2c528-43f4-4c43-a27a-be36d47c36b2-kube-api-access-njfl4\") pod \"maas-keycloak-0\" (UID: \"c0d2c528-43f4-4c43-a27a-be36d47c36b2\") " pod="keycloak-system/maas-keycloak-0" Jul 01 16:43:29.714732 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:43:29.714614 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="keycloak-system/maas-keycloak-0" Jul 01 16:43:29.840031 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:43:29.840009 2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["keycloak-system/maas-keycloak-0"] Jul 01 16:43:29.842676 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:43:29.842649 2545 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc0d2c528_43f4_4c43_a27a_be36d47c36b2.slice/crio-8408b2f7e2831745607f34cbd3af657da28811d79cc04cef1ff654e91fca5a94 WatchSource:0}: Error finding container 8408b2f7e2831745607f34cbd3af657da28811d79cc04cef1ff654e91fca5a94: Status 404 returned error can't find the container with id 8408b2f7e2831745607f34cbd3af657da28811d79cc04cef1ff654e91fca5a94 Jul 01 16:43:30.310551 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:43:30.310528 2545 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-59f49fd4f6-l52k7" Jul 01 16:43:30.370055 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:43:30.370029 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="keycloak-system/maas-keycloak-0" event={"ID":"c0d2c528-43f4-4c43-a27a-be36d47c36b2","Type":"ContainerStarted","Data":"835aa4fe79f601e6b6191834af663df65e14f63642fff8189107946b80d7ef84"} Jul 01 16:43:30.370263 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:43:30.370116 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="keycloak-system/maas-keycloak-0" event={"ID":"c0d2c528-43f4-4c43-a27a-be36d47c36b2","Type":"ContainerStarted","Data":"8408b2f7e2831745607f34cbd3af657da28811d79cc04cef1ff654e91fca5a94"} Jul 01 16:43:30.386911 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:43:30.386870 2545 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="keycloak-system/maas-keycloak-0" podStartSLOduration=0.93808835 podStartE2EDuration="1.386855373s" totalImagesPullingTime="448.767023ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 16:43:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 16:43:30.386377883 +0000 UTC m=+888.966995400" watchObservedRunningTime="2026-07-01 16:43:30.386855373 +0000 UTC m=+888.967472916" Jul 01 16:43:30.715829 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:43:30.715787 2545 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="keycloak-system/maas-keycloak-0" Jul 01 16:43:30.718011 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:43:30.717978 2545 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="c0d2c528-43f4-4c43-a27a-be36d47c36b2" containerName="keycloak" probeResult="failure" output="Get \"http://10.134.0.38:9000/health/started\": dial tcp 10.134.0.38:9000: connect: connection refused" Jul 01 16:43:30.937482 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:43:30.937449 2545 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="585fa80e-cdea-49cc-9fbf-86aa93a1fe94" path="/var/lib/kubelet/pods/585fa80e-cdea-49cc-9fbf-86aa93a1fe94/volumes" Jul 01 16:43:31.715340 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:43:31.715293 2545 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="c0d2c528-43f4-4c43-a27a-be36d47c36b2" containerName="keycloak" probeResult="failure" output="Get \"http://10.134.0.38:9000/health/started\": dial tcp 10.134.0.38:9000: connect: connection refused" Jul 01 16:43:32.715526 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:43:32.715468 2545 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="c0d2c528-43f4-4c43-a27a-be36d47c36b2" containerName="keycloak" probeResult="failure" output="Get \"http://10.134.0.38:9000/health/started\": dial tcp 10.134.0.38:9000: connect: connection refused" Jul 01 16:43:33.716004 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:43:33.715900 2545 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="c0d2c528-43f4-4c43-a27a-be36d47c36b2" containerName="keycloak" probeResult="failure" output="Get \"http://10.134.0.38:9000/health/started\": dial tcp 10.134.0.38:9000: connect: connection refused" Jul 01 16:43:34.715736 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:43:34.715653 2545 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="c0d2c528-43f4-4c43-a27a-be36d47c36b2" containerName="keycloak" probeResult="failure" output="Get \"http://10.134.0.38:9000/health/started\": dial tcp 10.134.0.38:9000: connect: connection refused" Jul 01 16:43:35.715661 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:43:35.715615 2545 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="c0d2c528-43f4-4c43-a27a-be36d47c36b2" containerName="keycloak" probeResult="failure" output="Get \"http://10.134.0.38:9000/health/started\": dial tcp 10.134.0.38:9000: connect: connection refused" Jul 01 16:43:36.715149 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:43:36.715043 2545 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="c0d2c528-43f4-4c43-a27a-be36d47c36b2" containerName="keycloak" probeResult="failure" output="Get \"http://10.134.0.38:9000/health/started\": dial tcp 10.134.0.38:9000: connect: connection refused" Jul 01 16:43:37.715598 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:43:37.715184 2545 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="c0d2c528-43f4-4c43-a27a-be36d47c36b2" containerName="keycloak" probeResult="failure" output="Get \"http://10.134.0.38:9000/health/started\": dial tcp 10.134.0.38:9000: connect: connection refused" Jul 01 16:43:38.715733 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:43:38.715663 2545 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="c0d2c528-43f4-4c43-a27a-be36d47c36b2" containerName="keycloak" probeResult="failure" output="Get \"http://10.134.0.38:9000/health/started\": dial tcp 10.134.0.38:9000: connect: connection refused" Jul 01 16:43:39.715685 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:43:39.715187 2545 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="keycloak-system/maas-keycloak-0" Jul 01 16:43:39.715921 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:43:39.715621 2545 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="c0d2c528-43f4-4c43-a27a-be36d47c36b2" containerName="keycloak" probeResult="failure" output="Get \"http://10.134.0.38:9000/health/started\": dial tcp 10.134.0.38:9000: connect: connection refused" Jul 01 16:43:40.715994 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:43:40.715950 2545 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="c0d2c528-43f4-4c43-a27a-be36d47c36b2" containerName="keycloak" probeResult="failure" output="Get \"http://10.134.0.38:9000/health/started\": dial tcp 10.134.0.38:9000: connect: connection refused" Jul 01 16:43:41.715304 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:43:41.715253 2545 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="c0d2c528-43f4-4c43-a27a-be36d47c36b2" containerName="keycloak" probeResult="failure" output="Get \"http://10.134.0.38:9000/health/started\": dial tcp 10.134.0.38:9000: connect: connection refused" Jul 01 16:43:42.715420 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:43:42.715381 2545 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="c0d2c528-43f4-4c43-a27a-be36d47c36b2" containerName="keycloak" probeResult="failure" output="Get \"http://10.134.0.38:9000/health/started\": dial tcp 10.134.0.38:9000: connect: connection refused" Jul 01 16:43:42.951297 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:43:42.951274 2545 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5cr75_6c03a7ab-a7cc-44ed-86f7-62a750c49b71/ovn-acl-logging/0.log" Jul 01 16:43:42.951877 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:43:42.951856 2545 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5cr75_6c03a7ab-a7cc-44ed-86f7-62a750c49b71/ovn-acl-logging/0.log" Jul 01 16:43:43.837327 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:43:43.837295 2545 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="keycloak-system/maas-keycloak-0" Jul 01 16:43:43.851406 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:43:43.851371 2545 prober.go:120] "Probe failed" probeType="Readiness" pod="keycloak-system/maas-keycloak-0" podUID="c0d2c528-43f4-4c43-a27a-be36d47c36b2" containerName="keycloak" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 01 16:43:53.843487 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:43:53.843447 2545 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="keycloak-system/maas-keycloak-0" Jul 01 16:44:30.791625 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:44:30.791590 2545 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-ff95456df-lzbkk"] Jul 01 16:44:30.795220 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:44:30.795201 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-ff95456df-lzbkk" Jul 01 16:44:30.798816 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:44:30.798796 2545 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 01 16:44:30.798950 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:44:30.798799 2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\"" Jul 01 16:44:30.798950 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:44:30.798829 2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-sjkr6\"" Jul 01 16:44:30.798950 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:44:30.798829 2545 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 01 16:44:30.805232 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:44:30.805206 2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-ff95456df-lzbkk"] Jul 01 16:44:30.925468 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:44:30.925434 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3aaccff4-7378-44d5-987f-be1463e72738-dshm\") pod \"e2e-distinct-simulated-kserve-ff95456df-lzbkk\" (UID: \"3aaccff4-7378-44d5-987f-be1463e72738\") " pod="llm/e2e-distinct-simulated-kserve-ff95456df-lzbkk" Jul 01 16:44:30.925643 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:44:30.925514 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4zrkn\" (UniqueName: \"kubernetes.io/projected/3aaccff4-7378-44d5-987f-be1463e72738-kube-api-access-4zrkn\") pod \"e2e-distinct-simulated-kserve-ff95456df-lzbkk\" (UID: \"3aaccff4-7378-44d5-987f-be1463e72738\") " pod="llm/e2e-distinct-simulated-kserve-ff95456df-lzbkk" Jul 01 16:44:30.925643 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:44:30.925552 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3aaccff4-7378-44d5-987f-be1463e72738-home\") pod \"e2e-distinct-simulated-kserve-ff95456df-lzbkk\" (UID: \"3aaccff4-7378-44d5-987f-be1463e72738\") " pod="llm/e2e-distinct-simulated-kserve-ff95456df-lzbkk" Jul 01 16:44:30.925643 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:44:30.925573 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3aaccff4-7378-44d5-987f-be1463e72738-tls-certs\") pod \"e2e-distinct-simulated-kserve-ff95456df-lzbkk\" (UID: \"3aaccff4-7378-44d5-987f-be1463e72738\") " pod="llm/e2e-distinct-simulated-kserve-ff95456df-lzbkk" Jul 01 16:44:30.925643 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:44:30.925601 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3aaccff4-7378-44d5-987f-be1463e72738-model-cache\") pod \"e2e-distinct-simulated-kserve-ff95456df-lzbkk\" (UID: \"3aaccff4-7378-44d5-987f-be1463e72738\") " pod="llm/e2e-distinct-simulated-kserve-ff95456df-lzbkk" Jul 01 16:44:31.026640 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:44:31.026609 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3aaccff4-7378-44d5-987f-be1463e72738-dshm\") pod \"e2e-distinct-simulated-kserve-ff95456df-lzbkk\" (UID: \"3aaccff4-7378-44d5-987f-be1463e72738\") " pod="llm/e2e-distinct-simulated-kserve-ff95456df-lzbkk" Jul 01 16:44:31.026825 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:44:31.026671 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4zrkn\" (UniqueName: \"kubernetes.io/projected/3aaccff4-7378-44d5-987f-be1463e72738-kube-api-access-4zrkn\") pod \"e2e-distinct-simulated-kserve-ff95456df-lzbkk\" (UID: \"3aaccff4-7378-44d5-987f-be1463e72738\") " pod="llm/e2e-distinct-simulated-kserve-ff95456df-lzbkk" Jul 01 16:44:31.026825 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:44:31.026754 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3aaccff4-7378-44d5-987f-be1463e72738-home\") pod \"e2e-distinct-simulated-kserve-ff95456df-lzbkk\" (UID: \"3aaccff4-7378-44d5-987f-be1463e72738\") " pod="llm/e2e-distinct-simulated-kserve-ff95456df-lzbkk" Jul 01 16:44:31.026825 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:44:31.026799 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3aaccff4-7378-44d5-987f-be1463e72738-tls-certs\") pod \"e2e-distinct-simulated-kserve-ff95456df-lzbkk\" (UID: \"3aaccff4-7378-44d5-987f-be1463e72738\") " pod="llm/e2e-distinct-simulated-kserve-ff95456df-lzbkk" Jul 01 16:44:31.026930 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:44:31.026853 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3aaccff4-7378-44d5-987f-be1463e72738-model-cache\") pod \"e2e-distinct-simulated-kserve-ff95456df-lzbkk\" (UID: \"3aaccff4-7378-44d5-987f-be1463e72738\") " pod="llm/e2e-distinct-simulated-kserve-ff95456df-lzbkk" Jul 01 16:44:31.027188 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:44:31.027161 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3aaccff4-7378-44d5-987f-be1463e72738-model-cache\") pod \"e2e-distinct-simulated-kserve-ff95456df-lzbkk\" (UID: \"3aaccff4-7378-44d5-987f-be1463e72738\") " pod="llm/e2e-distinct-simulated-kserve-ff95456df-lzbkk" Jul 01 16:44:31.027309 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:44:31.027249 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3aaccff4-7378-44d5-987f-be1463e72738-home\") pod \"e2e-distinct-simulated-kserve-ff95456df-lzbkk\" (UID: \"3aaccff4-7378-44d5-987f-be1463e72738\") " pod="llm/e2e-distinct-simulated-kserve-ff95456df-lzbkk" Jul 01 16:44:31.028859 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:44:31.028836 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3aaccff4-7378-44d5-987f-be1463e72738-dshm\") pod \"e2e-distinct-simulated-kserve-ff95456df-lzbkk\" (UID: \"3aaccff4-7378-44d5-987f-be1463e72738\") " pod="llm/e2e-distinct-simulated-kserve-ff95456df-lzbkk" Jul 01 16:44:31.029112 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:44:31.029095 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3aaccff4-7378-44d5-987f-be1463e72738-tls-certs\") pod \"e2e-distinct-simulated-kserve-ff95456df-lzbkk\" (UID: \"3aaccff4-7378-44d5-987f-be1463e72738\") " pod="llm/e2e-distinct-simulated-kserve-ff95456df-lzbkk" Jul 01 16:44:31.035254 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:44:31.035229 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4zrkn\" (UniqueName: \"kubernetes.io/projected/3aaccff4-7378-44d5-987f-be1463e72738-kube-api-access-4zrkn\") pod \"e2e-distinct-simulated-kserve-ff95456df-lzbkk\" (UID: \"3aaccff4-7378-44d5-987f-be1463e72738\") " pod="llm/e2e-distinct-simulated-kserve-ff95456df-lzbkk" Jul 01 16:44:31.109151 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:44:31.109110 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-ff95456df-lzbkk" Jul 01 16:44:31.240891 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:44:31.240865 2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-ff95456df-lzbkk"] Jul 01 16:44:31.243076 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:44:31.243036 2545 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3aaccff4_7378_44d5_987f_be1463e72738.slice/crio-cef876cf87402cb1027a1cfce61a45bcf5ecf4629fc2b2814cbf932ab8183b49 WatchSource:0}: Error finding container cef876cf87402cb1027a1cfce61a45bcf5ecf4629fc2b2814cbf932ab8183b49: Status 404 returned error can't find the container with id cef876cf87402cb1027a1cfce61a45bcf5ecf4629fc2b2814cbf932ab8183b49 Jul 01 16:44:31.698563 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:44:31.698524 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-ff95456df-lzbkk" event={"ID":"3aaccff4-7378-44d5-987f-be1463e72738","Type":"ContainerStarted","Data":"cef876cf87402cb1027a1cfce61a45bcf5ecf4629fc2b2814cbf932ab8183b49"} Jul 01 16:44:32.705218 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:44:32.705142 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-ff95456df-lzbkk" event={"ID":"3aaccff4-7378-44d5-987f-be1463e72738","Type":"ContainerStarted","Data":"282c66f8fe284d92fd8c74f221df5df1d6aadd1c488ab341e0fbd42b031bfda1"} Jul 01 16:44:32.721414 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:44:32.721366 2545 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-ff95456df-lzbkk" podStartSLOduration=1.655856364 podStartE2EDuration="2.721349445s" totalImagesPullingTime="1.065493081s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 16:44:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 16:44:32.719657809 +0000 UTC m=+951.300275312" watchObservedRunningTime="2026-07-01 16:44:32.721349445 +0000 UTC m=+951.301966949" Jul 01 16:44:41.109557 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:44:41.109521 2545 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-ff95456df-lzbkk" Jul 01 16:44:41.109983 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:44:41.109593 2545 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-ff95456df-lzbkk" Jul 01 16:44:41.121888 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:44:41.121861 2545 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-ff95456df-lzbkk" Jul 01 16:44:41.760945 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:44:41.760916 2545 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-ff95456df-lzbkk" Jul 01 16:45:16.385276 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:45:16.385195 2545 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-99c7dfb5c-5ksrf"] Jul 01 16:45:16.388548 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:45:16.388531 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-99c7dfb5c-5ksrf" Jul 01 16:45:16.390905 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:45:16.390888 2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\"" Jul 01 16:45:16.395834 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:45:16.395477 2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-99c7dfb5c-5ksrf"] Jul 01 16:45:16.499399 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:45:16.499369 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5xszt\" (UniqueName: \"kubernetes.io/projected/66275862-9d4c-41d9-b1d9-f51b8176811f-kube-api-access-5xszt\") pod \"facebook-opt-125m-simulated-kserve-99c7dfb5c-5ksrf\" (UID: \"66275862-9d4c-41d9-b1d9-f51b8176811f\") " pod="llm/facebook-opt-125m-simulated-kserve-99c7dfb5c-5ksrf" Jul 01 16:45:16.499559 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:45:16.499411 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/66275862-9d4c-41d9-b1d9-f51b8176811f-model-cache\") pod \"facebook-opt-125m-simulated-kserve-99c7dfb5c-5ksrf\" (UID: \"66275862-9d4c-41d9-b1d9-f51b8176811f\") " pod="llm/facebook-opt-125m-simulated-kserve-99c7dfb5c-5ksrf" Jul 01 16:45:16.499559 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:45:16.499482 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/66275862-9d4c-41d9-b1d9-f51b8176811f-home\") pod \"facebook-opt-125m-simulated-kserve-99c7dfb5c-5ksrf\" (UID: \"66275862-9d4c-41d9-b1d9-f51b8176811f\") " pod="llm/facebook-opt-125m-simulated-kserve-99c7dfb5c-5ksrf" Jul 01 16:45:16.499559 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:45:16.499527 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/66275862-9d4c-41d9-b1d9-f51b8176811f-dshm\") pod \"facebook-opt-125m-simulated-kserve-99c7dfb5c-5ksrf\" (UID: \"66275862-9d4c-41d9-b1d9-f51b8176811f\") " pod="llm/facebook-opt-125m-simulated-kserve-99c7dfb5c-5ksrf" Jul 01 16:45:16.499559 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:45:16.499549 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/66275862-9d4c-41d9-b1d9-f51b8176811f-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-99c7dfb5c-5ksrf\" (UID: \"66275862-9d4c-41d9-b1d9-f51b8176811f\") " pod="llm/facebook-opt-125m-simulated-kserve-99c7dfb5c-5ksrf" Jul 01 16:45:16.600467 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:45:16.600435 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5xszt\" (UniqueName: \"kubernetes.io/projected/66275862-9d4c-41d9-b1d9-f51b8176811f-kube-api-access-5xszt\") pod \"facebook-opt-125m-simulated-kserve-99c7dfb5c-5ksrf\" (UID: \"66275862-9d4c-41d9-b1d9-f51b8176811f\") " pod="llm/facebook-opt-125m-simulated-kserve-99c7dfb5c-5ksrf" Jul 01 16:45:16.600467 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:45:16.600477 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/66275862-9d4c-41d9-b1d9-f51b8176811f-model-cache\") pod \"facebook-opt-125m-simulated-kserve-99c7dfb5c-5ksrf\" (UID: \"66275862-9d4c-41d9-b1d9-f51b8176811f\") " pod="llm/facebook-opt-125m-simulated-kserve-99c7dfb5c-5ksrf" Jul 01 16:45:16.600686 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:45:16.600512 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/66275862-9d4c-41d9-b1d9-f51b8176811f-home\") pod \"facebook-opt-125m-simulated-kserve-99c7dfb5c-5ksrf\" (UID: \"66275862-9d4c-41d9-b1d9-f51b8176811f\") " pod="llm/facebook-opt-125m-simulated-kserve-99c7dfb5c-5ksrf" Jul 01 16:45:16.600686 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:45:16.600557 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/66275862-9d4c-41d9-b1d9-f51b8176811f-dshm\") pod \"facebook-opt-125m-simulated-kserve-99c7dfb5c-5ksrf\" (UID: \"66275862-9d4c-41d9-b1d9-f51b8176811f\") " pod="llm/facebook-opt-125m-simulated-kserve-99c7dfb5c-5ksrf" Jul 01 16:45:16.600686 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:45:16.600582 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/66275862-9d4c-41d9-b1d9-f51b8176811f-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-99c7dfb5c-5ksrf\" (UID: \"66275862-9d4c-41d9-b1d9-f51b8176811f\") " pod="llm/facebook-opt-125m-simulated-kserve-99c7dfb5c-5ksrf" Jul 01 16:45:16.600906 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:45:16.600887 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/66275862-9d4c-41d9-b1d9-f51b8176811f-model-cache\") pod \"facebook-opt-125m-simulated-kserve-99c7dfb5c-5ksrf\" (UID: \"66275862-9d4c-41d9-b1d9-f51b8176811f\") " pod="llm/facebook-opt-125m-simulated-kserve-99c7dfb5c-5ksrf" Jul 01 16:45:16.600968 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:45:16.600936 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/66275862-9d4c-41d9-b1d9-f51b8176811f-home\") pod \"facebook-opt-125m-simulated-kserve-99c7dfb5c-5ksrf\" (UID: \"66275862-9d4c-41d9-b1d9-f51b8176811f\") " pod="llm/facebook-opt-125m-simulated-kserve-99c7dfb5c-5ksrf" Jul 01 16:45:16.602862 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:45:16.602833 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/66275862-9d4c-41d9-b1d9-f51b8176811f-dshm\") pod \"facebook-opt-125m-simulated-kserve-99c7dfb5c-5ksrf\" (UID: \"66275862-9d4c-41d9-b1d9-f51b8176811f\") " pod="llm/facebook-opt-125m-simulated-kserve-99c7dfb5c-5ksrf" Jul 01 16:45:16.603099 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:45:16.603082 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/66275862-9d4c-41d9-b1d9-f51b8176811f-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-99c7dfb5c-5ksrf\" (UID: \"66275862-9d4c-41d9-b1d9-f51b8176811f\") " pod="llm/facebook-opt-125m-simulated-kserve-99c7dfb5c-5ksrf" Jul 01 16:45:16.607600 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:45:16.607576 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5xszt\" (UniqueName: \"kubernetes.io/projected/66275862-9d4c-41d9-b1d9-f51b8176811f-kube-api-access-5xszt\") pod \"facebook-opt-125m-simulated-kserve-99c7dfb5c-5ksrf\" (UID: \"66275862-9d4c-41d9-b1d9-f51b8176811f\") " pod="llm/facebook-opt-125m-simulated-kserve-99c7dfb5c-5ksrf" Jul 01 16:45:16.699370 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:45:16.699298 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-99c7dfb5c-5ksrf" Jul 01 16:45:16.832066 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:45:16.831951 2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-99c7dfb5c-5ksrf"] Jul 01 16:45:16.834855 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:45:16.834825 2545 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod66275862_9d4c_41d9_b1d9_f51b8176811f.slice/crio-a8e1373b2e4e66ced5f06595a5321368572b22b3f8e05ea957f7846a7f6caedd WatchSource:0}: Error finding container a8e1373b2e4e66ced5f06595a5321368572b22b3f8e05ea957f7846a7f6caedd: Status 404 returned error can't find the container with id a8e1373b2e4e66ced5f06595a5321368572b22b3f8e05ea957f7846a7f6caedd Jul 01 16:45:16.915245 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:45:16.915213 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-99c7dfb5c-5ksrf" event={"ID":"66275862-9d4c-41d9-b1d9-f51b8176811f","Type":"ContainerStarted","Data":"a8e1373b2e4e66ced5f06595a5321368572b22b3f8e05ea957f7846a7f6caedd"} Jul 01 16:45:17.921782 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:45:17.921742 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-99c7dfb5c-5ksrf" event={"ID":"66275862-9d4c-41d9-b1d9-f51b8176811f","Type":"ContainerStarted","Data":"59d94c76376005406c34aaf0a504aeb6daaae733d8ca124997fde50110aee2d1"} Jul 01 16:45:17.936594 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:45:17.936540 2545 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-99c7dfb5c-5ksrf" podStartSLOduration=1.724336992 podStartE2EDuration="1.936521576s" totalImagesPullingTime="212.184584ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 16:45:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 16:45:17.936083114 +0000 UTC m=+996.516700617" watchObservedRunningTime="2026-07-01 16:45:17.936521576 +0000 UTC m=+996.517139081" Jul 01 16:45:26.699745 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:45:26.699701 2545 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-99c7dfb5c-5ksrf" Jul 01 16:45:26.700132 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:45:26.699803 2545 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-99c7dfb5c-5ksrf" Jul 01 16:45:26.711996 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:45:26.711974 2545 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-99c7dfb5c-5ksrf" Jul 01 16:45:26.983355 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:45:26.983278 2545 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-99c7dfb5c-5ksrf" Jul 01 16:47:10.763613 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:47:10.763581 2545 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_dashboard-redirect-9497df747-zjqqj_676197d5-3145-4bfc-84f9-b2e2adc79d05/nginx/0.log" Jul 01 16:47:10.873960 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:47:10.873926 2545 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-69fc866d76-jptg4_4fcbbf9c-2b2b-4636-8c26-6b773c6c8ed1/manager/0.log" Jul 01 16:47:10.987207 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:47:10.987178 2545 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-864bfd7bc9-knn5v_9eae99c9-6daf-4a90-aada-126f22386acb/manager/0.log" Jul 01 16:47:11.094660 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:47:11.094623 2545 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-99968fc6d-v2jkl_60c3923d-5ea5-49e5-87e4-62f60638bb5b/maas-api/0.log" Jul 01 16:47:11.307352 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:47:11.307324 2545 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-5476db56c7-8vzlk_893752c2-7088-4970-8f1e-cc860de1e571/manager/0.log" Jul 01 16:47:11.430982 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:47:11.430906 2545 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-5dd488fc48-5pjl6_c9b15502-b5e0-4359-86a3-a58e6d4c20cd/server/0.log" Jul 01 16:47:11.545893 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:47:11.545865 2545 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-6dc94f689f-566hl_67da1920-a7f6-4870-85f5-260d7988ce4c/manager/0.log" Jul 01 16:47:11.655870 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:47:11.655840 2545 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-9bbcfc888-7867m_a93f7bc8-6c74-4635-a611-0bd42af7c080/copy-manifests/0.log" Jul 01 16:47:11.668149 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:47:11.668122 2545 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-9bbcfc888-7867m_a93f7bc8-6c74-4635-a611-0bd42af7c080/manager/0.log" Jul 01 16:47:13.373287 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:47:13.373251 2545 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-v9ql8_0ea1fe60-777b-4358-b06e-543850d67128/manager/0.log" Jul 01 16:47:13.690974 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:47:13.690900 2545 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-4xqln_ab6090cc-d5cf-42a7-b98a-4088b3a0d6e8/registry-server/0.log" Jul 01 16:47:13.813306 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:47:13.813276 2545 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-w4gt8_de017bac-8d83-4337-8d42-6692d57f07c2/manager/0.log" Jul 01 16:47:14.059382 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:47:14.059355 2545 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-8zqvn_6fbd1636-ade8-443e-8c81-e86ac53520f9/manager/0.log" Jul 01 16:47:14.392014 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:47:14.391935 2545 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4m8qbk_0a98d33b-fc67-4515-8c76-fafdc010917e/istio-proxy/0.log" Jul 01 16:47:14.618243 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:47:14.618215 2545 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-75f5b69fd7-bqrqm_e0ec6a34-fb56-4865-b460-d76ad8f8f3d7/kube-auth-proxy/0.log" Jul 01 16:47:14.828867 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:47:14.828843 2545 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-g8wlh_b18e0e25-6471-4e9d-bbe6-7f41f9390dc8/istio-proxy/1.log" Jul 01 16:47:14.937545 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:47:14.937517 2545 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59f49fd4f6-l52k7_ad581954-9eef-49a2-b40d-c6892641bb77/payload-pre-processing/0.log" Jul 01 16:47:15.596330 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:47:15.596301 2545 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-ff95456df-lzbkk_3aaccff4-7378-44d5-987f-be1463e72738/main/0.log" Jul 01 16:47:15.932246 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:47:15.932165 2545 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-99c7dfb5c-5ksrf_66275862-9d4c-41d9-b1d9-f51b8176811f/main/0.log" Jul 01 16:47:23.347807 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:47:23.347775 2545 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-npttx_779fced8-cf7c-49a7-a236-c10a83bbd53c/global-pull-secret-syncer/0.log" Jul 01 16:47:23.455486 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:47:23.455458 2545 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-cprkr_1df6652f-89cc-41a1-89f7-0ed8568b6ee2/konnectivity-agent/0.log" Jul 01 16:47:23.500551 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:47:23.500524 2545 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-130-227.ec2.internal_7c07f7dba9c4bc09c77f869b83b22cdb/haproxy/0.log" Jul 01 16:47:28.229848 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:47:28.229821 2545 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-v9ql8_0ea1fe60-777b-4358-b06e-543850d67128/manager/0.log" Jul 01 16:47:28.321285 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:47:28.321251 2545 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-4xqln_ab6090cc-d5cf-42a7-b98a-4088b3a0d6e8/registry-server/0.log" Jul 01 16:47:28.365067 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:47:28.365037 2545 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-w4gt8_de017bac-8d83-4337-8d42-6692d57f07c2/manager/0.log" Jul 01 16:47:28.421971 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:47:28.421945 2545 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-8zqvn_6fbd1636-ade8-443e-8c81-e86ac53520f9/manager/0.log" Jul 01 16:47:30.276773 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:47:30.276749 2545 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-845b954455-hbgb8_d3172b59-6654-4770-b297-db32819ec89b/kube-state-metrics/0.log" Jul 01 16:47:30.295820 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:47:30.295793 2545 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-845b954455-hbgb8_d3172b59-6654-4770-b297-db32819ec89b/kube-rbac-proxy-main/0.log" Jul 01 16:47:30.323847 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:47:30.323804 2545 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-845b954455-hbgb8_d3172b59-6654-4770-b297-db32819ec89b/kube-rbac-proxy-self/0.log" Jul 01 16:47:30.506338 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:47:30.506305 2545 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-rbnqk_631adfb3-d231-4507-a3d5-d315e3f81d94/node-exporter/0.log" Jul 01 16:47:30.537409 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:47:30.537343 2545 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-rbnqk_631adfb3-d231-4507-a3d5-d315e3f81d94/kube-rbac-proxy/0.log" Jul 01 16:47:30.560580 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:47:30.560559 2545 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-rbnqk_631adfb3-d231-4507-a3d5-d315e3f81d94/init-textfile/0.log" Jul 01 16:47:33.754903 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:47:33.754872 2545 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-t5k2s/perf-node-gather-daemonset-9d9c6"] Jul 01 16:47:33.758595 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:47:33.758575 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-t5k2s/perf-node-gather-daemonset-9d9c6" Jul 01 16:47:33.761012 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:47:33.760987 2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-t5k2s\"/\"default-dockercfg-w7424\"" Jul 01 16:47:33.761116 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:47:33.760996 2545 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-t5k2s\"/\"kube-root-ca.crt\"" Jul 01 16:47:33.761116 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:47:33.761031 2545 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-t5k2s\"/\"openshift-service-ca.crt\"" Jul 01 16:47:33.767845 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:47:33.767825 2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-t5k2s/perf-node-gather-daemonset-9d9c6"] Jul 01 16:47:33.844343 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:47:33.844311 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/db099ecf-bb10-4e19-8e03-e1d798c77bb6-podres\") pod \"perf-node-gather-daemonset-9d9c6\" (UID: \"db099ecf-bb10-4e19-8e03-e1d798c77bb6\") " pod="openshift-must-gather-t5k2s/perf-node-gather-daemonset-9d9c6" Jul 01 16:47:33.844343 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:47:33.844346 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cm9hb\" (UniqueName: \"kubernetes.io/projected/db099ecf-bb10-4e19-8e03-e1d798c77bb6-kube-api-access-cm9hb\") pod \"perf-node-gather-daemonset-9d9c6\" (UID: \"db099ecf-bb10-4e19-8e03-e1d798c77bb6\") " pod="openshift-must-gather-t5k2s/perf-node-gather-daemonset-9d9c6" Jul 01 16:47:33.844545 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:47:33.844418 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/db099ecf-bb10-4e19-8e03-e1d798c77bb6-lib-modules\") pod \"perf-node-gather-daemonset-9d9c6\" (UID: \"db099ecf-bb10-4e19-8e03-e1d798c77bb6\") " pod="openshift-must-gather-t5k2s/perf-node-gather-daemonset-9d9c6" Jul 01 16:47:33.844545 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:47:33.844443 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/db099ecf-bb10-4e19-8e03-e1d798c77bb6-proc\") pod \"perf-node-gather-daemonset-9d9c6\" (UID: \"db099ecf-bb10-4e19-8e03-e1d798c77bb6\") " pod="openshift-must-gather-t5k2s/perf-node-gather-daemonset-9d9c6" Jul 01 16:47:33.844545 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:47:33.844484 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/db099ecf-bb10-4e19-8e03-e1d798c77bb6-sys\") pod \"perf-node-gather-daemonset-9d9c6\" (UID: \"db099ecf-bb10-4e19-8e03-e1d798c77bb6\") " pod="openshift-must-gather-t5k2s/perf-node-gather-daemonset-9d9c6" Jul 01 16:47:33.945643 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:47:33.945609 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/db099ecf-bb10-4e19-8e03-e1d798c77bb6-lib-modules\") pod \"perf-node-gather-daemonset-9d9c6\" (UID: \"db099ecf-bb10-4e19-8e03-e1d798c77bb6\") " pod="openshift-must-gather-t5k2s/perf-node-gather-daemonset-9d9c6" Jul 01 16:47:33.945643 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:47:33.945649 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/db099ecf-bb10-4e19-8e03-e1d798c77bb6-proc\") pod \"perf-node-gather-daemonset-9d9c6\" (UID: \"db099ecf-bb10-4e19-8e03-e1d798c77bb6\") " pod="openshift-must-gather-t5k2s/perf-node-gather-daemonset-9d9c6" Jul 01 16:47:33.945896 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:47:33.945675 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/db099ecf-bb10-4e19-8e03-e1d798c77bb6-sys\") pod \"perf-node-gather-daemonset-9d9c6\" (UID: \"db099ecf-bb10-4e19-8e03-e1d798c77bb6\") " pod="openshift-must-gather-t5k2s/perf-node-gather-daemonset-9d9c6" Jul 01 16:47:33.945896 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:47:33.945740 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/db099ecf-bb10-4e19-8e03-e1d798c77bb6-podres\") pod \"perf-node-gather-daemonset-9d9c6\" (UID: \"db099ecf-bb10-4e19-8e03-e1d798c77bb6\") " pod="openshift-must-gather-t5k2s/perf-node-gather-daemonset-9d9c6" Jul 01 16:47:33.945896 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:47:33.945762 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cm9hb\" (UniqueName: \"kubernetes.io/projected/db099ecf-bb10-4e19-8e03-e1d798c77bb6-kube-api-access-cm9hb\") pod \"perf-node-gather-daemonset-9d9c6\" (UID: \"db099ecf-bb10-4e19-8e03-e1d798c77bb6\") " pod="openshift-must-gather-t5k2s/perf-node-gather-daemonset-9d9c6" Jul 01 16:47:33.945896 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:47:33.945764 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/db099ecf-bb10-4e19-8e03-e1d798c77bb6-proc\") pod \"perf-node-gather-daemonset-9d9c6\" (UID: \"db099ecf-bb10-4e19-8e03-e1d798c77bb6\") " pod="openshift-must-gather-t5k2s/perf-node-gather-daemonset-9d9c6" Jul 01 16:47:33.945896 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:47:33.945800 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/db099ecf-bb10-4e19-8e03-e1d798c77bb6-lib-modules\") pod \"perf-node-gather-daemonset-9d9c6\" (UID: \"db099ecf-bb10-4e19-8e03-e1d798c77bb6\") " pod="openshift-must-gather-t5k2s/perf-node-gather-daemonset-9d9c6" Jul 01 16:47:33.945896 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:47:33.945825 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/db099ecf-bb10-4e19-8e03-e1d798c77bb6-sys\") pod \"perf-node-gather-daemonset-9d9c6\" (UID: \"db099ecf-bb10-4e19-8e03-e1d798c77bb6\") " pod="openshift-must-gather-t5k2s/perf-node-gather-daemonset-9d9c6" Jul 01 16:47:33.945896 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:47:33.945856 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/db099ecf-bb10-4e19-8e03-e1d798c77bb6-podres\") pod \"perf-node-gather-daemonset-9d9c6\" (UID: \"db099ecf-bb10-4e19-8e03-e1d798c77bb6\") " pod="openshift-must-gather-t5k2s/perf-node-gather-daemonset-9d9c6" Jul 01 16:47:33.953396 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:47:33.953375 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cm9hb\" (UniqueName: \"kubernetes.io/projected/db099ecf-bb10-4e19-8e03-e1d798c77bb6-kube-api-access-cm9hb\") pod \"perf-node-gather-daemonset-9d9c6\" (UID: \"db099ecf-bb10-4e19-8e03-e1d798c77bb6\") " pod="openshift-must-gather-t5k2s/perf-node-gather-daemonset-9d9c6" Jul 01 16:47:34.069584 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:47:34.069556 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-t5k2s/perf-node-gather-daemonset-9d9c6" Jul 01 16:47:34.192493 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:47:34.192470 2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-t5k2s/perf-node-gather-daemonset-9d9c6"] Jul 01 16:47:34.194675 ip-10-0-130-227 kubenswrapper[2545]: W0701 16:47:34.194638 2545 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-poddb099ecf_bb10_4e19_8e03_e1d798c77bb6.slice/crio-399898846a7b957e2c13f2556539d50349a72f0e0d5f08f49acea697ac23653c WatchSource:0}: Error finding container 399898846a7b957e2c13f2556539d50349a72f0e0d5f08f49acea697ac23653c: Status 404 returned error can't find the container with id 399898846a7b957e2c13f2556539d50349a72f0e0d5f08f49acea697ac23653c Jul 01 16:47:34.575136 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:47:34.575100 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-t5k2s/perf-node-gather-daemonset-9d9c6" event={"ID":"db099ecf-bb10-4e19-8e03-e1d798c77bb6","Type":"ContainerStarted","Data":"94cfdd1b47a401bb84fc6b9f587de8559c4ea98562ca92903d55eb772346bfc4"} Jul 01 16:47:34.575311 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:47:34.575150 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-t5k2s/perf-node-gather-daemonset-9d9c6" event={"ID":"db099ecf-bb10-4e19-8e03-e1d798c77bb6","Type":"ContainerStarted","Data":"399898846a7b957e2c13f2556539d50349a72f0e0d5f08f49acea697ac23653c"} Jul 01 16:47:34.575311 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:47:34.575190 2545 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-t5k2s/perf-node-gather-daemonset-9d9c6" Jul 01 16:47:34.588650 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:47:34.588593 2545 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-t5k2s/perf-node-gather-daemonset-9d9c6" podStartSLOduration=1.588575116 podStartE2EDuration="1.588575116s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-01 16:47:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 16:47:34.588187842 +0000 UTC m=+1133.168805350" watchObservedRunningTime="2026-07-01 16:47:34.588575116 +0000 UTC m=+1133.169192624" Jul 01 16:47:35.357779 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:47:35.357753 2545 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-55sd4_c3cf9697-8b56-4162-b761-39c2a0557b3b/dns/0.log" Jul 01 16:47:35.378475 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:47:35.378455 2545 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-55sd4_c3cf9697-8b56-4162-b761-39c2a0557b3b/kube-rbac-proxy/0.log" Jul 01 16:47:35.530864 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:47:35.530841 2545 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-t7nrc_5be67da1-9165-4446-b3cb-2333db8c09cc/dns-node-resolver/0.log" Jul 01 16:47:36.010390 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:47:36.010359 2545 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-7bc8c8cb69-dt9sk_a97628e5-f93f-448e-8bae-83444e4fa34b/registry/0.log" Jul 01 16:47:36.013284 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:47:36.013265 2545 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-7bc8c8cb69-dt9sk_a97628e5-f93f-448e-8bae-83444e4fa34b/registry/1.log" Jul 01 16:47:36.077721 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:47:36.077694 2545 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-wb2hh_8242757c-9a33-4958-a860-0679d154b28c/node-ca/0.log" Jul 01 16:47:36.982236 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:47:36.982206 2545 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4m8qbk_0a98d33b-fc67-4515-8c76-fafdc010917e/istio-proxy/0.log" Jul 01 16:47:37.065223 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:47:37.065204 2545 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-75f5b69fd7-bqrqm_e0ec6a34-fb56-4865-b460-d76ad8f8f3d7/kube-auth-proxy/0.log" Jul 01 16:47:37.119336 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:47:37.119312 2545 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-g8wlh_b18e0e25-6471-4e9d-bbe6-7f41f9390dc8/istio-proxy/1.log" Jul 01 16:47:37.119624 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:47:37.119609 2545 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-g8wlh_b18e0e25-6471-4e9d-bbe6-7f41f9390dc8/istio-proxy/0.log" Jul 01 16:47:37.137555 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:47:37.137533 2545 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59f49fd4f6-l52k7_ad581954-9eef-49a2-b40d-c6892641bb77/payload-pre-processing/0.log" Jul 01 16:47:37.795667 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:47:37.795634 2545 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-2fxwv_06e3403f-0530-476b-b689-ffc0a7e50132/serve-healthcheck-canary/0.log" Jul 01 16:47:38.301556 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:47:38.301527 2545 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-2bzw5_059f2f9d-a756-4692-9452-9a9b0ffb55ce/kube-rbac-proxy/0.log" Jul 01 16:47:38.321323 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:47:38.321297 2545 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-2bzw5_059f2f9d-a756-4692-9452-9a9b0ffb55ce/exporter/0.log" Jul 01 16:47:38.341606 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:47:38.341580 2545 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-2bzw5_059f2f9d-a756-4692-9452-9a9b0ffb55ce/extractor/0.log" Jul 01 16:47:40.536200 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:47:40.536176 2545 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_dashboard-redirect-9497df747-zjqqj_676197d5-3145-4bfc-84f9-b2e2adc79d05/nginx/0.log" Jul 01 16:47:40.559513 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:47:40.559490 2545 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-69fc866d76-jptg4_4fcbbf9c-2b2b-4636-8c26-6b773c6c8ed1/manager/0.log" Jul 01 16:47:40.591641 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:47:40.591622 2545 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-t5k2s/perf-node-gather-daemonset-9d9c6" Jul 01 16:47:40.592205 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:47:40.592187 2545 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-864bfd7bc9-knn5v_9eae99c9-6daf-4a90-aada-126f22386acb/manager/0.log" Jul 01 16:47:40.612125 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:47:40.612105 2545 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-99968fc6d-v2jkl_60c3923d-5ea5-49e5-87e4-62f60638bb5b/maas-api/0.log" Jul 01 16:47:40.668282 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:47:40.668261 2545 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-5476db56c7-8vzlk_893752c2-7088-4970-8f1e-cc860de1e571/manager/0.log" Jul 01 16:47:40.687524 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:47:40.687501 2545 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-5dd488fc48-5pjl6_c9b15502-b5e0-4359-86a3-a58e6d4c20cd/server/0.log" Jul 01 16:47:40.737597 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:47:40.737577 2545 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-6dc94f689f-566hl_67da1920-a7f6-4870-85f5-260d7988ce4c/manager/0.log" Jul 01 16:47:40.762672 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:47:40.762640 2545 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-9bbcfc888-7867m_a93f7bc8-6c74-4635-a611-0bd42af7c080/manager/0.log" Jul 01 16:47:40.781288 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:47:40.781262 2545 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-9bbcfc888-7867m_a93f7bc8-6c74-4635-a611-0bd42af7c080/copy-manifests/0.log" Jul 01 16:47:42.404527 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:47:42.404506 2545 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-khfv4_24291043-3c8f-41d0-a01d-aabb3fcc0923/openshift-lws-operator/0.log" Jul 01 16:47:44.035086 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:47:44.035059 2545 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4m8qbk_0a98d33b-fc67-4515-8c76-fafdc010917e/istio-proxy/0.log" Jul 01 16:47:44.102233 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:47:44.102168 2545 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-75f5b69fd7-bqrqm_e0ec6a34-fb56-4865-b460-d76ad8f8f3d7/kube-auth-proxy/0.log" Jul 01 16:47:44.141701 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:47:44.141670 2545 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-g8wlh_b18e0e25-6471-4e9d-bbe6-7f41f9390dc8/istio-proxy/1.log" Jul 01 16:47:44.142011 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:47:44.141996 2545 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-g8wlh_b18e0e25-6471-4e9d-bbe6-7f41f9390dc8/istio-proxy/0.log" Jul 01 16:47:44.152479 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:47:44.152461 2545 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59f49fd4f6-l52k7_ad581954-9eef-49a2-b40d-c6892641bb77/payload-pre-processing/0.log" Jul 01 16:47:45.252626 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:47:45.252601 2545 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4m8qbk_0a98d33b-fc67-4515-8c76-fafdc010917e/istio-proxy/0.log" Jul 01 16:47:45.323460 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:47:45.323439 2545 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-75f5b69fd7-bqrqm_e0ec6a34-fb56-4865-b460-d76ad8f8f3d7/kube-auth-proxy/0.log" Jul 01 16:47:45.361997 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:47:45.361966 2545 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-g8wlh_b18e0e25-6471-4e9d-bbe6-7f41f9390dc8/istio-proxy/1.log" Jul 01 16:47:45.362445 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:47:45.362425 2545 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-g8wlh_b18e0e25-6471-4e9d-bbe6-7f41f9390dc8/istio-proxy/0.log" Jul 01 16:47:45.375860 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:47:45.375842 2545 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59f49fd4f6-l52k7_ad581954-9eef-49a2-b40d-c6892641bb77/payload-pre-processing/0.log" Jul 01 16:47:49.287089 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:47:49.287059 2545 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-gnsqt_48cdc46f-d909-463d-942b-4d300d212062/kube-multus-additional-cni-plugins/0.log" Jul 01 16:47:49.315417 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:47:49.315396 2545 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-gnsqt_48cdc46f-d909-463d-942b-4d300d212062/egress-router-binary-copy/0.log" Jul 01 16:47:49.342598 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:47:49.342580 2545 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-gnsqt_48cdc46f-d909-463d-942b-4d300d212062/cni-plugins/0.log" Jul 01 16:47:49.369430 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:47:49.369414 2545 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-gnsqt_48cdc46f-d909-463d-942b-4d300d212062/bond-cni-plugin/0.log" Jul 01 16:47:49.394638 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:47:49.394619 2545 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-gnsqt_48cdc46f-d909-463d-942b-4d300d212062/routeoverride-cni/0.log" Jul 01 16:47:49.417557 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:47:49.417537 2545 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-gnsqt_48cdc46f-d909-463d-942b-4d300d212062/whereabouts-cni-bincopy/0.log" Jul 01 16:47:49.441429 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:47:49.441411 2545 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-gnsqt_48cdc46f-d909-463d-942b-4d300d212062/whereabouts-cni/0.log" Jul 01 16:47:49.715060 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:47:49.715031 2545 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-t7n99_76faf2eb-5dae-48e1-ba27-f1b94c85920d/kube-multus/0.log" Jul 01 16:47:49.737405 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:47:49.737380 2545 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-87ztj_be2ddffa-466a-4a37-baba-5cb9ecf07e41/network-metrics-daemon/0.log" Jul 01 16:47:49.758146 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:47:49.758120 2545 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-87ztj_be2ddffa-466a-4a37-baba-5cb9ecf07e41/kube-rbac-proxy/0.log" Jul 01 16:47:50.648134 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:47:50.648110 2545 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5cr75_6c03a7ab-a7cc-44ed-86f7-62a750c49b71/ovn-controller/0.log" Jul 01 16:47:50.665053 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:47:50.665029 2545 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5cr75_6c03a7ab-a7cc-44ed-86f7-62a750c49b71/ovn-acl-logging/1.log" Jul 01 16:47:50.665350 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:47:50.665336 2545 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5cr75_6c03a7ab-a7cc-44ed-86f7-62a750c49b71/ovn-acl-logging/0.log" Jul 01 16:47:50.686537 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:47:50.686515 2545 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5cr75_6c03a7ab-a7cc-44ed-86f7-62a750c49b71/kube-rbac-proxy-node/0.log" Jul 01 16:47:50.705624 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:47:50.705601 2545 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5cr75_6c03a7ab-a7cc-44ed-86f7-62a750c49b71/kube-rbac-proxy-ovn-metrics/0.log" Jul 01 16:47:50.722570 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:47:50.722552 2545 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5cr75_6c03a7ab-a7cc-44ed-86f7-62a750c49b71/northd/0.log" Jul 01 16:47:50.742534 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:47:50.742519 2545 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5cr75_6c03a7ab-a7cc-44ed-86f7-62a750c49b71/nbdb/0.log" Jul 01 16:47:50.763053 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:47:50.763037 2545 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5cr75_6c03a7ab-a7cc-44ed-86f7-62a750c49b71/sbdb/0.log" Jul 01 16:47:50.805741 ip-10-0-130-227 kubenswrapper[2545]: I0701 16:47:50.805704 2545 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5cr75_6c03a7ab-a7cc-44ed-86f7-62a750c49b71/ovnkube-controller/0.log"