May 27 20:10:47.448507 ip-10-0-142-251 systemd[1]: Starting Kubernetes Kubelet... May 27 20:10:47.961166 ip-10-0-142-251 kubenswrapper[2571]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. May 27 20:10:47.961166 ip-10-0-142-251 kubenswrapper[2571]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. May 27 20:10:47.961166 ip-10-0-142-251 kubenswrapper[2571]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. May 27 20:10:47.961166 ip-10-0-142-251 kubenswrapper[2571]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. May 27 20:10:47.961166 ip-10-0-142-251 kubenswrapper[2571]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. May 27 20:10:47.963164 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:47.963071 2571 server.go:212] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" May 27 20:10:47.967807 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.967786 2571 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings May 27 20:10:47.967807 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.967805 2571 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas May 27 20:10:47.967807 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.967809 2571 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall May 27 20:10:47.967937 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.967813 2571 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk May 27 20:10:47.967937 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.967816 2571 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity May 27 20:10:47.967937 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.967819 2571 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota May 27 20:10:47.967937 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.967822 2571 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk May 27 20:10:47.967937 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.967826 2571 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup May 27 20:10:47.967937 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.967829 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure May 27 20:10:47.967937 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.967832 2571 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode May 27 20:10:47.967937 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.967835 2571 feature_gate.go:328] unrecognized feature gate: DualReplica May 27 20:10:47.967937 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.967838 2571 feature_gate.go:328] unrecognized feature gate: GatewayAPIController May 27 20:10:47.967937 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.967840 2571 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation May 27 20:10:47.967937 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.967843 2571 feature_gate.go:328] unrecognized feature gate: DNSNameResolver May 27 20:10:47.967937 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.967846 2571 feature_gate.go:328] unrecognized feature gate: PinnedImages May 27 20:10:47.967937 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.967849 2571 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. May 27 20:10:47.967937 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.967853 2571 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy May 27 20:10:47.967937 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.967856 2571 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud May 27 20:10:47.967937 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.967859 2571 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI May 27 20:10:47.967937 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.967861 2571 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv May 27 20:10:47.967937 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.967864 2571 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider May 27 20:10:47.967937 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.967866 2571 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController May 27 20:10:47.968453 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.967869 2571 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement May 27 20:10:47.968453 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.967872 2571 feature_gate.go:328] unrecognized feature gate: SignatureStores May 27 20:10:47.968453 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.967874 2571 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager May 27 20:10:47.968453 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.967884 2571 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode May 27 20:10:47.968453 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.967887 2571 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration May 27 20:10:47.968453 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.967890 2571 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig May 27 20:10:47.968453 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.967892 2571 feature_gate.go:328] unrecognized feature gate: GatewayAPI May 27 20:10:47.968453 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.967895 2571 feature_gate.go:328] unrecognized feature gate: OVNObservability May 27 20:10:47.968453 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.967897 2571 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration May 27 20:10:47.968453 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.967900 2571 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets May 27 20:10:47.968453 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.967902 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere May 27 20:10:47.968453 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.967905 2571 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy May 27 20:10:47.968453 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.967907 2571 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS May 27 20:10:47.968453 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.967910 2571 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration May 27 20:10:47.968453 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.967913 2571 feature_gate.go:328] unrecognized feature gate: ShortCertRotation May 27 20:10:47.968453 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.967916 2571 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure May 27 20:10:47.968453 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.967919 2571 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup May 27 20:10:47.968453 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.967921 2571 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA May 27 20:10:47.968453 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.967924 2571 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig May 27 20:10:47.968903 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.967928 2571 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. May 27 20:10:47.968903 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.967932 2571 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata May 27 20:10:47.968903 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.967935 2571 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup May 27 20:10:47.968903 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.967938 2571 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities May 27 20:10:47.968903 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.967941 2571 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall May 27 20:10:47.968903 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.967943 2571 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace May 27 20:10:47.968903 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.967946 2571 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints May 27 20:10:47.968903 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.967950 2571 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts May 27 20:10:47.968903 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.967952 2571 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles May 27 20:10:47.968903 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.967955 2571 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts May 27 20:10:47.968903 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.967959 2571 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification May 27 20:10:47.968903 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.967962 2571 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations May 27 20:10:47.968903 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.967965 2571 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS May 27 20:10:47.968903 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.967969 2571 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform May 27 20:10:47.968903 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.967972 2571 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS May 27 20:10:47.968903 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.967974 2571 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks May 27 20:10:47.968903 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.967977 2571 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot May 27 20:10:47.968903 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.967980 2571 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall May 27 20:10:47.968903 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.967982 2571 feature_gate.go:328] unrecognized feature gate: NewOLM May 27 20:10:47.969374 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.967985 2571 feature_gate.go:328] unrecognized feature gate: Example May 27 20:10:47.969374 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.967987 2571 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig May 27 20:10:47.969374 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.967990 2571 feature_gate.go:328] unrecognized feature gate: ExternalOIDC May 27 20:10:47.969374 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.967993 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImages May 27 20:10:47.969374 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.967995 2571 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements May 27 20:10:47.969374 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.967997 2571 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter May 27 20:10:47.969374 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.968000 2571 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController May 27 20:10:47.969374 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.968002 2571 feature_gate.go:328] unrecognized feature gate: Example2 May 27 20:10:47.969374 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.968007 2571 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses May 27 20:10:47.969374 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.968010 2571 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation May 27 20:10:47.969374 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.968012 2571 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks May 27 20:10:47.969374 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.968015 2571 feature_gate.go:328] unrecognized feature gate: InsightsConfig May 27 20:10:47.969374 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.968017 2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall May 27 20:10:47.969374 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.968020 2571 feature_gate.go:328] unrecognized feature gate: UpgradeStatus May 27 20:10:47.969374 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.968023 2571 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes May 27 20:10:47.969374 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.968025 2571 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission May 27 20:10:47.969374 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.968028 2571 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI May 27 20:10:47.969374 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.968030 2571 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes May 27 20:10:47.969374 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.968034 2571 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall May 27 20:10:47.969374 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.968036 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS May 27 20:10:47.969856 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.968039 2571 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix May 27 20:10:47.969856 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.968041 2571 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal May 27 20:10:47.969856 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.968044 2571 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement May 27 20:10:47.969856 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.968046 2571 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting May 27 20:10:47.969856 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.968048 2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud May 27 20:10:47.969856 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.968051 2571 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather May 27 20:10:47.969856 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.968463 2571 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement May 27 20:10:47.969856 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.968469 2571 feature_gate.go:328] unrecognized feature gate: GatewayAPI May 27 20:10:47.969856 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.968472 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure May 27 20:10:47.969856 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.968475 2571 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk May 27 20:10:47.969856 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.968477 2571 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup May 27 20:10:47.969856 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.968480 2571 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider May 27 20:10:47.969856 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.968483 2571 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS May 27 20:10:47.969856 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.968486 2571 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration May 27 20:10:47.969856 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.968488 2571 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot May 27 20:10:47.969856 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.968491 2571 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts May 27 20:10:47.969856 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.968493 2571 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter May 27 20:10:47.969856 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.968496 2571 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform May 27 20:10:47.969856 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.968498 2571 feature_gate.go:328] unrecognized feature gate: Example May 27 20:10:47.969856 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.968501 2571 feature_gate.go:328] unrecognized feature gate: DualReplica May 27 20:10:47.970352 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.968504 2571 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA May 27 20:10:47.970352 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.968507 2571 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy May 27 20:10:47.970352 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.968509 2571 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity May 27 20:10:47.970352 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.968512 2571 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks May 27 20:10:47.970352 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.968515 2571 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks May 27 20:10:47.970352 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.968518 2571 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall May 27 20:10:47.970352 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.968520 2571 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation May 27 20:10:47.970352 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.968523 2571 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting May 27 20:10:47.970352 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.968526 2571 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration May 27 20:10:47.970352 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.968528 2571 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode May 27 20:10:47.970352 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.968531 2571 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata May 27 20:10:47.970352 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.968534 2571 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles May 27 20:10:47.970352 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.968536 2571 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode May 27 20:10:47.970352 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.968539 2571 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal May 27 20:10:47.970352 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.968541 2571 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup May 27 20:10:47.970352 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.968544 2571 feature_gate.go:328] unrecognized feature gate: NewOLM May 27 20:10:47.970352 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.968546 2571 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager May 27 20:10:47.970352 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.968548 2571 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall May 27 20:10:47.970352 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.968551 2571 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix May 27 20:10:47.970843 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.968553 2571 feature_gate.go:328] unrecognized feature gate: PinnedImages May 27 20:10:47.970843 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.968556 2571 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig May 27 20:10:47.970843 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.968558 2571 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace May 27 20:10:47.970843 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.968561 2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall May 27 20:10:47.970843 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.968563 2571 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv May 27 20:10:47.970843 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.968567 2571 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets May 27 20:10:47.970843 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.968569 2571 feature_gate.go:328] unrecognized feature gate: UpgradeStatus May 27 20:10:47.970843 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.968571 2571 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities May 27 20:10:47.970843 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.968574 2571 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration May 27 20:10:47.970843 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.968576 2571 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup May 27 20:10:47.970843 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.968579 2571 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud May 27 20:10:47.970843 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.968581 2571 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts May 27 20:10:47.970843 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.968584 2571 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations May 27 20:10:47.970843 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.968587 2571 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController May 27 20:10:47.970843 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.968589 2571 feature_gate.go:328] unrecognized feature gate: GatewayAPIController May 27 20:10:47.970843 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.968593 2571 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. May 27 20:10:47.970843 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.968597 2571 feature_gate.go:328] unrecognized feature gate: InsightsConfig May 27 20:10:47.970843 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.968600 2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud May 27 20:10:47.970843 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.968603 2571 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure May 27 20:10:47.970843 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.968606 2571 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes May 27 20:10:47.971367 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.968609 2571 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation May 27 20:10:47.971367 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.968611 2571 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS May 27 20:10:47.971367 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.968614 2571 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController May 27 20:10:47.971367 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.968617 2571 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota May 27 20:10:47.971367 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.968620 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS May 27 20:10:47.971367 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.968623 2571 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission May 27 20:10:47.971367 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.968625 2571 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification May 27 20:10:47.971367 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.968628 2571 feature_gate.go:328] unrecognized feature gate: ShortCertRotation May 27 20:10:47.971367 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.968630 2571 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy May 27 20:10:47.971367 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.968634 2571 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS May 27 20:10:47.971367 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.968636 2571 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI May 27 20:10:47.971367 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.968638 2571 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig May 27 20:10:47.971367 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.968641 2571 feature_gate.go:328] unrecognized feature gate: ExternalOIDC May 27 20:10:47.971367 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.968644 2571 feature_gate.go:328] unrecognized feature gate: Example2 May 27 20:10:47.971367 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.968646 2571 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings May 27 20:10:47.971367 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.968649 2571 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall May 27 20:10:47.971367 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.968651 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImages May 27 20:10:47.971367 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.968654 2571 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall May 27 20:10:47.971367 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.968657 2571 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig May 27 20:10:47.971816 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.968660 2571 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement May 27 20:10:47.971816 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.968662 2571 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes May 27 20:10:47.971816 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.968665 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere May 27 20:10:47.971816 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.968667 2571 feature_gate.go:328] unrecognized feature gate: OVNObservability May 27 20:10:47.971816 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.968670 2571 feature_gate.go:328] unrecognized feature gate: SignatureStores May 27 20:10:47.971816 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.968672 2571 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI May 27 20:10:47.971816 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.968675 2571 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints May 27 20:10:47.971816 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.968677 2571 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather May 27 20:10:47.971816 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.968680 2571 feature_gate.go:328] unrecognized feature gate: DNSNameResolver May 27 20:10:47.971816 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.968684 2571 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. May 27 20:10:47.971816 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.968687 2571 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas May 27 20:10:47.971816 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.968690 2571 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses May 27 20:10:47.971816 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.968692 2571 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk May 27 20:10:47.971816 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.968695 2571 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements May 27 20:10:47.971816 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:47.970155 2571 flags.go:64] FLAG: --address="0.0.0.0" May 27 20:10:47.971816 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:47.970166 2571 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" May 27 20:10:47.971816 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:47.970174 2571 flags.go:64] FLAG: --anonymous-auth="true" May 27 20:10:47.971816 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:47.970178 2571 flags.go:64] FLAG: --application-metrics-count-limit="100" May 27 20:10:47.971816 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:47.970183 2571 flags.go:64] FLAG: --authentication-token-webhook="false" May 27 20:10:47.971816 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:47.970187 2571 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" May 27 20:10:47.971816 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:47.970191 2571 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" May 27 20:10:47.972348 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:47.970195 2571 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" May 27 20:10:47.972348 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:47.970198 2571 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" May 27 20:10:47.972348 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:47.970201 2571 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" May 27 20:10:47.972348 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:47.970204 2571 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" May 27 20:10:47.972348 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:47.970208 2571 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" May 27 20:10:47.972348 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:47.970211 2571 flags.go:64] FLAG: --cgroup-driver="cgroupfs" May 27 20:10:47.972348 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:47.970213 2571 flags.go:64] FLAG: --cgroup-root="" May 27 20:10:47.972348 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:47.970216 2571 flags.go:64] FLAG: --cgroups-per-qos="true" May 27 20:10:47.972348 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:47.970219 2571 flags.go:64] FLAG: --client-ca-file="" May 27 20:10:47.972348 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:47.970223 2571 flags.go:64] FLAG: --cloud-config="" May 27 20:10:47.972348 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:47.970225 2571 flags.go:64] FLAG: --cloud-provider="external" May 27 20:10:47.972348 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:47.970228 2571 flags.go:64] FLAG: --cluster-dns="[]" May 27 20:10:47.972348 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:47.970232 2571 flags.go:64] FLAG: --cluster-domain="" May 27 20:10:47.972348 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:47.970234 2571 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" May 27 20:10:47.972348 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:47.970238 2571 flags.go:64] FLAG: --config-dir="" May 27 20:10:47.972348 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:47.970241 2571 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" May 27 20:10:47.972348 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:47.970244 2571 flags.go:64] FLAG: --container-log-max-files="5" May 27 20:10:47.972348 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:47.970255 2571 flags.go:64] FLAG: --container-log-max-size="10Mi" May 27 20:10:47.972348 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:47.970258 2571 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" May 27 20:10:47.972348 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:47.970261 2571 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" May 27 20:10:47.972348 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:47.970264 2571 flags.go:64] FLAG: --containerd-namespace="k8s.io" May 27 20:10:47.972348 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:47.970267 2571 flags.go:64] FLAG: --contention-profiling="false" May 27 20:10:47.972348 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:47.970270 2571 flags.go:64] FLAG: --cpu-cfs-quota="true" May 27 20:10:47.972348 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:47.970273 2571 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" May 27 20:10:47.972925 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:47.970276 2571 flags.go:64] FLAG: --cpu-manager-policy="none" May 27 20:10:47.972925 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:47.970279 2571 flags.go:64] FLAG: --cpu-manager-policy-options="" May 27 20:10:47.972925 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:47.970283 2571 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" May 27 20:10:47.972925 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:47.970286 2571 flags.go:64] FLAG: --enable-controller-attach-detach="true" May 27 20:10:47.972925 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:47.970289 2571 flags.go:64] FLAG: --enable-debugging-handlers="true" May 27 20:10:47.972925 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:47.970291 2571 flags.go:64] FLAG: --enable-load-reader="false" May 27 20:10:47.972925 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:47.970295 2571 flags.go:64] FLAG: --enable-server="true" May 27 20:10:47.972925 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:47.970298 2571 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" May 27 20:10:47.972925 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:47.970302 2571 flags.go:64] FLAG: --event-burst="100" May 27 20:10:47.972925 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:47.970306 2571 flags.go:64] FLAG: --event-qps="50" May 27 20:10:47.972925 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:47.970309 2571 flags.go:64] FLAG: --event-storage-age-limit="default=0" May 27 20:10:47.972925 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:47.970312 2571 flags.go:64] FLAG: --event-storage-event-limit="default=0" May 27 20:10:47.972925 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:47.970315 2571 flags.go:64] FLAG: --eviction-hard="" May 27 20:10:47.972925 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:47.970319 2571 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" May 27 20:10:47.972925 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:47.970322 2571 flags.go:64] FLAG: --eviction-minimum-reclaim="" May 27 20:10:47.972925 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:47.970325 2571 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" May 27 20:10:47.972925 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:47.970327 2571 flags.go:64] FLAG: --eviction-soft="" May 27 20:10:47.972925 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:47.970331 2571 flags.go:64] FLAG: --eviction-soft-grace-period="" May 27 20:10:47.972925 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:47.970334 2571 flags.go:64] FLAG: --exit-on-lock-contention="false" May 27 20:10:47.972925 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:47.970337 2571 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" May 27 20:10:47.972925 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:47.970340 2571 flags.go:64] FLAG: --experimental-mounter-path="" May 27 20:10:47.972925 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:47.970343 2571 flags.go:64] FLAG: --fail-cgroupv1="false" May 27 20:10:47.972925 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:47.970345 2571 flags.go:64] FLAG: --fail-swap-on="true" May 27 20:10:47.972925 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:47.970348 2571 flags.go:64] FLAG: --feature-gates="" May 27 20:10:47.972925 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:47.970352 2571 flags.go:64] FLAG: --file-check-frequency="20s" May 27 20:10:47.973534 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:47.970355 2571 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" May 27 20:10:47.973534 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:47.970358 2571 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" May 27 20:10:47.973534 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:47.970361 2571 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" May 27 20:10:47.973534 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:47.970364 2571 flags.go:64] FLAG: --healthz-port="10248" May 27 20:10:47.973534 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:47.970367 2571 flags.go:64] FLAG: --help="false" May 27 20:10:47.973534 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:47.970370 2571 flags.go:64] FLAG: --hostname-override="ip-10-0-142-251.ec2.internal" May 27 20:10:47.973534 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:47.970373 2571 flags.go:64] FLAG: --housekeeping-interval="10s" May 27 20:10:47.973534 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:47.970376 2571 flags.go:64] FLAG: --http-check-frequency="20s" May 27 20:10:47.973534 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:47.970379 2571 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" May 27 20:10:47.973534 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:47.970382 2571 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" May 27 20:10:47.973534 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:47.970386 2571 flags.go:64] FLAG: --image-gc-high-threshold="85" May 27 20:10:47.973534 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:47.970388 2571 flags.go:64] FLAG: --image-gc-low-threshold="80" May 27 20:10:47.973534 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:47.970391 2571 flags.go:64] FLAG: --image-service-endpoint="" May 27 20:10:47.973534 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:47.970394 2571 flags.go:64] FLAG: --kernel-memcg-notification="false" May 27 20:10:47.973534 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:47.970399 2571 flags.go:64] FLAG: --kube-api-burst="100" May 27 20:10:47.973534 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:47.970402 2571 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" May 27 20:10:47.973534 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:47.970405 2571 flags.go:64] FLAG: --kube-api-qps="50" May 27 20:10:47.973534 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:47.970409 2571 flags.go:64] FLAG: --kube-reserved="" May 27 20:10:47.973534 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:47.970411 2571 flags.go:64] FLAG: --kube-reserved-cgroup="" May 27 20:10:47.973534 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:47.970414 2571 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" May 27 20:10:47.973534 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:47.970417 2571 flags.go:64] FLAG: --kubelet-cgroups="" May 27 20:10:47.973534 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:47.970420 2571 flags.go:64] FLAG: --local-storage-capacity-isolation="true" May 27 20:10:47.973534 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:47.970423 2571 flags.go:64] FLAG: --lock-file="" May 27 20:10:47.973534 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:47.970426 2571 flags.go:64] FLAG: --log-cadvisor-usage="false" May 27 20:10:47.974125 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:47.970429 2571 flags.go:64] FLAG: --log-flush-frequency="5s" May 27 20:10:47.974125 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:47.970432 2571 flags.go:64] FLAG: --log-json-info-buffer-size="0" May 27 20:10:47.974125 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:47.970437 2571 flags.go:64] FLAG: --log-json-split-stream="false" May 27 20:10:47.974125 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:47.970440 2571 flags.go:64] FLAG: --log-text-info-buffer-size="0" May 27 20:10:47.974125 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:47.970443 2571 flags.go:64] FLAG: --log-text-split-stream="false" May 27 20:10:47.974125 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:47.970445 2571 flags.go:64] FLAG: --logging-format="text" May 27 20:10:47.974125 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:47.970448 2571 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" May 27 20:10:47.974125 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:47.970451 2571 flags.go:64] FLAG: --make-iptables-util-chains="true" May 27 20:10:47.974125 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:47.970454 2571 flags.go:64] FLAG: --manifest-url="" May 27 20:10:47.974125 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:47.970457 2571 flags.go:64] FLAG: --manifest-url-header="" May 27 20:10:47.974125 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:47.970461 2571 flags.go:64] FLAG: --max-housekeeping-interval="15s" May 27 20:10:47.974125 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:47.970464 2571 flags.go:64] FLAG: --max-open-files="1000000" May 27 20:10:47.974125 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:47.970468 2571 flags.go:64] FLAG: --max-pods="110" May 27 20:10:47.974125 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:47.970471 2571 flags.go:64] FLAG: --maximum-dead-containers="-1" May 27 20:10:47.974125 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:47.970474 2571 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" May 27 20:10:47.974125 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:47.970477 2571 flags.go:64] FLAG: --memory-manager-policy="None" May 27 20:10:47.974125 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:47.970480 2571 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" May 27 20:10:47.974125 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:47.970483 2571 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" May 27 20:10:47.974125 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:47.970485 2571 flags.go:64] FLAG: --node-ip="0.0.0.0" May 27 20:10:47.974125 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:47.970488 2571 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" May 27 20:10:47.974125 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:47.970495 2571 flags.go:64] FLAG: --node-status-max-images="50" May 27 20:10:47.974125 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:47.970498 2571 flags.go:64] FLAG: --node-status-update-frequency="10s" May 27 20:10:47.974125 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:47.970501 2571 flags.go:64] FLAG: --oom-score-adj="-999" May 27 20:10:47.974125 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:47.970504 2571 flags.go:64] FLAG: --pod-cidr="" May 27 20:10:47.974705 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:47.970507 2571 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:afe88671bf163c42b467d159dc6f0b784dbe28b1ed44e171edc9c06fd99cc6fc" May 27 20:10:47.974705 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:47.970512 2571 flags.go:64] FLAG: --pod-manifest-path="" May 27 20:10:47.974705 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:47.970515 2571 flags.go:64] FLAG: --pod-max-pids="-1" May 27 20:10:47.974705 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:47.970519 2571 flags.go:64] FLAG: --pods-per-core="0" May 27 20:10:47.974705 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:47.970522 2571 flags.go:64] FLAG: --port="10250" May 27 20:10:47.974705 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:47.970525 2571 flags.go:64] FLAG: --protect-kernel-defaults="false" May 27 20:10:47.974705 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:47.970527 2571 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0c3b54ffeeada3e82" May 27 20:10:47.974705 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:47.970531 2571 flags.go:64] FLAG: --qos-reserved="" May 27 20:10:47.974705 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:47.970534 2571 flags.go:64] FLAG: --read-only-port="10255" May 27 20:10:47.974705 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:47.970537 2571 flags.go:64] FLAG: --register-node="true" May 27 20:10:47.974705 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:47.970540 2571 flags.go:64] FLAG: --register-schedulable="true" May 27 20:10:47.974705 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:47.970543 2571 flags.go:64] FLAG: --register-with-taints="" May 27 20:10:47.974705 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:47.970546 2571 flags.go:64] FLAG: --registry-burst="10" May 27 20:10:47.974705 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:47.970549 2571 flags.go:64] FLAG: --registry-qps="5" May 27 20:10:47.974705 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:47.970552 2571 flags.go:64] FLAG: --reserved-cpus="" May 27 20:10:47.974705 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:47.970554 2571 flags.go:64] FLAG: --reserved-memory="" May 27 20:10:47.974705 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:47.970558 2571 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" May 27 20:10:47.974705 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:47.970561 2571 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" May 27 20:10:47.974705 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:47.970564 2571 flags.go:64] FLAG: --rotate-certificates="false" May 27 20:10:47.974705 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:47.970567 2571 flags.go:64] FLAG: --rotate-server-certificates="false" May 27 20:10:47.974705 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:47.970570 2571 flags.go:64] FLAG: --runonce="false" May 27 20:10:47.974705 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:47.970573 2571 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" May 27 20:10:47.974705 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:47.970576 2571 flags.go:64] FLAG: --runtime-request-timeout="2m0s" May 27 20:10:47.974705 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:47.970579 2571 flags.go:64] FLAG: --seccomp-default="false" May 27 20:10:47.974705 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:47.970582 2571 flags.go:64] FLAG: --serialize-image-pulls="true" May 27 20:10:47.975313 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:47.970584 2571 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" May 27 20:10:47.975313 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:47.970587 2571 flags.go:64] FLAG: --storage-driver-db="cadvisor" May 27 20:10:47.975313 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:47.970590 2571 flags.go:64] FLAG: --storage-driver-host="localhost:8086" May 27 20:10:47.975313 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:47.970593 2571 flags.go:64] FLAG: --storage-driver-password="root" May 27 20:10:47.975313 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:47.970595 2571 flags.go:64] FLAG: --storage-driver-secure="false" May 27 20:10:47.975313 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:47.970598 2571 flags.go:64] FLAG: --storage-driver-table="stats" May 27 20:10:47.975313 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:47.970601 2571 flags.go:64] FLAG: --storage-driver-user="root" May 27 20:10:47.975313 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:47.970604 2571 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" May 27 20:10:47.975313 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:47.970610 2571 flags.go:64] FLAG: --sync-frequency="1m0s" May 27 20:10:47.975313 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:47.970613 2571 flags.go:64] FLAG: --system-cgroups="" May 27 20:10:47.975313 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:47.970616 2571 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" May 27 20:10:47.975313 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:47.970621 2571 flags.go:64] FLAG: --system-reserved-cgroup="" May 27 20:10:47.975313 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:47.970624 2571 flags.go:64] FLAG: --tls-cert-file="" May 27 20:10:47.975313 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:47.970626 2571 flags.go:64] FLAG: --tls-cipher-suites="[]" May 27 20:10:47.975313 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:47.970630 2571 flags.go:64] FLAG: --tls-min-version="" May 27 20:10:47.975313 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:47.970634 2571 flags.go:64] FLAG: --tls-private-key-file="" May 27 20:10:47.975313 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:47.970637 2571 flags.go:64] FLAG: --topology-manager-policy="none" May 27 20:10:47.975313 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:47.970640 2571 flags.go:64] FLAG: --topology-manager-policy-options="" May 27 20:10:47.975313 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:47.970643 2571 flags.go:64] FLAG: --topology-manager-scope="container" May 27 20:10:47.975313 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:47.970646 2571 flags.go:64] FLAG: --v="2" May 27 20:10:47.975313 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:47.970649 2571 flags.go:64] FLAG: --version="false" May 27 20:10:47.975313 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:47.970653 2571 flags.go:64] FLAG: --vmodule="" May 27 20:10:47.975313 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:47.970657 2571 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" May 27 20:10:47.975313 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:47.970661 2571 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" May 27 20:10:47.975313 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.970754 2571 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix May 27 20:10:47.975891 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.970758 2571 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission May 27 20:10:47.975891 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.970760 2571 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig May 27 20:10:47.975891 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.970763 2571 feature_gate.go:328] unrecognized feature gate: Example2 May 27 20:10:47.975891 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.970765 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImages May 27 20:10:47.975891 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.970768 2571 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata May 27 20:10:47.975891 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.970771 2571 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement May 27 20:10:47.975891 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.970773 2571 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes May 27 20:10:47.975891 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.970776 2571 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts May 27 20:10:47.975891 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.970778 2571 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk May 27 20:10:47.975891 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.970781 2571 feature_gate.go:328] unrecognized feature gate: ExternalOIDC May 27 20:10:47.975891 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.970784 2571 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall May 27 20:10:47.975891 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.970787 2571 feature_gate.go:328] unrecognized feature gate: SignatureStores May 27 20:10:47.975891 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.970789 2571 feature_gate.go:328] unrecognized feature gate: GatewayAPIController May 27 20:10:47.975891 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.970792 2571 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI May 27 20:10:47.975891 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.970794 2571 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements May 27 20:10:47.975891 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.970799 2571 feature_gate.go:328] unrecognized feature gate: GatewayAPI May 27 20:10:47.975891 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.970801 2571 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter May 27 20:10:47.975891 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.970804 2571 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS May 27 20:10:47.975891 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.970806 2571 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification May 27 20:10:47.975891 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.970809 2571 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity May 27 20:10:47.976433 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.970812 2571 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig May 27 20:10:47.976433 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.970814 2571 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas May 27 20:10:47.976433 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.970818 2571 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall May 27 20:10:47.976433 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.970821 2571 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy May 27 20:10:47.976433 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.970823 2571 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes May 27 20:10:47.976433 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.970826 2571 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform May 27 20:10:47.976433 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.970828 2571 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration May 27 20:10:47.976433 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.970831 2571 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager May 27 20:10:47.976433 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.970834 2571 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup May 27 20:10:47.976433 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.970836 2571 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController May 27 20:10:47.976433 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.970838 2571 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks May 27 20:10:47.976433 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.970841 2571 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI May 27 20:10:47.976433 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.970843 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure May 27 20:10:47.976433 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.970846 2571 feature_gate.go:328] unrecognized feature gate: Example May 27 20:10:47.976433 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.970849 2571 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation May 27 20:10:47.976433 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.970851 2571 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations May 27 20:10:47.976433 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.970854 2571 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk May 27 20:10:47.976433 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.970857 2571 feature_gate.go:328] unrecognized feature gate: PinnedImages May 27 20:10:47.976433 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.970859 2571 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints May 27 20:10:47.976433 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.970862 2571 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities May 27 20:10:47.976912 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.970864 2571 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure May 27 20:10:47.976912 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.970867 2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall May 27 20:10:47.976912 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.970869 2571 feature_gate.go:328] unrecognized feature gate: NewOLM May 27 20:10:47.976912 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.970871 2571 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets May 27 20:10:47.976912 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.970874 2571 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration May 27 20:10:47.976912 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.970876 2571 feature_gate.go:328] unrecognized feature gate: ShortCertRotation May 27 20:10:47.976912 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.970879 2571 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud May 27 20:10:47.976912 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.970882 2571 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles May 27 20:10:47.976912 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.970885 2571 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather May 27 20:10:47.976912 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.970889 2571 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. May 27 20:10:47.976912 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.970892 2571 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall May 27 20:10:47.976912 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.970896 2571 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup May 27 20:10:47.976912 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.970898 2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud May 27 20:10:47.976912 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.970901 2571 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts May 27 20:10:47.976912 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.970905 2571 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks May 27 20:10:47.976912 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.970908 2571 feature_gate.go:328] unrecognized feature gate: DualReplica May 27 20:10:47.976912 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.970911 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS May 27 20:10:47.976912 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.970914 2571 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS May 27 20:10:47.976912 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.970917 2571 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings May 27 20:10:47.977380 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.970919 2571 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal May 27 20:10:47.977380 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.970922 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere May 27 20:10:47.977380 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.970924 2571 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot May 27 20:10:47.977380 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.970926 2571 feature_gate.go:328] unrecognized feature gate: InsightsConfig May 27 20:10:47.977380 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.970929 2571 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider May 27 20:10:47.977380 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.970932 2571 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration May 27 20:10:47.977380 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.970934 2571 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS May 27 20:10:47.977380 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.970936 2571 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController May 27 20:10:47.977380 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.970939 2571 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA May 27 20:10:47.977380 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.970941 2571 feature_gate.go:328] unrecognized feature gate: OVNObservability May 27 20:10:47.977380 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.970944 2571 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall May 27 20:10:47.977380 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.970946 2571 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation May 27 20:10:47.977380 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.970949 2571 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement May 27 20:10:47.977380 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.970951 2571 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv May 27 20:10:47.977380 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.970954 2571 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup May 27 20:10:47.977380 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.970956 2571 feature_gate.go:328] unrecognized feature gate: UpgradeStatus May 27 20:10:47.977380 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.970958 2571 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting May 27 20:10:47.977380 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.970961 2571 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace May 27 20:10:47.977380 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.970963 2571 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode May 27 20:10:47.977380 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.970965 2571 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota May 27 20:10:47.977864 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.970969 2571 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. May 27 20:10:47.977864 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.970972 2571 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig May 27 20:10:47.977864 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.970976 2571 feature_gate.go:328] unrecognized feature gate: DNSNameResolver May 27 20:10:47.977864 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.970978 2571 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy May 27 20:10:47.977864 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.970980 2571 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode May 27 20:10:47.977864 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.970983 2571 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses May 27 20:10:47.977864 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:47.970991 2571 feature_gate.go:384] feature gates: {map[DynamicResourceAllocation:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutatingAdmissionPolicy:false NodeSwap:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:false]} May 27 20:10:47.977864 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:47.977806 2571 server.go:530] "Kubelet version" kubeletVersion="v1.33.11" May 27 20:10:47.977864 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:47.977821 2571 server.go:532] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" May 27 20:10:47.978121 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.977895 2571 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement May 27 20:10:47.978121 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.977900 2571 feature_gate.go:328] unrecognized feature gate: GatewayAPI May 27 20:10:47.978121 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.977903 2571 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot May 27 20:10:47.978121 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.977906 2571 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform May 27 20:10:47.978121 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.977909 2571 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager May 27 20:10:47.978121 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.977911 2571 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration May 27 20:10:47.978121 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.977914 2571 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud May 27 20:10:47.978121 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.977916 2571 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter May 27 20:10:47.978121 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.977919 2571 feature_gate.go:328] unrecognized feature gate: DualReplica May 27 20:10:47.978121 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.977921 2571 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup May 27 20:10:47.978121 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.977924 2571 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes May 27 20:10:47.978121 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.977926 2571 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification May 27 20:10:47.978121 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.977929 2571 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI May 27 20:10:47.978121 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.977932 2571 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes May 27 20:10:47.978121 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.977935 2571 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. May 27 20:10:47.978121 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.977940 2571 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider May 27 20:10:47.978121 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.977942 2571 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata May 27 20:10:47.978121 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.977945 2571 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets May 27 20:10:47.978121 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.977947 2571 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController May 27 20:10:47.978121 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.977950 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS May 27 20:10:47.978617 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.977952 2571 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig May 27 20:10:47.978617 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.977955 2571 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas May 27 20:10:47.978617 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.977958 2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall May 27 20:10:47.978617 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.977960 2571 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations May 27 20:10:47.978617 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.977963 2571 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure May 27 20:10:47.978617 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.977966 2571 feature_gate.go:328] unrecognized feature gate: ShortCertRotation May 27 20:10:47.978617 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.977969 2571 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy May 27 20:10:47.978617 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.977971 2571 feature_gate.go:328] unrecognized feature gate: SignatureStores May 27 20:10:47.978617 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.977974 2571 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup May 27 20:10:47.978617 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.977976 2571 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts May 27 20:10:47.978617 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.977980 2571 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. May 27 20:10:47.978617 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.977984 2571 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS May 27 20:10:47.978617 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.977987 2571 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall May 27 20:10:47.978617 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.977991 2571 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration May 27 20:10:47.978617 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.977994 2571 feature_gate.go:328] unrecognized feature gate: PinnedImages May 27 20:10:47.978617 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.977996 2571 feature_gate.go:328] unrecognized feature gate: GatewayAPIController May 27 20:10:47.978617 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.977999 2571 feature_gate.go:328] unrecognized feature gate: DNSNameResolver May 27 20:10:47.978617 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.978002 2571 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace May 27 20:10:47.978617 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.978004 2571 feature_gate.go:328] unrecognized feature gate: UpgradeStatus May 27 20:10:47.979086 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.978007 2571 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings May 27 20:10:47.979086 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.978010 2571 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation May 27 20:10:47.979086 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.978012 2571 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting May 27 20:10:47.979086 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.978015 2571 feature_gate.go:328] unrecognized feature gate: NewOLM May 27 20:10:47.979086 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.978017 2571 feature_gate.go:328] unrecognized feature gate: InsightsConfig May 27 20:10:47.979086 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.978020 2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud May 27 20:10:47.979086 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.978023 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImages May 27 20:10:47.979086 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.978025 2571 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk May 27 20:10:47.979086 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.978028 2571 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS May 27 20:10:47.979086 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.978030 2571 feature_gate.go:328] unrecognized feature gate: Example May 27 20:10:47.979086 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.978033 2571 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity May 27 20:10:47.979086 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.978035 2571 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup May 27 20:10:47.979086 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.978037 2571 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities May 27 20:10:47.979086 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.978040 2571 feature_gate.go:328] unrecognized feature gate: Example2 May 27 20:10:47.979086 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.978042 2571 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather May 27 20:10:47.979086 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.978045 2571 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles May 27 20:10:47.979086 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.978047 2571 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal May 27 20:10:47.979086 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.978050 2571 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS May 27 20:10:47.979086 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.978052 2571 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement May 27 20:10:47.979086 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.978055 2571 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall May 27 20:10:47.979591 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.978058 2571 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv May 27 20:10:47.979591 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.978061 2571 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation May 27 20:10:47.979591 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.978063 2571 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota May 27 20:10:47.979591 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.978066 2571 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig May 27 20:10:47.979591 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.978068 2571 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA May 27 20:10:47.979591 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.978071 2571 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode May 27 20:10:47.979591 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.978074 2571 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission May 27 20:10:47.979591 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.978077 2571 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI May 27 20:10:47.979591 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.978079 2571 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses May 27 20:10:47.979591 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.978082 2571 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy May 27 20:10:47.979591 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.978084 2571 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode May 27 20:10:47.979591 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.978087 2571 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig May 27 20:10:47.979591 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.978089 2571 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall May 27 20:10:47.979591 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.978092 2571 feature_gate.go:328] unrecognized feature gate: OVNObservability May 27 20:10:47.979591 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.978094 2571 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints May 27 20:10:47.979591 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.978096 2571 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk May 27 20:10:47.979591 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.978115 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure May 27 20:10:47.979591 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.978120 2571 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks May 27 20:10:47.979591 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.978123 2571 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall May 27 20:10:47.980048 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.978126 2571 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix May 27 20:10:47.980048 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.978128 2571 feature_gate.go:328] unrecognized feature gate: ExternalOIDC May 27 20:10:47.980048 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.978131 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere May 27 20:10:47.980048 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.978134 2571 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration May 27 20:10:47.980048 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.978136 2571 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks May 27 20:10:47.980048 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.978139 2571 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts May 27 20:10:47.980048 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.978141 2571 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController May 27 20:10:47.980048 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.978145 2571 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements May 27 20:10:47.980048 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:47.978149 2571 feature_gate.go:384] feature gates: {map[DynamicResourceAllocation:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutatingAdmissionPolicy:false NodeSwap:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:false]} May 27 20:10:47.980048 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.978268 2571 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk May 27 20:10:47.980048 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.978272 2571 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall May 27 20:10:47.980048 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.978275 2571 feature_gate.go:328] unrecognized feature gate: UpgradeStatus May 27 20:10:47.980048 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.978278 2571 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController May 27 20:10:47.980048 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.978293 2571 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration May 27 20:10:47.980048 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.978297 2571 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission May 27 20:10:47.980424 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.978300 2571 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks May 27 20:10:47.980424 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.978302 2571 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup May 27 20:10:47.980424 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.978305 2571 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles May 27 20:10:47.980424 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.978308 2571 feature_gate.go:328] unrecognized feature gate: NewOLM May 27 20:10:47.980424 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.978311 2571 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets May 27 20:10:47.980424 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.978314 2571 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement May 27 20:10:47.980424 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.978317 2571 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController May 27 20:10:47.980424 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.978319 2571 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk May 27 20:10:47.980424 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.978322 2571 feature_gate.go:328] unrecognized feature gate: InsightsConfig May 27 20:10:47.980424 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.978325 2571 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses May 27 20:10:47.980424 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.978328 2571 feature_gate.go:328] unrecognized feature gate: SignatureStores May 27 20:10:47.980424 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.978331 2571 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig May 27 20:10:47.980424 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.978333 2571 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata May 27 20:10:47.980424 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.978336 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS May 27 20:10:47.980424 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.978338 2571 feature_gate.go:328] unrecognized feature gate: ExternalOIDC May 27 20:10:47.980424 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.978340 2571 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration May 27 20:10:47.980424 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.978343 2571 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity May 27 20:10:47.980424 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.978345 2571 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy May 27 20:10:47.980424 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.978348 2571 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement May 27 20:10:47.980424 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.978351 2571 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. May 27 20:10:47.980914 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.978355 2571 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall May 27 20:10:47.980914 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.978357 2571 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform May 27 20:10:47.980914 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.978359 2571 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup May 27 20:10:47.980914 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.978362 2571 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter May 27 20:10:47.980914 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.978364 2571 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot May 27 20:10:47.980914 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.978366 2571 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall May 27 20:10:47.980914 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.978369 2571 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities May 27 20:10:47.980914 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.978371 2571 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS May 27 20:10:47.980914 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.978374 2571 feature_gate.go:328] unrecognized feature gate: GatewayAPIController May 27 20:10:47.980914 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.978376 2571 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI May 27 20:10:47.980914 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.978379 2571 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations May 27 20:10:47.980914 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.978382 2571 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA May 27 20:10:47.980914 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.978384 2571 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure May 27 20:10:47.980914 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.978387 2571 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager May 27 20:10:47.980914 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.978389 2571 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud May 27 20:10:47.980914 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.978392 2571 feature_gate.go:328] unrecognized feature gate: GatewayAPI May 27 20:10:47.980914 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.978395 2571 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas May 27 20:10:47.980914 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.978398 2571 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig May 27 20:10:47.980914 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.978400 2571 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall May 27 20:10:47.980914 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.978403 2571 feature_gate.go:328] unrecognized feature gate: DualReplica May 27 20:10:47.981405 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.978405 2571 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification May 27 20:10:47.981405 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.978407 2571 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode May 27 20:10:47.981405 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.978409 2571 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS May 27 20:10:47.981405 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.978412 2571 feature_gate.go:328] unrecognized feature gate: DNSNameResolver May 27 20:10:47.981405 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.978414 2571 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider May 27 20:10:47.981405 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.978417 2571 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation May 27 20:10:47.981405 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.978419 2571 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts May 27 20:10:47.981405 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.978421 2571 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings May 27 20:10:47.981405 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.978424 2571 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts May 27 20:10:47.981405 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.978426 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImages May 27 20:10:47.981405 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.978428 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere May 27 20:10:47.981405 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.978431 2571 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy May 27 20:10:47.981405 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.978433 2571 feature_gate.go:328] unrecognized feature gate: Example May 27 20:10:47.981405 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.978436 2571 feature_gate.go:328] unrecognized feature gate: PinnedImages May 27 20:10:47.981405 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.978438 2571 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather May 27 20:10:47.981405 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.978440 2571 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting May 27 20:10:47.981405 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.978443 2571 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation May 27 20:10:47.981405 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.978445 2571 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota May 27 20:10:47.981405 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.978448 2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall May 27 20:10:47.981854 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.978451 2571 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. May 27 20:10:47.981854 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.978455 2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure May 27 20:10:47.981854 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.978458 2571 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig May 27 20:10:47.981854 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.978461 2571 feature_gate.go:328] unrecognized feature gate: OVNObservability May 27 20:10:47.981854 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.978464 2571 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode May 27 20:10:47.981854 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.978467 2571 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration May 27 20:10:47.981854 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.978470 2571 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS May 27 20:10:47.981854 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.978473 2571 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace May 27 20:10:47.981854 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.978476 2571 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv May 27 20:10:47.981854 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.978478 2571 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes May 27 20:10:47.981854 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.978481 2571 feature_gate.go:328] unrecognized feature gate: Example2 May 27 20:10:47.981854 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.978484 2571 feature_gate.go:328] unrecognized feature gate: ShortCertRotation May 27 20:10:47.981854 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.978486 2571 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal May 27 20:10:47.981854 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.978489 2571 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix May 27 20:10:47.981854 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.978491 2571 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements May 27 20:10:47.981854 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.978494 2571 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes May 27 20:10:47.981854 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.978496 2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud May 27 20:10:47.981854 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.978498 2571 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints May 27 20:10:47.981854 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.978501 2571 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks May 27 20:10:47.982323 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.978503 2571 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI May 27 20:10:47.982323 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:47.978505 2571 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup May 27 20:10:47.982323 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:47.978510 2571 feature_gate.go:384] feature gates: {map[DynamicResourceAllocation:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutatingAdmissionPolicy:false NodeSwap:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:false]} May 27 20:10:47.982323 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:47.979392 2571 server.go:962] "Client rotation is on, will bootstrap in background" May 27 20:10:47.982911 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:47.982898 2571 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" May 27 20:10:47.984067 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:47.984057 2571 server.go:1019] "Starting client certificate rotation" May 27 20:10:47.984182 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:47.984165 2571 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" May 27 20:10:47.984225 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:47.984199 2571 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" May 27 20:10:48.014005 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:48.013988 2571 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" May 27 20:10:48.015890 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:48.015873 2571 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" May 27 20:10:48.033296 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:48.033277 2571 log.go:25] "Validated CRI v1 runtime API" May 27 20:10:48.039824 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:48.039810 2571 log.go:25] "Validated CRI v1 image API" May 27 20:10:48.042899 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:48.042878 2571 server.go:1452] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" May 27 20:10:48.046541 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:48.046524 2571 reflector.go:430] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" May 27 20:10:48.047839 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:48.047818 2571 fs.go:135] Filesystem UUIDs: map[6652f3c6-bf8a-4a65-a0e4-fcbd4179b5cb:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 80793a8a-37c5-4eb0-9990-7168149254da:/dev/nvme0n1p3] May 27 20:10:48.047883 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:48.047841 2571 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] May 27 20:10:48.051865 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:48.051764 2571 manager.go:217] Machine: {Timestamp:2026-05-27 20:10:48.051288889 +0000 UTC m=+0.460915210 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3099302 MemoryCapacity:32812175360 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2e7ea1c06576c7122b99500bcc450d SystemUUID:ec2e7ea1-c065-76c7-122b-99500bcc450d BootID:7e5806d6-31f1-481f-8e38-8f3aa9b1555f Filesystems:[{Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562435072 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6103040 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406089728 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406085632 Type:vfs Inodes:4005392 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:7b:f7:7e:d8:a3 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:7b:f7:7e:d8:a3 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:7e:2b:b4:8f:16:68 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812175360 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} May 27 20:10:48.052520 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:48.052509 2571 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. May 27 20:10:48.052618 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:48.052607 2571 manager.go:233] Version: {KernelVersion:5.14.0-570.114.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260512-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} May 27 20:10:48.055871 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:48.055843 2571 container_manager_linux.go:270] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] May 27 20:10:48.056006 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:48.055875 2571 container_manager_linux.go:275] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-142-251.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} May 27 20:10:48.056048 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:48.056013 2571 topology_manager.go:138] "Creating topology manager with none policy" May 27 20:10:48.056048 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:48.056022 2571 container_manager_linux.go:306] "Creating device plugin manager" May 27 20:10:48.056048 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:48.056029 2571 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" May 27 20:10:48.056795 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:48.056785 2571 server.go:72] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" May 27 20:10:48.058051 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:48.058041 2571 state_mem.go:36] "Initialized new in-memory state store" May 27 20:10:48.058299 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:48.058289 2571 server.go:1267] "Using root directory" path="/var/lib/kubelet" May 27 20:10:48.061002 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:48.060992 2571 kubelet.go:491] "Attempting to sync node with API server" May 27 20:10:48.061033 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:48.061011 2571 kubelet.go:386] "Adding static pod path" path="/etc/kubernetes/manifests" May 27 20:10:48.061033 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:48.061029 2571 file.go:69] "Watching path" path="/etc/kubernetes/manifests" May 27 20:10:48.061093 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:48.061044 2571 kubelet.go:397] "Adding apiserver pod source" May 27 20:10:48.061093 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:48.061057 2571 apiserver.go:42] "Waiting for node sync before watching apiserver pods" May 27 20:10:48.062245 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:48.062228 2571 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" May 27 20:10:48.062245 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:48.062249 2571 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" May 27 20:10:48.070193 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:48.070176 2571 kuberuntime_manager.go:279] "Container runtime initialized" containerRuntime="cri-o" version="1.33.11-2.rhaos4.20.gitb2a8320.el9" apiVersion="v1" May 27 20:10:48.071226 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:48.071211 2571 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-rndbg" May 27 20:10:48.072578 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:48.072564 2571 kubelet.go:953] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" May 27 20:10:48.072798 ip-10-0-142-251 kubenswrapper[2571]: E0527 20:10:48.072775 2571 reflector.go:200] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" May 27 20:10:48.072926 ip-10-0-142-251 kubenswrapper[2571]: E0527 20:10:48.072911 2571 reflector.go:200] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-142-251.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" May 27 20:10:48.075709 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:48.075698 2571 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" May 27 20:10:48.075746 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:48.075715 2571 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" May 27 20:10:48.075746 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:48.075721 2571 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" May 27 20:10:48.075746 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:48.075726 2571 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/host-path" May 27 20:10:48.075746 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:48.075736 2571 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/nfs" May 27 20:10:48.075746 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:48.075745 2571 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/secret" May 27 20:10:48.075880 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:48.075752 2571 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" May 27 20:10:48.075880 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:48.075757 2571 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" May 27 20:10:48.075880 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:48.075766 2571 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/fc" May 27 20:10:48.075880 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:48.075771 2571 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/configmap" May 27 20:10:48.075880 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:48.075784 2571 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/projected" May 27 20:10:48.075880 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:48.075790 2571 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" May 27 20:10:48.076823 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:48.076812 2571 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/csi" May 27 20:10:48.076856 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:48.076825 2571 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/image" May 27 20:10:48.078436 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:48.078420 2571 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-rndbg" May 27 20:10:48.079752 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:48.079583 2571 csi_plugin.go:988] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-142-251.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope May 27 20:10:48.079916 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:48.079906 2571 watchdog_linux.go:99] "Systemd watchdog is not enabled" May 27 20:10:48.079952 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:48.079945 2571 server.go:1295] "Started kubelet" May 27 20:10:48.080092 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:48.080061 2571 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 May 27 20:10:48.080190 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:48.080076 2571 ratelimit.go:55] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 May 27 20:10:48.082066 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:48.082041 2571 server_v1.go:47] "podresources" method="list" useActivePods=true May 27 20:10:48.082701 ip-10-0-142-251 systemd[1]: Started Kubernetes Kubelet. May 27 20:10:48.083552 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:48.083506 2571 server.go:255] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" May 27 20:10:48.085243 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:48.084424 2571 server.go:317] "Adding debug handlers to kubelet server" May 27 20:10:48.092740 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:48.092721 2571 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" May 27 20:10:48.092820 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:48.092743 2571 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" May 27 20:10:48.092820 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:48.092775 2571 volume_manager.go:295] "The desired_state_of_world populator starts" May 27 20:10:48.092820 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:48.092786 2571 volume_manager.go:297] "Starting Kubelet Volume Manager" May 27 20:10:48.092820 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:48.092810 2571 desired_state_of_world_populator.go:150] "Desired state populator starts to run" May 27 20:10:48.092954 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:48.092903 2571 reconstruct.go:97] "Volume reconstruction finished" May 27 20:10:48.092954 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:48.092913 2571 reconciler.go:26] "Reconciler: start to sync state" May 27 20:10:48.093714 ip-10-0-142-251 kubenswrapper[2571]: E0527 20:10:48.093671 2571 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-142-251.ec2.internal\" not found" May 27 20:10:48.093884 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:48.093868 2571 factory.go:153] Registering CRI-O factory May 27 20:10:48.093884 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:48.093886 2571 factory.go:223] Registration of the crio container factory successfully May 27 20:10:48.094095 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:48.094083 2571 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory May 27 20:10:48.094095 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:48.094096 2571 factory.go:55] Registering systemd factory May 27 20:10:48.094216 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:48.094123 2571 factory.go:223] Registration of the systemd container factory successfully May 27 20:10:48.094216 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:48.094140 2571 factory.go:103] Registering Raw factory May 27 20:10:48.094216 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:48.094157 2571 manager.go:1196] Started watching for new ooms in manager May 27 20:10:48.094586 ip-10-0-142-251 kubenswrapper[2571]: E0527 20:10:48.094488 2571 kubelet.go:1618] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" May 27 20:10:48.094693 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:48.094679 2571 manager.go:319] Starting recovery of all containers May 27 20:10:48.102208 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:48.102187 2571 reflector.go:430] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" May 27 20:10:48.104293 ip-10-0-142-251 kubenswrapper[2571]: E0527 20:10:48.104274 2571 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-142-251.ec2.internal\" not found" node="ip-10-0-142-251.ec2.internal" May 27 20:10:48.106276 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:48.106262 2571 manager.go:324] Recovery completed May 27 20:10:48.109980 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:48.109970 2571 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach" May 27 20:10:48.111601 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:48.111586 2571 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-142-251.ec2.internal" event="NodeHasSufficientMemory" May 27 20:10:48.111663 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:48.111612 2571 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-142-251.ec2.internal" event="NodeHasNoDiskPressure" May 27 20:10:48.111663 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:48.111622 2571 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-142-251.ec2.internal" event="NodeHasSufficientPID" May 27 20:10:48.112084 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:48.112070 2571 cpu_manager.go:222] "Starting CPU manager" policy="none" May 27 20:10:48.112154 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:48.112084 2571 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" May 27 20:10:48.112154 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:48.112117 2571 state_mem.go:36] "Initialized new in-memory state store" May 27 20:10:48.116138 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:48.116125 2571 policy_none.go:49] "None policy: Start" May 27 20:10:48.116173 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:48.116142 2571 memory_manager.go:186] "Starting memorymanager" policy="None" May 27 20:10:48.116173 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:48.116152 2571 state_mem.go:35] "Initializing new in-memory state store" May 27 20:10:48.158670 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:48.156803 2571 manager.go:341] "Starting Device Plugin manager" May 27 20:10:48.158670 ip-10-0-142-251 kubenswrapper[2571]: E0527 20:10:48.156870 2571 manager.go:517] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" May 27 20:10:48.158670 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:48.156888 2571 server.go:85] "Starting device plugin registration server" May 27 20:10:48.158670 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:48.157052 2571 eviction_manager.go:189] "Eviction manager: starting control loop" May 27 20:10:48.158670 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:48.157060 2571 container_log_manager.go:189] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" May 27 20:10:48.158670 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:48.157522 2571 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" May 27 20:10:48.158670 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:48.157594 2571 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" May 27 20:10:48.158670 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:48.157603 2571 plugin_manager.go:118] "Starting Kubelet Plugin Manager" May 27 20:10:48.158670 ip-10-0-142-251 kubenswrapper[2571]: E0527 20:10:48.157967 2571 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" May 27 20:10:48.158670 ip-10-0-142-251 kubenswrapper[2571]: E0527 20:10:48.158011 2571 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-142-251.ec2.internal\" not found" May 27 20:10:48.222621 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:48.222569 2571 kubelet_network_linux.go:49] "Initialized iptables rules." protocol="IPv4" May 27 20:10:48.223858 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:48.223840 2571 kubelet_network_linux.go:49] "Initialized iptables rules." protocol="IPv6" May 27 20:10:48.223858 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:48.223858 2571 status_manager.go:230] "Starting to sync pod status with apiserver" May 27 20:10:48.223972 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:48.223871 2571 watchdog_linux.go:127] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." May 27 20:10:48.223972 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:48.223878 2571 kubelet.go:2451] "Starting kubelet main sync loop" May 27 20:10:48.223972 ip-10-0-142-251 kubenswrapper[2571]: E0527 20:10:48.223915 2571 kubelet.go:2475] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" May 27 20:10:48.230693 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:48.230675 2571 reflector.go:430] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" May 27 20:10:48.257596 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:48.257582 2571 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach" May 27 20:10:48.258384 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:48.258369 2571 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-142-251.ec2.internal" event="NodeHasSufficientMemory" May 27 20:10:48.258475 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:48.258401 2571 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-142-251.ec2.internal" event="NodeHasNoDiskPressure" May 27 20:10:48.258475 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:48.258414 2571 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-142-251.ec2.internal" event="NodeHasSufficientPID" May 27 20:10:48.258475 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:48.258442 2571 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-142-251.ec2.internal" May 27 20:10:48.266448 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:48.266434 2571 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-142-251.ec2.internal" May 27 20:10:48.266527 ip-10-0-142-251 kubenswrapper[2571]: E0527 20:10:48.266455 2571 kubelet_node_status.go:597] "Error updating node status, will retry" err="error getting node \"ip-10-0-142-251.ec2.internal\": node \"ip-10-0-142-251.ec2.internal\" not found" May 27 20:10:48.285184 ip-10-0-142-251 kubenswrapper[2571]: E0527 20:10:48.285162 2571 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-142-251.ec2.internal\" not found" May 27 20:10:48.324656 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:48.324637 2571 kubelet.go:2537] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-251.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-142-251.ec2.internal"] May 27 20:10:48.324711 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:48.324689 2571 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach" May 27 20:10:48.325954 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:48.325937 2571 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-142-251.ec2.internal" event="NodeHasSufficientMemory" May 27 20:10:48.326050 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:48.325969 2571 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-142-251.ec2.internal" event="NodeHasNoDiskPressure" May 27 20:10:48.326050 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:48.325986 2571 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-142-251.ec2.internal" event="NodeHasSufficientPID" May 27 20:10:48.327173 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:48.327159 2571 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach" May 27 20:10:48.327320 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:48.327306 2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-251.ec2.internal" May 27 20:10:48.327366 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:48.327335 2571 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach" May 27 20:10:48.327822 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:48.327800 2571 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-142-251.ec2.internal" event="NodeHasSufficientMemory" May 27 20:10:48.327822 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:48.327812 2571 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-142-251.ec2.internal" event="NodeHasSufficientMemory" May 27 20:10:48.327896 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:48.327832 2571 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-142-251.ec2.internal" event="NodeHasNoDiskPressure" May 27 20:10:48.327896 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:48.327848 2571 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-142-251.ec2.internal" event="NodeHasSufficientPID" May 27 20:10:48.327972 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:48.327836 2571 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-142-251.ec2.internal" event="NodeHasNoDiskPressure" May 27 20:10:48.328011 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:48.327988 2571 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-142-251.ec2.internal" event="NodeHasSufficientPID" May 27 20:10:48.328852 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:48.328839 2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-251.ec2.internal" May 27 20:10:48.328897 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:48.328861 2571 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach" May 27 20:10:48.329458 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:48.329445 2571 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-142-251.ec2.internal" event="NodeHasSufficientMemory" May 27 20:10:48.329515 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:48.329464 2571 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-142-251.ec2.internal" event="NodeHasNoDiskPressure" May 27 20:10:48.329515 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:48.329472 2571 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-142-251.ec2.internal" event="NodeHasSufficientPID" May 27 20:10:48.349595 ip-10-0-142-251 kubenswrapper[2571]: E0527 20:10:48.349573 2571 kubelet.go:3336] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-142-251.ec2.internal\" not found" node="ip-10-0-142-251.ec2.internal" May 27 20:10:48.353514 ip-10-0-142-251 kubenswrapper[2571]: E0527 20:10:48.353500 2571 kubelet.go:3336] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-142-251.ec2.internal\" not found" node="ip-10-0-142-251.ec2.internal" May 27 20:10:48.385477 ip-10-0-142-251 kubenswrapper[2571]: E0527 20:10:48.385449 2571 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-142-251.ec2.internal\" not found" May 27 20:10:48.394753 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:48.394738 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/5541ecea1d6385d1bbfe3fb4b8556b5e-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-142-251.ec2.internal\" (UID: \"5541ecea1d6385d1bbfe3fb4b8556b5e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-251.ec2.internal" May 27 20:10:48.394838 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:48.394768 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5541ecea1d6385d1bbfe3fb4b8556b5e-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-142-251.ec2.internal\" (UID: \"5541ecea1d6385d1bbfe3fb4b8556b5e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-251.ec2.internal" May 27 20:10:48.394838 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:48.394795 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/e8c054cb9a59ce3874e2199aacfd5039-config\") pod \"kube-apiserver-proxy-ip-10-0-142-251.ec2.internal\" (UID: \"e8c054cb9a59ce3874e2199aacfd5039\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-142-251.ec2.internal" May 27 20:10:48.485881 ip-10-0-142-251 kubenswrapper[2571]: E0527 20:10:48.485830 2571 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-142-251.ec2.internal\" not found" May 27 20:10:48.495180 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:48.495160 2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/5541ecea1d6385d1bbfe3fb4b8556b5e-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-142-251.ec2.internal\" (UID: \"5541ecea1d6385d1bbfe3fb4b8556b5e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-251.ec2.internal" May 27 20:10:48.495245 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:48.495192 2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5541ecea1d6385d1bbfe3fb4b8556b5e-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-142-251.ec2.internal\" (UID: \"5541ecea1d6385d1bbfe3fb4b8556b5e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-251.ec2.internal" May 27 20:10:48.495245 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:48.495209 2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/e8c054cb9a59ce3874e2199aacfd5039-config\") pod \"kube-apiserver-proxy-ip-10-0-142-251.ec2.internal\" (UID: \"e8c054cb9a59ce3874e2199aacfd5039\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-142-251.ec2.internal" May 27 20:10:48.495245 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:48.495242 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/e8c054cb9a59ce3874e2199aacfd5039-config\") pod \"kube-apiserver-proxy-ip-10-0-142-251.ec2.internal\" (UID: \"e8c054cb9a59ce3874e2199aacfd5039\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-142-251.ec2.internal" May 27 20:10:48.495364 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:48.495272 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5541ecea1d6385d1bbfe3fb4b8556b5e-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-142-251.ec2.internal\" (UID: \"5541ecea1d6385d1bbfe3fb4b8556b5e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-251.ec2.internal" May 27 20:10:48.495364 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:48.495273 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/5541ecea1d6385d1bbfe3fb4b8556b5e-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-142-251.ec2.internal\" (UID: \"5541ecea1d6385d1bbfe3fb4b8556b5e\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-251.ec2.internal" May 27 20:10:48.586545 ip-10-0-142-251 kubenswrapper[2571]: E0527 20:10:48.586526 2571 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-142-251.ec2.internal\" not found" May 27 20:10:48.651972 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:48.651956 2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-251.ec2.internal" May 27 20:10:48.655903 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:48.655885 2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-251.ec2.internal" May 27 20:10:48.687608 ip-10-0-142-251 kubenswrapper[2571]: E0527 20:10:48.687586 2571 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-142-251.ec2.internal\" not found" May 27 20:10:48.788030 ip-10-0-142-251 kubenswrapper[2571]: E0527 20:10:48.788014 2571 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-142-251.ec2.internal\" not found" May 27 20:10:48.888506 ip-10-0-142-251 kubenswrapper[2571]: E0527 20:10:48.888484 2571 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-142-251.ec2.internal\" not found" May 27 20:10:48.984984 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:48.984954 2571 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" May 27 20:10:48.985556 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:48.985091 2571 reflector.go:556] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" May 27 20:10:48.985556 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:48.985132 2571 reflector.go:556] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" May 27 20:10:48.989098 ip-10-0-142-251 kubenswrapper[2571]: E0527 20:10:48.989086 2571 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-142-251.ec2.internal\" not found" May 27 20:10:49.080184 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:49.080142 2571 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-05-26 20:05:48 +0000 UTC" deadline="2027-12-19 15:18:37.6726934 +0000 UTC" May 27 20:10:49.080184 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:49.080183 2571 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13699h7m48.592514304s" May 27 20:10:49.089790 ip-10-0-142-251 kubenswrapper[2571]: E0527 20:10:49.089768 2571 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-142-251.ec2.internal\" not found" May 27 20:10:49.093442 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:49.093429 2571 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" May 27 20:10:49.104144 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:49.104119 2571 reflector.go:430] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" May 27 20:10:49.128783 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:49.128761 2571 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-mbkdg" May 27 20:10:49.135036 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:49.135020 2571 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-mbkdg" May 27 20:10:49.184381 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:49.184356 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode8c054cb9a59ce3874e2199aacfd5039.slice/crio-37f59d5784db3435441d91cb73d9eacce8d732a2b20e354979904cf8954e952b WatchSource:0}: Error finding container 37f59d5784db3435441d91cb73d9eacce8d732a2b20e354979904cf8954e952b: Status 404 returned error can't find the container with id 37f59d5784db3435441d91cb73d9eacce8d732a2b20e354979904cf8954e952b May 27 20:10:49.184629 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:49.184615 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5541ecea1d6385d1bbfe3fb4b8556b5e.slice/crio-7caae858aaae1b1176edf849107b506709342dcc7ce9a92c91ac8a7d4bb31320 WatchSource:0}: Error finding container 7caae858aaae1b1176edf849107b506709342dcc7ce9a92c91ac8a7d4bb31320: Status 404 returned error can't find the container with id 7caae858aaae1b1176edf849107b506709342dcc7ce9a92c91ac8a7d4bb31320 May 27 20:10:49.187976 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:49.187964 2571 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider May 27 20:10:49.190570 ip-10-0-142-251 kubenswrapper[2571]: E0527 20:10:49.190556 2571 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-142-251.ec2.internal\" not found" May 27 20:10:49.227195 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:49.227155 2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-251.ec2.internal" event={"ID":"5541ecea1d6385d1bbfe3fb4b8556b5e","Type":"ContainerStarted","Data":"7caae858aaae1b1176edf849107b506709342dcc7ce9a92c91ac8a7d4bb31320"} May 27 20:10:49.228073 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:49.228045 2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-251.ec2.internal" event={"ID":"e8c054cb9a59ce3874e2199aacfd5039","Type":"ContainerStarted","Data":"37f59d5784db3435441d91cb73d9eacce8d732a2b20e354979904cf8954e952b"} May 27 20:10:49.287803 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:49.287778 2571 reflector.go:430] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" May 27 20:10:49.290929 ip-10-0-142-251 kubenswrapper[2571]: E0527 20:10:49.290914 2571 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-142-251.ec2.internal\" not found" May 27 20:10:49.391417 ip-10-0-142-251 kubenswrapper[2571]: E0527 20:10:49.391373 2571 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-142-251.ec2.internal\" not found" May 27 20:10:49.491870 ip-10-0-142-251 kubenswrapper[2571]: E0527 20:10:49.491850 2571 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-142-251.ec2.internal\" not found" May 27 20:10:49.592667 ip-10-0-142-251 kubenswrapper[2571]: E0527 20:10:49.592635 2571 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-142-251.ec2.internal\" not found" May 27 20:10:49.634940 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:49.634907 2571 reflector.go:430] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" May 27 20:10:49.693093 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:49.693030 2571 kubelet.go:3340] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-251.ec2.internal" May 27 20:10:49.705870 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:49.705838 2571 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" May 27 20:10:49.707073 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:49.707051 2571 kubelet.go:3340] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-251.ec2.internal" May 27 20:10:49.716524 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:49.716501 2571 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" May 27 20:10:49.912620 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:49.912597 2571 reflector.go:430] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" May 27 20:10:49.976355 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:49.976276 2571 reflector.go:430] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" May 27 20:10:50.061962 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:50.061934 2571 apiserver.go:52] "Watching apiserver" May 27 20:10:50.068042 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:50.068013 2571 reflector.go:430] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" May 27 20:10:50.069763 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:50.069732 2571 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-image-registry/node-ca-f4p5s","openshift-multus/multus-265pt","openshift-multus/multus-additional-cni-plugins-zhh65","openshift-network-operator/iptables-alerter-gbwkk","openshift-ovn-kubernetes/ovnkube-node-rcvbq","kube-system/kube-apiserver-proxy-ip-10-0-142-251.ec2.internal","openshift-cluster-node-tuning-operator/tuned-tzz6t","openshift-dns/node-resolver-n4brm","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-251.ec2.internal","openshift-multus/network-metrics-daemon-zsx5p","openshift-network-diagnostics/network-check-target-jdzbr","kube-system/konnectivity-agent-j46mn","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zpwdj"] May 27 20:10:50.072524 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:50.072503 2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-tzz6t" May 27 20:10:50.074607 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:50.074580 2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-265pt" May 27 20:10:50.075700 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:50.075676 2571 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" May 27 20:10:50.075794 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:50.075777 2571 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-jt48q\"" May 27 20:10:50.075986 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:50.075967 2571 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" May 27 20:10:50.076949 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:50.076929 2571 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" May 27 20:10:50.077041 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:50.076934 2571 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" May 27 20:10:50.077126 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:50.077066 2571 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" May 27 20:10:50.077586 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:50.077560 2571 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-jzs29\"" May 27 20:10:50.078177 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:50.077990 2571 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" May 27 20:10:50.082029 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:50.082008 2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-zhh65" May 27 20:10:50.084235 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:50.084169 2571 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" May 27 20:10:50.084402 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:50.084380 2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-gbwkk" May 27 20:10:50.084576 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:50.084560 2571 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-4nm2p\"" May 27 20:10:50.084773 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:50.084751 2571 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" May 27 20:10:50.086701 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:50.086684 2571 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" May 27 20:10:50.086947 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:50.086730 2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-rcvbq" May 27 20:10:50.087220 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:50.087186 2571 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-kmqbn\"" May 27 20:10:50.087390 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:50.087374 2571 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" May 27 20:10:50.087548 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:50.087537 2571 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" May 27 20:10:50.089139 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:50.089118 2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-n4brm" May 27 20:10:50.089868 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:50.089850 2571 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" May 27 20:10:50.090281 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:50.090261 2571 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" May 27 20:10:50.090485 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:50.090470 2571 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-86w4z\"" May 27 20:10:50.090652 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:50.090638 2571 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" May 27 20:10:50.091276 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:50.090989 2571 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" May 27 20:10:50.091276 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:50.091023 2571 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" May 27 20:10:50.091276 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:50.091159 2571 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" May 27 20:10:50.091529 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:50.091516 2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-f4p5s" May 27 20:10:50.091671 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:50.091652 2571 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" May 27 20:10:50.092045 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:50.092022 2571 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" May 27 20:10:50.092170 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:50.092060 2571 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-mxmgh\"" May 27 20:10:50.094267 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:50.094250 2571 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" May 27 20:10:50.095214 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:50.094732 2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jdzbr" May 27 20:10:50.095214 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:50.094768 2571 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" May 27 20:10:50.095214 ip-10-0-142-251 kubenswrapper[2571]: E0527 20:10:50.094793 2571 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jdzbr" podUID="de3a4bee-c44a-4da9-833f-7e04d40c1f82" May 27 20:10:50.095214 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:50.094939 2571 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-zrv2b\"" May 27 20:10:50.095214 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:50.095032 2571 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" May 27 20:10:50.097195 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:50.097132 2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zsx5p" May 27 20:10:50.097295 ip-10-0-142-251 kubenswrapper[2571]: E0527 20:10:50.097209 2571 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zsx5p" podUID="96207b98-716e-4c63-847f-fc976f5563f2" May 27 20:10:50.097295 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:50.097239 2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-j46mn" May 27 20:10:50.099655 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:50.099627 2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zpwdj" May 27 20:10:50.099871 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:50.099855 2571 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-p949f\"" May 27 20:10:50.099975 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:50.099963 2571 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" May 27 20:10:50.100076 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:50.100058 2571 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" May 27 20:10:50.102354 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:50.102333 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/2efad6a2-ff87-479a-894d-d2397edf22d9-host-var-lib-cni-multus\") pod \"multus-265pt\" (UID: \"2efad6a2-ff87-479a-894d-d2397edf22d9\") " pod="openshift-multus/multus-265pt" May 27 20:10:50.102438 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:50.102370 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b546aece-5207-4a3f-98a5-cda0a8333121-sys\") pod \"tuned-tzz6t\" (UID: \"b546aece-5207-4a3f-98a5-cda0a8333121\") " pod="openshift-cluster-node-tuning-operator/tuned-tzz6t" May 27 20:10:50.102438 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:50.102408 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/d4f71add-cfe6-4636-a5d9-fb823d22a61d-os-release\") pod \"multus-additional-cni-plugins-zhh65\" (UID: \"d4f71add-cfe6-4636-a5d9-fb823d22a61d\") " pod="openshift-multus/multus-additional-cni-plugins-zhh65" May 27 20:10:50.102438 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:50.102435 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/2efad6a2-ff87-479a-894d-d2397edf22d9-host-run-k8s-cni-cncf-io\") pod \"multus-265pt\" (UID: \"2efad6a2-ff87-479a-894d-d2397edf22d9\") " pod="openshift-multus/multus-265pt" May 27 20:10:50.102593 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:50.102461 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/b546aece-5207-4a3f-98a5-cda0a8333121-etc-sysconfig\") pod \"tuned-tzz6t\" (UID: \"b546aece-5207-4a3f-98a5-cda0a8333121\") " pod="openshift-cluster-node-tuning-operator/tuned-tzz6t" May 27 20:10:50.102593 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:50.102500 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/d4f71add-cfe6-4636-a5d9-fb823d22a61d-tuning-conf-dir\") pod \"multus-additional-cni-plugins-zhh65\" (UID: \"d4f71add-cfe6-4636-a5d9-fb823d22a61d\") " pod="openshift-multus/multus-additional-cni-plugins-zhh65" May 27 20:10:50.102593 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:50.102522 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/d4f71add-cfe6-4636-a5d9-fb823d22a61d-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-zhh65\" (UID: \"d4f71add-cfe6-4636-a5d9-fb823d22a61d\") " pod="openshift-multus/multus-additional-cni-plugins-zhh65" May 27 20:10:50.102593 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:50.102546 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/92933172-b100-42e6-8018-dcb430647ae2-etc-openvswitch\") pod \"ovnkube-node-rcvbq\" (UID: \"92933172-b100-42e6-8018-dcb430647ae2\") " pod="openshift-ovn-kubernetes/ovnkube-node-rcvbq" May 27 20:10:50.102593 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:50.102573 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/92933172-b100-42e6-8018-dcb430647ae2-run-ovn\") pod \"ovnkube-node-rcvbq\" (UID: \"92933172-b100-42e6-8018-dcb430647ae2\") " pod="openshift-ovn-kubernetes/ovnkube-node-rcvbq" May 27 20:10:50.102748 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:50.102613 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/b546aece-5207-4a3f-98a5-cda0a8333121-etc-systemd\") pod \"tuned-tzz6t\" (UID: \"b546aece-5207-4a3f-98a5-cda0a8333121\") " pod="openshift-cluster-node-tuning-operator/tuned-tzz6t" May 27 20:10:50.102748 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:50.102675 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b546aece-5207-4a3f-98a5-cda0a8333121-var-lib-kubelet\") pod \"tuned-tzz6t\" (UID: \"b546aece-5207-4a3f-98a5-cda0a8333121\") " pod="openshift-cluster-node-tuning-operator/tuned-tzz6t" May 27 20:10:50.102748 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:50.102707 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/2efad6a2-ff87-479a-894d-d2397edf22d9-cni-binary-copy\") pod \"multus-265pt\" (UID: \"2efad6a2-ff87-479a-894d-d2397edf22d9\") " pod="openshift-multus/multus-265pt" May 27 20:10:50.102843 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:50.102753 2571 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" May 27 20:10:50.102843 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:50.102761 2571 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-8rlzw\"" May 27 20:10:50.102843 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:50.102763 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/1b900804-03e8-43bb-b424-cbc50cc6805e-iptables-alerter-script\") pod \"iptables-alerter-gbwkk\" (UID: \"1b900804-03e8-43bb-b424-cbc50cc6805e\") " pod="openshift-network-operator/iptables-alerter-gbwkk" May 27 20:10:50.102843 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:50.102710 2571 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" May 27 20:10:50.103000 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:50.102899 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/92933172-b100-42e6-8018-dcb430647ae2-ovnkube-script-lib\") pod \"ovnkube-node-rcvbq\" (UID: \"92933172-b100-42e6-8018-dcb430647ae2\") " pod="openshift-ovn-kubernetes/ovnkube-node-rcvbq" May 27 20:10:50.103000 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:50.102935 2571 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" May 27 20:10:50.103000 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:50.102927 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/b546aece-5207-4a3f-98a5-cda0a8333121-etc-modprobe-d\") pod \"tuned-tzz6t\" (UID: \"b546aece-5207-4a3f-98a5-cda0a8333121\") " pod="openshift-cluster-node-tuning-operator/tuned-tzz6t" May 27 20:10:50.103000 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:50.102973 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/b546aece-5207-4a3f-98a5-cda0a8333121-etc-sysctl-d\") pod \"tuned-tzz6t\" (UID: \"b546aece-5207-4a3f-98a5-cda0a8333121\") " pod="openshift-cluster-node-tuning-operator/tuned-tzz6t" May 27 20:10:50.103000 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:50.102996 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/b546aece-5207-4a3f-98a5-cda0a8333121-etc-tuned\") pod \"tuned-tzz6t\" (UID: \"b546aece-5207-4a3f-98a5-cda0a8333121\") " pod="openshift-cluster-node-tuning-operator/tuned-tzz6t" May 27 20:10:50.103311 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:50.103039 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/2efad6a2-ff87-479a-894d-d2397edf22d9-os-release\") pod \"multus-265pt\" (UID: \"2efad6a2-ff87-479a-894d-d2397edf22d9\") " pod="openshift-multus/multus-265pt" May 27 20:10:50.103311 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:50.103065 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/92933172-b100-42e6-8018-dcb430647ae2-log-socket\") pod \"ovnkube-node-rcvbq\" (UID: \"92933172-b100-42e6-8018-dcb430647ae2\") " pod="openshift-ovn-kubernetes/ovnkube-node-rcvbq" May 27 20:10:50.103311 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:50.103092 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/92933172-b100-42e6-8018-dcb430647ae2-host-cni-bin\") pod \"ovnkube-node-rcvbq\" (UID: \"92933172-b100-42e6-8018-dcb430647ae2\") " pod="openshift-ovn-kubernetes/ovnkube-node-rcvbq" May 27 20:10:50.103311 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:50.103153 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/d4f71add-cfe6-4636-a5d9-fb823d22a61d-cni-binary-copy\") pod \"multus-additional-cni-plugins-zhh65\" (UID: \"d4f71add-cfe6-4636-a5d9-fb823d22a61d\") " pod="openshift-multus/multus-additional-cni-plugins-zhh65" May 27 20:10:50.103311 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:50.103185 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gpj4z\" (UniqueName: \"kubernetes.io/projected/de3a4bee-c44a-4da9-833f-7e04d40c1f82-kube-api-access-gpj4z\") pod \"network-check-target-jdzbr\" (UID: \"de3a4bee-c44a-4da9-833f-7e04d40c1f82\") " pod="openshift-network-diagnostics/network-check-target-jdzbr" May 27 20:10:50.103311 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:50.103219 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/2efad6a2-ff87-479a-894d-d2397edf22d9-multus-socket-dir-parent\") pod \"multus-265pt\" (UID: \"2efad6a2-ff87-479a-894d-d2397edf22d9\") " pod="openshift-multus/multus-265pt" May 27 20:10:50.103311 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:50.103249 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/2efad6a2-ff87-479a-894d-d2397edf22d9-host-var-lib-cni-bin\") pod \"multus-265pt\" (UID: \"2efad6a2-ff87-479a-894d-d2397edf22d9\") " pod="openshift-multus/multus-265pt" May 27 20:10:50.103311 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:50.103282 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/96207b98-716e-4c63-847f-fc976f5563f2-metrics-certs\") pod \"network-metrics-daemon-zsx5p\" (UID: \"96207b98-716e-4c63-847f-fc976f5563f2\") " pod="openshift-multus/network-metrics-daemon-zsx5p" May 27 20:10:50.103603 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:50.103319 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2efad6a2-ff87-479a-894d-d2397edf22d9-system-cni-dir\") pod \"multus-265pt\" (UID: \"2efad6a2-ff87-479a-894d-d2397edf22d9\") " pod="openshift-multus/multus-265pt" May 27 20:10:50.103603 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:50.103341 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/2efad6a2-ff87-479a-894d-d2397edf22d9-cnibin\") pod \"multus-265pt\" (UID: \"2efad6a2-ff87-479a-894d-d2397edf22d9\") " pod="openshift-multus/multus-265pt" May 27 20:10:50.103603 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:50.103367 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2mcbq\" (UniqueName: \"kubernetes.io/projected/d4f71add-cfe6-4636-a5d9-fb823d22a61d-kube-api-access-2mcbq\") pod \"multus-additional-cni-plugins-zhh65\" (UID: \"d4f71add-cfe6-4636-a5d9-fb823d22a61d\") " pod="openshift-multus/multus-additional-cni-plugins-zhh65" May 27 20:10:50.103603 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:50.103391 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/92933172-b100-42e6-8018-dcb430647ae2-systemd-units\") pod \"ovnkube-node-rcvbq\" (UID: \"92933172-b100-42e6-8018-dcb430647ae2\") " pod="openshift-ovn-kubernetes/ovnkube-node-rcvbq" May 27 20:10:50.103603 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:50.103414 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b546aece-5207-4a3f-98a5-cda0a8333121-etc-kubernetes\") pod \"tuned-tzz6t\" (UID: \"b546aece-5207-4a3f-98a5-cda0a8333121\") " pod="openshift-cluster-node-tuning-operator/tuned-tzz6t" May 27 20:10:50.103603 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:50.103437 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/d4f71add-cfe6-4636-a5d9-fb823d22a61d-cnibin\") pod \"multus-additional-cni-plugins-zhh65\" (UID: \"d4f71add-cfe6-4636-a5d9-fb823d22a61d\") " pod="openshift-multus/multus-additional-cni-plugins-zhh65" May 27 20:10:50.103603 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:50.103455 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/92933172-b100-42e6-8018-dcb430647ae2-host-run-ovn-kubernetes\") pod \"ovnkube-node-rcvbq\" (UID: \"92933172-b100-42e6-8018-dcb430647ae2\") " pod="openshift-ovn-kubernetes/ovnkube-node-rcvbq" May 27 20:10:50.103603 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:50.103473 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fmxtt\" (UniqueName: \"kubernetes.io/projected/4937a11c-a8df-40eb-9a62-0b9fa6ea34ef-kube-api-access-fmxtt\") pod \"node-resolver-n4brm\" (UID: \"4937a11c-a8df-40eb-9a62-0b9fa6ea34ef\") " pod="openshift-dns/node-resolver-n4brm" May 27 20:10:50.103603 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:50.103492 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/2efad6a2-ff87-479a-894d-d2397edf22d9-hostroot\") pod \"multus-265pt\" (UID: \"2efad6a2-ff87-479a-894d-d2397edf22d9\") " pod="openshift-multus/multus-265pt" May 27 20:10:50.103603 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:50.103512 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2efad6a2-ff87-479a-894d-d2397edf22d9-etc-kubernetes\") pod \"multus-265pt\" (UID: \"2efad6a2-ff87-479a-894d-d2397edf22d9\") " pod="openshift-multus/multus-265pt" May 27 20:10:50.103603 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:50.103529 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tzvrg\" (UniqueName: \"kubernetes.io/projected/2efad6a2-ff87-479a-894d-d2397edf22d9-kube-api-access-tzvrg\") pod \"multus-265pt\" (UID: \"2efad6a2-ff87-479a-894d-d2397edf22d9\") " pod="openshift-multus/multus-265pt" May 27 20:10:50.103603 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:50.103542 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d4f71add-cfe6-4636-a5d9-fb823d22a61d-system-cni-dir\") pod \"multus-additional-cni-plugins-zhh65\" (UID: \"d4f71add-cfe6-4636-a5d9-fb823d22a61d\") " pod="openshift-multus/multus-additional-cni-plugins-zhh65" May 27 20:10:50.103603 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:50.103558 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/d4f71add-cfe6-4636-a5d9-fb823d22a61d-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-zhh65\" (UID: \"d4f71add-cfe6-4636-a5d9-fb823d22a61d\") " pod="openshift-multus/multus-additional-cni-plugins-zhh65" May 27 20:10:50.103603 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:50.103583 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/4937a11c-a8df-40eb-9a62-0b9fa6ea34ef-hosts-file\") pod \"node-resolver-n4brm\" (UID: \"4937a11c-a8df-40eb-9a62-0b9fa6ea34ef\") " pod="openshift-dns/node-resolver-n4brm" May 27 20:10:50.103603 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:50.103606 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4937a11c-a8df-40eb-9a62-0b9fa6ea34ef-tmp-dir\") pod \"node-resolver-n4brm\" (UID: \"4937a11c-a8df-40eb-9a62-0b9fa6ea34ef\") " pod="openshift-dns/node-resolver-n4brm" May 27 20:10:50.104155 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:50.103627 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b546aece-5207-4a3f-98a5-cda0a8333121-tmp\") pod \"tuned-tzz6t\" (UID: \"b546aece-5207-4a3f-98a5-cda0a8333121\") " pod="openshift-cluster-node-tuning-operator/tuned-tzz6t" May 27 20:10:50.104155 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:50.103652 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2efad6a2-ff87-479a-894d-d2397edf22d9-multus-cni-dir\") pod \"multus-265pt\" (UID: \"2efad6a2-ff87-479a-894d-d2397edf22d9\") " pod="openshift-multus/multus-265pt" May 27 20:10:50.104155 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:50.103674 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/2efad6a2-ff87-479a-894d-d2397edf22d9-host-run-netns\") pod \"multus-265pt\" (UID: \"2efad6a2-ff87-479a-894d-d2397edf22d9\") " pod="openshift-multus/multus-265pt" May 27 20:10:50.104155 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:50.103698 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/2efad6a2-ff87-479a-894d-d2397edf22d9-multus-conf-dir\") pod \"multus-265pt\" (UID: \"2efad6a2-ff87-479a-894d-d2397edf22d9\") " pod="openshift-multus/multus-265pt" May 27 20:10:50.104155 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:50.103720 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/2efad6a2-ff87-479a-894d-d2397edf22d9-host-run-multus-certs\") pod \"multus-265pt\" (UID: \"2efad6a2-ff87-479a-894d-d2397edf22d9\") " pod="openshift-multus/multus-265pt" May 27 20:10:50.104155 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:50.103757 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/92933172-b100-42e6-8018-dcb430647ae2-var-lib-openvswitch\") pod \"ovnkube-node-rcvbq\" (UID: \"92933172-b100-42e6-8018-dcb430647ae2\") " pod="openshift-ovn-kubernetes/ovnkube-node-rcvbq" May 27 20:10:50.104155 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:50.103780 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/b546aece-5207-4a3f-98a5-cda0a8333121-run\") pod \"tuned-tzz6t\" (UID: \"b546aece-5207-4a3f-98a5-cda0a8333121\") " pod="openshift-cluster-node-tuning-operator/tuned-tzz6t" May 27 20:10:50.104155 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:50.103799 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b546aece-5207-4a3f-98a5-cda0a8333121-host\") pod \"tuned-tzz6t\" (UID: \"b546aece-5207-4a3f-98a5-cda0a8333121\") " pod="openshift-cluster-node-tuning-operator/tuned-tzz6t" May 27 20:10:50.104155 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:50.103818 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/1b900804-03e8-43bb-b424-cbc50cc6805e-host-slash\") pod \"iptables-alerter-gbwkk\" (UID: \"1b900804-03e8-43bb-b424-cbc50cc6805e\") " pod="openshift-network-operator/iptables-alerter-gbwkk" May 27 20:10:50.104155 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:50.103834 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/92933172-b100-42e6-8018-dcb430647ae2-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-rcvbq\" (UID: \"92933172-b100-42e6-8018-dcb430647ae2\") " pod="openshift-ovn-kubernetes/ovnkube-node-rcvbq" May 27 20:10:50.104155 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:50.103849 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b546aece-5207-4a3f-98a5-cda0a8333121-lib-modules\") pod \"tuned-tzz6t\" (UID: \"b546aece-5207-4a3f-98a5-cda0a8333121\") " pod="openshift-cluster-node-tuning-operator/tuned-tzz6t" May 27 20:10:50.104155 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:50.103866 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mbdwg\" (UniqueName: \"kubernetes.io/projected/b546aece-5207-4a3f-98a5-cda0a8333121-kube-api-access-mbdwg\") pod \"tuned-tzz6t\" (UID: \"b546aece-5207-4a3f-98a5-cda0a8333121\") " pod="openshift-cluster-node-tuning-operator/tuned-tzz6t" May 27 20:10:50.104155 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:50.103899 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2efad6a2-ff87-479a-894d-d2397edf22d9-host-var-lib-kubelet\") pod \"multus-265pt\" (UID: \"2efad6a2-ff87-479a-894d-d2397edf22d9\") " pod="openshift-multus/multus-265pt" May 27 20:10:50.104155 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:50.103921 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cldt6\" (UniqueName: \"kubernetes.io/projected/92933172-b100-42e6-8018-dcb430647ae2-kube-api-access-cldt6\") pod \"ovnkube-node-rcvbq\" (UID: \"92933172-b100-42e6-8018-dcb430647ae2\") " pod="openshift-ovn-kubernetes/ovnkube-node-rcvbq" May 27 20:10:50.104155 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:50.103943 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/92933172-b100-42e6-8018-dcb430647ae2-run-openvswitch\") pod \"ovnkube-node-rcvbq\" (UID: \"92933172-b100-42e6-8018-dcb430647ae2\") " pod="openshift-ovn-kubernetes/ovnkube-node-rcvbq" May 27 20:10:50.104155 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:50.103968 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/cdf5a20d-c5e1-4d30-a6ce-85c91afd58e2-serviceca\") pod \"node-ca-f4p5s\" (UID: \"cdf5a20d-c5e1-4d30-a6ce-85c91afd58e2\") " pod="openshift-image-registry/node-ca-f4p5s" May 27 20:10:50.104155 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:50.104001 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pxsl5\" (UniqueName: \"kubernetes.io/projected/cdf5a20d-c5e1-4d30-a6ce-85c91afd58e2-kube-api-access-pxsl5\") pod \"node-ca-f4p5s\" (UID: \"cdf5a20d-c5e1-4d30-a6ce-85c91afd58e2\") " pod="openshift-image-registry/node-ca-f4p5s" May 27 20:10:50.104900 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:50.104027 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/2efad6a2-ff87-479a-894d-d2397edf22d9-multus-daemon-config\") pod \"multus-265pt\" (UID: \"2efad6a2-ff87-479a-894d-d2397edf22d9\") " pod="openshift-multus/multus-265pt" May 27 20:10:50.104900 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:50.104051 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/92933172-b100-42e6-8018-dcb430647ae2-host-slash\") pod \"ovnkube-node-rcvbq\" (UID: \"92933172-b100-42e6-8018-dcb430647ae2\") " pod="openshift-ovn-kubernetes/ovnkube-node-rcvbq" May 27 20:10:50.104900 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:50.104073 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/cdf5a20d-c5e1-4d30-a6ce-85c91afd58e2-host\") pod \"node-ca-f4p5s\" (UID: \"cdf5a20d-c5e1-4d30-a6ce-85c91afd58e2\") " pod="openshift-image-registry/node-ca-f4p5s" May 27 20:10:50.104900 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:50.104097 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/b546aece-5207-4a3f-98a5-cda0a8333121-etc-sysctl-conf\") pod \"tuned-tzz6t\" (UID: \"b546aece-5207-4a3f-98a5-cda0a8333121\") " pod="openshift-cluster-node-tuning-operator/tuned-tzz6t" May 27 20:10:50.104900 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:50.104141 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j9mnj\" (UniqueName: \"kubernetes.io/projected/1b900804-03e8-43bb-b424-cbc50cc6805e-kube-api-access-j9mnj\") pod \"iptables-alerter-gbwkk\" (UID: \"1b900804-03e8-43bb-b424-cbc50cc6805e\") " pod="openshift-network-operator/iptables-alerter-gbwkk" May 27 20:10:50.104900 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:50.104165 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/92933172-b100-42e6-8018-dcb430647ae2-run-systemd\") pod \"ovnkube-node-rcvbq\" (UID: \"92933172-b100-42e6-8018-dcb430647ae2\") " pod="openshift-ovn-kubernetes/ovnkube-node-rcvbq" May 27 20:10:50.104900 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:50.104189 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/92933172-b100-42e6-8018-dcb430647ae2-node-log\") pod \"ovnkube-node-rcvbq\" (UID: \"92933172-b100-42e6-8018-dcb430647ae2\") " pod="openshift-ovn-kubernetes/ovnkube-node-rcvbq" May 27 20:10:50.104900 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:50.104213 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/92933172-b100-42e6-8018-dcb430647ae2-ovnkube-config\") pod \"ovnkube-node-rcvbq\" (UID: \"92933172-b100-42e6-8018-dcb430647ae2\") " pod="openshift-ovn-kubernetes/ovnkube-node-rcvbq" May 27 20:10:50.104900 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:50.104237 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/92933172-b100-42e6-8018-dcb430647ae2-env-overrides\") pod \"ovnkube-node-rcvbq\" (UID: \"92933172-b100-42e6-8018-dcb430647ae2\") " pod="openshift-ovn-kubernetes/ovnkube-node-rcvbq" May 27 20:10:50.104900 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:50.104260 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/92933172-b100-42e6-8018-dcb430647ae2-ovn-node-metrics-cert\") pod \"ovnkube-node-rcvbq\" (UID: \"92933172-b100-42e6-8018-dcb430647ae2\") " pod="openshift-ovn-kubernetes/ovnkube-node-rcvbq" May 27 20:10:50.104900 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:50.104285 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4k76z\" (UniqueName: \"kubernetes.io/projected/96207b98-716e-4c63-847f-fc976f5563f2-kube-api-access-4k76z\") pod \"network-metrics-daemon-zsx5p\" (UID: \"96207b98-716e-4c63-847f-fc976f5563f2\") " pod="openshift-multus/network-metrics-daemon-zsx5p" May 27 20:10:50.104900 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:50.104313 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/92933172-b100-42e6-8018-dcb430647ae2-host-kubelet\") pod \"ovnkube-node-rcvbq\" (UID: \"92933172-b100-42e6-8018-dcb430647ae2\") " pod="openshift-ovn-kubernetes/ovnkube-node-rcvbq" May 27 20:10:50.104900 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:50.104337 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/92933172-b100-42e6-8018-dcb430647ae2-host-run-netns\") pod \"ovnkube-node-rcvbq\" (UID: \"92933172-b100-42e6-8018-dcb430647ae2\") " pod="openshift-ovn-kubernetes/ovnkube-node-rcvbq" May 27 20:10:50.104900 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:50.104361 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/92933172-b100-42e6-8018-dcb430647ae2-host-cni-netd\") pod \"ovnkube-node-rcvbq\" (UID: \"92933172-b100-42e6-8018-dcb430647ae2\") " pod="openshift-ovn-kubernetes/ovnkube-node-rcvbq" May 27 20:10:50.136418 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:50.136391 2571 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-05-26 20:05:49 +0000 UTC" deadline="2028-01-26 07:49:49.946263976 +0000 UTC" May 27 20:10:50.136418 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:50.136415 2571 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14603h38m59.809851275s" May 27 20:10:50.194338 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:50.194313 2571 desired_state_of_world_populator.go:158] "Finished populating initial desired state of world" May 27 20:10:50.204723 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:50.204697 2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/2efad6a2-ff87-479a-894d-d2397edf22d9-host-run-k8s-cni-cncf-io\") pod \"multus-265pt\" (UID: \"2efad6a2-ff87-479a-894d-d2397edf22d9\") " pod="openshift-multus/multus-265pt" May 27 20:10:50.204840 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:50.204734 2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/b546aece-5207-4a3f-98a5-cda0a8333121-etc-sysconfig\") pod \"tuned-tzz6t\" (UID: \"b546aece-5207-4a3f-98a5-cda0a8333121\") " pod="openshift-cluster-node-tuning-operator/tuned-tzz6t" May 27 20:10:50.204840 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:50.204764 2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/d4f71add-cfe6-4636-a5d9-fb823d22a61d-tuning-conf-dir\") pod \"multus-additional-cni-plugins-zhh65\" (UID: \"d4f71add-cfe6-4636-a5d9-fb823d22a61d\") " pod="openshift-multus/multus-additional-cni-plugins-zhh65" May 27 20:10:50.204840 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:50.204789 2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/d4f71add-cfe6-4636-a5d9-fb823d22a61d-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-zhh65\" (UID: \"d4f71add-cfe6-4636-a5d9-fb823d22a61d\") " pod="openshift-multus/multus-additional-cni-plugins-zhh65" May 27 20:10:50.204840 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:50.204805 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/2efad6a2-ff87-479a-894d-d2397edf22d9-host-run-k8s-cni-cncf-io\") pod \"multus-265pt\" (UID: \"2efad6a2-ff87-479a-894d-d2397edf22d9\") " pod="openshift-multus/multus-265pt" May 27 20:10:50.204840 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:50.204819 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/b546aece-5207-4a3f-98a5-cda0a8333121-etc-sysconfig\") pod \"tuned-tzz6t\" (UID: \"b546aece-5207-4a3f-98a5-cda0a8333121\") " pod="openshift-cluster-node-tuning-operator/tuned-tzz6t" May 27 20:10:50.204840 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:50.204814 2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/92933172-b100-42e6-8018-dcb430647ae2-etc-openvswitch\") pod \"ovnkube-node-rcvbq\" (UID: \"92933172-b100-42e6-8018-dcb430647ae2\") " pod="openshift-ovn-kubernetes/ovnkube-node-rcvbq" May 27 20:10:50.205173 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:50.204856 2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/92933172-b100-42e6-8018-dcb430647ae2-run-ovn\") pod \"ovnkube-node-rcvbq\" (UID: \"92933172-b100-42e6-8018-dcb430647ae2\") " pod="openshift-ovn-kubernetes/ovnkube-node-rcvbq" May 27 20:10:50.205173 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:50.204863 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/92933172-b100-42e6-8018-dcb430647ae2-etc-openvswitch\") pod \"ovnkube-node-rcvbq\" (UID: \"92933172-b100-42e6-8018-dcb430647ae2\") " pod="openshift-ovn-kubernetes/ovnkube-node-rcvbq" May 27 20:10:50.205173 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:50.204879 2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/b546aece-5207-4a3f-98a5-cda0a8333121-etc-systemd\") pod \"tuned-tzz6t\" (UID: \"b546aece-5207-4a3f-98a5-cda0a8333121\") " pod="openshift-cluster-node-tuning-operator/tuned-tzz6t" May 27 20:10:50.205173 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:50.204902 2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b546aece-5207-4a3f-98a5-cda0a8333121-var-lib-kubelet\") pod \"tuned-tzz6t\" (UID: \"b546aece-5207-4a3f-98a5-cda0a8333121\") " pod="openshift-cluster-node-tuning-operator/tuned-tzz6t" May 27 20:10:50.205173 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:50.204907 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/92933172-b100-42e6-8018-dcb430647ae2-run-ovn\") pod \"ovnkube-node-rcvbq\" (UID: \"92933172-b100-42e6-8018-dcb430647ae2\") " pod="openshift-ovn-kubernetes/ovnkube-node-rcvbq" May 27 20:10:50.205173 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:50.204927 2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/2efad6a2-ff87-479a-894d-d2397edf22d9-cni-binary-copy\") pod \"multus-265pt\" (UID: \"2efad6a2-ff87-479a-894d-d2397edf22d9\") " pod="openshift-multus/multus-265pt" May 27 20:10:50.205173 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:50.204951 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/b546aece-5207-4a3f-98a5-cda0a8333121-etc-systemd\") pod \"tuned-tzz6t\" (UID: \"b546aece-5207-4a3f-98a5-cda0a8333121\") " pod="openshift-cluster-node-tuning-operator/tuned-tzz6t" May 27 20:10:50.205173 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:50.204953 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/d4f71add-cfe6-4636-a5d9-fb823d22a61d-tuning-conf-dir\") pod \"multus-additional-cni-plugins-zhh65\" (UID: \"d4f71add-cfe6-4636-a5d9-fb823d22a61d\") " pod="openshift-multus/multus-additional-cni-plugins-zhh65" May 27 20:10:50.205173 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:50.204960 2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/1b900804-03e8-43bb-b424-cbc50cc6805e-iptables-alerter-script\") pod \"iptables-alerter-gbwkk\" (UID: \"1b900804-03e8-43bb-b424-cbc50cc6805e\") " pod="openshift-network-operator/iptables-alerter-gbwkk" May 27 20:10:50.205173 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:50.204982 2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/92933172-b100-42e6-8018-dcb430647ae2-ovnkube-script-lib\") pod \"ovnkube-node-rcvbq\" (UID: \"92933172-b100-42e6-8018-dcb430647ae2\") " pod="openshift-ovn-kubernetes/ovnkube-node-rcvbq" May 27 20:10:50.205173 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:50.205000 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b546aece-5207-4a3f-98a5-cda0a8333121-var-lib-kubelet\") pod \"tuned-tzz6t\" (UID: \"b546aece-5207-4a3f-98a5-cda0a8333121\") " pod="openshift-cluster-node-tuning-operator/tuned-tzz6t" May 27 20:10:50.205173 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:50.205003 2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/b546aece-5207-4a3f-98a5-cda0a8333121-etc-modprobe-d\") pod \"tuned-tzz6t\" (UID: \"b546aece-5207-4a3f-98a5-cda0a8333121\") " pod="openshift-cluster-node-tuning-operator/tuned-tzz6t" May 27 20:10:50.205173 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:50.205042 2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/b546aece-5207-4a3f-98a5-cda0a8333121-etc-sysctl-d\") pod \"tuned-tzz6t\" (UID: \"b546aece-5207-4a3f-98a5-cda0a8333121\") " pod="openshift-cluster-node-tuning-operator/tuned-tzz6t" May 27 20:10:50.205173 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:50.205070 2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/b546aece-5207-4a3f-98a5-cda0a8333121-etc-tuned\") pod \"tuned-tzz6t\" (UID: \"b546aece-5207-4a3f-98a5-cda0a8333121\") " pod="openshift-cluster-node-tuning-operator/tuned-tzz6t" May 27 20:10:50.205173 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:50.205088 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/b546aece-5207-4a3f-98a5-cda0a8333121-etc-modprobe-d\") pod \"tuned-tzz6t\" (UID: \"b546aece-5207-4a3f-98a5-cda0a8333121\") " pod="openshift-cluster-node-tuning-operator/tuned-tzz6t" May 27 20:10:50.205173 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:50.205093 2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/2efad6a2-ff87-479a-894d-d2397edf22d9-os-release\") pod \"multus-265pt\" (UID: \"2efad6a2-ff87-479a-894d-d2397edf22d9\") " pod="openshift-multus/multus-265pt" May 27 20:10:50.205905 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:50.205231 2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/92933172-b100-42e6-8018-dcb430647ae2-log-socket\") pod \"ovnkube-node-rcvbq\" (UID: \"92933172-b100-42e6-8018-dcb430647ae2\") " pod="openshift-ovn-kubernetes/ovnkube-node-rcvbq" May 27 20:10:50.205905 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:50.205259 2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/92933172-b100-42e6-8018-dcb430647ae2-host-cni-bin\") pod \"ovnkube-node-rcvbq\" (UID: \"92933172-b100-42e6-8018-dcb430647ae2\") " pod="openshift-ovn-kubernetes/ovnkube-node-rcvbq" May 27 20:10:50.205905 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:50.205283 2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/d4f71add-cfe6-4636-a5d9-fb823d22a61d-cni-binary-copy\") pod \"multus-additional-cni-plugins-zhh65\" (UID: \"d4f71add-cfe6-4636-a5d9-fb823d22a61d\") " pod="openshift-multus/multus-additional-cni-plugins-zhh65" May 27 20:10:50.205905 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:50.205308 2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-gpj4z\" (UniqueName: \"kubernetes.io/projected/de3a4bee-c44a-4da9-833f-7e04d40c1f82-kube-api-access-gpj4z\") pod \"network-check-target-jdzbr\" (UID: \"de3a4bee-c44a-4da9-833f-7e04d40c1f82\") " pod="openshift-network-diagnostics/network-check-target-jdzbr" May 27 20:10:50.205905 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:50.205335 2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/2efad6a2-ff87-479a-894d-d2397edf22d9-multus-socket-dir-parent\") pod \"multus-265pt\" (UID: \"2efad6a2-ff87-479a-894d-d2397edf22d9\") " pod="openshift-multus/multus-265pt" May 27 20:10:50.205905 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:50.205361 2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/2efad6a2-ff87-479a-894d-d2397edf22d9-host-var-lib-cni-bin\") pod \"multus-265pt\" (UID: \"2efad6a2-ff87-479a-894d-d2397edf22d9\") " pod="openshift-multus/multus-265pt" May 27 20:10:50.205905 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:50.205389 2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/96207b98-716e-4c63-847f-fc976f5563f2-metrics-certs\") pod \"network-metrics-daemon-zsx5p\" (UID: \"96207b98-716e-4c63-847f-fc976f5563f2\") " pod="openshift-multus/network-metrics-daemon-zsx5p" May 27 20:10:50.205905 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:50.205413 2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2efad6a2-ff87-479a-894d-d2397edf22d9-system-cni-dir\") pod \"multus-265pt\" (UID: \"2efad6a2-ff87-479a-894d-d2397edf22d9\") " pod="openshift-multus/multus-265pt" May 27 20:10:50.205905 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:50.205427 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/d4f71add-cfe6-4636-a5d9-fb823d22a61d-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-zhh65\" (UID: \"d4f71add-cfe6-4636-a5d9-fb823d22a61d\") " pod="openshift-multus/multus-additional-cni-plugins-zhh65" May 27 20:10:50.205905 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:50.205438 2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/2efad6a2-ff87-479a-894d-d2397edf22d9-cnibin\") pod \"multus-265pt\" (UID: \"2efad6a2-ff87-479a-894d-d2397edf22d9\") " pod="openshift-multus/multus-265pt" May 27 20:10:50.205905 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:50.205444 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/2efad6a2-ff87-479a-894d-d2397edf22d9-cni-binary-copy\") pod \"multus-265pt\" (UID: \"2efad6a2-ff87-479a-894d-d2397edf22d9\") " pod="openshift-multus/multus-265pt" May 27 20:10:50.205905 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:50.205463 2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-2mcbq\" (UniqueName: \"kubernetes.io/projected/d4f71add-cfe6-4636-a5d9-fb823d22a61d-kube-api-access-2mcbq\") pod \"multus-additional-cni-plugins-zhh65\" (UID: \"d4f71add-cfe6-4636-a5d9-fb823d22a61d\") " pod="openshift-multus/multus-additional-cni-plugins-zhh65" May 27 20:10:50.205905 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:50.205491 2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/92933172-b100-42e6-8018-dcb430647ae2-systemd-units\") pod \"ovnkube-node-rcvbq\" (UID: \"92933172-b100-42e6-8018-dcb430647ae2\") " pod="openshift-ovn-kubernetes/ovnkube-node-rcvbq" May 27 20:10:50.205905 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:50.205517 2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b546aece-5207-4a3f-98a5-cda0a8333121-etc-kubernetes\") pod \"tuned-tzz6t\" (UID: \"b546aece-5207-4a3f-98a5-cda0a8333121\") " pod="openshift-cluster-node-tuning-operator/tuned-tzz6t" May 27 20:10:50.205905 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:50.205541 2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/d4f71add-cfe6-4636-a5d9-fb823d22a61d-cnibin\") pod \"multus-additional-cni-plugins-zhh65\" (UID: \"d4f71add-cfe6-4636-a5d9-fb823d22a61d\") " pod="openshift-multus/multus-additional-cni-plugins-zhh65" May 27 20:10:50.205905 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:50.205558 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/1b900804-03e8-43bb-b424-cbc50cc6805e-iptables-alerter-script\") pod \"iptables-alerter-gbwkk\" (UID: \"1b900804-03e8-43bb-b424-cbc50cc6805e\") " pod="openshift-network-operator/iptables-alerter-gbwkk" May 27 20:10:50.205905 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:50.205567 2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/92933172-b100-42e6-8018-dcb430647ae2-host-run-ovn-kubernetes\") pod \"ovnkube-node-rcvbq\" (UID: \"92933172-b100-42e6-8018-dcb430647ae2\") " pod="openshift-ovn-kubernetes/ovnkube-node-rcvbq" May 27 20:10:50.206600 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:50.205609 2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-fmxtt\" (UniqueName: \"kubernetes.io/projected/4937a11c-a8df-40eb-9a62-0b9fa6ea34ef-kube-api-access-fmxtt\") pod \"node-resolver-n4brm\" (UID: \"4937a11c-a8df-40eb-9a62-0b9fa6ea34ef\") " pod="openshift-dns/node-resolver-n4brm" May 27 20:10:50.206600 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:50.205619 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/92933172-b100-42e6-8018-dcb430647ae2-host-run-ovn-kubernetes\") pod \"ovnkube-node-rcvbq\" (UID: \"92933172-b100-42e6-8018-dcb430647ae2\") " pod="openshift-ovn-kubernetes/ovnkube-node-rcvbq" May 27 20:10:50.206600 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:50.205639 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/5590928d-9a47-49d9-8626-780744dfd2fc-registration-dir\") pod \"aws-ebs-csi-driver-node-zpwdj\" (UID: \"5590928d-9a47-49d9-8626-780744dfd2fc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zpwdj" May 27 20:10:50.206600 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:50.205656 2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/2efad6a2-ff87-479a-894d-d2397edf22d9-hostroot\") pod \"multus-265pt\" (UID: \"2efad6a2-ff87-479a-894d-d2397edf22d9\") " pod="openshift-multus/multus-265pt" May 27 20:10:50.206600 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:50.205671 2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2efad6a2-ff87-479a-894d-d2397edf22d9-etc-kubernetes\") pod \"multus-265pt\" (UID: \"2efad6a2-ff87-479a-894d-d2397edf22d9\") " pod="openshift-multus/multus-265pt" May 27 20:10:50.206600 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:50.205671 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/2efad6a2-ff87-479a-894d-d2397edf22d9-multus-socket-dir-parent\") pod \"multus-265pt\" (UID: \"2efad6a2-ff87-479a-894d-d2397edf22d9\") " pod="openshift-multus/multus-265pt" May 27 20:10:50.206600 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:50.205717 2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-tzvrg\" (UniqueName: \"kubernetes.io/projected/2efad6a2-ff87-479a-894d-d2397edf22d9-kube-api-access-tzvrg\") pod \"multus-265pt\" (UID: \"2efad6a2-ff87-479a-894d-d2397edf22d9\") " pod="openshift-multus/multus-265pt" May 27 20:10:50.206600 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:50.205735 2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d4f71add-cfe6-4636-a5d9-fb823d22a61d-system-cni-dir\") pod \"multus-additional-cni-plugins-zhh65\" (UID: \"d4f71add-cfe6-4636-a5d9-fb823d22a61d\") " pod="openshift-multus/multus-additional-cni-plugins-zhh65" May 27 20:10:50.206600 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:50.205751 2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/d4f71add-cfe6-4636-a5d9-fb823d22a61d-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-zhh65\" (UID: \"d4f71add-cfe6-4636-a5d9-fb823d22a61d\") " pod="openshift-multus/multus-additional-cni-plugins-zhh65" May 27 20:10:50.206600 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:50.205768 2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/4937a11c-a8df-40eb-9a62-0b9fa6ea34ef-hosts-file\") pod \"node-resolver-n4brm\" (UID: \"4937a11c-a8df-40eb-9a62-0b9fa6ea34ef\") " pod="openshift-dns/node-resolver-n4brm" May 27 20:10:50.206600 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:50.205783 2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4937a11c-a8df-40eb-9a62-0b9fa6ea34ef-tmp-dir\") pod \"node-resolver-n4brm\" (UID: \"4937a11c-a8df-40eb-9a62-0b9fa6ea34ef\") " pod="openshift-dns/node-resolver-n4brm" May 27 20:10:50.206600 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:50.205796 2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b546aece-5207-4a3f-98a5-cda0a8333121-tmp\") pod \"tuned-tzz6t\" (UID: \"b546aece-5207-4a3f-98a5-cda0a8333121\") " pod="openshift-cluster-node-tuning-operator/tuned-tzz6t" May 27 20:10:50.206600 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:50.205811 2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2efad6a2-ff87-479a-894d-d2397edf22d9-multus-cni-dir\") pod \"multus-265pt\" (UID: \"2efad6a2-ff87-479a-894d-d2397edf22d9\") " pod="openshift-multus/multus-265pt" May 27 20:10:50.206600 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:50.205810 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/b546aece-5207-4a3f-98a5-cda0a8333121-etc-sysctl-d\") pod \"tuned-tzz6t\" (UID: \"b546aece-5207-4a3f-98a5-cda0a8333121\") " pod="openshift-cluster-node-tuning-operator/tuned-tzz6t" May 27 20:10:50.206600 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:50.205848 2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/2efad6a2-ff87-479a-894d-d2397edf22d9-host-run-netns\") pod \"multus-265pt\" (UID: \"2efad6a2-ff87-479a-894d-d2397edf22d9\") " pod="openshift-multus/multus-265pt" May 27 20:10:50.206600 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:50.205864 2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/2efad6a2-ff87-479a-894d-d2397edf22d9-multus-conf-dir\") pod \"multus-265pt\" (UID: \"2efad6a2-ff87-479a-894d-d2397edf22d9\") " pod="openshift-multus/multus-265pt" May 27 20:10:50.206600 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:50.205865 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/92933172-b100-42e6-8018-dcb430647ae2-host-cni-bin\") pod \"ovnkube-node-rcvbq\" (UID: \"92933172-b100-42e6-8018-dcb430647ae2\") " pod="openshift-ovn-kubernetes/ovnkube-node-rcvbq" May 27 20:10:50.206600 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:50.205879 2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/2efad6a2-ff87-479a-894d-d2397edf22d9-host-run-multus-certs\") pod \"multus-265pt\" (UID: \"2efad6a2-ff87-479a-894d-d2397edf22d9\") " pod="openshift-multus/multus-265pt" May 27 20:10:50.207297 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:50.205871 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2efad6a2-ff87-479a-894d-d2397edf22d9-system-cni-dir\") pod \"multus-265pt\" (UID: \"2efad6a2-ff87-479a-894d-d2397edf22d9\") " pod="openshift-multus/multus-265pt" May 27 20:10:50.207297 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:50.205898 2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/92933172-b100-42e6-8018-dcb430647ae2-var-lib-openvswitch\") pod \"ovnkube-node-rcvbq\" (UID: \"92933172-b100-42e6-8018-dcb430647ae2\") " pod="openshift-ovn-kubernetes/ovnkube-node-rcvbq" May 27 20:10:50.207297 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:50.205907 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/2efad6a2-ff87-479a-894d-d2397edf22d9-cnibin\") pod \"multus-265pt\" (UID: \"2efad6a2-ff87-479a-894d-d2397edf22d9\") " pod="openshift-multus/multus-265pt" May 27 20:10:50.207297 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:50.205913 2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/b546aece-5207-4a3f-98a5-cda0a8333121-run\") pod \"tuned-tzz6t\" (UID: \"b546aece-5207-4a3f-98a5-cda0a8333121\") " pod="openshift-cluster-node-tuning-operator/tuned-tzz6t" May 27 20:10:50.207297 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:50.205943 2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b546aece-5207-4a3f-98a5-cda0a8333121-host\") pod \"tuned-tzz6t\" (UID: \"b546aece-5207-4a3f-98a5-cda0a8333121\") " pod="openshift-cluster-node-tuning-operator/tuned-tzz6t" May 27 20:10:50.207297 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:50.205958 2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/1b900804-03e8-43bb-b424-cbc50cc6805e-host-slash\") pod \"iptables-alerter-gbwkk\" (UID: \"1b900804-03e8-43bb-b424-cbc50cc6805e\") " pod="openshift-network-operator/iptables-alerter-gbwkk" May 27 20:10:50.207297 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:50.205974 2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/92933172-b100-42e6-8018-dcb430647ae2-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-rcvbq\" (UID: \"92933172-b100-42e6-8018-dcb430647ae2\") " pod="openshift-ovn-kubernetes/ovnkube-node-rcvbq" May 27 20:10:50.207297 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:50.205993 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/5590928d-9a47-49d9-8626-780744dfd2fc-kubelet-dir\") pod \"aws-ebs-csi-driver-node-zpwdj\" (UID: \"5590928d-9a47-49d9-8626-780744dfd2fc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zpwdj" May 27 20:10:50.207297 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:50.205999 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/d4f71add-cfe6-4636-a5d9-fb823d22a61d-cni-binary-copy\") pod \"multus-additional-cni-plugins-zhh65\" (UID: \"d4f71add-cfe6-4636-a5d9-fb823d22a61d\") " pod="openshift-multus/multus-additional-cni-plugins-zhh65" May 27 20:10:50.207297 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:50.206008 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/5590928d-9a47-49d9-8626-780744dfd2fc-device-dir\") pod \"aws-ebs-csi-driver-node-zpwdj\" (UID: \"5590928d-9a47-49d9-8626-780744dfd2fc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zpwdj" May 27 20:10:50.207297 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:50.205715 2571 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" May 27 20:10:50.207297 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:50.206038 2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b546aece-5207-4a3f-98a5-cda0a8333121-lib-modules\") pod \"tuned-tzz6t\" (UID: \"b546aece-5207-4a3f-98a5-cda0a8333121\") " pod="openshift-cluster-node-tuning-operator/tuned-tzz6t" May 27 20:10:50.207297 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:50.206044 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/92933172-b100-42e6-8018-dcb430647ae2-systemd-units\") pod \"ovnkube-node-rcvbq\" (UID: \"92933172-b100-42e6-8018-dcb430647ae2\") " pod="openshift-ovn-kubernetes/ovnkube-node-rcvbq" May 27 20:10:50.207297 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:50.205733 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/2efad6a2-ff87-479a-894d-d2397edf22d9-host-var-lib-cni-bin\") pod \"multus-265pt\" (UID: \"2efad6a2-ff87-479a-894d-d2397edf22d9\") " pod="openshift-multus/multus-265pt" May 27 20:10:50.207297 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:50.206075 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/92933172-b100-42e6-8018-dcb430647ae2-log-socket\") pod \"ovnkube-node-rcvbq\" (UID: \"92933172-b100-42e6-8018-dcb430647ae2\") " pod="openshift-ovn-kubernetes/ovnkube-node-rcvbq" May 27 20:10:50.207297 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:50.206142 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/4937a11c-a8df-40eb-9a62-0b9fa6ea34ef-hosts-file\") pod \"node-resolver-n4brm\" (UID: \"4937a11c-a8df-40eb-9a62-0b9fa6ea34ef\") " pod="openshift-dns/node-resolver-n4brm" May 27 20:10:50.207297 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:50.206176 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/b546aece-5207-4a3f-98a5-cda0a8333121-run\") pod \"tuned-tzz6t\" (UID: \"b546aece-5207-4a3f-98a5-cda0a8333121\") " pod="openshift-cluster-node-tuning-operator/tuned-tzz6t" May 27 20:10:50.207297 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:50.206205 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/2efad6a2-ff87-479a-894d-d2397edf22d9-os-release\") pod \"multus-265pt\" (UID: \"2efad6a2-ff87-479a-894d-d2397edf22d9\") " pod="openshift-multus/multus-265pt" May 27 20:10:50.208097 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:50.206219 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/2efad6a2-ff87-479a-894d-d2397edf22d9-multus-conf-dir\") pod \"multus-265pt\" (UID: \"2efad6a2-ff87-479a-894d-d2397edf22d9\") " pod="openshift-multus/multus-265pt" May 27 20:10:50.208097 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:50.206268 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/2efad6a2-ff87-479a-894d-d2397edf22d9-hostroot\") pod \"multus-265pt\" (UID: \"2efad6a2-ff87-479a-894d-d2397edf22d9\") " pod="openshift-multus/multus-265pt" May 27 20:10:50.208097 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:50.206303 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2efad6a2-ff87-479a-894d-d2397edf22d9-etc-kubernetes\") pod \"multus-265pt\" (UID: \"2efad6a2-ff87-479a-894d-d2397edf22d9\") " pod="openshift-multus/multus-265pt" May 27 20:10:50.208097 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:50.206324 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b546aece-5207-4a3f-98a5-cda0a8333121-etc-kubernetes\") pod \"tuned-tzz6t\" (UID: \"b546aece-5207-4a3f-98a5-cda0a8333121\") " pod="openshift-cluster-node-tuning-operator/tuned-tzz6t" May 27 20:10:50.208097 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:50.206326 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b546aece-5207-4a3f-98a5-cda0a8333121-lib-modules\") pod \"tuned-tzz6t\" (UID: \"b546aece-5207-4a3f-98a5-cda0a8333121\") " pod="openshift-cluster-node-tuning-operator/tuned-tzz6t" May 27 20:10:50.208097 ip-10-0-142-251 kubenswrapper[2571]: E0527 20:10:50.205816 2571 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered May 27 20:10:50.208097 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:50.206354 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/2efad6a2-ff87-479a-894d-d2397edf22d9-host-run-multus-certs\") pod \"multus-265pt\" (UID: \"2efad6a2-ff87-479a-894d-d2397edf22d9\") " pod="openshift-multus/multus-265pt" May 27 20:10:50.208097 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:50.206388 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/92933172-b100-42e6-8018-dcb430647ae2-var-lib-openvswitch\") pod \"ovnkube-node-rcvbq\" (UID: \"92933172-b100-42e6-8018-dcb430647ae2\") " pod="openshift-ovn-kubernetes/ovnkube-node-rcvbq" May 27 20:10:50.208097 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:50.206427 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/92933172-b100-42e6-8018-dcb430647ae2-ovnkube-script-lib\") pod \"ovnkube-node-rcvbq\" (UID: \"92933172-b100-42e6-8018-dcb430647ae2\") " pod="openshift-ovn-kubernetes/ovnkube-node-rcvbq" May 27 20:10:50.208097 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:50.206516 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d4f71add-cfe6-4636-a5d9-fb823d22a61d-system-cni-dir\") pod \"multus-additional-cni-plugins-zhh65\" (UID: \"d4f71add-cfe6-4636-a5d9-fb823d22a61d\") " pod="openshift-multus/multus-additional-cni-plugins-zhh65" May 27 20:10:50.208097 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:50.206519 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2efad6a2-ff87-479a-894d-d2397edf22d9-multus-cni-dir\") pod \"multus-265pt\" (UID: \"2efad6a2-ff87-479a-894d-d2397edf22d9\") " pod="openshift-multus/multus-265pt" May 27 20:10:50.208097 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:50.206543 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b546aece-5207-4a3f-98a5-cda0a8333121-host\") pod \"tuned-tzz6t\" (UID: \"b546aece-5207-4a3f-98a5-cda0a8333121\") " pod="openshift-cluster-node-tuning-operator/tuned-tzz6t" May 27 20:10:50.208097 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:50.206550 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/1b900804-03e8-43bb-b424-cbc50cc6805e-host-slash\") pod \"iptables-alerter-gbwkk\" (UID: \"1b900804-03e8-43bb-b424-cbc50cc6805e\") " pod="openshift-network-operator/iptables-alerter-gbwkk" May 27 20:10:50.208097 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:50.206568 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/92933172-b100-42e6-8018-dcb430647ae2-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-rcvbq\" (UID: \"92933172-b100-42e6-8018-dcb430647ae2\") " pod="openshift-ovn-kubernetes/ovnkube-node-rcvbq" May 27 20:10:50.208097 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:50.206585 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/d4f71add-cfe6-4636-a5d9-fb823d22a61d-cnibin\") pod \"multus-additional-cni-plugins-zhh65\" (UID: \"d4f71add-cfe6-4636-a5d9-fb823d22a61d\") " pod="openshift-multus/multus-additional-cni-plugins-zhh65" May 27 20:10:50.208097 ip-10-0-142-251 kubenswrapper[2571]: E0527 20:10:50.206593 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/96207b98-716e-4c63-847f-fc976f5563f2-metrics-certs podName:96207b98-716e-4c63-847f-fc976f5563f2 nodeName:}" failed. No retries permitted until 2026-05-27 20:10:50.706573705 +0000 UTC m=+3.116200031 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/96207b98-716e-4c63-847f-fc976f5563f2-metrics-certs") pod "network-metrics-daemon-zsx5p" (UID: "96207b98-716e-4c63-847f-fc976f5563f2") : object "openshift-multus"/"metrics-daemon-secret" not registered May 27 20:10:50.208097 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:50.206616 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/2efad6a2-ff87-479a-894d-d2397edf22d9-host-run-netns\") pod \"multus-265pt\" (UID: \"2efad6a2-ff87-479a-894d-d2397edf22d9\") " pod="openshift-multus/multus-265pt" May 27 20:10:50.208840 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:50.206644 2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-mbdwg\" (UniqueName: \"kubernetes.io/projected/b546aece-5207-4a3f-98a5-cda0a8333121-kube-api-access-mbdwg\") pod \"tuned-tzz6t\" (UID: \"b546aece-5207-4a3f-98a5-cda0a8333121\") " pod="openshift-cluster-node-tuning-operator/tuned-tzz6t" May 27 20:10:50.208840 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:50.206796 2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2efad6a2-ff87-479a-894d-d2397edf22d9-host-var-lib-kubelet\") pod \"multus-265pt\" (UID: \"2efad6a2-ff87-479a-894d-d2397edf22d9\") " pod="openshift-multus/multus-265pt" May 27 20:10:50.208840 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:50.206826 2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-cldt6\" (UniqueName: \"kubernetes.io/projected/92933172-b100-42e6-8018-dcb430647ae2-kube-api-access-cldt6\") pod \"ovnkube-node-rcvbq\" (UID: \"92933172-b100-42e6-8018-dcb430647ae2\") " pod="openshift-ovn-kubernetes/ovnkube-node-rcvbq" May 27 20:10:50.208840 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:50.206856 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8msc9\" (UniqueName: \"kubernetes.io/projected/5590928d-9a47-49d9-8626-780744dfd2fc-kube-api-access-8msc9\") pod \"aws-ebs-csi-driver-node-zpwdj\" (UID: \"5590928d-9a47-49d9-8626-780744dfd2fc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zpwdj" May 27 20:10:50.208840 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:50.206882 2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/92933172-b100-42e6-8018-dcb430647ae2-run-openvswitch\") pod \"ovnkube-node-rcvbq\" (UID: \"92933172-b100-42e6-8018-dcb430647ae2\") " pod="openshift-ovn-kubernetes/ovnkube-node-rcvbq" May 27 20:10:50.208840 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:50.206992 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4937a11c-a8df-40eb-9a62-0b9fa6ea34ef-tmp-dir\") pod \"node-resolver-n4brm\" (UID: \"4937a11c-a8df-40eb-9a62-0b9fa6ea34ef\") " pod="openshift-dns/node-resolver-n4brm" May 27 20:10:50.208840 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:50.206997 2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/cdf5a20d-c5e1-4d30-a6ce-85c91afd58e2-serviceca\") pod \"node-ca-f4p5s\" (UID: \"cdf5a20d-c5e1-4d30-a6ce-85c91afd58e2\") " pod="openshift-image-registry/node-ca-f4p5s" May 27 20:10:50.208840 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:50.207013 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2efad6a2-ff87-479a-894d-d2397edf22d9-host-var-lib-kubelet\") pod \"multus-265pt\" (UID: \"2efad6a2-ff87-479a-894d-d2397edf22d9\") " pod="openshift-multus/multus-265pt" May 27 20:10:50.208840 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:50.207035 2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-pxsl5\" (UniqueName: \"kubernetes.io/projected/cdf5a20d-c5e1-4d30-a6ce-85c91afd58e2-kube-api-access-pxsl5\") pod \"node-ca-f4p5s\" (UID: \"cdf5a20d-c5e1-4d30-a6ce-85c91afd58e2\") " pod="openshift-image-registry/node-ca-f4p5s" May 27 20:10:50.208840 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:50.206997 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/92933172-b100-42e6-8018-dcb430647ae2-run-openvswitch\") pod \"ovnkube-node-rcvbq\" (UID: \"92933172-b100-42e6-8018-dcb430647ae2\") " pod="openshift-ovn-kubernetes/ovnkube-node-rcvbq" May 27 20:10:50.208840 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:50.207077 2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/2efad6a2-ff87-479a-894d-d2397edf22d9-multus-daemon-config\") pod \"multus-265pt\" (UID: \"2efad6a2-ff87-479a-894d-d2397edf22d9\") " pod="openshift-multus/multus-265pt" May 27 20:10:50.208840 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:50.207130 2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/92933172-b100-42e6-8018-dcb430647ae2-host-slash\") pod \"ovnkube-node-rcvbq\" (UID: \"92933172-b100-42e6-8018-dcb430647ae2\") " pod="openshift-ovn-kubernetes/ovnkube-node-rcvbq" May 27 20:10:50.208840 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:50.207158 2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/cdf5a20d-c5e1-4d30-a6ce-85c91afd58e2-host\") pod \"node-ca-f4p5s\" (UID: \"cdf5a20d-c5e1-4d30-a6ce-85c91afd58e2\") " pod="openshift-image-registry/node-ca-f4p5s" May 27 20:10:50.208840 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:50.207187 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/5590928d-9a47-49d9-8626-780744dfd2fc-socket-dir\") pod \"aws-ebs-csi-driver-node-zpwdj\" (UID: \"5590928d-9a47-49d9-8626-780744dfd2fc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zpwdj" May 27 20:10:50.208840 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:50.207215 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/5590928d-9a47-49d9-8626-780744dfd2fc-etc-selinux\") pod \"aws-ebs-csi-driver-node-zpwdj\" (UID: \"5590928d-9a47-49d9-8626-780744dfd2fc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zpwdj" May 27 20:10:50.208840 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:50.207239 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/5590928d-9a47-49d9-8626-780744dfd2fc-sys-fs\") pod \"aws-ebs-csi-driver-node-zpwdj\" (UID: \"5590928d-9a47-49d9-8626-780744dfd2fc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zpwdj" May 27 20:10:50.208840 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:50.207246 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/92933172-b100-42e6-8018-dcb430647ae2-host-slash\") pod \"ovnkube-node-rcvbq\" (UID: \"92933172-b100-42e6-8018-dcb430647ae2\") " pod="openshift-ovn-kubernetes/ovnkube-node-rcvbq" May 27 20:10:50.209491 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:50.207254 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/cdf5a20d-c5e1-4d30-a6ce-85c91afd58e2-host\") pod \"node-ca-f4p5s\" (UID: \"cdf5a20d-c5e1-4d30-a6ce-85c91afd58e2\") " pod="openshift-image-registry/node-ca-f4p5s" May 27 20:10:50.209491 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:50.207264 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/27613ab0-7fd5-4d50-a40c-58aa4b58ff6a-agent-certs\") pod \"konnectivity-agent-j46mn\" (UID: \"27613ab0-7fd5-4d50-a40c-58aa4b58ff6a\") " pod="kube-system/konnectivity-agent-j46mn" May 27 20:10:50.209491 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:50.207291 2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/b546aece-5207-4a3f-98a5-cda0a8333121-etc-sysctl-conf\") pod \"tuned-tzz6t\" (UID: \"b546aece-5207-4a3f-98a5-cda0a8333121\") " pod="openshift-cluster-node-tuning-operator/tuned-tzz6t" May 27 20:10:50.209491 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:50.207319 2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-j9mnj\" (UniqueName: \"kubernetes.io/projected/1b900804-03e8-43bb-b424-cbc50cc6805e-kube-api-access-j9mnj\") pod \"iptables-alerter-gbwkk\" (UID: \"1b900804-03e8-43bb-b424-cbc50cc6805e\") " pod="openshift-network-operator/iptables-alerter-gbwkk" May 27 20:10:50.209491 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:50.207342 2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/92933172-b100-42e6-8018-dcb430647ae2-run-systemd\") pod \"ovnkube-node-rcvbq\" (UID: \"92933172-b100-42e6-8018-dcb430647ae2\") " pod="openshift-ovn-kubernetes/ovnkube-node-rcvbq" May 27 20:10:50.209491 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:50.207364 2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/92933172-b100-42e6-8018-dcb430647ae2-node-log\") pod \"ovnkube-node-rcvbq\" (UID: \"92933172-b100-42e6-8018-dcb430647ae2\") " pod="openshift-ovn-kubernetes/ovnkube-node-rcvbq" May 27 20:10:50.209491 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:50.207386 2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/92933172-b100-42e6-8018-dcb430647ae2-ovnkube-config\") pod \"ovnkube-node-rcvbq\" (UID: \"92933172-b100-42e6-8018-dcb430647ae2\") " pod="openshift-ovn-kubernetes/ovnkube-node-rcvbq" May 27 20:10:50.209491 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:50.207392 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/b546aece-5207-4a3f-98a5-cda0a8333121-etc-sysctl-conf\") pod \"tuned-tzz6t\" (UID: \"b546aece-5207-4a3f-98a5-cda0a8333121\") " pod="openshift-cluster-node-tuning-operator/tuned-tzz6t" May 27 20:10:50.209491 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:50.207408 2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/92933172-b100-42e6-8018-dcb430647ae2-env-overrides\") pod \"ovnkube-node-rcvbq\" (UID: \"92933172-b100-42e6-8018-dcb430647ae2\") " pod="openshift-ovn-kubernetes/ovnkube-node-rcvbq" May 27 20:10:50.209491 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:50.207431 2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/92933172-b100-42e6-8018-dcb430647ae2-ovn-node-metrics-cert\") pod \"ovnkube-node-rcvbq\" (UID: \"92933172-b100-42e6-8018-dcb430647ae2\") " pod="openshift-ovn-kubernetes/ovnkube-node-rcvbq" May 27 20:10:50.209491 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:50.207438 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/92933172-b100-42e6-8018-dcb430647ae2-node-log\") pod \"ovnkube-node-rcvbq\" (UID: \"92933172-b100-42e6-8018-dcb430647ae2\") " pod="openshift-ovn-kubernetes/ovnkube-node-rcvbq" May 27 20:10:50.209491 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:50.207479 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/92933172-b100-42e6-8018-dcb430647ae2-run-systemd\") pod \"ovnkube-node-rcvbq\" (UID: \"92933172-b100-42e6-8018-dcb430647ae2\") " pod="openshift-ovn-kubernetes/ovnkube-node-rcvbq" May 27 20:10:50.209491 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:50.207515 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/cdf5a20d-c5e1-4d30-a6ce-85c91afd58e2-serviceca\") pod \"node-ca-f4p5s\" (UID: \"cdf5a20d-c5e1-4d30-a6ce-85c91afd58e2\") " pod="openshift-image-registry/node-ca-f4p5s" May 27 20:10:50.209491 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:50.207606 2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-4k76z\" (UniqueName: \"kubernetes.io/projected/96207b98-716e-4c63-847f-fc976f5563f2-kube-api-access-4k76z\") pod \"network-metrics-daemon-zsx5p\" (UID: \"96207b98-716e-4c63-847f-fc976f5563f2\") " pod="openshift-multus/network-metrics-daemon-zsx5p" May 27 20:10:50.209491 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:50.207650 2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/92933172-b100-42e6-8018-dcb430647ae2-host-kubelet\") pod \"ovnkube-node-rcvbq\" (UID: \"92933172-b100-42e6-8018-dcb430647ae2\") " pod="openshift-ovn-kubernetes/ovnkube-node-rcvbq" May 27 20:10:50.209491 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:50.207675 2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/92933172-b100-42e6-8018-dcb430647ae2-host-run-netns\") pod \"ovnkube-node-rcvbq\" (UID: \"92933172-b100-42e6-8018-dcb430647ae2\") " pod="openshift-ovn-kubernetes/ovnkube-node-rcvbq" May 27 20:10:50.209491 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:50.207700 2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/92933172-b100-42e6-8018-dcb430647ae2-host-cni-netd\") pod \"ovnkube-node-rcvbq\" (UID: \"92933172-b100-42e6-8018-dcb430647ae2\") " pod="openshift-ovn-kubernetes/ovnkube-node-rcvbq" May 27 20:10:50.209491 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:50.207726 2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/2efad6a2-ff87-479a-894d-d2397edf22d9-host-var-lib-cni-multus\") pod \"multus-265pt\" (UID: \"2efad6a2-ff87-479a-894d-d2397edf22d9\") " pod="openshift-multus/multus-265pt" May 27 20:10:50.210266 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:50.207748 2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b546aece-5207-4a3f-98a5-cda0a8333121-sys\") pod \"tuned-tzz6t\" (UID: \"b546aece-5207-4a3f-98a5-cda0a8333121\") " pod="openshift-cluster-node-tuning-operator/tuned-tzz6t" May 27 20:10:50.210266 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:50.207772 2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/d4f71add-cfe6-4636-a5d9-fb823d22a61d-os-release\") pod \"multus-additional-cni-plugins-zhh65\" (UID: \"d4f71add-cfe6-4636-a5d9-fb823d22a61d\") " pod="openshift-multus/multus-additional-cni-plugins-zhh65" May 27 20:10:50.210266 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:50.207798 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/27613ab0-7fd5-4d50-a40c-58aa4b58ff6a-konnectivity-ca\") pod \"konnectivity-agent-j46mn\" (UID: \"27613ab0-7fd5-4d50-a40c-58aa4b58ff6a\") " pod="kube-system/konnectivity-agent-j46mn" May 27 20:10:50.210266 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:50.207890 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/92933172-b100-42e6-8018-dcb430647ae2-host-kubelet\") pod \"ovnkube-node-rcvbq\" (UID: \"92933172-b100-42e6-8018-dcb430647ae2\") " pod="openshift-ovn-kubernetes/ovnkube-node-rcvbq" May 27 20:10:50.210266 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:50.207923 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/92933172-b100-42e6-8018-dcb430647ae2-host-run-netns\") pod \"ovnkube-node-rcvbq\" (UID: \"92933172-b100-42e6-8018-dcb430647ae2\") " pod="openshift-ovn-kubernetes/ovnkube-node-rcvbq" May 27 20:10:50.210266 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:50.207954 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/92933172-b100-42e6-8018-dcb430647ae2-ovnkube-config\") pod \"ovnkube-node-rcvbq\" (UID: \"92933172-b100-42e6-8018-dcb430647ae2\") " pod="openshift-ovn-kubernetes/ovnkube-node-rcvbq" May 27 20:10:50.210266 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:50.207962 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/92933172-b100-42e6-8018-dcb430647ae2-host-cni-netd\") pod \"ovnkube-node-rcvbq\" (UID: \"92933172-b100-42e6-8018-dcb430647ae2\") " pod="openshift-ovn-kubernetes/ovnkube-node-rcvbq" May 27 20:10:50.210266 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:50.207960 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/92933172-b100-42e6-8018-dcb430647ae2-env-overrides\") pod \"ovnkube-node-rcvbq\" (UID: \"92933172-b100-42e6-8018-dcb430647ae2\") " pod="openshift-ovn-kubernetes/ovnkube-node-rcvbq" May 27 20:10:50.210266 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:50.208013 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b546aece-5207-4a3f-98a5-cda0a8333121-sys\") pod \"tuned-tzz6t\" (UID: \"b546aece-5207-4a3f-98a5-cda0a8333121\") " pod="openshift-cluster-node-tuning-operator/tuned-tzz6t" May 27 20:10:50.210266 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:50.208024 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/2efad6a2-ff87-479a-894d-d2397edf22d9-host-var-lib-cni-multus\") pod \"multus-265pt\" (UID: \"2efad6a2-ff87-479a-894d-d2397edf22d9\") " pod="openshift-multus/multus-265pt" May 27 20:10:50.210266 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:50.208031 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/d4f71add-cfe6-4636-a5d9-fb823d22a61d-os-release\") pod \"multus-additional-cni-plugins-zhh65\" (UID: \"d4f71add-cfe6-4636-a5d9-fb823d22a61d\") " pod="openshift-multus/multus-additional-cni-plugins-zhh65" May 27 20:10:50.210266 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:50.208078 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/d4f71add-cfe6-4636-a5d9-fb823d22a61d-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-zhh65\" (UID: \"d4f71add-cfe6-4636-a5d9-fb823d22a61d\") " pod="openshift-multus/multus-additional-cni-plugins-zhh65" May 27 20:10:50.210266 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:50.208279 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/2efad6a2-ff87-479a-894d-d2397edf22d9-multus-daemon-config\") pod \"multus-265pt\" (UID: \"2efad6a2-ff87-479a-894d-d2397edf22d9\") " pod="openshift-multus/multus-265pt" May 27 20:10:50.210266 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:50.209211 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b546aece-5207-4a3f-98a5-cda0a8333121-tmp\") pod \"tuned-tzz6t\" (UID: \"b546aece-5207-4a3f-98a5-cda0a8333121\") " pod="openshift-cluster-node-tuning-operator/tuned-tzz6t" May 27 20:10:50.210886 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:50.210437 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/92933172-b100-42e6-8018-dcb430647ae2-ovn-node-metrics-cert\") pod \"ovnkube-node-rcvbq\" (UID: \"92933172-b100-42e6-8018-dcb430647ae2\") " pod="openshift-ovn-kubernetes/ovnkube-node-rcvbq" May 27 20:10:50.213469 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:50.213450 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/b546aece-5207-4a3f-98a5-cda0a8333121-etc-tuned\") pod \"tuned-tzz6t\" (UID: \"b546aece-5207-4a3f-98a5-cda0a8333121\") " pod="openshift-cluster-node-tuning-operator/tuned-tzz6t" May 27 20:10:50.216391 ip-10-0-142-251 kubenswrapper[2571]: E0527 20:10:50.215908 2571 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered May 27 20:10:50.216391 ip-10-0-142-251 kubenswrapper[2571]: E0527 20:10:50.215932 2571 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered May 27 20:10:50.216391 ip-10-0-142-251 kubenswrapper[2571]: E0527 20:10:50.215945 2571 projected.go:194] Error preparing data for projected volume kube-api-access-gpj4z for pod openshift-network-diagnostics/network-check-target-jdzbr: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] May 27 20:10:50.216391 ip-10-0-142-251 kubenswrapper[2571]: E0527 20:10:50.216013 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/de3a4bee-c44a-4da9-833f-7e04d40c1f82-kube-api-access-gpj4z podName:de3a4bee-c44a-4da9-833f-7e04d40c1f82 nodeName:}" failed. No retries permitted until 2026-05-27 20:10:50.715989351 +0000 UTC m=+3.125615657 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-gpj4z" (UniqueName: "kubernetes.io/projected/de3a4bee-c44a-4da9-833f-7e04d40c1f82-kube-api-access-gpj4z") pod "network-check-target-jdzbr" (UID: "de3a4bee-c44a-4da9-833f-7e04d40c1f82") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] May 27 20:10:50.219771 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:50.217786 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cldt6\" (UniqueName: \"kubernetes.io/projected/92933172-b100-42e6-8018-dcb430647ae2-kube-api-access-cldt6\") pod \"ovnkube-node-rcvbq\" (UID: \"92933172-b100-42e6-8018-dcb430647ae2\") " pod="openshift-ovn-kubernetes/ovnkube-node-rcvbq" May 27 20:10:50.219771 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:50.218062 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4k76z\" (UniqueName: \"kubernetes.io/projected/96207b98-716e-4c63-847f-fc976f5563f2-kube-api-access-4k76z\") pod \"network-metrics-daemon-zsx5p\" (UID: \"96207b98-716e-4c63-847f-fc976f5563f2\") " pod="openshift-multus/network-metrics-daemon-zsx5p" May 27 20:10:50.220780 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:50.220754 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tzvrg\" (UniqueName: \"kubernetes.io/projected/2efad6a2-ff87-479a-894d-d2397edf22d9-kube-api-access-tzvrg\") pod \"multus-265pt\" (UID: \"2efad6a2-ff87-479a-894d-d2397edf22d9\") " pod="openshift-multus/multus-265pt" May 27 20:10:50.220780 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:50.220769 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pxsl5\" (UniqueName: \"kubernetes.io/projected/cdf5a20d-c5e1-4d30-a6ce-85c91afd58e2-kube-api-access-pxsl5\") pod \"node-ca-f4p5s\" (UID: \"cdf5a20d-c5e1-4d30-a6ce-85c91afd58e2\") " pod="openshift-image-registry/node-ca-f4p5s" May 27 20:10:50.221550 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:50.221528 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j9mnj\" (UniqueName: \"kubernetes.io/projected/1b900804-03e8-43bb-b424-cbc50cc6805e-kube-api-access-j9mnj\") pod \"iptables-alerter-gbwkk\" (UID: \"1b900804-03e8-43bb-b424-cbc50cc6805e\") " pod="openshift-network-operator/iptables-alerter-gbwkk" May 27 20:10:50.221762 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:50.221566 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2mcbq\" (UniqueName: \"kubernetes.io/projected/d4f71add-cfe6-4636-a5d9-fb823d22a61d-kube-api-access-2mcbq\") pod \"multus-additional-cni-plugins-zhh65\" (UID: \"d4f71add-cfe6-4636-a5d9-fb823d22a61d\") " pod="openshift-multus/multus-additional-cni-plugins-zhh65" May 27 20:10:50.221762 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:50.221591 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mbdwg\" (UniqueName: \"kubernetes.io/projected/b546aece-5207-4a3f-98a5-cda0a8333121-kube-api-access-mbdwg\") pod \"tuned-tzz6t\" (UID: \"b546aece-5207-4a3f-98a5-cda0a8333121\") " pod="openshift-cluster-node-tuning-operator/tuned-tzz6t" May 27 20:10:50.227414 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:50.226177 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fmxtt\" (UniqueName: \"kubernetes.io/projected/4937a11c-a8df-40eb-9a62-0b9fa6ea34ef-kube-api-access-fmxtt\") pod \"node-resolver-n4brm\" (UID: \"4937a11c-a8df-40eb-9a62-0b9fa6ea34ef\") " pod="openshift-dns/node-resolver-n4brm" May 27 20:10:50.308424 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:50.308392 2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/5590928d-9a47-49d9-8626-780744dfd2fc-kubelet-dir\") pod \"aws-ebs-csi-driver-node-zpwdj\" (UID: \"5590928d-9a47-49d9-8626-780744dfd2fc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zpwdj" May 27 20:10:50.308566 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:50.308438 2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/5590928d-9a47-49d9-8626-780744dfd2fc-device-dir\") pod \"aws-ebs-csi-driver-node-zpwdj\" (UID: \"5590928d-9a47-49d9-8626-780744dfd2fc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zpwdj" May 27 20:10:50.308566 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:50.308467 2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-8msc9\" (UniqueName: \"kubernetes.io/projected/5590928d-9a47-49d9-8626-780744dfd2fc-kube-api-access-8msc9\") pod \"aws-ebs-csi-driver-node-zpwdj\" (UID: \"5590928d-9a47-49d9-8626-780744dfd2fc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zpwdj" May 27 20:10:50.308566 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:50.308499 2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/5590928d-9a47-49d9-8626-780744dfd2fc-socket-dir\") pod \"aws-ebs-csi-driver-node-zpwdj\" (UID: \"5590928d-9a47-49d9-8626-780744dfd2fc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zpwdj" May 27 20:10:50.308566 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:50.308522 2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/5590928d-9a47-49d9-8626-780744dfd2fc-etc-selinux\") pod \"aws-ebs-csi-driver-node-zpwdj\" (UID: \"5590928d-9a47-49d9-8626-780744dfd2fc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zpwdj" May 27 20:10:50.308566 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:50.308546 2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/5590928d-9a47-49d9-8626-780744dfd2fc-sys-fs\") pod \"aws-ebs-csi-driver-node-zpwdj\" (UID: \"5590928d-9a47-49d9-8626-780744dfd2fc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zpwdj" May 27 20:10:50.308566 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:50.308518 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/5590928d-9a47-49d9-8626-780744dfd2fc-kubelet-dir\") pod \"aws-ebs-csi-driver-node-zpwdj\" (UID: \"5590928d-9a47-49d9-8626-780744dfd2fc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zpwdj" May 27 20:10:50.308876 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:50.308578 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/5590928d-9a47-49d9-8626-780744dfd2fc-device-dir\") pod \"aws-ebs-csi-driver-node-zpwdj\" (UID: \"5590928d-9a47-49d9-8626-780744dfd2fc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zpwdj" May 27 20:10:50.308876 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:50.308591 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/5590928d-9a47-49d9-8626-780744dfd2fc-etc-selinux\") pod \"aws-ebs-csi-driver-node-zpwdj\" (UID: \"5590928d-9a47-49d9-8626-780744dfd2fc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zpwdj" May 27 20:10:50.308876 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:50.308598 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/5590928d-9a47-49d9-8626-780744dfd2fc-sys-fs\") pod \"aws-ebs-csi-driver-node-zpwdj\" (UID: \"5590928d-9a47-49d9-8626-780744dfd2fc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zpwdj" May 27 20:10:50.308876 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:50.308624 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/5590928d-9a47-49d9-8626-780744dfd2fc-socket-dir\") pod \"aws-ebs-csi-driver-node-zpwdj\" (UID: \"5590928d-9a47-49d9-8626-780744dfd2fc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zpwdj" May 27 20:10:50.308876 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:50.308698 2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/27613ab0-7fd5-4d50-a40c-58aa4b58ff6a-agent-certs\") pod \"konnectivity-agent-j46mn\" (UID: \"27613ab0-7fd5-4d50-a40c-58aa4b58ff6a\") " pod="kube-system/konnectivity-agent-j46mn" May 27 20:10:50.308876 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:50.308758 2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/27613ab0-7fd5-4d50-a40c-58aa4b58ff6a-konnectivity-ca\") pod \"konnectivity-agent-j46mn\" (UID: \"27613ab0-7fd5-4d50-a40c-58aa4b58ff6a\") " pod="kube-system/konnectivity-agent-j46mn" May 27 20:10:50.308876 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:50.308840 2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/5590928d-9a47-49d9-8626-780744dfd2fc-registration-dir\") pod \"aws-ebs-csi-driver-node-zpwdj\" (UID: \"5590928d-9a47-49d9-8626-780744dfd2fc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zpwdj" May 27 20:10:50.309171 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:50.308930 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/5590928d-9a47-49d9-8626-780744dfd2fc-registration-dir\") pod \"aws-ebs-csi-driver-node-zpwdj\" (UID: \"5590928d-9a47-49d9-8626-780744dfd2fc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zpwdj" May 27 20:10:50.310193 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:50.310169 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/27613ab0-7fd5-4d50-a40c-58aa4b58ff6a-konnectivity-ca\") pod \"konnectivity-agent-j46mn\" (UID: \"27613ab0-7fd5-4d50-a40c-58aa4b58ff6a\") " pod="kube-system/konnectivity-agent-j46mn" May 27 20:10:50.311607 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:50.311581 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/27613ab0-7fd5-4d50-a40c-58aa4b58ff6a-agent-certs\") pod \"konnectivity-agent-j46mn\" (UID: \"27613ab0-7fd5-4d50-a40c-58aa4b58ff6a\") " pod="kube-system/konnectivity-agent-j46mn" May 27 20:10:50.317117 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:50.317084 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8msc9\" (UniqueName: \"kubernetes.io/projected/5590928d-9a47-49d9-8626-780744dfd2fc-kube-api-access-8msc9\") pod \"aws-ebs-csi-driver-node-zpwdj\" (UID: \"5590928d-9a47-49d9-8626-780744dfd2fc\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zpwdj" May 27 20:10:50.385945 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:50.385917 2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-tzz6t" May 27 20:10:50.396704 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:50.396686 2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-265pt" May 27 20:10:50.408042 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:50.408016 2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-zhh65" May 27 20:10:50.412676 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:50.412655 2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-j46mn" May 27 20:10:50.422186 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:50.422163 2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-gbwkk" May 27 20:10:50.428824 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:50.428805 2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-rcvbq" May 27 20:10:50.435366 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:50.435348 2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-n4brm" May 27 20:10:50.442831 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:50.442813 2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-f4p5s" May 27 20:10:50.447382 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:50.447365 2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zpwdj" May 27 20:10:50.712072 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:50.712047 2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/96207b98-716e-4c63-847f-fc976f5563f2-metrics-certs\") pod \"network-metrics-daemon-zsx5p\" (UID: \"96207b98-716e-4c63-847f-fc976f5563f2\") " pod="openshift-multus/network-metrics-daemon-zsx5p" May 27 20:10:50.712235 ip-10-0-142-251 kubenswrapper[2571]: E0527 20:10:50.712177 2571 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered May 27 20:10:50.712318 ip-10-0-142-251 kubenswrapper[2571]: E0527 20:10:50.712243 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/96207b98-716e-4c63-847f-fc976f5563f2-metrics-certs podName:96207b98-716e-4c63-847f-fc976f5563f2 nodeName:}" failed. No retries permitted until 2026-05-27 20:10:51.712223971 +0000 UTC m=+4.121850265 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/96207b98-716e-4c63-847f-fc976f5563f2-metrics-certs") pod "network-metrics-daemon-zsx5p" (UID: "96207b98-716e-4c63-847f-fc976f5563f2") : object "openshift-multus"/"metrics-daemon-secret" not registered May 27 20:10:50.766954 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:50.766923 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1b900804_03e8_43bb_b424_cbc50cc6805e.slice/crio-f27f0c5760388d0c2d2ce37a6a333256d173b01c3c24bd5c2bcdada5c2c3cc23 WatchSource:0}: Error finding container f27f0c5760388d0c2d2ce37a6a333256d173b01c3c24bd5c2bcdada5c2c3cc23: Status 404 returned error can't find the container with id f27f0c5760388d0c2d2ce37a6a333256d173b01c3c24bd5c2bcdada5c2c3cc23 May 27 20:10:50.768161 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:50.768130 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcdf5a20d_c5e1_4d30_a6ce_85c91afd58e2.slice/crio-a516985a52471d370e42c980a1a2b2b72dda900af7411581809421ca705c4b96 WatchSource:0}: Error finding container a516985a52471d370e42c980a1a2b2b72dda900af7411581809421ca705c4b96: Status 404 returned error can't find the container with id a516985a52471d370e42c980a1a2b2b72dda900af7411581809421ca705c4b96 May 27 20:10:50.769131 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:50.769026 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod27613ab0_7fd5_4d50_a40c_58aa4b58ff6a.slice/crio-9b3be142920e18297455eeb05c244bff4d786b879cc768ae6ccff5220da00f39 WatchSource:0}: Error finding container 9b3be142920e18297455eeb05c244bff4d786b879cc768ae6ccff5220da00f39: Status 404 returned error can't find the container with id 9b3be142920e18297455eeb05c244bff4d786b879cc768ae6ccff5220da00f39 May 27 20:10:50.771688 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:50.771601 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2efad6a2_ff87_479a_894d_d2397edf22d9.slice/crio-37be14a0b10c96b2d813f69f4a08913845a34d9f38627eaf1d6c0f09656c3d1d WatchSource:0}: Error finding container 37be14a0b10c96b2d813f69f4a08913845a34d9f38627eaf1d6c0f09656c3d1d: Status 404 returned error can't find the container with id 37be14a0b10c96b2d813f69f4a08913845a34d9f38627eaf1d6c0f09656c3d1d May 27 20:10:50.772906 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:50.772884 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb546aece_5207_4a3f_98a5_cda0a8333121.slice/crio-d6ef46556714d539c103b82b21ac17831eb81813630a8f5e52ab4cab51e076b4 WatchSource:0}: Error finding container d6ef46556714d539c103b82b21ac17831eb81813630a8f5e52ab4cab51e076b4: Status 404 returned error can't find the container with id d6ef46556714d539c103b82b21ac17831eb81813630a8f5e52ab4cab51e076b4 May 27 20:10:50.774182 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:50.774164 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5590928d_9a47_49d9_8626_780744dfd2fc.slice/crio-3b4a97c89c832b68d8a0eb8c16615ec98136fd35fdb61d3257344189ed978692 WatchSource:0}: Error finding container 3b4a97c89c832b68d8a0eb8c16615ec98136fd35fdb61d3257344189ed978692: Status 404 returned error can't find the container with id 3b4a97c89c832b68d8a0eb8c16615ec98136fd35fdb61d3257344189ed978692 May 27 20:10:50.774786 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:50.774764 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4937a11c_a8df_40eb_9a62_0b9fa6ea34ef.slice/crio-c6d9543aa138877a415ecbddaa77bb28df8eac0a0c7a54c79f4492e9b46db040 WatchSource:0}: Error finding container c6d9543aa138877a415ecbddaa77bb28df8eac0a0c7a54c79f4492e9b46db040: Status 404 returned error can't find the container with id c6d9543aa138877a415ecbddaa77bb28df8eac0a0c7a54c79f4492e9b46db040 May 27 20:10:50.775848 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:50.775829 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd4f71add_cfe6_4636_a5d9_fb823d22a61d.slice/crio-787042f6a11087e2039aea2fb8d1a9e98d25e43d64f4afe87e98aebe0df3a7fe WatchSource:0}: Error finding container 787042f6a11087e2039aea2fb8d1a9e98d25e43d64f4afe87e98aebe0df3a7fe: Status 404 returned error can't find the container with id 787042f6a11087e2039aea2fb8d1a9e98d25e43d64f4afe87e98aebe0df3a7fe May 27 20:10:50.777692 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:10:50.777665 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod92933172_b100_42e6_8018_dcb430647ae2.slice/crio-1abadd13853f65380b1b8223d1b0e270a4c0fbbf62a7207155d69bf3d2ed9747 WatchSource:0}: Error finding container 1abadd13853f65380b1b8223d1b0e270a4c0fbbf62a7207155d69bf3d2ed9747: Status 404 returned error can't find the container with id 1abadd13853f65380b1b8223d1b0e270a4c0fbbf62a7207155d69bf3d2ed9747 May 27 20:10:50.812319 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:50.812294 2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-gpj4z\" (UniqueName: \"kubernetes.io/projected/de3a4bee-c44a-4da9-833f-7e04d40c1f82-kube-api-access-gpj4z\") pod \"network-check-target-jdzbr\" (UID: \"de3a4bee-c44a-4da9-833f-7e04d40c1f82\") " pod="openshift-network-diagnostics/network-check-target-jdzbr" May 27 20:10:50.812429 ip-10-0-142-251 kubenswrapper[2571]: E0527 20:10:50.812409 2571 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered May 27 20:10:50.812503 ip-10-0-142-251 kubenswrapper[2571]: E0527 20:10:50.812434 2571 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered May 27 20:10:50.812503 ip-10-0-142-251 kubenswrapper[2571]: E0527 20:10:50.812446 2571 projected.go:194] Error preparing data for projected volume kube-api-access-gpj4z for pod openshift-network-diagnostics/network-check-target-jdzbr: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] May 27 20:10:50.812503 ip-10-0-142-251 kubenswrapper[2571]: E0527 20:10:50.812491 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/de3a4bee-c44a-4da9-833f-7e04d40c1f82-kube-api-access-gpj4z podName:de3a4bee-c44a-4da9-833f-7e04d40c1f82 nodeName:}" failed. No retries permitted until 2026-05-27 20:10:51.812475788 +0000 UTC m=+4.222102084 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-gpj4z" (UniqueName: "kubernetes.io/projected/de3a4bee-c44a-4da9-833f-7e04d40c1f82-kube-api-access-gpj4z") pod "network-check-target-jdzbr" (UID: "de3a4bee-c44a-4da9-833f-7e04d40c1f82") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] May 27 20:10:51.137354 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:51.137321 2571 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-05-26 20:05:49 +0000 UTC" deadline="2027-10-23 18:10:38.419533363 +0000 UTC" May 27 20:10:51.137354 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:51.137348 2571 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12333h59m47.282187581s" May 27 20:10:51.244740 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:51.244649 2571 generic.go:358] "Generic (PLEG): container finished" podID="5541ecea1d6385d1bbfe3fb4b8556b5e" containerID="4ccbfa7b1035eca1ff89dd4094ede8249ad16f98e23598bcee08483c7b3cd294" exitCode=0 May 27 20:10:51.244880 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:51.244737 2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-251.ec2.internal" event={"ID":"5541ecea1d6385d1bbfe3fb4b8556b5e","Type":"ContainerDied","Data":"4ccbfa7b1035eca1ff89dd4094ede8249ad16f98e23598bcee08483c7b3cd294"} May 27 20:10:51.248137 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:51.248074 2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-251.ec2.internal" event={"ID":"e8c054cb9a59ce3874e2199aacfd5039","Type":"ContainerStarted","Data":"09cb11fd945a2338503d95d9cbe0ebd90a59ab4f9d255d334f90926835bf079d"} May 27 20:10:51.249181 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:51.249154 2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-rcvbq" event={"ID":"92933172-b100-42e6-8018-dcb430647ae2","Type":"ContainerStarted","Data":"1abadd13853f65380b1b8223d1b0e270a4c0fbbf62a7207155d69bf3d2ed9747"} May 27 20:10:51.251988 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:51.251944 2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zhh65" event={"ID":"d4f71add-cfe6-4636-a5d9-fb823d22a61d","Type":"ContainerStarted","Data":"787042f6a11087e2039aea2fb8d1a9e98d25e43d64f4afe87e98aebe0df3a7fe"} May 27 20:10:51.254173 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:51.254148 2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-n4brm" event={"ID":"4937a11c-a8df-40eb-9a62-0b9fa6ea34ef","Type":"ContainerStarted","Data":"c6d9543aa138877a415ecbddaa77bb28df8eac0a0c7a54c79f4492e9b46db040"} May 27 20:10:51.257467 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:51.257408 2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-f4p5s" event={"ID":"cdf5a20d-c5e1-4d30-a6ce-85c91afd58e2","Type":"ContainerStarted","Data":"a516985a52471d370e42c980a1a2b2b72dda900af7411581809421ca705c4b96"} May 27 20:10:51.259937 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:51.259558 2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-gbwkk" event={"ID":"1b900804-03e8-43bb-b424-cbc50cc6805e","Type":"ContainerStarted","Data":"f27f0c5760388d0c2d2ce37a6a333256d173b01c3c24bd5c2bcdada5c2c3cc23"} May 27 20:10:51.262087 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:51.262059 2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zpwdj" event={"ID":"5590928d-9a47-49d9-8626-780744dfd2fc","Type":"ContainerStarted","Data":"3b4a97c89c832b68d8a0eb8c16615ec98136fd35fdb61d3257344189ed978692"} May 27 20:10:51.264088 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:51.264046 2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-tzz6t" event={"ID":"b546aece-5207-4a3f-98a5-cda0a8333121","Type":"ContainerStarted","Data":"d6ef46556714d539c103b82b21ac17831eb81813630a8f5e52ab4cab51e076b4"} May 27 20:10:51.266034 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:51.266011 2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-j46mn" event={"ID":"27613ab0-7fd5-4d50-a40c-58aa4b58ff6a","Type":"ContainerStarted","Data":"9b3be142920e18297455eeb05c244bff4d786b879cc768ae6ccff5220da00f39"} May 27 20:10:51.271967 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:51.271945 2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-265pt" event={"ID":"2efad6a2-ff87-479a-894d-d2397edf22d9","Type":"ContainerStarted","Data":"37be14a0b10c96b2d813f69f4a08913845a34d9f38627eaf1d6c0f09656c3d1d"} May 27 20:10:51.719489 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:51.719387 2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/96207b98-716e-4c63-847f-fc976f5563f2-metrics-certs\") pod \"network-metrics-daemon-zsx5p\" (UID: \"96207b98-716e-4c63-847f-fc976f5563f2\") " pod="openshift-multus/network-metrics-daemon-zsx5p" May 27 20:10:51.719656 ip-10-0-142-251 kubenswrapper[2571]: E0527 20:10:51.719532 2571 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered May 27 20:10:51.719656 ip-10-0-142-251 kubenswrapper[2571]: E0527 20:10:51.719594 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/96207b98-716e-4c63-847f-fc976f5563f2-metrics-certs podName:96207b98-716e-4c63-847f-fc976f5563f2 nodeName:}" failed. No retries permitted until 2026-05-27 20:10:53.719574403 +0000 UTC m=+6.129200697 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/96207b98-716e-4c63-847f-fc976f5563f2-metrics-certs") pod "network-metrics-daemon-zsx5p" (UID: "96207b98-716e-4c63-847f-fc976f5563f2") : object "openshift-multus"/"metrics-daemon-secret" not registered May 27 20:10:51.820439 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:51.820401 2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-gpj4z\" (UniqueName: \"kubernetes.io/projected/de3a4bee-c44a-4da9-833f-7e04d40c1f82-kube-api-access-gpj4z\") pod \"network-check-target-jdzbr\" (UID: \"de3a4bee-c44a-4da9-833f-7e04d40c1f82\") " pod="openshift-network-diagnostics/network-check-target-jdzbr" May 27 20:10:51.820615 ip-10-0-142-251 kubenswrapper[2571]: E0527 20:10:51.820578 2571 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered May 27 20:10:51.820615 ip-10-0-142-251 kubenswrapper[2571]: E0527 20:10:51.820597 2571 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered May 27 20:10:51.820615 ip-10-0-142-251 kubenswrapper[2571]: E0527 20:10:51.820608 2571 projected.go:194] Error preparing data for projected volume kube-api-access-gpj4z for pod openshift-network-diagnostics/network-check-target-jdzbr: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] May 27 20:10:51.820860 ip-10-0-142-251 kubenswrapper[2571]: E0527 20:10:51.820662 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/de3a4bee-c44a-4da9-833f-7e04d40c1f82-kube-api-access-gpj4z podName:de3a4bee-c44a-4da9-833f-7e04d40c1f82 nodeName:}" failed. No retries permitted until 2026-05-27 20:10:53.820644211 +0000 UTC m=+6.230270502 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-gpj4z" (UniqueName: "kubernetes.io/projected/de3a4bee-c44a-4da9-833f-7e04d40c1f82-kube-api-access-gpj4z") pod "network-check-target-jdzbr" (UID: "de3a4bee-c44a-4da9-833f-7e04d40c1f82") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] May 27 20:10:52.225318 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:52.225284 2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zsx5p" May 27 20:10:52.225812 ip-10-0-142-251 kubenswrapper[2571]: E0527 20:10:52.225447 2571 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zsx5p" podUID="96207b98-716e-4c63-847f-fc976f5563f2" May 27 20:10:52.226008 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:52.225988 2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jdzbr" May 27 20:10:52.226165 ip-10-0-142-251 kubenswrapper[2571]: E0527 20:10:52.226090 2571 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jdzbr" podUID="de3a4bee-c44a-4da9-833f-7e04d40c1f82" May 27 20:10:52.309248 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:52.309181 2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-251.ec2.internal" event={"ID":"5541ecea1d6385d1bbfe3fb4b8556b5e","Type":"ContainerStarted","Data":"73421f122f3a3563685fef468a2b17d694863eae446b20a690da8f873e752ad8"} May 27 20:10:52.326761 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:52.326301 2571 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-251.ec2.internal" podStartSLOduration=3.326283905 podStartE2EDuration="3.326283905s" podCreationTimestamp="2026-05-27 20:10:49 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-27 20:10:51.279466052 +0000 UTC m=+3.689092363" watchObservedRunningTime="2026-05-27 20:10:52.326283905 +0000 UTC m=+4.735910219" May 27 20:10:52.326761 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:52.326509 2571 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-251.ec2.internal" podStartSLOduration=3.3265023400000002 podStartE2EDuration="3.32650234s" podCreationTimestamp="2026-05-27 20:10:49 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-27 20:10:52.325585047 +0000 UTC m=+4.735211339" watchObservedRunningTime="2026-05-27 20:10:52.32650234 +0000 UTC m=+4.736128654" May 27 20:10:53.736789 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:53.736751 2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/96207b98-716e-4c63-847f-fc976f5563f2-metrics-certs\") pod \"network-metrics-daemon-zsx5p\" (UID: \"96207b98-716e-4c63-847f-fc976f5563f2\") " pod="openshift-multus/network-metrics-daemon-zsx5p" May 27 20:10:53.737362 ip-10-0-142-251 kubenswrapper[2571]: E0527 20:10:53.736914 2571 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered May 27 20:10:53.737362 ip-10-0-142-251 kubenswrapper[2571]: E0527 20:10:53.736973 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/96207b98-716e-4c63-847f-fc976f5563f2-metrics-certs podName:96207b98-716e-4c63-847f-fc976f5563f2 nodeName:}" failed. No retries permitted until 2026-05-27 20:10:57.736955712 +0000 UTC m=+10.146582017 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/96207b98-716e-4c63-847f-fc976f5563f2-metrics-certs") pod "network-metrics-daemon-zsx5p" (UID: "96207b98-716e-4c63-847f-fc976f5563f2") : object "openshift-multus"/"metrics-daemon-secret" not registered May 27 20:10:53.838174 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:53.837515 2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-gpj4z\" (UniqueName: \"kubernetes.io/projected/de3a4bee-c44a-4da9-833f-7e04d40c1f82-kube-api-access-gpj4z\") pod \"network-check-target-jdzbr\" (UID: \"de3a4bee-c44a-4da9-833f-7e04d40c1f82\") " pod="openshift-network-diagnostics/network-check-target-jdzbr" May 27 20:10:53.838174 ip-10-0-142-251 kubenswrapper[2571]: E0527 20:10:53.837724 2571 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered May 27 20:10:53.838174 ip-10-0-142-251 kubenswrapper[2571]: E0527 20:10:53.837744 2571 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered May 27 20:10:53.838174 ip-10-0-142-251 kubenswrapper[2571]: E0527 20:10:53.837757 2571 projected.go:194] Error preparing data for projected volume kube-api-access-gpj4z for pod openshift-network-diagnostics/network-check-target-jdzbr: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] May 27 20:10:53.838174 ip-10-0-142-251 kubenswrapper[2571]: E0527 20:10:53.837814 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/de3a4bee-c44a-4da9-833f-7e04d40c1f82-kube-api-access-gpj4z podName:de3a4bee-c44a-4da9-833f-7e04d40c1f82 nodeName:}" failed. No retries permitted until 2026-05-27 20:10:57.837795204 +0000 UTC m=+10.247421507 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-gpj4z" (UniqueName: "kubernetes.io/projected/de3a4bee-c44a-4da9-833f-7e04d40c1f82-kube-api-access-gpj4z") pod "network-check-target-jdzbr" (UID: "de3a4bee-c44a-4da9-833f-7e04d40c1f82") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] May 27 20:10:54.224299 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:54.224261 2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zsx5p" May 27 20:10:54.240300 ip-10-0-142-251 kubenswrapper[2571]: E0527 20:10:54.240255 2571 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zsx5p" podUID="96207b98-716e-4c63-847f-fc976f5563f2" May 27 20:10:54.240976 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:54.240951 2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jdzbr" May 27 20:10:54.241720 ip-10-0-142-251 kubenswrapper[2571]: E0527 20:10:54.241693 2571 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jdzbr" podUID="de3a4bee-c44a-4da9-833f-7e04d40c1f82" May 27 20:10:56.224678 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:56.224647 2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zsx5p" May 27 20:10:56.225097 ip-10-0-142-251 kubenswrapper[2571]: E0527 20:10:56.224791 2571 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zsx5p" podUID="96207b98-716e-4c63-847f-fc976f5563f2" May 27 20:10:56.225234 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:56.225216 2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jdzbr" May 27 20:10:56.225328 ip-10-0-142-251 kubenswrapper[2571]: E0527 20:10:56.225308 2571 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jdzbr" podUID="de3a4bee-c44a-4da9-833f-7e04d40c1f82" May 27 20:10:57.770393 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:57.770362 2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/96207b98-716e-4c63-847f-fc976f5563f2-metrics-certs\") pod \"network-metrics-daemon-zsx5p\" (UID: \"96207b98-716e-4c63-847f-fc976f5563f2\") " pod="openshift-multus/network-metrics-daemon-zsx5p" May 27 20:10:57.770828 ip-10-0-142-251 kubenswrapper[2571]: E0527 20:10:57.770528 2571 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered May 27 20:10:57.770828 ip-10-0-142-251 kubenswrapper[2571]: E0527 20:10:57.770583 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/96207b98-716e-4c63-847f-fc976f5563f2-metrics-certs podName:96207b98-716e-4c63-847f-fc976f5563f2 nodeName:}" failed. No retries permitted until 2026-05-27 20:11:05.770568365 +0000 UTC m=+18.180194654 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/96207b98-716e-4c63-847f-fc976f5563f2-metrics-certs") pod "network-metrics-daemon-zsx5p" (UID: "96207b98-716e-4c63-847f-fc976f5563f2") : object "openshift-multus"/"metrics-daemon-secret" not registered May 27 20:10:57.871279 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:57.871221 2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-gpj4z\" (UniqueName: \"kubernetes.io/projected/de3a4bee-c44a-4da9-833f-7e04d40c1f82-kube-api-access-gpj4z\") pod \"network-check-target-jdzbr\" (UID: \"de3a4bee-c44a-4da9-833f-7e04d40c1f82\") " pod="openshift-network-diagnostics/network-check-target-jdzbr" May 27 20:10:57.871461 ip-10-0-142-251 kubenswrapper[2571]: E0527 20:10:57.871380 2571 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered May 27 20:10:57.871461 ip-10-0-142-251 kubenswrapper[2571]: E0527 20:10:57.871399 2571 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered May 27 20:10:57.871461 ip-10-0-142-251 kubenswrapper[2571]: E0527 20:10:57.871412 2571 projected.go:194] Error preparing data for projected volume kube-api-access-gpj4z for pod openshift-network-diagnostics/network-check-target-jdzbr: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] May 27 20:10:57.871627 ip-10-0-142-251 kubenswrapper[2571]: E0527 20:10:57.871469 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/de3a4bee-c44a-4da9-833f-7e04d40c1f82-kube-api-access-gpj4z podName:de3a4bee-c44a-4da9-833f-7e04d40c1f82 nodeName:}" failed. No retries permitted until 2026-05-27 20:11:05.871448973 +0000 UTC m=+18.281075276 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-gpj4z" (UniqueName: "kubernetes.io/projected/de3a4bee-c44a-4da9-833f-7e04d40c1f82-kube-api-access-gpj4z") pod "network-check-target-jdzbr" (UID: "de3a4bee-c44a-4da9-833f-7e04d40c1f82") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] May 27 20:10:58.224725 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:58.224647 2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jdzbr" May 27 20:10:58.224873 ip-10-0-142-251 kubenswrapper[2571]: E0527 20:10:58.224778 2571 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jdzbr" podUID="de3a4bee-c44a-4da9-833f-7e04d40c1f82" May 27 20:10:58.226326 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:10:58.226300 2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zsx5p" May 27 20:10:58.226437 ip-10-0-142-251 kubenswrapper[2571]: E0527 20:10:58.226418 2571 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zsx5p" podUID="96207b98-716e-4c63-847f-fc976f5563f2" May 27 20:11:00.224169 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:11:00.224130 2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zsx5p" May 27 20:11:00.224602 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:11:00.224178 2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jdzbr" May 27 20:11:00.224602 ip-10-0-142-251 kubenswrapper[2571]: E0527 20:11:00.224273 2571 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zsx5p" podUID="96207b98-716e-4c63-847f-fc976f5563f2" May 27 20:11:00.224602 ip-10-0-142-251 kubenswrapper[2571]: E0527 20:11:00.224388 2571 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jdzbr" podUID="de3a4bee-c44a-4da9-833f-7e04d40c1f82" May 27 20:11:02.224991 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:11:02.224953 2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zsx5p" May 27 20:11:02.225466 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:11:02.225007 2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jdzbr" May 27 20:11:02.225466 ip-10-0-142-251 kubenswrapper[2571]: E0527 20:11:02.225126 2571 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zsx5p" podUID="96207b98-716e-4c63-847f-fc976f5563f2" May 27 20:11:02.225466 ip-10-0-142-251 kubenswrapper[2571]: E0527 20:11:02.225222 2571 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jdzbr" podUID="de3a4bee-c44a-4da9-833f-7e04d40c1f82" May 27 20:11:04.225037 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:11:04.224999 2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jdzbr" May 27 20:11:04.225619 ip-10-0-142-251 kubenswrapper[2571]: E0527 20:11:04.225140 2571 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jdzbr" podUID="de3a4bee-c44a-4da9-833f-7e04d40c1f82" May 27 20:11:04.225619 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:11:04.225204 2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zsx5p" May 27 20:11:04.225619 ip-10-0-142-251 kubenswrapper[2571]: E0527 20:11:04.225334 2571 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zsx5p" podUID="96207b98-716e-4c63-847f-fc976f5563f2" May 27 20:11:05.830049 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:11:05.830008 2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/96207b98-716e-4c63-847f-fc976f5563f2-metrics-certs\") pod \"network-metrics-daemon-zsx5p\" (UID: \"96207b98-716e-4c63-847f-fc976f5563f2\") " pod="openshift-multus/network-metrics-daemon-zsx5p" May 27 20:11:05.830550 ip-10-0-142-251 kubenswrapper[2571]: E0527 20:11:05.830185 2571 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered May 27 20:11:05.830550 ip-10-0-142-251 kubenswrapper[2571]: E0527 20:11:05.830261 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/96207b98-716e-4c63-847f-fc976f5563f2-metrics-certs podName:96207b98-716e-4c63-847f-fc976f5563f2 nodeName:}" failed. No retries permitted until 2026-05-27 20:11:21.83024024 +0000 UTC m=+34.239866552 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/96207b98-716e-4c63-847f-fc976f5563f2-metrics-certs") pod "network-metrics-daemon-zsx5p" (UID: "96207b98-716e-4c63-847f-fc976f5563f2") : object "openshift-multus"/"metrics-daemon-secret" not registered May 27 20:11:05.931153 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:11:05.931115 2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-gpj4z\" (UniqueName: \"kubernetes.io/projected/de3a4bee-c44a-4da9-833f-7e04d40c1f82-kube-api-access-gpj4z\") pod \"network-check-target-jdzbr\" (UID: \"de3a4bee-c44a-4da9-833f-7e04d40c1f82\") " pod="openshift-network-diagnostics/network-check-target-jdzbr" May 27 20:11:05.931345 ip-10-0-142-251 kubenswrapper[2571]: E0527 20:11:05.931257 2571 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered May 27 20:11:05.931345 ip-10-0-142-251 kubenswrapper[2571]: E0527 20:11:05.931277 2571 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered May 27 20:11:05.931345 ip-10-0-142-251 kubenswrapper[2571]: E0527 20:11:05.931291 2571 projected.go:194] Error preparing data for projected volume kube-api-access-gpj4z for pod openshift-network-diagnostics/network-check-target-jdzbr: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] May 27 20:11:05.931496 ip-10-0-142-251 kubenswrapper[2571]: E0527 20:11:05.931354 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/de3a4bee-c44a-4da9-833f-7e04d40c1f82-kube-api-access-gpj4z podName:de3a4bee-c44a-4da9-833f-7e04d40c1f82 nodeName:}" failed. No retries permitted until 2026-05-27 20:11:21.931338063 +0000 UTC m=+34.340964351 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-gpj4z" (UniqueName: "kubernetes.io/projected/de3a4bee-c44a-4da9-833f-7e04d40c1f82-kube-api-access-gpj4z") pod "network-check-target-jdzbr" (UID: "de3a4bee-c44a-4da9-833f-7e04d40c1f82") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] May 27 20:11:06.225149 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:11:06.225043 2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jdzbr" May 27 20:11:06.225318 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:11:06.225047 2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zsx5p" May 27 20:11:06.225318 ip-10-0-142-251 kubenswrapper[2571]: E0527 20:11:06.225189 2571 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jdzbr" podUID="de3a4bee-c44a-4da9-833f-7e04d40c1f82" May 27 20:11:06.225318 ip-10-0-142-251 kubenswrapper[2571]: E0527 20:11:06.225253 2571 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zsx5p" podUID="96207b98-716e-4c63-847f-fc976f5563f2" May 27 20:11:08.225601 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:11:08.225557 2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jdzbr" May 27 20:11:08.226043 ip-10-0-142-251 kubenswrapper[2571]: E0527 20:11:08.225675 2571 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jdzbr" podUID="de3a4bee-c44a-4da9-833f-7e04d40c1f82" May 27 20:11:08.226043 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:11:08.225738 2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zsx5p" May 27 20:11:08.226043 ip-10-0-142-251 kubenswrapper[2571]: E0527 20:11:08.225869 2571 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zsx5p" podUID="96207b98-716e-4c63-847f-fc976f5563f2" May 27 20:11:10.227029 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:11:10.226651 2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jdzbr" May 27 20:11:10.227926 ip-10-0-142-251 kubenswrapper[2571]: E0527 20:11:10.227049 2571 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jdzbr" podUID="de3a4bee-c44a-4da9-833f-7e04d40c1f82" May 27 20:11:10.227926 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:11:10.227467 2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zsx5p" May 27 20:11:10.227926 ip-10-0-142-251 kubenswrapper[2571]: E0527 20:11:10.227562 2571 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zsx5p" podUID="96207b98-716e-4c63-847f-fc976f5563f2" May 27 20:11:10.343242 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:11:10.343210 2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zpwdj" event={"ID":"5590928d-9a47-49d9-8626-780744dfd2fc","Type":"ContainerStarted","Data":"afbd5c5f6e270281b6a2da389d7508dd86d3d5d9b6acd47ddf788d23a173afe6"} May 27 20:11:10.344331 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:11:10.344290 2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-tzz6t" event={"ID":"b546aece-5207-4a3f-98a5-cda0a8333121","Type":"ContainerStarted","Data":"b83162d29146ce9efa803c627aa915838b5e05d40ec39d5b2266ff26bc2a2368"} May 27 20:11:10.345512 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:11:10.345487 2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-j46mn" event={"ID":"27613ab0-7fd5-4d50-a40c-58aa4b58ff6a","Type":"ContainerStarted","Data":"d1d9b247dc9c1bb8789b8fafb9a898aaec7901c11dd2ca86065946be74746f58"} May 27 20:11:10.346713 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:11:10.346693 2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-265pt" event={"ID":"2efad6a2-ff87-479a-894d-d2397edf22d9","Type":"ContainerStarted","Data":"b3c20c62610364394bf175da72a07a5c43325e85c618c95cc1842297fbb4bfa7"} May 27 20:11:10.348910 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:11:10.348894 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rcvbq_92933172-b100-42e6-8018-dcb430647ae2/ovn-acl-logging/0.log" May 27 20:11:10.349198 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:11:10.349177 2571 generic.go:358] "Generic (PLEG): container finished" podID="92933172-b100-42e6-8018-dcb430647ae2" containerID="58feaf3f32c93c033c867ce27105fd238734e37bd284f63c49da73f64bb2b1ff" exitCode=1 May 27 20:11:10.349277 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:11:10.349242 2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-rcvbq" event={"ID":"92933172-b100-42e6-8018-dcb430647ae2","Type":"ContainerStarted","Data":"3566f615635c15e0b55879d48774a2c0ce035d3a89ce2f75a9a590c887f3bd6a"} May 27 20:11:10.349277 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:11:10.349268 2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-rcvbq" event={"ID":"92933172-b100-42e6-8018-dcb430647ae2","Type":"ContainerStarted","Data":"02c6c3fa0b47eabf2ed3cabe1424003a65fc9bf8f9d2c217d94d6aa633e7c838"} May 27 20:11:10.349345 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:11:10.349277 2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-rcvbq" event={"ID":"92933172-b100-42e6-8018-dcb430647ae2","Type":"ContainerStarted","Data":"ffd80cb91f6f67c4ee45af97c08af754da3fea20d08d249ca233664501d372f7"} May 27 20:11:10.349345 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:11:10.349287 2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-rcvbq" event={"ID":"92933172-b100-42e6-8018-dcb430647ae2","Type":"ContainerStarted","Data":"031fdb133e8f29db0baa04b440893b75dcffe8af7a02fbc83154552def6e1121"} May 27 20:11:10.349345 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:11:10.349295 2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-rcvbq" event={"ID":"92933172-b100-42e6-8018-dcb430647ae2","Type":"ContainerDied","Data":"58feaf3f32c93c033c867ce27105fd238734e37bd284f63c49da73f64bb2b1ff"} May 27 20:11:10.349345 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:11:10.349306 2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-rcvbq" event={"ID":"92933172-b100-42e6-8018-dcb430647ae2","Type":"ContainerStarted","Data":"8b00e7c05fa81b8019720cb440d37b686ccf9b223832872d2584c039c120308b"} May 27 20:11:10.350478 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:11:10.350457 2571 generic.go:358] "Generic (PLEG): container finished" podID="d4f71add-cfe6-4636-a5d9-fb823d22a61d" containerID="36a45059c922b8bc368c5054de13418801a45c4f58a252fd4dff146d5b193820" exitCode=0 May 27 20:11:10.350570 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:11:10.350538 2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zhh65" event={"ID":"d4f71add-cfe6-4636-a5d9-fb823d22a61d","Type":"ContainerDied","Data":"36a45059c922b8bc368c5054de13418801a45c4f58a252fd4dff146d5b193820"} May 27 20:11:10.351707 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:11:10.351688 2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-n4brm" event={"ID":"4937a11c-a8df-40eb-9a62-0b9fa6ea34ef","Type":"ContainerStarted","Data":"5cc29a8dc1c0d323b55b7168586877e15698486e805a4a8cb21d31a3cf692b79"} May 27 20:11:10.352910 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:11:10.352889 2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-f4p5s" event={"ID":"cdf5a20d-c5e1-4d30-a6ce-85c91afd58e2","Type":"ContainerStarted","Data":"9f259ea6e17b48200f6983205cf15213ad2914535aef81b8e6598e664bbe65b0"} May 27 20:11:10.361756 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:11:10.361711 2571 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-tzz6t" podStartSLOduration=3.758347773 podStartE2EDuration="22.361698595s" podCreationTimestamp="2026-05-27 20:10:48 +0000 UTC" firstStartedPulling="2026-05-27 20:10:50.774935216 +0000 UTC m=+3.184561520" lastFinishedPulling="2026-05-27 20:11:09.378286038 +0000 UTC m=+21.787912342" observedRunningTime="2026-05-27 20:11:10.361261555 +0000 UTC m=+22.770887876" watchObservedRunningTime="2026-05-27 20:11:10.361698595 +0000 UTC m=+22.771324969" May 27 20:11:10.376252 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:11:10.376196 2571 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-dns/node-resolver-n4brm" podStartSLOduration=3.832222975 podStartE2EDuration="22.376180843s" podCreationTimestamp="2026-05-27 20:10:48 +0000 UTC" firstStartedPulling="2026-05-27 20:10:50.77790294 +0000 UTC m=+3.187529229" lastFinishedPulling="2026-05-27 20:11:09.321860809 +0000 UTC m=+21.731487097" observedRunningTime="2026-05-27 20:11:10.37603493 +0000 UTC m=+22.785661240" watchObservedRunningTime="2026-05-27 20:11:10.376180843 +0000 UTC m=+22.785807171" May 27 20:11:10.390990 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:11:10.390940 2571 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-image-registry/node-ca-f4p5s" podStartSLOduration=3.839969159 podStartE2EDuration="22.390925071s" podCreationTimestamp="2026-05-27 20:10:48 +0000 UTC" firstStartedPulling="2026-05-27 20:10:50.770726794 +0000 UTC m=+3.180353097" lastFinishedPulling="2026-05-27 20:11:09.321682721 +0000 UTC m=+21.731309009" observedRunningTime="2026-05-27 20:11:10.39071705 +0000 UTC m=+22.800343361" watchObservedRunningTime="2026-05-27 20:11:10.390925071 +0000 UTC m=+22.800551392" May 27 20:11:10.404610 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:11:10.404556 2571 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kube-system/konnectivity-agent-j46mn" podStartSLOduration=3.854720416 podStartE2EDuration="22.404537216s" podCreationTimestamp="2026-05-27 20:10:48 +0000 UTC" firstStartedPulling="2026-05-27 20:10:50.771959213 +0000 UTC m=+3.181585503" lastFinishedPulling="2026-05-27 20:11:09.321776007 +0000 UTC m=+21.731402303" observedRunningTime="2026-05-27 20:11:10.404533583 +0000 UTC m=+22.814159895" watchObservedRunningTime="2026-05-27 20:11:10.404537216 +0000 UTC m=+22.814163706" May 27 20:11:10.426003 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:11:10.425948 2571 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-multus/multus-265pt" podStartSLOduration=3.820454452 podStartE2EDuration="22.425930343s" podCreationTimestamp="2026-05-27 20:10:48 +0000 UTC" firstStartedPulling="2026-05-27 20:10:50.773503056 +0000 UTC m=+3.183129346" lastFinishedPulling="2026-05-27 20:11:09.378978933 +0000 UTC m=+21.788605237" observedRunningTime="2026-05-27 20:11:10.425662256 +0000 UTC m=+22.835288567" watchObservedRunningTime="2026-05-27 20:11:10.425930343 +0000 UTC m=+22.835556654" May 27 20:11:11.130811 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:11:11.130649 2571 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" May 27 20:11:11.171256 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:11:11.171096 2571 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-05-27T20:11:11.130807087Z","UUID":"e64bdc8b-cb6c-4756-990e-6e19f3c1ddc2","Handler":null,"Name":"","Endpoint":""} May 27 20:11:11.172809 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:11:11.172785 2571 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 May 27 20:11:11.172809 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:11:11.172812 2571 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock May 27 20:11:11.355481 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:11:11.355441 2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-gbwkk" event={"ID":"1b900804-03e8-43bb-b424-cbc50cc6805e","Type":"ContainerStarted","Data":"a100a72b84b722eca473d67b2c8c38b4d072b3358069152096e827ee36a785ee"} May 27 20:11:11.357085 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:11:11.357053 2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zpwdj" event={"ID":"5590928d-9a47-49d9-8626-780744dfd2fc","Type":"ContainerStarted","Data":"f553d8b8652f1fc85c899930831747c8808e5977a1d3f7bce683072e71a44baa"} May 27 20:11:11.371966 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:11:11.371929 2571 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-gbwkk" podStartSLOduration=4.759500094 podStartE2EDuration="23.371918998s" podCreationTimestamp="2026-05-27 20:10:48 +0000 UTC" firstStartedPulling="2026-05-27 20:10:50.769981373 +0000 UTC m=+3.179607663" lastFinishedPulling="2026-05-27 20:11:09.382400279 +0000 UTC m=+21.792026567" observedRunningTime="2026-05-27 20:11:11.371542348 +0000 UTC m=+23.781168658" watchObservedRunningTime="2026-05-27 20:11:11.371918998 +0000 UTC m=+23.781545309" May 27 20:11:12.224953 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:11:12.224422 2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jdzbr" May 27 20:11:12.224953 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:11:12.224433 2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zsx5p" May 27 20:11:12.224953 ip-10-0-142-251 kubenswrapper[2571]: E0527 20:11:12.224556 2571 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jdzbr" podUID="de3a4bee-c44a-4da9-833f-7e04d40c1f82" May 27 20:11:12.224953 ip-10-0-142-251 kubenswrapper[2571]: E0527 20:11:12.224636 2571 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zsx5p" podUID="96207b98-716e-4c63-847f-fc976f5563f2" May 27 20:11:12.361423 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:11:12.361383 2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zpwdj" event={"ID":"5590928d-9a47-49d9-8626-780744dfd2fc","Type":"ContainerStarted","Data":"9c9e41818e35baf90c084b3210aabc5a2e8e9b45deb9f26f27d711136516d648"} May 27 20:11:12.364757 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:11:12.364731 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rcvbq_92933172-b100-42e6-8018-dcb430647ae2/ovn-acl-logging/0.log" May 27 20:11:12.366198 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:11:12.366166 2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-rcvbq" event={"ID":"92933172-b100-42e6-8018-dcb430647ae2","Type":"ContainerStarted","Data":"26e99080a760e4fb9d4dbd5623e5a56ad43b5ecd6c188b26ab60418a6f99a0ea"} May 27 20:11:12.379628 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:11:12.379573 2571 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zpwdj" podStartSLOduration=2.9626750729999998 podStartE2EDuration="24.3795578s" podCreationTimestamp="2026-05-27 20:10:48 +0000 UTC" firstStartedPulling="2026-05-27 20:10:50.776198709 +0000 UTC m=+3.185824997" lastFinishedPulling="2026-05-27 20:11:12.193081435 +0000 UTC m=+24.602707724" observedRunningTime="2026-05-27 20:11:12.37940212 +0000 UTC m=+24.789028432" watchObservedRunningTime="2026-05-27 20:11:12.3795578 +0000 UTC m=+24.789184112" May 27 20:11:13.401621 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:11:13.401580 2571 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-j46mn" May 27 20:11:13.402224 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:11:13.402198 2571 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-j46mn" May 27 20:11:14.224483 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:11:14.224447 2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jdzbr" May 27 20:11:14.224680 ip-10-0-142-251 kubenswrapper[2571]: E0527 20:11:14.224567 2571 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jdzbr" podUID="de3a4bee-c44a-4da9-833f-7e04d40c1f82" May 27 20:11:14.224680 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:11:14.224630 2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zsx5p" May 27 20:11:14.224787 ip-10-0-142-251 kubenswrapper[2571]: E0527 20:11:14.224740 2571 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zsx5p" podUID="96207b98-716e-4c63-847f-fc976f5563f2" May 27 20:11:15.374577 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:11:15.374397 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rcvbq_92933172-b100-42e6-8018-dcb430647ae2/ovn-acl-logging/0.log" May 27 20:11:15.375122 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:11:15.374889 2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-rcvbq" event={"ID":"92933172-b100-42e6-8018-dcb430647ae2","Type":"ContainerStarted","Data":"13e2f0d7741f7aa8be58a936022cc2af6ea9a27997a23db77917a6f6fead41ae"} May 27 20:11:15.375238 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:11:15.375218 2571 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-rcvbq" May 27 20:11:15.375297 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:11:15.375253 2571 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-rcvbq" May 27 20:11:15.375497 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:11:15.375478 2571 scope.go:117] "RemoveContainer" containerID="58feaf3f32c93c033c867ce27105fd238734e37bd284f63c49da73f64bb2b1ff" May 27 20:11:15.377088 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:11:15.377067 2571 generic.go:358] "Generic (PLEG): container finished" podID="d4f71add-cfe6-4636-a5d9-fb823d22a61d" containerID="03ebcf6547bb1b8a7e1ac21a947d79b1c436accab6d36d463ae498e61288f019" exitCode=0 May 27 20:11:15.377208 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:11:15.377134 2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zhh65" event={"ID":"d4f71add-cfe6-4636-a5d9-fb823d22a61d","Type":"ContainerDied","Data":"03ebcf6547bb1b8a7e1ac21a947d79b1c436accab6d36d463ae498e61288f019"} May 27 20:11:15.393469 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:11:15.393446 2571 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-rcvbq" May 27 20:11:16.224891 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:11:16.224865 2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jdzbr" May 27 20:11:16.225036 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:11:16.224958 2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zsx5p" May 27 20:11:16.225080 ip-10-0-142-251 kubenswrapper[2571]: E0527 20:11:16.225051 2571 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jdzbr" podUID="de3a4bee-c44a-4da9-833f-7e04d40c1f82" May 27 20:11:16.225228 ip-10-0-142-251 kubenswrapper[2571]: E0527 20:11:16.225207 2571 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zsx5p" podUID="96207b98-716e-4c63-847f-fc976f5563f2" May 27 20:11:16.382219 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:11:16.382155 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rcvbq_92933172-b100-42e6-8018-dcb430647ae2/ovn-acl-logging/0.log" May 27 20:11:16.382605 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:11:16.382499 2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-rcvbq" event={"ID":"92933172-b100-42e6-8018-dcb430647ae2","Type":"ContainerStarted","Data":"c47159c930fa0885576d9927b54f5db01be000e282741eedb6b2ba02e86f80c4"} May 27 20:11:16.382822 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:11:16.382803 2571 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-rcvbq" May 27 20:11:16.384481 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:11:16.384457 2571 generic.go:358] "Generic (PLEG): container finished" podID="d4f71add-cfe6-4636-a5d9-fb823d22a61d" containerID="68f04f1d6a7b1e073ce62a956cec7e1009ffb2044cd1b5053cd202c2881a0941" exitCode=0 May 27 20:11:16.384588 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:11:16.384491 2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zhh65" event={"ID":"d4f71add-cfe6-4636-a5d9-fb823d22a61d","Type":"ContainerDied","Data":"68f04f1d6a7b1e073ce62a956cec7e1009ffb2044cd1b5053cd202c2881a0941"} May 27 20:11:16.398798 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:11:16.398774 2571 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-rcvbq" May 27 20:11:16.414122 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:11:16.414037 2571 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-rcvbq" podStartSLOduration=9.793546463 podStartE2EDuration="28.414025895s" podCreationTimestamp="2026-05-27 20:10:48 +0000 UTC" firstStartedPulling="2026-05-27 20:10:50.779492141 +0000 UTC m=+3.189118430" lastFinishedPulling="2026-05-27 20:11:09.399971574 +0000 UTC m=+21.809597862" observedRunningTime="2026-05-27 20:11:16.413149305 +0000 UTC m=+28.822775615" watchObservedRunningTime="2026-05-27 20:11:16.414025895 +0000 UTC m=+28.823652206" May 27 20:11:16.834183 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:11:16.833921 2571 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-zsx5p"] May 27 20:11:16.834328 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:11:16.834303 2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zsx5p" May 27 20:11:16.834411 ip-10-0-142-251 kubenswrapper[2571]: E0527 20:11:16.834394 2571 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zsx5p" podUID="96207b98-716e-4c63-847f-fc976f5563f2" May 27 20:11:16.836252 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:11:16.836228 2571 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-jdzbr"] May 27 20:11:16.836352 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:11:16.836310 2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jdzbr" May 27 20:11:16.836394 ip-10-0-142-251 kubenswrapper[2571]: E0527 20:11:16.836375 2571 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jdzbr" podUID="de3a4bee-c44a-4da9-833f-7e04d40c1f82" May 27 20:11:17.295452 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:11:17.295425 2571 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-j46mn" May 27 20:11:17.295608 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:11:17.295561 2571 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" May 27 20:11:17.296077 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:11:17.296060 2571 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-j46mn" May 27 20:11:17.388713 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:11:17.388686 2571 generic.go:358] "Generic (PLEG): container finished" podID="d4f71add-cfe6-4636-a5d9-fb823d22a61d" containerID="e8cbc4d6d677491f2e3d3d085229e99ba1657b973ee089d0916635c92b8f24d2" exitCode=0 May 27 20:11:17.389014 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:11:17.388753 2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zhh65" event={"ID":"d4f71add-cfe6-4636-a5d9-fb823d22a61d","Type":"ContainerDied","Data":"e8cbc4d6d677491f2e3d3d085229e99ba1657b973ee089d0916635c92b8f24d2"} May 27 20:11:18.228247 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:11:18.228213 2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jdzbr" May 27 20:11:18.228421 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:11:18.228221 2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zsx5p" May 27 20:11:18.228421 ip-10-0-142-251 kubenswrapper[2571]: E0527 20:11:18.228310 2571 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jdzbr" podUID="de3a4bee-c44a-4da9-833f-7e04d40c1f82" May 27 20:11:18.228566 ip-10-0-142-251 kubenswrapper[2571]: E0527 20:11:18.228412 2571 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zsx5p" podUID="96207b98-716e-4c63-847f-fc976f5563f2" May 27 20:11:20.224772 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:11:20.224739 2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zsx5p" May 27 20:11:20.225671 ip-10-0-142-251 kubenswrapper[2571]: E0527 20:11:20.224884 2571 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zsx5p" podUID="96207b98-716e-4c63-847f-fc976f5563f2" May 27 20:11:20.225671 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:11:20.224739 2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jdzbr" May 27 20:11:20.225671 ip-10-0-142-251 kubenswrapper[2571]: E0527 20:11:20.225520 2571 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jdzbr" podUID="de3a4bee-c44a-4da9-833f-7e04d40c1f82" May 27 20:11:21.850055 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:11:21.850019 2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/96207b98-716e-4c63-847f-fc976f5563f2-metrics-certs\") pod \"network-metrics-daemon-zsx5p\" (UID: \"96207b98-716e-4c63-847f-fc976f5563f2\") " pod="openshift-multus/network-metrics-daemon-zsx5p" May 27 20:11:21.850673 ip-10-0-142-251 kubenswrapper[2571]: E0527 20:11:21.850180 2571 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered May 27 20:11:21.850673 ip-10-0-142-251 kubenswrapper[2571]: E0527 20:11:21.850249 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/96207b98-716e-4c63-847f-fc976f5563f2-metrics-certs podName:96207b98-716e-4c63-847f-fc976f5563f2 nodeName:}" failed. No retries permitted until 2026-05-27 20:11:53.850228738 +0000 UTC m=+66.259855030 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/96207b98-716e-4c63-847f-fc976f5563f2-metrics-certs") pod "network-metrics-daemon-zsx5p" (UID: "96207b98-716e-4c63-847f-fc976f5563f2") : object "openshift-multus"/"metrics-daemon-secret" not registered May 27 20:11:21.951250 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:11:21.950935 2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-gpj4z\" (UniqueName: \"kubernetes.io/projected/de3a4bee-c44a-4da9-833f-7e04d40c1f82-kube-api-access-gpj4z\") pod \"network-check-target-jdzbr\" (UID: \"de3a4bee-c44a-4da9-833f-7e04d40c1f82\") " pod="openshift-network-diagnostics/network-check-target-jdzbr" May 27 20:11:21.951250 ip-10-0-142-251 kubenswrapper[2571]: E0527 20:11:21.951067 2571 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered May 27 20:11:21.951250 ip-10-0-142-251 kubenswrapper[2571]: E0527 20:11:21.951084 2571 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered May 27 20:11:21.951250 ip-10-0-142-251 kubenswrapper[2571]: E0527 20:11:21.951093 2571 projected.go:194] Error preparing data for projected volume kube-api-access-gpj4z for pod openshift-network-diagnostics/network-check-target-jdzbr: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] May 27 20:11:21.951250 ip-10-0-142-251 kubenswrapper[2571]: E0527 20:11:21.951169 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/de3a4bee-c44a-4da9-833f-7e04d40c1f82-kube-api-access-gpj4z podName:de3a4bee-c44a-4da9-833f-7e04d40c1f82 nodeName:}" failed. No retries permitted until 2026-05-27 20:11:53.951151897 +0000 UTC m=+66.360778188 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-gpj4z" (UniqueName: "kubernetes.io/projected/de3a4bee-c44a-4da9-833f-7e04d40c1f82-kube-api-access-gpj4z") pod "network-check-target-jdzbr" (UID: "de3a4bee-c44a-4da9-833f-7e04d40c1f82") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] May 27 20:11:22.224329 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:11:22.224238 2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zsx5p" May 27 20:11:22.224553 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:11:22.224238 2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jdzbr" May 27 20:11:22.224553 ip-10-0-142-251 kubenswrapper[2571]: E0527 20:11:22.224381 2571 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-zsx5p" podUID="96207b98-716e-4c63-847f-fc976f5563f2" May 27 20:11:22.224553 ip-10-0-142-251 kubenswrapper[2571]: E0527 20:11:22.224458 2571 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jdzbr" podUID="de3a4bee-c44a-4da9-833f-7e04d40c1f82" May 27 20:11:22.429768 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:11:22.429736 2571 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-142-251.ec2.internal" event="NodeReady" May 27 20:11:22.429929 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:11:22.429880 2571 kubelet_node_status.go:550] "Fast updating node status as it just became ready" May 27 20:11:22.471779 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:11:22.471745 2571 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-58qkv"] May 27 20:11:22.503472 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:11:22.503399 2571 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-jxjsb"] May 27 20:11:22.503613 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:11:22.503585 2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-58qkv" May 27 20:11:22.506476 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:11:22.506451 2571 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" May 27 20:11:22.506624 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:11:22.506477 2571 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-jcg7q\"" May 27 20:11:22.506624 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:11:22.506480 2571 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" May 27 20:11:22.519893 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:11:22.519866 2571 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-58qkv"] May 27 20:11:22.519893 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:11:22.519896 2571 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-jxjsb"] May 27 20:11:22.520134 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:11:22.520000 2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-jxjsb" May 27 20:11:22.522594 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:11:22.522563 2571 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" May 27 20:11:22.522719 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:11:22.522606 2571 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-c8wsm\"" May 27 20:11:22.522719 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:11:22.522690 2571 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" May 27 20:11:22.522839 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:11:22.522690 2571 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" May 27 20:11:22.654389 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:11:22.654358 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/cf585445-0773-47ca-ba45-3bad1a2d5b88-metrics-tls\") pod \"dns-default-58qkv\" (UID: \"cf585445-0773-47ca-ba45-3bad1a2d5b88\") " pod="openshift-dns/dns-default-58qkv" May 27 20:11:22.654586 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:11:22.654432 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cf585445-0773-47ca-ba45-3bad1a2d5b88-tmp-dir\") pod \"dns-default-58qkv\" (UID: \"cf585445-0773-47ca-ba45-3bad1a2d5b88\") " pod="openshift-dns/dns-default-58qkv" May 27 20:11:22.654586 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:11:22.654498 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/cf585445-0773-47ca-ba45-3bad1a2d5b88-config-volume\") pod \"dns-default-58qkv\" (UID: \"cf585445-0773-47ca-ba45-3bad1a2d5b88\") " pod="openshift-dns/dns-default-58qkv" May 27 20:11:22.654586 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:11:22.654526 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gnl9t\" (UniqueName: \"kubernetes.io/projected/d6fb3e28-cbfd-47a3-8104-bbfa86006ad6-kube-api-access-gnl9t\") pod \"ingress-canary-jxjsb\" (UID: \"d6fb3e28-cbfd-47a3-8104-bbfa86006ad6\") " pod="openshift-ingress-canary/ingress-canary-jxjsb" May 27 20:11:22.654586 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:11:22.654548 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d6fb3e28-cbfd-47a3-8104-bbfa86006ad6-cert\") pod \"ingress-canary-jxjsb\" (UID: \"d6fb3e28-cbfd-47a3-8104-bbfa86006ad6\") " pod="openshift-ingress-canary/ingress-canary-jxjsb" May 27 20:11:22.654586 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:11:22.654575 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-694ms\" (UniqueName: \"kubernetes.io/projected/cf585445-0773-47ca-ba45-3bad1a2d5b88-kube-api-access-694ms\") pod \"dns-default-58qkv\" (UID: \"cf585445-0773-47ca-ba45-3bad1a2d5b88\") " pod="openshift-dns/dns-default-58qkv" May 27 20:11:22.755504 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:11:22.755420 2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-694ms\" (UniqueName: \"kubernetes.io/projected/cf585445-0773-47ca-ba45-3bad1a2d5b88-kube-api-access-694ms\") pod \"dns-default-58qkv\" (UID: \"cf585445-0773-47ca-ba45-3bad1a2d5b88\") " pod="openshift-dns/dns-default-58qkv" May 27 20:11:22.755504 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:11:22.755474 2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/cf585445-0773-47ca-ba45-3bad1a2d5b88-metrics-tls\") pod \"dns-default-58qkv\" (UID: \"cf585445-0773-47ca-ba45-3bad1a2d5b88\") " pod="openshift-dns/dns-default-58qkv" May 27 20:11:22.755730 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:11:22.755531 2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cf585445-0773-47ca-ba45-3bad1a2d5b88-tmp-dir\") pod \"dns-default-58qkv\" (UID: \"cf585445-0773-47ca-ba45-3bad1a2d5b88\") " pod="openshift-dns/dns-default-58qkv" May 27 20:11:22.755730 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:11:22.755568 2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/cf585445-0773-47ca-ba45-3bad1a2d5b88-config-volume\") pod \"dns-default-58qkv\" (UID: \"cf585445-0773-47ca-ba45-3bad1a2d5b88\") " pod="openshift-dns/dns-default-58qkv" May 27 20:11:22.755730 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:11:22.755590 2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-gnl9t\" (UniqueName: \"kubernetes.io/projected/d6fb3e28-cbfd-47a3-8104-bbfa86006ad6-kube-api-access-gnl9t\") pod \"ingress-canary-jxjsb\" (UID: \"d6fb3e28-cbfd-47a3-8104-bbfa86006ad6\") " pod="openshift-ingress-canary/ingress-canary-jxjsb" May 27 20:11:22.755730 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:11:22.755619 2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d6fb3e28-cbfd-47a3-8104-bbfa86006ad6-cert\") pod \"ingress-canary-jxjsb\" (UID: \"d6fb3e28-cbfd-47a3-8104-bbfa86006ad6\") " pod="openshift-ingress-canary/ingress-canary-jxjsb" May 27 20:11:22.755730 ip-10-0-142-251 kubenswrapper[2571]: E0527 20:11:22.755635 2571 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found May 27 20:11:22.755730 ip-10-0-142-251 kubenswrapper[2571]: E0527 20:11:22.755701 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cf585445-0773-47ca-ba45-3bad1a2d5b88-metrics-tls podName:cf585445-0773-47ca-ba45-3bad1a2d5b88 nodeName:}" failed. No retries permitted until 2026-05-27 20:11:23.255677606 +0000 UTC m=+35.665303898 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/cf585445-0773-47ca-ba45-3bad1a2d5b88-metrics-tls") pod "dns-default-58qkv" (UID: "cf585445-0773-47ca-ba45-3bad1a2d5b88") : secret "dns-default-metrics-tls" not found May 27 20:11:22.755730 ip-10-0-142-251 kubenswrapper[2571]: E0527 20:11:22.755707 2571 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found May 27 20:11:22.756027 ip-10-0-142-251 kubenswrapper[2571]: E0527 20:11:22.755751 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d6fb3e28-cbfd-47a3-8104-bbfa86006ad6-cert podName:d6fb3e28-cbfd-47a3-8104-bbfa86006ad6 nodeName:}" failed. No retries permitted until 2026-05-27 20:11:23.255736014 +0000 UTC m=+35.665362302 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/d6fb3e28-cbfd-47a3-8104-bbfa86006ad6-cert") pod "ingress-canary-jxjsb" (UID: "d6fb3e28-cbfd-47a3-8104-bbfa86006ad6") : secret "canary-serving-cert" not found May 27 20:11:22.756027 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:11:22.755939 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cf585445-0773-47ca-ba45-3bad1a2d5b88-tmp-dir\") pod \"dns-default-58qkv\" (UID: \"cf585445-0773-47ca-ba45-3bad1a2d5b88\") " pod="openshift-dns/dns-default-58qkv" May 27 20:11:22.756271 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:11:22.756251 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/cf585445-0773-47ca-ba45-3bad1a2d5b88-config-volume\") pod \"dns-default-58qkv\" (UID: \"cf585445-0773-47ca-ba45-3bad1a2d5b88\") " pod="openshift-dns/dns-default-58qkv" May 27 20:11:22.767792 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:11:22.767671 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-694ms\" (UniqueName: \"kubernetes.io/projected/cf585445-0773-47ca-ba45-3bad1a2d5b88-kube-api-access-694ms\") pod \"dns-default-58qkv\" (UID: \"cf585445-0773-47ca-ba45-3bad1a2d5b88\") " pod="openshift-dns/dns-default-58qkv" May 27 20:11:22.767938 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:11:22.767738 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gnl9t\" (UniqueName: \"kubernetes.io/projected/d6fb3e28-cbfd-47a3-8104-bbfa86006ad6-kube-api-access-gnl9t\") pod \"ingress-canary-jxjsb\" (UID: \"d6fb3e28-cbfd-47a3-8104-bbfa86006ad6\") " pod="openshift-ingress-canary/ingress-canary-jxjsb" May 27 20:11:23.258416 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:11:23.258382 2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/cf585445-0773-47ca-ba45-3bad1a2d5b88-metrics-tls\") pod \"dns-default-58qkv\" (UID: \"cf585445-0773-47ca-ba45-3bad1a2d5b88\") " pod="openshift-dns/dns-default-58qkv" May 27 20:11:23.258854 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:11:23.258452 2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d6fb3e28-cbfd-47a3-8104-bbfa86006ad6-cert\") pod \"ingress-canary-jxjsb\" (UID: \"d6fb3e28-cbfd-47a3-8104-bbfa86006ad6\") " pod="openshift-ingress-canary/ingress-canary-jxjsb" May 27 20:11:23.258854 ip-10-0-142-251 kubenswrapper[2571]: E0527 20:11:23.258541 2571 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found May 27 20:11:23.258854 ip-10-0-142-251 kubenswrapper[2571]: E0527 20:11:23.258611 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d6fb3e28-cbfd-47a3-8104-bbfa86006ad6-cert podName:d6fb3e28-cbfd-47a3-8104-bbfa86006ad6 nodeName:}" failed. No retries permitted until 2026-05-27 20:11:24.258596106 +0000 UTC m=+36.668222395 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/d6fb3e28-cbfd-47a3-8104-bbfa86006ad6-cert") pod "ingress-canary-jxjsb" (UID: "d6fb3e28-cbfd-47a3-8104-bbfa86006ad6") : secret "canary-serving-cert" not found May 27 20:11:23.258854 ip-10-0-142-251 kubenswrapper[2571]: E0527 20:11:23.258544 2571 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found May 27 20:11:23.258854 ip-10-0-142-251 kubenswrapper[2571]: E0527 20:11:23.258693 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cf585445-0773-47ca-ba45-3bad1a2d5b88-metrics-tls podName:cf585445-0773-47ca-ba45-3bad1a2d5b88 nodeName:}" failed. No retries permitted until 2026-05-27 20:11:24.258675251 +0000 UTC m=+36.668301543 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/cf585445-0773-47ca-ba45-3bad1a2d5b88-metrics-tls") pod "dns-default-58qkv" (UID: "cf585445-0773-47ca-ba45-3bad1a2d5b88") : secret "dns-default-metrics-tls" not found May 27 20:11:24.224457 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:11:24.224418 2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zsx5p" May 27 20:11:24.224457 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:11:24.224450 2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jdzbr" May 27 20:11:24.229301 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:11:24.229276 2571 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" May 27 20:11:24.229301 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:11:24.229296 2571 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" May 27 20:11:24.229489 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:11:24.229317 2571 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-b7jlg\"" May 27 20:11:24.229489 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:11:24.229300 2571 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-g79gn\"" May 27 20:11:24.229489 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:11:24.229401 2571 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" May 27 20:11:24.263648 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:11:24.263620 2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d6fb3e28-cbfd-47a3-8104-bbfa86006ad6-cert\") pod \"ingress-canary-jxjsb\" (UID: \"d6fb3e28-cbfd-47a3-8104-bbfa86006ad6\") " pod="openshift-ingress-canary/ingress-canary-jxjsb" May 27 20:11:24.263971 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:11:24.263666 2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/cf585445-0773-47ca-ba45-3bad1a2d5b88-metrics-tls\") pod \"dns-default-58qkv\" (UID: \"cf585445-0773-47ca-ba45-3bad1a2d5b88\") " pod="openshift-dns/dns-default-58qkv" May 27 20:11:24.263971 ip-10-0-142-251 kubenswrapper[2571]: E0527 20:11:24.263755 2571 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found May 27 20:11:24.263971 ip-10-0-142-251 kubenswrapper[2571]: E0527 20:11:24.263781 2571 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found May 27 20:11:24.263971 ip-10-0-142-251 kubenswrapper[2571]: E0527 20:11:24.263809 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d6fb3e28-cbfd-47a3-8104-bbfa86006ad6-cert podName:d6fb3e28-cbfd-47a3-8104-bbfa86006ad6 nodeName:}" failed. No retries permitted until 2026-05-27 20:11:26.263793847 +0000 UTC m=+38.673420136 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/d6fb3e28-cbfd-47a3-8104-bbfa86006ad6-cert") pod "ingress-canary-jxjsb" (UID: "d6fb3e28-cbfd-47a3-8104-bbfa86006ad6") : secret "canary-serving-cert" not found May 27 20:11:24.263971 ip-10-0-142-251 kubenswrapper[2571]: E0527 20:11:24.263832 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cf585445-0773-47ca-ba45-3bad1a2d5b88-metrics-tls podName:cf585445-0773-47ca-ba45-3bad1a2d5b88 nodeName:}" failed. No retries permitted until 2026-05-27 20:11:26.263816396 +0000 UTC m=+38.673442705 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/cf585445-0773-47ca-ba45-3bad1a2d5b88-metrics-tls") pod "dns-default-58qkv" (UID: "cf585445-0773-47ca-ba45-3bad1a2d5b88") : secret "dns-default-metrics-tls" not found May 27 20:11:24.406058 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:11:24.406030 2571 generic.go:358] "Generic (PLEG): container finished" podID="d4f71add-cfe6-4636-a5d9-fb823d22a61d" containerID="6fab5b0befc43caf99bda7687ab97eac10ad4d019e3a51ee527ee8304146dc70" exitCode=0 May 27 20:11:24.406226 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:11:24.406081 2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zhh65" event={"ID":"d4f71add-cfe6-4636-a5d9-fb823d22a61d","Type":"ContainerDied","Data":"6fab5b0befc43caf99bda7687ab97eac10ad4d019e3a51ee527ee8304146dc70"} May 27 20:11:25.410695 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:11:25.410666 2571 generic.go:358] "Generic (PLEG): container finished" podID="d4f71add-cfe6-4636-a5d9-fb823d22a61d" containerID="fa8cd5227a116789012cb83ad7099406151c0d5d6b1ae1381c248e56d5469cee" exitCode=0 May 27 20:11:25.411035 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:11:25.410716 2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zhh65" event={"ID":"d4f71add-cfe6-4636-a5d9-fb823d22a61d","Type":"ContainerDied","Data":"fa8cd5227a116789012cb83ad7099406151c0d5d6b1ae1381c248e56d5469cee"} May 27 20:11:26.275808 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:11:26.275777 2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d6fb3e28-cbfd-47a3-8104-bbfa86006ad6-cert\") pod \"ingress-canary-jxjsb\" (UID: \"d6fb3e28-cbfd-47a3-8104-bbfa86006ad6\") " pod="openshift-ingress-canary/ingress-canary-jxjsb" May 27 20:11:26.275940 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:11:26.275817 2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/cf585445-0773-47ca-ba45-3bad1a2d5b88-metrics-tls\") pod \"dns-default-58qkv\" (UID: \"cf585445-0773-47ca-ba45-3bad1a2d5b88\") " pod="openshift-dns/dns-default-58qkv" May 27 20:11:26.275940 ip-10-0-142-251 kubenswrapper[2571]: E0527 20:11:26.275911 2571 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found May 27 20:11:26.275940 ip-10-0-142-251 kubenswrapper[2571]: E0527 20:11:26.275913 2571 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found May 27 20:11:26.276030 ip-10-0-142-251 kubenswrapper[2571]: E0527 20:11:26.275962 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d6fb3e28-cbfd-47a3-8104-bbfa86006ad6-cert podName:d6fb3e28-cbfd-47a3-8104-bbfa86006ad6 nodeName:}" failed. No retries permitted until 2026-05-27 20:11:30.275948589 +0000 UTC m=+42.685574877 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/d6fb3e28-cbfd-47a3-8104-bbfa86006ad6-cert") pod "ingress-canary-jxjsb" (UID: "d6fb3e28-cbfd-47a3-8104-bbfa86006ad6") : secret "canary-serving-cert" not found May 27 20:11:26.276030 ip-10-0-142-251 kubenswrapper[2571]: E0527 20:11:26.275976 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cf585445-0773-47ca-ba45-3bad1a2d5b88-metrics-tls podName:cf585445-0773-47ca-ba45-3bad1a2d5b88 nodeName:}" failed. No retries permitted until 2026-05-27 20:11:30.275969683 +0000 UTC m=+42.685595971 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/cf585445-0773-47ca-ba45-3bad1a2d5b88-metrics-tls") pod "dns-default-58qkv" (UID: "cf585445-0773-47ca-ba45-3bad1a2d5b88") : secret "dns-default-metrics-tls" not found May 27 20:11:26.415427 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:11:26.415349 2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zhh65" event={"ID":"d4f71add-cfe6-4636-a5d9-fb823d22a61d","Type":"ContainerStarted","Data":"e13ab4c1cb125b5eb97d351ef3c5501df2c3934a8c40031b914a879caced236e"} May 27 20:11:26.440356 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:11:26.440302 2571 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-zhh65" podStartSLOduration=5.752681847 podStartE2EDuration="38.440284179s" podCreationTimestamp="2026-05-27 20:10:48 +0000 UTC" firstStartedPulling="2026-05-27 20:10:50.777685029 +0000 UTC m=+3.187311324" lastFinishedPulling="2026-05-27 20:11:23.465287362 +0000 UTC m=+35.874913656" observedRunningTime="2026-05-27 20:11:26.438517129 +0000 UTC m=+38.848143441" watchObservedRunningTime="2026-05-27 20:11:26.440284179 +0000 UTC m=+38.849910492" May 27 20:11:30.302483 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:11:30.302442 2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/cf585445-0773-47ca-ba45-3bad1a2d5b88-metrics-tls\") pod \"dns-default-58qkv\" (UID: \"cf585445-0773-47ca-ba45-3bad1a2d5b88\") " pod="openshift-dns/dns-default-58qkv" May 27 20:11:30.302894 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:11:30.302532 2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d6fb3e28-cbfd-47a3-8104-bbfa86006ad6-cert\") pod \"ingress-canary-jxjsb\" (UID: \"d6fb3e28-cbfd-47a3-8104-bbfa86006ad6\") " pod="openshift-ingress-canary/ingress-canary-jxjsb" May 27 20:11:30.302894 ip-10-0-142-251 kubenswrapper[2571]: E0527 20:11:30.302586 2571 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found May 27 20:11:30.302894 ip-10-0-142-251 kubenswrapper[2571]: E0527 20:11:30.302624 2571 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found May 27 20:11:30.302894 ip-10-0-142-251 kubenswrapper[2571]: E0527 20:11:30.302646 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cf585445-0773-47ca-ba45-3bad1a2d5b88-metrics-tls podName:cf585445-0773-47ca-ba45-3bad1a2d5b88 nodeName:}" failed. No retries permitted until 2026-05-27 20:11:38.302629975 +0000 UTC m=+50.712256264 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/cf585445-0773-47ca-ba45-3bad1a2d5b88-metrics-tls") pod "dns-default-58qkv" (UID: "cf585445-0773-47ca-ba45-3bad1a2d5b88") : secret "dns-default-metrics-tls" not found May 27 20:11:30.302894 ip-10-0-142-251 kubenswrapper[2571]: E0527 20:11:30.302672 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d6fb3e28-cbfd-47a3-8104-bbfa86006ad6-cert podName:d6fb3e28-cbfd-47a3-8104-bbfa86006ad6 nodeName:}" failed. No retries permitted until 2026-05-27 20:11:38.302655482 +0000 UTC m=+50.712281777 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/d6fb3e28-cbfd-47a3-8104-bbfa86006ad6-cert") pod "ingress-canary-jxjsb" (UID: "d6fb3e28-cbfd-47a3-8104-bbfa86006ad6") : secret "canary-serving-cert" not found May 27 20:11:31.445670 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:11:31.445639 2571 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kube-system/global-pull-secret-syncer-9lgmk"] May 27 20:11:31.470950 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:11:31.470919 2571 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-9lgmk"] May 27 20:11:31.471093 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:11:31.471041 2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9lgmk" May 27 20:11:31.473770 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:11:31.473751 2571 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" May 27 20:11:31.613863 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:11:31.613825 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/87d3dcf4-4db8-4d23-a9be-dac49b48ab6a-kubelet-config\") pod \"global-pull-secret-syncer-9lgmk\" (UID: \"87d3dcf4-4db8-4d23-a9be-dac49b48ab6a\") " pod="kube-system/global-pull-secret-syncer-9lgmk" May 27 20:11:31.613863 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:11:31.613868 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/87d3dcf4-4db8-4d23-a9be-dac49b48ab6a-dbus\") pod \"global-pull-secret-syncer-9lgmk\" (UID: \"87d3dcf4-4db8-4d23-a9be-dac49b48ab6a\") " pod="kube-system/global-pull-secret-syncer-9lgmk" May 27 20:11:31.614068 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:11:31.613899 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/87d3dcf4-4db8-4d23-a9be-dac49b48ab6a-original-pull-secret\") pod \"global-pull-secret-syncer-9lgmk\" (UID: \"87d3dcf4-4db8-4d23-a9be-dac49b48ab6a\") " pod="kube-system/global-pull-secret-syncer-9lgmk" May 27 20:11:31.715133 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:11:31.715052 2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/87d3dcf4-4db8-4d23-a9be-dac49b48ab6a-kubelet-config\") pod \"global-pull-secret-syncer-9lgmk\" (UID: \"87d3dcf4-4db8-4d23-a9be-dac49b48ab6a\") " pod="kube-system/global-pull-secret-syncer-9lgmk" May 27 20:11:31.715133 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:11:31.715092 2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/87d3dcf4-4db8-4d23-a9be-dac49b48ab6a-dbus\") pod \"global-pull-secret-syncer-9lgmk\" (UID: \"87d3dcf4-4db8-4d23-a9be-dac49b48ab6a\") " pod="kube-system/global-pull-secret-syncer-9lgmk" May 27 20:11:31.715282 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:11:31.715142 2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/87d3dcf4-4db8-4d23-a9be-dac49b48ab6a-original-pull-secret\") pod \"global-pull-secret-syncer-9lgmk\" (UID: \"87d3dcf4-4db8-4d23-a9be-dac49b48ab6a\") " pod="kube-system/global-pull-secret-syncer-9lgmk" May 27 20:11:31.715282 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:11:31.715190 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/87d3dcf4-4db8-4d23-a9be-dac49b48ab6a-kubelet-config\") pod \"global-pull-secret-syncer-9lgmk\" (UID: \"87d3dcf4-4db8-4d23-a9be-dac49b48ab6a\") " pod="kube-system/global-pull-secret-syncer-9lgmk" May 27 20:11:31.715377 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:11:31.715354 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/87d3dcf4-4db8-4d23-a9be-dac49b48ab6a-dbus\") pod \"global-pull-secret-syncer-9lgmk\" (UID: \"87d3dcf4-4db8-4d23-a9be-dac49b48ab6a\") " pod="kube-system/global-pull-secret-syncer-9lgmk" May 27 20:11:31.718561 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:11:31.718541 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/87d3dcf4-4db8-4d23-a9be-dac49b48ab6a-original-pull-secret\") pod \"global-pull-secret-syncer-9lgmk\" (UID: \"87d3dcf4-4db8-4d23-a9be-dac49b48ab6a\") " pod="kube-system/global-pull-secret-syncer-9lgmk" May 27 20:11:31.780238 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:11:31.780203 2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9lgmk" May 27 20:11:31.958157 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:11:31.958092 2571 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-9lgmk"] May 27 20:11:31.961260 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:11:31.961225 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod87d3dcf4_4db8_4d23_a9be_dac49b48ab6a.slice/crio-a7312258170ccc26e78d5af4cdf5778134e4d96464520b02678bf1bfab5ff0ce WatchSource:0}: Error finding container a7312258170ccc26e78d5af4cdf5778134e4d96464520b02678bf1bfab5ff0ce: Status 404 returned error can't find the container with id a7312258170ccc26e78d5af4cdf5778134e4d96464520b02678bf1bfab5ff0ce May 27 20:11:32.428287 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:11:32.428254 2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-9lgmk" event={"ID":"87d3dcf4-4db8-4d23-a9be-dac49b48ab6a","Type":"ContainerStarted","Data":"a7312258170ccc26e78d5af4cdf5778134e4d96464520b02678bf1bfab5ff0ce"} May 27 20:11:37.439906 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:11:37.439871 2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-9lgmk" event={"ID":"87d3dcf4-4db8-4d23-a9be-dac49b48ab6a","Type":"ContainerStarted","Data":"7b45eefa610d72fd1b339cc0c654f0ea4a07ce305097751ea3d2f91ee2835d9c"} May 27 20:11:37.454930 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:11:37.454881 2571 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-9lgmk" podStartSLOduration=1.606222262 podStartE2EDuration="6.45486809s" podCreationTimestamp="2026-05-27 20:11:31 +0000 UTC" firstStartedPulling="2026-05-27 20:11:31.962947443 +0000 UTC m=+44.372573732" lastFinishedPulling="2026-05-27 20:11:36.811593268 +0000 UTC m=+49.221219560" observedRunningTime="2026-05-27 20:11:37.454828271 +0000 UTC m=+49.864454582" watchObservedRunningTime="2026-05-27 20:11:37.45486809 +0000 UTC m=+49.864494401" May 27 20:11:37.502646 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:11:37.502614 2571 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6fc8f5cd4b-wrbm9"] May 27 20:11:37.523601 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:11:37.523578 2571 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6fc8f5cd4b-wrbm9"] May 27 20:11:37.523601 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:11:37.523602 2571 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-dccc57588-twsww"] May 27 20:11:37.523795 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:11:37.523738 2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6fc8f5cd4b-wrbm9" May 27 20:11:37.527789 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:11:37.526789 2571 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" May 27 20:11:37.527789 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:11:37.527089 2571 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\"" May 27 20:11:37.527789 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:11:37.527332 2571 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" May 27 20:11:37.527789 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:11:37.527464 2571 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" May 27 20:11:37.527789 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:11:37.527580 2571 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-4wsqw\"" May 27 20:11:37.539604 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:11:37.539584 2571 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-dccc57588-twsww"] May 27 20:11:37.539714 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:11:37.539681 2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-dccc57588-twsww" May 27 20:11:37.542528 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:11:37.542506 2571 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\"" May 27 20:11:37.542528 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:11:37.542509 2571 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\"" May 27 20:11:37.542675 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:11:37.542509 2571 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\"" May 27 20:11:37.542675 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:11:37.542544 2571 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\"" May 27 20:11:37.659529 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:11:37.659501 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/f5a33912-2fd4-4b0f-b2b6-0730dbd0d82d-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-dccc57588-twsww\" (UID: \"f5a33912-2fd4-4b0f-b2b6-0730dbd0d82d\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-dccc57588-twsww" May 27 20:11:37.659529 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:11:37.659536 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/f5a33912-2fd4-4b0f-b2b6-0730dbd0d82d-ca\") pod \"cluster-proxy-proxy-agent-dccc57588-twsww\" (UID: \"f5a33912-2fd4-4b0f-b2b6-0730dbd0d82d\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-dccc57588-twsww" May 27 20:11:37.659721 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:11:37.659557 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nnv4w\" (UniqueName: \"kubernetes.io/projected/35683361-3a05-46ce-8d13-43a5a027c40f-kube-api-access-nnv4w\") pod \"managed-serviceaccount-addon-agent-6fc8f5cd4b-wrbm9\" (UID: \"35683361-3a05-46ce-8d13-43a5a027c40f\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6fc8f5cd4b-wrbm9" May 27 20:11:37.659721 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:11:37.659624 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/f5a33912-2fd4-4b0f-b2b6-0730dbd0d82d-hub\") pod \"cluster-proxy-proxy-agent-dccc57588-twsww\" (UID: \"f5a33912-2fd4-4b0f-b2b6-0730dbd0d82d\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-dccc57588-twsww" May 27 20:11:37.659721 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:11:37.659698 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/35683361-3a05-46ce-8d13-43a5a027c40f-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-6fc8f5cd4b-wrbm9\" (UID: \"35683361-3a05-46ce-8d13-43a5a027c40f\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6fc8f5cd4b-wrbm9" May 27 20:11:37.659813 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:11:37.659722 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/f5a33912-2fd4-4b0f-b2b6-0730dbd0d82d-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-dccc57588-twsww\" (UID: \"f5a33912-2fd4-4b0f-b2b6-0730dbd0d82d\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-dccc57588-twsww" May 27 20:11:37.659813 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:11:37.659742 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ocpservice-ca\" (UniqueName: \"kubernetes.io/configmap/f5a33912-2fd4-4b0f-b2b6-0730dbd0d82d-ocpservice-ca\") pod \"cluster-proxy-proxy-agent-dccc57588-twsww\" (UID: \"f5a33912-2fd4-4b0f-b2b6-0730dbd0d82d\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-dccc57588-twsww" May 27 20:11:37.659813 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:11:37.659757 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cbk4d\" (UniqueName: \"kubernetes.io/projected/f5a33912-2fd4-4b0f-b2b6-0730dbd0d82d-kube-api-access-cbk4d\") pod \"cluster-proxy-proxy-agent-dccc57588-twsww\" (UID: \"f5a33912-2fd4-4b0f-b2b6-0730dbd0d82d\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-dccc57588-twsww" May 27 20:11:37.760670 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:11:37.760601 2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/f5a33912-2fd4-4b0f-b2b6-0730dbd0d82d-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-dccc57588-twsww\" (UID: \"f5a33912-2fd4-4b0f-b2b6-0730dbd0d82d\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-dccc57588-twsww" May 27 20:11:37.760670 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:11:37.760640 2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/f5a33912-2fd4-4b0f-b2b6-0730dbd0d82d-ca\") pod \"cluster-proxy-proxy-agent-dccc57588-twsww\" (UID: \"f5a33912-2fd4-4b0f-b2b6-0730dbd0d82d\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-dccc57588-twsww" May 27 20:11:37.760670 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:11:37.760668 2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-nnv4w\" (UniqueName: \"kubernetes.io/projected/35683361-3a05-46ce-8d13-43a5a027c40f-kube-api-access-nnv4w\") pod \"managed-serviceaccount-addon-agent-6fc8f5cd4b-wrbm9\" (UID: \"35683361-3a05-46ce-8d13-43a5a027c40f\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6fc8f5cd4b-wrbm9" May 27 20:11:37.760881 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:11:37.760704 2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/f5a33912-2fd4-4b0f-b2b6-0730dbd0d82d-hub\") pod \"cluster-proxy-proxy-agent-dccc57588-twsww\" (UID: \"f5a33912-2fd4-4b0f-b2b6-0730dbd0d82d\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-dccc57588-twsww" May 27 20:11:37.760881 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:11:37.760758 2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/35683361-3a05-46ce-8d13-43a5a027c40f-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-6fc8f5cd4b-wrbm9\" (UID: \"35683361-3a05-46ce-8d13-43a5a027c40f\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6fc8f5cd4b-wrbm9" May 27 20:11:37.760881 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:11:37.760788 2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/f5a33912-2fd4-4b0f-b2b6-0730dbd0d82d-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-dccc57588-twsww\" (UID: \"f5a33912-2fd4-4b0f-b2b6-0730dbd0d82d\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-dccc57588-twsww" May 27 20:11:37.760881 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:11:37.760813 2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ocpservice-ca\" (UniqueName: \"kubernetes.io/configmap/f5a33912-2fd4-4b0f-b2b6-0730dbd0d82d-ocpservice-ca\") pod \"cluster-proxy-proxy-agent-dccc57588-twsww\" (UID: \"f5a33912-2fd4-4b0f-b2b6-0730dbd0d82d\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-dccc57588-twsww" May 27 20:11:37.760881 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:11:37.760836 2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-cbk4d\" (UniqueName: \"kubernetes.io/projected/f5a33912-2fd4-4b0f-b2b6-0730dbd0d82d-kube-api-access-cbk4d\") pod \"cluster-proxy-proxy-agent-dccc57588-twsww\" (UID: \"f5a33912-2fd4-4b0f-b2b6-0730dbd0d82d\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-dccc57588-twsww" May 27 20:11:37.761643 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:11:37.761619 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ocpservice-ca\" (UniqueName: \"kubernetes.io/configmap/f5a33912-2fd4-4b0f-b2b6-0730dbd0d82d-ocpservice-ca\") pod \"cluster-proxy-proxy-agent-dccc57588-twsww\" (UID: \"f5a33912-2fd4-4b0f-b2b6-0730dbd0d82d\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-dccc57588-twsww" May 27 20:11:37.763232 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:11:37.763210 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/f5a33912-2fd4-4b0f-b2b6-0730dbd0d82d-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-dccc57588-twsww\" (UID: \"f5a33912-2fd4-4b0f-b2b6-0730dbd0d82d\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-dccc57588-twsww" May 27 20:11:37.763792 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:11:37.763770 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/f5a33912-2fd4-4b0f-b2b6-0730dbd0d82d-ca\") pod \"cluster-proxy-proxy-agent-dccc57588-twsww\" (UID: \"f5a33912-2fd4-4b0f-b2b6-0730dbd0d82d\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-dccc57588-twsww" May 27 20:11:37.763904 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:11:37.763885 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/35683361-3a05-46ce-8d13-43a5a027c40f-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-6fc8f5cd4b-wrbm9\" (UID: \"35683361-3a05-46ce-8d13-43a5a027c40f\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6fc8f5cd4b-wrbm9" May 27 20:11:37.768924 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:11:37.768899 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nnv4w\" (UniqueName: \"kubernetes.io/projected/35683361-3a05-46ce-8d13-43a5a027c40f-kube-api-access-nnv4w\") pod \"managed-serviceaccount-addon-agent-6fc8f5cd4b-wrbm9\" (UID: \"35683361-3a05-46ce-8d13-43a5a027c40f\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6fc8f5cd4b-wrbm9" May 27 20:11:37.772705 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:11:37.772677 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/f5a33912-2fd4-4b0f-b2b6-0730dbd0d82d-hub\") pod \"cluster-proxy-proxy-agent-dccc57588-twsww\" (UID: \"f5a33912-2fd4-4b0f-b2b6-0730dbd0d82d\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-dccc57588-twsww" May 27 20:11:37.772821 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:11:37.772719 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/f5a33912-2fd4-4b0f-b2b6-0730dbd0d82d-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-dccc57588-twsww\" (UID: \"f5a33912-2fd4-4b0f-b2b6-0730dbd0d82d\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-dccc57588-twsww" May 27 20:11:37.774491 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:11:37.774471 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cbk4d\" (UniqueName: \"kubernetes.io/projected/f5a33912-2fd4-4b0f-b2b6-0730dbd0d82d-kube-api-access-cbk4d\") pod \"cluster-proxy-proxy-agent-dccc57588-twsww\" (UID: \"f5a33912-2fd4-4b0f-b2b6-0730dbd0d82d\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-dccc57588-twsww" May 27 20:11:37.844562 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:11:37.844537 2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6fc8f5cd4b-wrbm9" May 27 20:11:37.851220 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:11:37.851201 2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-dccc57588-twsww" May 27 20:11:37.993980 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:11:37.993952 2571 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-dccc57588-twsww"] May 27 20:11:37.996716 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:11:37.996682 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf5a33912_2fd4_4b0f_b2b6_0730dbd0d82d.slice/crio-d6b0cb0308d2f809642c415095b85ea01cceadfdf0bc36993935d1fd0c765300 WatchSource:0}: Error finding container d6b0cb0308d2f809642c415095b85ea01cceadfdf0bc36993935d1fd0c765300: Status 404 returned error can't find the container with id d6b0cb0308d2f809642c415095b85ea01cceadfdf0bc36993935d1fd0c765300 May 27 20:11:38.009738 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:11:38.009715 2571 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6fc8f5cd4b-wrbm9"] May 27 20:11:38.012433 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:11:38.012411 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod35683361_3a05_46ce_8d13_43a5a027c40f.slice/crio-fda3f82db287e0fb965c95ed389f435240f9b941e638619be8669235dc144324 WatchSource:0}: Error finding container fda3f82db287e0fb965c95ed389f435240f9b941e638619be8669235dc144324: Status 404 returned error can't find the container with id fda3f82db287e0fb965c95ed389f435240f9b941e638619be8669235dc144324 May 27 20:11:38.365394 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:11:38.365301 2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/cf585445-0773-47ca-ba45-3bad1a2d5b88-metrics-tls\") pod \"dns-default-58qkv\" (UID: \"cf585445-0773-47ca-ba45-3bad1a2d5b88\") " pod="openshift-dns/dns-default-58qkv" May 27 20:11:38.365679 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:11:38.365413 2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d6fb3e28-cbfd-47a3-8104-bbfa86006ad6-cert\") pod \"ingress-canary-jxjsb\" (UID: \"d6fb3e28-cbfd-47a3-8104-bbfa86006ad6\") " pod="openshift-ingress-canary/ingress-canary-jxjsb" May 27 20:11:38.365679 ip-10-0-142-251 kubenswrapper[2571]: E0527 20:11:38.365463 2571 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found May 27 20:11:38.365679 ip-10-0-142-251 kubenswrapper[2571]: E0527 20:11:38.365500 2571 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found May 27 20:11:38.365679 ip-10-0-142-251 kubenswrapper[2571]: E0527 20:11:38.365532 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cf585445-0773-47ca-ba45-3bad1a2d5b88-metrics-tls podName:cf585445-0773-47ca-ba45-3bad1a2d5b88 nodeName:}" failed. No retries permitted until 2026-05-27 20:11:54.365512869 +0000 UTC m=+66.775139175 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/cf585445-0773-47ca-ba45-3bad1a2d5b88-metrics-tls") pod "dns-default-58qkv" (UID: "cf585445-0773-47ca-ba45-3bad1a2d5b88") : secret "dns-default-metrics-tls" not found May 27 20:11:38.365679 ip-10-0-142-251 kubenswrapper[2571]: E0527 20:11:38.365548 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d6fb3e28-cbfd-47a3-8104-bbfa86006ad6-cert podName:d6fb3e28-cbfd-47a3-8104-bbfa86006ad6 nodeName:}" failed. No retries permitted until 2026-05-27 20:11:54.365542203 +0000 UTC m=+66.775168491 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/d6fb3e28-cbfd-47a3-8104-bbfa86006ad6-cert") pod "ingress-canary-jxjsb" (UID: "d6fb3e28-cbfd-47a3-8104-bbfa86006ad6") : secret "canary-serving-cert" not found May 27 20:11:38.442746 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:11:38.442708 2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6fc8f5cd4b-wrbm9" event={"ID":"35683361-3a05-46ce-8d13-43a5a027c40f","Type":"ContainerStarted","Data":"fda3f82db287e0fb965c95ed389f435240f9b941e638619be8669235dc144324"} May 27 20:11:38.443654 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:11:38.443630 2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-dccc57588-twsww" event={"ID":"f5a33912-2fd4-4b0f-b2b6-0730dbd0d82d","Type":"ContainerStarted","Data":"d6b0cb0308d2f809642c415095b85ea01cceadfdf0bc36993935d1fd0c765300"} May 27 20:11:43.454206 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:11:43.454164 2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6fc8f5cd4b-wrbm9" event={"ID":"35683361-3a05-46ce-8d13-43a5a027c40f","Type":"ContainerStarted","Data":"53c02d02551eb980548cb2bff3a76f681b10497255709b75b70ef0d652af58cd"} May 27 20:11:43.455476 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:11:43.455450 2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-dccc57588-twsww" event={"ID":"f5a33912-2fd4-4b0f-b2b6-0730dbd0d82d","Type":"ContainerStarted","Data":"1d1d0fb92af4f7bf69e117f8bcd26f400590bdb0c27d8450c91b316f9fdbe93f"} May 27 20:11:43.470011 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:11:43.469968 2571 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6fc8f5cd4b-wrbm9" podStartSLOduration=1.880831852 podStartE2EDuration="6.469956315s" podCreationTimestamp="2026-05-27 20:11:37 +0000 UTC" firstStartedPulling="2026-05-27 20:11:38.014346924 +0000 UTC m=+50.423973213" lastFinishedPulling="2026-05-27 20:11:42.603471386 +0000 UTC m=+55.013097676" observedRunningTime="2026-05-27 20:11:43.469442952 +0000 UTC m=+55.879069266" watchObservedRunningTime="2026-05-27 20:11:43.469956315 +0000 UTC m=+55.879582626" May 27 20:11:46.463006 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:11:46.462967 2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-dccc57588-twsww" event={"ID":"f5a33912-2fd4-4b0f-b2b6-0730dbd0d82d","Type":"ContainerStarted","Data":"4e633d4e18fad441fce3c40369a9601f1863e46d03201fabf3e190dc7b0ced97"} May 27 20:11:46.463006 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:11:46.463005 2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-dccc57588-twsww" event={"ID":"f5a33912-2fd4-4b0f-b2b6-0730dbd0d82d","Type":"ContainerStarted","Data":"40b5ed068fe9d4c5da1711006328a4cad6aafbddda6f4b9e5eb70fa5defbcd6b"} May 27 20:11:46.482663 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:11:46.482604 2571 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-dccc57588-twsww" podStartSLOduration=2.042748794 podStartE2EDuration="9.482592403s" podCreationTimestamp="2026-05-27 20:11:37 +0000 UTC" firstStartedPulling="2026-05-27 20:11:37.998625582 +0000 UTC m=+50.408251871" lastFinishedPulling="2026-05-27 20:11:45.438469176 +0000 UTC m=+57.848095480" observedRunningTime="2026-05-27 20:11:46.480667064 +0000 UTC m=+58.890293377" watchObservedRunningTime="2026-05-27 20:11:46.482592403 +0000 UTC m=+58.892218734" May 27 20:11:48.403387 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:11:48.403358 2571 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-rcvbq" May 27 20:11:53.880876 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:11:53.880834 2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/96207b98-716e-4c63-847f-fc976f5563f2-metrics-certs\") pod \"network-metrics-daemon-zsx5p\" (UID: \"96207b98-716e-4c63-847f-fc976f5563f2\") " pod="openshift-multus/network-metrics-daemon-zsx5p" May 27 20:11:53.883871 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:11:53.883848 2571 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" May 27 20:11:53.892001 ip-10-0-142-251 kubenswrapper[2571]: E0527 20:11:53.891973 2571 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found May 27 20:11:53.892132 ip-10-0-142-251 kubenswrapper[2571]: E0527 20:11:53.892047 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/96207b98-716e-4c63-847f-fc976f5563f2-metrics-certs podName:96207b98-716e-4c63-847f-fc976f5563f2 nodeName:}" failed. No retries permitted until 2026-05-27 20:12:57.892028392 +0000 UTC m=+130.301654681 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/96207b98-716e-4c63-847f-fc976f5563f2-metrics-certs") pod "network-metrics-daemon-zsx5p" (UID: "96207b98-716e-4c63-847f-fc976f5563f2") : secret "metrics-daemon-secret" not found May 27 20:11:53.982076 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:11:53.982036 2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-gpj4z\" (UniqueName: \"kubernetes.io/projected/de3a4bee-c44a-4da9-833f-7e04d40c1f82-kube-api-access-gpj4z\") pod \"network-check-target-jdzbr\" (UID: \"de3a4bee-c44a-4da9-833f-7e04d40c1f82\") " pod="openshift-network-diagnostics/network-check-target-jdzbr" May 27 20:11:53.984673 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:11:53.984657 2571 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" May 27 20:11:53.995441 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:11:53.995419 2571 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" May 27 20:11:54.006048 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:11:54.006024 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gpj4z\" (UniqueName: \"kubernetes.io/projected/de3a4bee-c44a-4da9-833f-7e04d40c1f82-kube-api-access-gpj4z\") pod \"network-check-target-jdzbr\" (UID: \"de3a4bee-c44a-4da9-833f-7e04d40c1f82\") " pod="openshift-network-diagnostics/network-check-target-jdzbr" May 27 20:11:54.243406 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:11:54.243328 2571 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-g79gn\"" May 27 20:11:54.251561 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:11:54.251535 2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jdzbr" May 27 20:11:54.371191 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:11:54.371160 2571 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-jdzbr"] May 27 20:11:54.374050 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:11:54.374017 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podde3a4bee_c44a_4da9_833f_7e04d40c1f82.slice/crio-c4f789d2c332f3200fc780918854ca460e4a46260f8f9f92651d979d9ec40fe1 WatchSource:0}: Error finding container c4f789d2c332f3200fc780918854ca460e4a46260f8f9f92651d979d9ec40fe1: Status 404 returned error can't find the container with id c4f789d2c332f3200fc780918854ca460e4a46260f8f9f92651d979d9ec40fe1 May 27 20:11:54.384193 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:11:54.384171 2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d6fb3e28-cbfd-47a3-8104-bbfa86006ad6-cert\") pod \"ingress-canary-jxjsb\" (UID: \"d6fb3e28-cbfd-47a3-8104-bbfa86006ad6\") " pod="openshift-ingress-canary/ingress-canary-jxjsb" May 27 20:11:54.384322 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:11:54.384216 2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/cf585445-0773-47ca-ba45-3bad1a2d5b88-metrics-tls\") pod \"dns-default-58qkv\" (UID: \"cf585445-0773-47ca-ba45-3bad1a2d5b88\") " pod="openshift-dns/dns-default-58qkv" May 27 20:11:54.384381 ip-10-0-142-251 kubenswrapper[2571]: E0527 20:11:54.384340 2571 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found May 27 20:11:54.384381 ip-10-0-142-251 kubenswrapper[2571]: E0527 20:11:54.384341 2571 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found May 27 20:11:54.384476 ip-10-0-142-251 kubenswrapper[2571]: E0527 20:11:54.384405 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cf585445-0773-47ca-ba45-3bad1a2d5b88-metrics-tls podName:cf585445-0773-47ca-ba45-3bad1a2d5b88 nodeName:}" failed. No retries permitted until 2026-05-27 20:12:26.384386117 +0000 UTC m=+98.794012424 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/cf585445-0773-47ca-ba45-3bad1a2d5b88-metrics-tls") pod "dns-default-58qkv" (UID: "cf585445-0773-47ca-ba45-3bad1a2d5b88") : secret "dns-default-metrics-tls" not found May 27 20:11:54.384476 ip-10-0-142-251 kubenswrapper[2571]: E0527 20:11:54.384426 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d6fb3e28-cbfd-47a3-8104-bbfa86006ad6-cert podName:d6fb3e28-cbfd-47a3-8104-bbfa86006ad6 nodeName:}" failed. No retries permitted until 2026-05-27 20:12:26.384415136 +0000 UTC m=+98.794041431 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/d6fb3e28-cbfd-47a3-8104-bbfa86006ad6-cert") pod "ingress-canary-jxjsb" (UID: "d6fb3e28-cbfd-47a3-8104-bbfa86006ad6") : secret "canary-serving-cert" not found May 27 20:11:54.478759 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:11:54.478719 2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-jdzbr" event={"ID":"de3a4bee-c44a-4da9-833f-7e04d40c1f82","Type":"ContainerStarted","Data":"c4f789d2c332f3200fc780918854ca460e4a46260f8f9f92651d979d9ec40fe1"} May 27 20:11:58.487910 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:11:58.487872 2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-jdzbr" event={"ID":"de3a4bee-c44a-4da9-833f-7e04d40c1f82","Type":"ContainerStarted","Data":"340a7300e1f01483ca0d069d3096e36a054764606c389472912092c37af5e38d"} May 27 20:11:58.488299 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:11:58.488053 2571 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-jdzbr" May 27 20:11:58.507137 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:11:58.507066 2571 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-jdzbr" podStartSLOduration=67.411401619 podStartE2EDuration="1m10.507053736s" podCreationTimestamp="2026-05-27 20:10:48 +0000 UTC" firstStartedPulling="2026-05-27 20:11:54.375923753 +0000 UTC m=+66.785550041" lastFinishedPulling="2026-05-27 20:11:57.471575858 +0000 UTC m=+69.881202158" observedRunningTime="2026-05-27 20:11:58.506335201 +0000 UTC m=+70.915961512" watchObservedRunningTime="2026-05-27 20:11:58.507053736 +0000 UTC m=+70.916680046" May 27 20:12:24.279365 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:12:24.279335 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-n4brm_4937a11c-a8df-40eb-9a62-0b9fa6ea34ef/dns-node-resolver/0.log" May 27 20:12:25.279203 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:12:25.279174 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-f4p5s_cdf5a20d-c5e1-4d30-a6ce-85c91afd58e2/node-ca/0.log" May 27 20:12:26.408963 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:12:26.408885 2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d6fb3e28-cbfd-47a3-8104-bbfa86006ad6-cert\") pod \"ingress-canary-jxjsb\" (UID: \"d6fb3e28-cbfd-47a3-8104-bbfa86006ad6\") " pod="openshift-ingress-canary/ingress-canary-jxjsb" May 27 20:12:26.408963 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:12:26.408925 2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/cf585445-0773-47ca-ba45-3bad1a2d5b88-metrics-tls\") pod \"dns-default-58qkv\" (UID: \"cf585445-0773-47ca-ba45-3bad1a2d5b88\") " pod="openshift-dns/dns-default-58qkv" May 27 20:12:26.409409 ip-10-0-142-251 kubenswrapper[2571]: E0527 20:12:26.409023 2571 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found May 27 20:12:26.409409 ip-10-0-142-251 kubenswrapper[2571]: E0527 20:12:26.409044 2571 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found May 27 20:12:26.409409 ip-10-0-142-251 kubenswrapper[2571]: E0527 20:12:26.409090 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cf585445-0773-47ca-ba45-3bad1a2d5b88-metrics-tls podName:cf585445-0773-47ca-ba45-3bad1a2d5b88 nodeName:}" failed. No retries permitted until 2026-05-27 20:13:30.409066996 +0000 UTC m=+162.818693288 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/cf585445-0773-47ca-ba45-3bad1a2d5b88-metrics-tls") pod "dns-default-58qkv" (UID: "cf585445-0773-47ca-ba45-3bad1a2d5b88") : secret "dns-default-metrics-tls" not found May 27 20:12:26.409409 ip-10-0-142-251 kubenswrapper[2571]: E0527 20:12:26.409123 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d6fb3e28-cbfd-47a3-8104-bbfa86006ad6-cert podName:d6fb3e28-cbfd-47a3-8104-bbfa86006ad6 nodeName:}" failed. No retries permitted until 2026-05-27 20:13:30.409097341 +0000 UTC m=+162.818723629 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/d6fb3e28-cbfd-47a3-8104-bbfa86006ad6-cert") pod "ingress-canary-jxjsb" (UID: "d6fb3e28-cbfd-47a3-8104-bbfa86006ad6") : secret "canary-serving-cert" not found May 27 20:12:29.493081 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:12:29.493050 2571 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-jdzbr" May 27 20:12:44.943157 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:12:44.943127 2571 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-625bj"] May 27 20:12:44.946023 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:12:44.946006 2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-625bj" May 27 20:12:44.949038 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:12:44.949016 2571 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" May 27 20:12:44.950021 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:12:44.949995 2571 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-4nk26\"" May 27 20:12:44.950021 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:12:44.950004 2571 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" May 27 20:12:44.950219 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:12:44.950198 2571 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" May 27 20:12:44.950281 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:12:44.950206 2571 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" May 27 20:12:44.957116 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:12:44.957070 2571 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-625bj"] May 27 20:12:45.042746 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:12:45.042716 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/66ac589d-841c-4b3d-87f5-e57a079b95e3-data-volume\") pod \"insights-runtime-extractor-625bj\" (UID: \"66ac589d-841c-4b3d-87f5-e57a079b95e3\") " pod="openshift-insights/insights-runtime-extractor-625bj" May 27 20:12:45.042913 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:12:45.042798 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/66ac589d-841c-4b3d-87f5-e57a079b95e3-crio-socket\") pod \"insights-runtime-extractor-625bj\" (UID: \"66ac589d-841c-4b3d-87f5-e57a079b95e3\") " pod="openshift-insights/insights-runtime-extractor-625bj" May 27 20:12:45.042913 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:12:45.042818 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/66ac589d-841c-4b3d-87f5-e57a079b95e3-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-625bj\" (UID: \"66ac589d-841c-4b3d-87f5-e57a079b95e3\") " pod="openshift-insights/insights-runtime-extractor-625bj" May 27 20:12:45.042913 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:12:45.042840 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/66ac589d-841c-4b3d-87f5-e57a079b95e3-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-625bj\" (UID: \"66ac589d-841c-4b3d-87f5-e57a079b95e3\") " pod="openshift-insights/insights-runtime-extractor-625bj" May 27 20:12:45.042913 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:12:45.042855 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-44s5r\" (UniqueName: \"kubernetes.io/projected/66ac589d-841c-4b3d-87f5-e57a079b95e3-kube-api-access-44s5r\") pod \"insights-runtime-extractor-625bj\" (UID: \"66ac589d-841c-4b3d-87f5-e57a079b95e3\") " pod="openshift-insights/insights-runtime-extractor-625bj" May 27 20:12:45.134595 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:12:45.134556 2571 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-6bc4d5d5c8-7lb24"] May 27 20:12:45.137340 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:12:45.137316 2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6bc4d5d5c8-7lb24" May 27 20:12:45.140482 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:12:45.140462 2571 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" May 27 20:12:45.140560 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:12:45.140501 2571 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" May 27 20:12:45.142043 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:12:45.142014 2571 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-vxj6j\"" May 27 20:12:45.143671 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:12:45.143207 2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/66ac589d-841c-4b3d-87f5-e57a079b95e3-data-volume\") pod \"insights-runtime-extractor-625bj\" (UID: \"66ac589d-841c-4b3d-87f5-e57a079b95e3\") " pod="openshift-insights/insights-runtime-extractor-625bj" May 27 20:12:45.143671 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:12:45.143312 2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/66ac589d-841c-4b3d-87f5-e57a079b95e3-crio-socket\") pod \"insights-runtime-extractor-625bj\" (UID: \"66ac589d-841c-4b3d-87f5-e57a079b95e3\") " pod="openshift-insights/insights-runtime-extractor-625bj" May 27 20:12:45.143671 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:12:45.143342 2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/66ac589d-841c-4b3d-87f5-e57a079b95e3-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-625bj\" (UID: \"66ac589d-841c-4b3d-87f5-e57a079b95e3\") " pod="openshift-insights/insights-runtime-extractor-625bj" May 27 20:12:45.143671 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:12:45.143386 2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/66ac589d-841c-4b3d-87f5-e57a079b95e3-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-625bj\" (UID: \"66ac589d-841c-4b3d-87f5-e57a079b95e3\") " pod="openshift-insights/insights-runtime-extractor-625bj" May 27 20:12:45.143671 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:12:45.143411 2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-44s5r\" (UniqueName: \"kubernetes.io/projected/66ac589d-841c-4b3d-87f5-e57a079b95e3-kube-api-access-44s5r\") pod \"insights-runtime-extractor-625bj\" (UID: \"66ac589d-841c-4b3d-87f5-e57a079b95e3\") " pod="openshift-insights/insights-runtime-extractor-625bj" May 27 20:12:45.143671 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:12:45.143555 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/66ac589d-841c-4b3d-87f5-e57a079b95e3-data-volume\") pod \"insights-runtime-extractor-625bj\" (UID: \"66ac589d-841c-4b3d-87f5-e57a079b95e3\") " pod="openshift-insights/insights-runtime-extractor-625bj" May 27 20:12:45.143945 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:12:45.143847 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/66ac589d-841c-4b3d-87f5-e57a079b95e3-crio-socket\") pod \"insights-runtime-extractor-625bj\" (UID: \"66ac589d-841c-4b3d-87f5-e57a079b95e3\") " pod="openshift-insights/insights-runtime-extractor-625bj" May 27 20:12:45.144011 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:12:45.143990 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/66ac589d-841c-4b3d-87f5-e57a079b95e3-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-625bj\" (UID: \"66ac589d-841c-4b3d-87f5-e57a079b95e3\") " pod="openshift-insights/insights-runtime-extractor-625bj" May 27 20:12:45.144584 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:12:45.144566 2571 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" May 27 20:12:45.145796 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:12:45.145779 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/66ac589d-841c-4b3d-87f5-e57a079b95e3-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-625bj\" (UID: \"66ac589d-841c-4b3d-87f5-e57a079b95e3\") " pod="openshift-insights/insights-runtime-extractor-625bj" May 27 20:12:45.146092 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:12:45.146074 2571 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" May 27 20:12:45.154281 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:12:45.154262 2571 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6bc4d5d5c8-7lb24"] May 27 20:12:45.160405 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:12:45.160383 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-44s5r\" (UniqueName: \"kubernetes.io/projected/66ac589d-841c-4b3d-87f5-e57a079b95e3-kube-api-access-44s5r\") pod \"insights-runtime-extractor-625bj\" (UID: \"66ac589d-841c-4b3d-87f5-e57a079b95e3\") " pod="openshift-insights/insights-runtime-extractor-625bj" May 27 20:12:45.244603 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:12:45.244528 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/d292858c-6349-4ea3-8019-fa4bbf14d6a9-trusted-ca\") pod \"image-registry-6bc4d5d5c8-7lb24\" (UID: \"d292858c-6349-4ea3-8019-fa4bbf14d6a9\") " pod="openshift-image-registry/image-registry-6bc4d5d5c8-7lb24" May 27 20:12:45.244959 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:12:45.244564 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5kksd\" (UniqueName: \"kubernetes.io/projected/d292858c-6349-4ea3-8019-fa4bbf14d6a9-kube-api-access-5kksd\") pod \"image-registry-6bc4d5d5c8-7lb24\" (UID: \"d292858c-6349-4ea3-8019-fa4bbf14d6a9\") " pod="openshift-image-registry/image-registry-6bc4d5d5c8-7lb24" May 27 20:12:45.247805 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:12:45.245073 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d292858c-6349-4ea3-8019-fa4bbf14d6a9-registry-tls\") pod \"image-registry-6bc4d5d5c8-7lb24\" (UID: \"d292858c-6349-4ea3-8019-fa4bbf14d6a9\") " pod="openshift-image-registry/image-registry-6bc4d5d5c8-7lb24" May 27 20:12:45.247805 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:12:45.245198 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/d292858c-6349-4ea3-8019-fa4bbf14d6a9-registry-certificates\") pod \"image-registry-6bc4d5d5c8-7lb24\" (UID: \"d292858c-6349-4ea3-8019-fa4bbf14d6a9\") " pod="openshift-image-registry/image-registry-6bc4d5d5c8-7lb24" May 27 20:12:45.247805 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:12:45.245270 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/d292858c-6349-4ea3-8019-fa4bbf14d6a9-installation-pull-secrets\") pod \"image-registry-6bc4d5d5c8-7lb24\" (UID: \"d292858c-6349-4ea3-8019-fa4bbf14d6a9\") " pod="openshift-image-registry/image-registry-6bc4d5d5c8-7lb24" May 27 20:12:45.247805 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:12:45.245315 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/d292858c-6349-4ea3-8019-fa4bbf14d6a9-ca-trust-extracted\") pod \"image-registry-6bc4d5d5c8-7lb24\" (UID: \"d292858c-6349-4ea3-8019-fa4bbf14d6a9\") " pod="openshift-image-registry/image-registry-6bc4d5d5c8-7lb24" May 27 20:12:45.247805 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:12:45.245370 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d292858c-6349-4ea3-8019-fa4bbf14d6a9-bound-sa-token\") pod \"image-registry-6bc4d5d5c8-7lb24\" (UID: \"d292858c-6349-4ea3-8019-fa4bbf14d6a9\") " pod="openshift-image-registry/image-registry-6bc4d5d5c8-7lb24" May 27 20:12:45.247805 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:12:45.245402 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/d292858c-6349-4ea3-8019-fa4bbf14d6a9-image-registry-private-configuration\") pod \"image-registry-6bc4d5d5c8-7lb24\" (UID: \"d292858c-6349-4ea3-8019-fa4bbf14d6a9\") " pod="openshift-image-registry/image-registry-6bc4d5d5c8-7lb24" May 27 20:12:45.254246 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:12:45.254225 2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-625bj" May 27 20:12:45.346545 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:12:45.346514 2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/d292858c-6349-4ea3-8019-fa4bbf14d6a9-trusted-ca\") pod \"image-registry-6bc4d5d5c8-7lb24\" (UID: \"d292858c-6349-4ea3-8019-fa4bbf14d6a9\") " pod="openshift-image-registry/image-registry-6bc4d5d5c8-7lb24" May 27 20:12:45.346545 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:12:45.346550 2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-5kksd\" (UniqueName: \"kubernetes.io/projected/d292858c-6349-4ea3-8019-fa4bbf14d6a9-kube-api-access-5kksd\") pod \"image-registry-6bc4d5d5c8-7lb24\" (UID: \"d292858c-6349-4ea3-8019-fa4bbf14d6a9\") " pod="openshift-image-registry/image-registry-6bc4d5d5c8-7lb24" May 27 20:12:45.346719 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:12:45.346599 2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d292858c-6349-4ea3-8019-fa4bbf14d6a9-registry-tls\") pod \"image-registry-6bc4d5d5c8-7lb24\" (UID: \"d292858c-6349-4ea3-8019-fa4bbf14d6a9\") " pod="openshift-image-registry/image-registry-6bc4d5d5c8-7lb24" May 27 20:12:45.346719 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:12:45.346624 2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/d292858c-6349-4ea3-8019-fa4bbf14d6a9-registry-certificates\") pod \"image-registry-6bc4d5d5c8-7lb24\" (UID: \"d292858c-6349-4ea3-8019-fa4bbf14d6a9\") " pod="openshift-image-registry/image-registry-6bc4d5d5c8-7lb24" May 27 20:12:45.346719 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:12:45.346677 2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/d292858c-6349-4ea3-8019-fa4bbf14d6a9-installation-pull-secrets\") pod \"image-registry-6bc4d5d5c8-7lb24\" (UID: \"d292858c-6349-4ea3-8019-fa4bbf14d6a9\") " pod="openshift-image-registry/image-registry-6bc4d5d5c8-7lb24" May 27 20:12:45.346848 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:12:45.346717 2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/d292858c-6349-4ea3-8019-fa4bbf14d6a9-ca-trust-extracted\") pod \"image-registry-6bc4d5d5c8-7lb24\" (UID: \"d292858c-6349-4ea3-8019-fa4bbf14d6a9\") " pod="openshift-image-registry/image-registry-6bc4d5d5c8-7lb24" May 27 20:12:45.346848 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:12:45.346759 2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d292858c-6349-4ea3-8019-fa4bbf14d6a9-bound-sa-token\") pod \"image-registry-6bc4d5d5c8-7lb24\" (UID: \"d292858c-6349-4ea3-8019-fa4bbf14d6a9\") " pod="openshift-image-registry/image-registry-6bc4d5d5c8-7lb24" May 27 20:12:45.346848 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:12:45.346789 2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/d292858c-6349-4ea3-8019-fa4bbf14d6a9-image-registry-private-configuration\") pod \"image-registry-6bc4d5d5c8-7lb24\" (UID: \"d292858c-6349-4ea3-8019-fa4bbf14d6a9\") " pod="openshift-image-registry/image-registry-6bc4d5d5c8-7lb24" May 27 20:12:45.347505 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:12:45.347474 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/d292858c-6349-4ea3-8019-fa4bbf14d6a9-ca-trust-extracted\") pod \"image-registry-6bc4d5d5c8-7lb24\" (UID: \"d292858c-6349-4ea3-8019-fa4bbf14d6a9\") " pod="openshift-image-registry/image-registry-6bc4d5d5c8-7lb24" May 27 20:12:45.347620 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:12:45.347519 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/d292858c-6349-4ea3-8019-fa4bbf14d6a9-registry-certificates\") pod \"image-registry-6bc4d5d5c8-7lb24\" (UID: \"d292858c-6349-4ea3-8019-fa4bbf14d6a9\") " pod="openshift-image-registry/image-registry-6bc4d5d5c8-7lb24" May 27 20:12:45.347773 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:12:45.347751 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/d292858c-6349-4ea3-8019-fa4bbf14d6a9-trusted-ca\") pod \"image-registry-6bc4d5d5c8-7lb24\" (UID: \"d292858c-6349-4ea3-8019-fa4bbf14d6a9\") " pod="openshift-image-registry/image-registry-6bc4d5d5c8-7lb24" May 27 20:12:45.349347 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:12:45.349319 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/d292858c-6349-4ea3-8019-fa4bbf14d6a9-image-registry-private-configuration\") pod \"image-registry-6bc4d5d5c8-7lb24\" (UID: \"d292858c-6349-4ea3-8019-fa4bbf14d6a9\") " pod="openshift-image-registry/image-registry-6bc4d5d5c8-7lb24" May 27 20:12:45.349444 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:12:45.349392 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/d292858c-6349-4ea3-8019-fa4bbf14d6a9-installation-pull-secrets\") pod \"image-registry-6bc4d5d5c8-7lb24\" (UID: \"d292858c-6349-4ea3-8019-fa4bbf14d6a9\") " pod="openshift-image-registry/image-registry-6bc4d5d5c8-7lb24" May 27 20:12:45.349689 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:12:45.349671 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d292858c-6349-4ea3-8019-fa4bbf14d6a9-registry-tls\") pod \"image-registry-6bc4d5d5c8-7lb24\" (UID: \"d292858c-6349-4ea3-8019-fa4bbf14d6a9\") " pod="openshift-image-registry/image-registry-6bc4d5d5c8-7lb24" May 27 20:12:45.361736 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:12:45.361704 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5kksd\" (UniqueName: \"kubernetes.io/projected/d292858c-6349-4ea3-8019-fa4bbf14d6a9-kube-api-access-5kksd\") pod \"image-registry-6bc4d5d5c8-7lb24\" (UID: \"d292858c-6349-4ea3-8019-fa4bbf14d6a9\") " pod="openshift-image-registry/image-registry-6bc4d5d5c8-7lb24" May 27 20:12:45.365420 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:12:45.365400 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d292858c-6349-4ea3-8019-fa4bbf14d6a9-bound-sa-token\") pod \"image-registry-6bc4d5d5c8-7lb24\" (UID: \"d292858c-6349-4ea3-8019-fa4bbf14d6a9\") " pod="openshift-image-registry/image-registry-6bc4d5d5c8-7lb24" May 27 20:12:45.408070 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:12:45.408039 2571 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-625bj"] May 27 20:12:45.446938 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:12:45.446901 2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6bc4d5d5c8-7lb24" May 27 20:12:45.580409 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:12:45.580376 2571 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6bc4d5d5c8-7lb24"] May 27 20:12:45.583535 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:12:45.583509 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd292858c_6349_4ea3_8019_fa4bbf14d6a9.slice/crio-3514751709a123b910d4e9fb465afd6c642ea09108c97c1a4b171ae208f14bd7 WatchSource:0}: Error finding container 3514751709a123b910d4e9fb465afd6c642ea09108c97c1a4b171ae208f14bd7: Status 404 returned error can't find the container with id 3514751709a123b910d4e9fb465afd6c642ea09108c97c1a4b171ae208f14bd7 May 27 20:12:45.600822 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:12:45.600790 2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6bc4d5d5c8-7lb24" event={"ID":"d292858c-6349-4ea3-8019-fa4bbf14d6a9","Type":"ContainerStarted","Data":"3514751709a123b910d4e9fb465afd6c642ea09108c97c1a4b171ae208f14bd7"} May 27 20:12:45.602444 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:12:45.602412 2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-625bj" event={"ID":"66ac589d-841c-4b3d-87f5-e57a079b95e3","Type":"ContainerStarted","Data":"3e57c2e2bce1305465c6224417f45c21842914ded232bc625ab8b6b86f50861a"} May 27 20:12:45.602444 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:12:45.602442 2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-625bj" event={"ID":"66ac589d-841c-4b3d-87f5-e57a079b95e3","Type":"ContainerStarted","Data":"25d1d2ebbe1f897a5af5c013a654dca35ed0a51b09d903d16f9747de98705542"} May 27 20:12:46.606049 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:12:46.606010 2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6bc4d5d5c8-7lb24" event={"ID":"d292858c-6349-4ea3-8019-fa4bbf14d6a9","Type":"ContainerStarted","Data":"f46e3dadda5e2fdca2222c44e59760e63f60f53ec66691e810079957267c44a5"} May 27 20:12:46.606541 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:12:46.606077 2571 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-6bc4d5d5c8-7lb24" May 27 20:12:46.607797 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:12:46.607763 2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-625bj" event={"ID":"66ac589d-841c-4b3d-87f5-e57a079b95e3","Type":"ContainerStarted","Data":"a69a8bd098075f630ec5180be4f1a81ad435fabeb1419c39ac6b46b1d03bdd0d"} May 27 20:12:46.626122 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:12:46.626057 2571 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-image-registry/image-registry-6bc4d5d5c8-7lb24" podStartSLOduration=1.626041386 podStartE2EDuration="1.626041386s" podCreationTimestamp="2026-05-27 20:12:45 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-27 20:12:46.625029534 +0000 UTC m=+119.034655881" watchObservedRunningTime="2026-05-27 20:12:46.626041386 +0000 UTC m=+119.035667701" May 27 20:12:47.612275 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:12:47.612238 2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-625bj" event={"ID":"66ac589d-841c-4b3d-87f5-e57a079b95e3","Type":"ContainerStarted","Data":"65b5cf1eece08bd7cf3c8a592d88c3037c90338475c29557e4b561995cdd113d"} May 27 20:12:47.631030 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:12:47.630986 2571 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-625bj" podStartSLOduration=1.646788399 podStartE2EDuration="3.630971801s" podCreationTimestamp="2026-05-27 20:12:44 +0000 UTC" firstStartedPulling="2026-05-27 20:12:45.47690131 +0000 UTC m=+117.886527602" lastFinishedPulling="2026-05-27 20:12:47.461084697 +0000 UTC m=+119.870711004" observedRunningTime="2026-05-27 20:12:47.629429371 +0000 UTC m=+120.039055683" watchObservedRunningTime="2026-05-27 20:12:47.630971801 +0000 UTC m=+120.040598113" May 27 20:12:57.003134 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:12:57.003083 2571 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-9c959559f-wgtzh"] May 27 20:12:57.007886 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:12:57.007863 2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-9c959559f-wgtzh" May 27 20:12:57.011068 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:12:57.011048 2571 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\"" May 27 20:12:57.012096 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:12:57.012070 2571 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" May 27 20:12:57.012224 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:12:57.012137 2571 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-bzwpw\"" May 27 20:12:57.012224 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:12:57.012176 2571 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\"" May 27 20:12:57.012224 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:12:57.012145 2571 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" May 27 20:12:57.012388 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:12:57.012208 2571 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" May 27 20:12:57.022253 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:12:57.022232 2571 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-9c959559f-wgtzh"] May 27 20:12:57.041650 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:12:57.041623 2571 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-r9grk"] May 27 20:12:57.044740 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:12:57.044726 2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-r9grk" May 27 20:12:57.048839 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:12:57.048816 2571 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" May 27 20:12:57.048960 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:12:57.048945 2571 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-mflgp\"" May 27 20:12:57.049273 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:12:57.049255 2571 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" May 27 20:12:57.049338 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:12:57.049261 2571 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" May 27 20:12:57.139616 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:12:57.139577 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/bb42b85b-4372-4700-ba7f-d5e91b80e3f1-node-exporter-textfile\") pod \"node-exporter-r9grk\" (UID: \"bb42b85b-4372-4700-ba7f-d5e91b80e3f1\") " pod="openshift-monitoring/node-exporter-r9grk" May 27 20:12:57.139793 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:12:57.139627 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/bb42b85b-4372-4700-ba7f-d5e91b80e3f1-node-exporter-wtmp\") pod \"node-exporter-r9grk\" (UID: \"bb42b85b-4372-4700-ba7f-d5e91b80e3f1\") " pod="openshift-monitoring/node-exporter-r9grk" May 27 20:12:57.139793 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:12:57.139681 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2gmr8\" (UniqueName: \"kubernetes.io/projected/5710019c-8d7a-446e-b750-85706ec7bf14-kube-api-access-2gmr8\") pod \"openshift-state-metrics-9c959559f-wgtzh\" (UID: \"5710019c-8d7a-446e-b750-85706ec7bf14\") " pod="openshift-monitoring/openshift-state-metrics-9c959559f-wgtzh" May 27 20:12:57.139793 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:12:57.139715 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/bb42b85b-4372-4700-ba7f-d5e91b80e3f1-metrics-client-ca\") pod \"node-exporter-r9grk\" (UID: \"bb42b85b-4372-4700-ba7f-d5e91b80e3f1\") " pod="openshift-monitoring/node-exporter-r9grk" May 27 20:12:57.139793 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:12:57.139739 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/bb42b85b-4372-4700-ba7f-d5e91b80e3f1-sys\") pod \"node-exporter-r9grk\" (UID: \"bb42b85b-4372-4700-ba7f-d5e91b80e3f1\") " pod="openshift-monitoring/node-exporter-r9grk" May 27 20:12:57.139793 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:12:57.139761 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/5710019c-8d7a-446e-b750-85706ec7bf14-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-9c959559f-wgtzh\" (UID: \"5710019c-8d7a-446e-b750-85706ec7bf14\") " pod="openshift-monitoring/openshift-state-metrics-9c959559f-wgtzh" May 27 20:12:57.139974 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:12:57.139828 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9drmq\" (UniqueName: \"kubernetes.io/projected/bb42b85b-4372-4700-ba7f-d5e91b80e3f1-kube-api-access-9drmq\") pod \"node-exporter-r9grk\" (UID: \"bb42b85b-4372-4700-ba7f-d5e91b80e3f1\") " pod="openshift-monitoring/node-exporter-r9grk" May 27 20:12:57.139974 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:12:57.139854 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/bb42b85b-4372-4700-ba7f-d5e91b80e3f1-node-exporter-tls\") pod \"node-exporter-r9grk\" (UID: \"bb42b85b-4372-4700-ba7f-d5e91b80e3f1\") " pod="openshift-monitoring/node-exporter-r9grk" May 27 20:12:57.139974 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:12:57.139881 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/bb42b85b-4372-4700-ba7f-d5e91b80e3f1-node-exporter-accelerators-collector-config\") pod \"node-exporter-r9grk\" (UID: \"bb42b85b-4372-4700-ba7f-d5e91b80e3f1\") " pod="openshift-monitoring/node-exporter-r9grk" May 27 20:12:57.139974 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:12:57.139903 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/bb42b85b-4372-4700-ba7f-d5e91b80e3f1-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-r9grk\" (UID: \"bb42b85b-4372-4700-ba7f-d5e91b80e3f1\") " pod="openshift-monitoring/node-exporter-r9grk" May 27 20:12:57.139974 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:12:57.139924 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/5710019c-8d7a-446e-b750-85706ec7bf14-openshift-state-metrics-tls\") pod \"openshift-state-metrics-9c959559f-wgtzh\" (UID: \"5710019c-8d7a-446e-b750-85706ec7bf14\") " pod="openshift-monitoring/openshift-state-metrics-9c959559f-wgtzh" May 27 20:12:57.139974 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:12:57.139942 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5710019c-8d7a-446e-b750-85706ec7bf14-metrics-client-ca\") pod \"openshift-state-metrics-9c959559f-wgtzh\" (UID: \"5710019c-8d7a-446e-b750-85706ec7bf14\") " pod="openshift-monitoring/openshift-state-metrics-9c959559f-wgtzh" May 27 20:12:57.140245 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:12:57.139981 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/bb42b85b-4372-4700-ba7f-d5e91b80e3f1-root\") pod \"node-exporter-r9grk\" (UID: \"bb42b85b-4372-4700-ba7f-d5e91b80e3f1\") " pod="openshift-monitoring/node-exporter-r9grk" May 27 20:12:57.241304 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:12:57.241271 2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-9drmq\" (UniqueName: \"kubernetes.io/projected/bb42b85b-4372-4700-ba7f-d5e91b80e3f1-kube-api-access-9drmq\") pod \"node-exporter-r9grk\" (UID: \"bb42b85b-4372-4700-ba7f-d5e91b80e3f1\") " pod="openshift-monitoring/node-exporter-r9grk" May 27 20:12:57.241304 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:12:57.241305 2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/bb42b85b-4372-4700-ba7f-d5e91b80e3f1-node-exporter-tls\") pod \"node-exporter-r9grk\" (UID: \"bb42b85b-4372-4700-ba7f-d5e91b80e3f1\") " pod="openshift-monitoring/node-exporter-r9grk" May 27 20:12:57.241535 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:12:57.241324 2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/bb42b85b-4372-4700-ba7f-d5e91b80e3f1-node-exporter-accelerators-collector-config\") pod \"node-exporter-r9grk\" (UID: \"bb42b85b-4372-4700-ba7f-d5e91b80e3f1\") " pod="openshift-monitoring/node-exporter-r9grk" May 27 20:12:57.241535 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:12:57.241344 2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/bb42b85b-4372-4700-ba7f-d5e91b80e3f1-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-r9grk\" (UID: \"bb42b85b-4372-4700-ba7f-d5e91b80e3f1\") " pod="openshift-monitoring/node-exporter-r9grk" May 27 20:12:57.241535 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:12:57.241393 2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/5710019c-8d7a-446e-b750-85706ec7bf14-openshift-state-metrics-tls\") pod \"openshift-state-metrics-9c959559f-wgtzh\" (UID: \"5710019c-8d7a-446e-b750-85706ec7bf14\") " pod="openshift-monitoring/openshift-state-metrics-9c959559f-wgtzh" May 27 20:12:57.241535 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:12:57.241421 2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5710019c-8d7a-446e-b750-85706ec7bf14-metrics-client-ca\") pod \"openshift-state-metrics-9c959559f-wgtzh\" (UID: \"5710019c-8d7a-446e-b750-85706ec7bf14\") " pod="openshift-monitoring/openshift-state-metrics-9c959559f-wgtzh" May 27 20:12:57.241535 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:12:57.241466 2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/bb42b85b-4372-4700-ba7f-d5e91b80e3f1-root\") pod \"node-exporter-r9grk\" (UID: \"bb42b85b-4372-4700-ba7f-d5e91b80e3f1\") " pod="openshift-monitoring/node-exporter-r9grk" May 27 20:12:57.241535 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:12:57.241497 2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/bb42b85b-4372-4700-ba7f-d5e91b80e3f1-node-exporter-textfile\") pod \"node-exporter-r9grk\" (UID: \"bb42b85b-4372-4700-ba7f-d5e91b80e3f1\") " pod="openshift-monitoring/node-exporter-r9grk" May 27 20:12:57.241853 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:12:57.241539 2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/bb42b85b-4372-4700-ba7f-d5e91b80e3f1-node-exporter-wtmp\") pod \"node-exporter-r9grk\" (UID: \"bb42b85b-4372-4700-ba7f-d5e91b80e3f1\") " pod="openshift-monitoring/node-exporter-r9grk" May 27 20:12:57.241853 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:12:57.241567 2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-2gmr8\" (UniqueName: \"kubernetes.io/projected/5710019c-8d7a-446e-b750-85706ec7bf14-kube-api-access-2gmr8\") pod \"openshift-state-metrics-9c959559f-wgtzh\" (UID: \"5710019c-8d7a-446e-b750-85706ec7bf14\") " pod="openshift-monitoring/openshift-state-metrics-9c959559f-wgtzh" May 27 20:12:57.241853 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:12:57.241594 2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/bb42b85b-4372-4700-ba7f-d5e91b80e3f1-metrics-client-ca\") pod \"node-exporter-r9grk\" (UID: \"bb42b85b-4372-4700-ba7f-d5e91b80e3f1\") " pod="openshift-monitoring/node-exporter-r9grk" May 27 20:12:57.241853 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:12:57.241596 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/bb42b85b-4372-4700-ba7f-d5e91b80e3f1-root\") pod \"node-exporter-r9grk\" (UID: \"bb42b85b-4372-4700-ba7f-d5e91b80e3f1\") " pod="openshift-monitoring/node-exporter-r9grk" May 27 20:12:57.241853 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:12:57.241631 2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/bb42b85b-4372-4700-ba7f-d5e91b80e3f1-sys\") pod \"node-exporter-r9grk\" (UID: \"bb42b85b-4372-4700-ba7f-d5e91b80e3f1\") " pod="openshift-monitoring/node-exporter-r9grk" May 27 20:12:57.241853 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:12:57.241668 2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/5710019c-8d7a-446e-b750-85706ec7bf14-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-9c959559f-wgtzh\" (UID: \"5710019c-8d7a-446e-b750-85706ec7bf14\") " pod="openshift-monitoring/openshift-state-metrics-9c959559f-wgtzh" May 27 20:12:57.241853 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:12:57.241710 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/bb42b85b-4372-4700-ba7f-d5e91b80e3f1-node-exporter-wtmp\") pod \"node-exporter-r9grk\" (UID: \"bb42b85b-4372-4700-ba7f-d5e91b80e3f1\") " pod="openshift-monitoring/node-exporter-r9grk" May 27 20:12:57.242399 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:12:57.242015 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/bb42b85b-4372-4700-ba7f-d5e91b80e3f1-node-exporter-textfile\") pod \"node-exporter-r9grk\" (UID: \"bb42b85b-4372-4700-ba7f-d5e91b80e3f1\") " pod="openshift-monitoring/node-exporter-r9grk" May 27 20:12:57.242399 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:12:57.242080 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/bb42b85b-4372-4700-ba7f-d5e91b80e3f1-sys\") pod \"node-exporter-r9grk\" (UID: \"bb42b85b-4372-4700-ba7f-d5e91b80e3f1\") " pod="openshift-monitoring/node-exporter-r9grk" May 27 20:12:57.242399 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:12:57.242127 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/bb42b85b-4372-4700-ba7f-d5e91b80e3f1-node-exporter-accelerators-collector-config\") pod \"node-exporter-r9grk\" (UID: \"bb42b85b-4372-4700-ba7f-d5e91b80e3f1\") " pod="openshift-monitoring/node-exporter-r9grk" May 27 20:12:57.242399 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:12:57.242346 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/bb42b85b-4372-4700-ba7f-d5e91b80e3f1-metrics-client-ca\") pod \"node-exporter-r9grk\" (UID: \"bb42b85b-4372-4700-ba7f-d5e91b80e3f1\") " pod="openshift-monitoring/node-exporter-r9grk" May 27 20:12:57.242689 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:12:57.242671 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5710019c-8d7a-446e-b750-85706ec7bf14-metrics-client-ca\") pod \"openshift-state-metrics-9c959559f-wgtzh\" (UID: \"5710019c-8d7a-446e-b750-85706ec7bf14\") " pod="openshift-monitoring/openshift-state-metrics-9c959559f-wgtzh" May 27 20:12:57.243923 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:12:57.243900 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/bb42b85b-4372-4700-ba7f-d5e91b80e3f1-node-exporter-tls\") pod \"node-exporter-r9grk\" (UID: \"bb42b85b-4372-4700-ba7f-d5e91b80e3f1\") " pod="openshift-monitoring/node-exporter-r9grk" May 27 20:12:57.244027 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:12:57.244008 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/5710019c-8d7a-446e-b750-85706ec7bf14-openshift-state-metrics-tls\") pod \"openshift-state-metrics-9c959559f-wgtzh\" (UID: \"5710019c-8d7a-446e-b750-85706ec7bf14\") " pod="openshift-monitoring/openshift-state-metrics-9c959559f-wgtzh" May 27 20:12:57.244322 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:12:57.244303 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/bb42b85b-4372-4700-ba7f-d5e91b80e3f1-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-r9grk\" (UID: \"bb42b85b-4372-4700-ba7f-d5e91b80e3f1\") " pod="openshift-monitoring/node-exporter-r9grk" May 27 20:12:57.244439 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:12:57.244421 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/5710019c-8d7a-446e-b750-85706ec7bf14-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-9c959559f-wgtzh\" (UID: \"5710019c-8d7a-446e-b750-85706ec7bf14\") " pod="openshift-monitoring/openshift-state-metrics-9c959559f-wgtzh" May 27 20:12:57.250124 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:12:57.250073 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9drmq\" (UniqueName: \"kubernetes.io/projected/bb42b85b-4372-4700-ba7f-d5e91b80e3f1-kube-api-access-9drmq\") pod \"node-exporter-r9grk\" (UID: \"bb42b85b-4372-4700-ba7f-d5e91b80e3f1\") " pod="openshift-monitoring/node-exporter-r9grk" May 27 20:12:57.250341 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:12:57.250322 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2gmr8\" (UniqueName: \"kubernetes.io/projected/5710019c-8d7a-446e-b750-85706ec7bf14-kube-api-access-2gmr8\") pod \"openshift-state-metrics-9c959559f-wgtzh\" (UID: \"5710019c-8d7a-446e-b750-85706ec7bf14\") " pod="openshift-monitoring/openshift-state-metrics-9c959559f-wgtzh" May 27 20:12:57.316756 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:12:57.316731 2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-9c959559f-wgtzh" May 27 20:12:57.353803 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:12:57.353770 2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-r9grk" May 27 20:12:57.361724 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:12:57.361693 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbb42b85b_4372_4700_ba7f_d5e91b80e3f1.slice/crio-f3f1fba11bc2165612d6c7d1246cc6ff4c17d49d35e8d76d8780979a33c8127f WatchSource:0}: Error finding container f3f1fba11bc2165612d6c7d1246cc6ff4c17d49d35e8d76d8780979a33c8127f: Status 404 returned error can't find the container with id f3f1fba11bc2165612d6c7d1246cc6ff4c17d49d35e8d76d8780979a33c8127f May 27 20:12:57.448724 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:12:57.448557 2571 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-9c959559f-wgtzh"] May 27 20:12:57.450938 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:12:57.450912 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5710019c_8d7a_446e_b750_85706ec7bf14.slice/crio-f91b298fcf66809e1933c2e8ccef0fe56818bae62aea9daec74e97736a06364f WatchSource:0}: Error finding container f91b298fcf66809e1933c2e8ccef0fe56818bae62aea9daec74e97736a06364f: Status 404 returned error can't find the container with id f91b298fcf66809e1933c2e8ccef0fe56818bae62aea9daec74e97736a06364f May 27 20:12:57.637274 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:12:57.637240 2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-9c959559f-wgtzh" event={"ID":"5710019c-8d7a-446e-b750-85706ec7bf14","Type":"ContainerStarted","Data":"9f23d4ab0d061a964b9f89ffca774eb51735d9ebff7c75504c1d902c47d9c46f"} May 27 20:12:57.637400 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:12:57.637282 2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-9c959559f-wgtzh" event={"ID":"5710019c-8d7a-446e-b750-85706ec7bf14","Type":"ContainerStarted","Data":"3a456695abf6ebaa4ef6b156fbffb08fea22b89f76b82e8b8f9a54083d5a98fb"} May 27 20:12:57.637400 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:12:57.637296 2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-9c959559f-wgtzh" event={"ID":"5710019c-8d7a-446e-b750-85706ec7bf14","Type":"ContainerStarted","Data":"f91b298fcf66809e1933c2e8ccef0fe56818bae62aea9daec74e97736a06364f"} May 27 20:12:57.638212 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:12:57.638188 2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-r9grk" event={"ID":"bb42b85b-4372-4700-ba7f-d5e91b80e3f1","Type":"ContainerStarted","Data":"f3f1fba11bc2165612d6c7d1246cc6ff4c17d49d35e8d76d8780979a33c8127f"} May 27 20:12:57.949226 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:12:57.949142 2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/96207b98-716e-4c63-847f-fc976f5563f2-metrics-certs\") pod \"network-metrics-daemon-zsx5p\" (UID: \"96207b98-716e-4c63-847f-fc976f5563f2\") " pod="openshift-multus/network-metrics-daemon-zsx5p" May 27 20:12:57.951714 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:12:57.951686 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/96207b98-716e-4c63-847f-fc976f5563f2-metrics-certs\") pod \"network-metrics-daemon-zsx5p\" (UID: \"96207b98-716e-4c63-847f-fc976f5563f2\") " pod="openshift-multus/network-metrics-daemon-zsx5p" May 27 20:12:58.139000 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:12:58.138967 2571 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-b7jlg\"" May 27 20:12:58.146574 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:12:58.146541 2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-zsx5p" May 27 20:12:58.300556 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:12:58.300525 2571 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-zsx5p"] May 27 20:12:58.400479 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:12:58.400444 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod96207b98_716e_4c63_847f_fc976f5563f2.slice/crio-489cfda1fe72db695b1c300a47ebe7b42b204971bc7da0f0978e1948c9f082ff WatchSource:0}: Error finding container 489cfda1fe72db695b1c300a47ebe7b42b204971bc7da0f0978e1948c9f082ff: Status 404 returned error can't find the container with id 489cfda1fe72db695b1c300a47ebe7b42b204971bc7da0f0978e1948c9f082ff May 27 20:12:58.642208 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:12:58.642165 2571 generic.go:358] "Generic (PLEG): container finished" podID="bb42b85b-4372-4700-ba7f-d5e91b80e3f1" containerID="1b3f3e159ca77177fd3d802ddbece96be80deb1e29ec3e4bb3f638a0ab91410c" exitCode=0 May 27 20:12:58.642391 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:12:58.642248 2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-r9grk" event={"ID":"bb42b85b-4372-4700-ba7f-d5e91b80e3f1","Type":"ContainerDied","Data":"1b3f3e159ca77177fd3d802ddbece96be80deb1e29ec3e4bb3f638a0ab91410c"} May 27 20:12:58.644116 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:12:58.644079 2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-9c959559f-wgtzh" event={"ID":"5710019c-8d7a-446e-b750-85706ec7bf14","Type":"ContainerStarted","Data":"33273bd78f8e6399e32f1f17cb4c08e344206beea7910160e3760224f22d6bd3"} May 27 20:12:58.645027 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:12:58.645006 2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-zsx5p" event={"ID":"96207b98-716e-4c63-847f-fc976f5563f2","Type":"ContainerStarted","Data":"489cfda1fe72db695b1c300a47ebe7b42b204971bc7da0f0978e1948c9f082ff"} May 27 20:12:58.677703 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:12:58.677653 2571 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-9c959559f-wgtzh" podStartSLOduration=1.80147383 podStartE2EDuration="2.677637046s" podCreationTimestamp="2026-05-27 20:12:56 +0000 UTC" firstStartedPulling="2026-05-27 20:12:57.569210731 +0000 UTC m=+129.978837020" lastFinishedPulling="2026-05-27 20:12:58.445373934 +0000 UTC m=+130.855000236" observedRunningTime="2026-05-27 20:12:58.676327913 +0000 UTC m=+131.085954226" watchObservedRunningTime="2026-05-27 20:12:58.677637046 +0000 UTC m=+131.087263358" May 27 20:12:59.649948 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:12:59.649905 2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-r9grk" event={"ID":"bb42b85b-4372-4700-ba7f-d5e91b80e3f1","Type":"ContainerStarted","Data":"94e7a641f3a9d5af15d8aa7ae0c089c291995558c2fe1ae510869a59c64fdf13"} May 27 20:12:59.650451 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:12:59.649960 2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-r9grk" event={"ID":"bb42b85b-4372-4700-ba7f-d5e91b80e3f1","Type":"ContainerStarted","Data":"a758be66bd1a99764ba3c266422171facabbf74f21feb0841069542f61f42378"} May 27 20:12:59.651448 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:12:59.651421 2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-zsx5p" event={"ID":"96207b98-716e-4c63-847f-fc976f5563f2","Type":"ContainerStarted","Data":"acad9ef5330958aab12ed3116b49b616cad6902c19ff6879a6c883ea98220604"} May 27 20:12:59.651539 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:12:59.651456 2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-zsx5p" event={"ID":"96207b98-716e-4c63-847f-fc976f5563f2","Type":"ContainerStarted","Data":"36f247bc428b4b72b654d23f6fca1936bf49abc2d88ddc476ae2cb24ae66f523"} May 27 20:12:59.673744 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:12:59.673647 2571 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-r9grk" podStartSLOduration=1.867151642 podStartE2EDuration="2.673630935s" podCreationTimestamp="2026-05-27 20:12:57 +0000 UTC" firstStartedPulling="2026-05-27 20:12:57.364412447 +0000 UTC m=+129.774038754" lastFinishedPulling="2026-05-27 20:12:58.170891743 +0000 UTC m=+130.580518047" observedRunningTime="2026-05-27 20:12:59.672122877 +0000 UTC m=+132.081749183" watchObservedRunningTime="2026-05-27 20:12:59.673630935 +0000 UTC m=+132.083257247" May 27 20:12:59.693453 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:12:59.693408 2571 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-zsx5p" podStartSLOduration=130.679706817 podStartE2EDuration="2m11.693397601s" podCreationTimestamp="2026-05-27 20:10:48 +0000 UTC" firstStartedPulling="2026-05-27 20:12:58.40276796 +0000 UTC m=+130.812394252" lastFinishedPulling="2026-05-27 20:12:59.416458743 +0000 UTC m=+131.826085036" observedRunningTime="2026-05-27 20:12:59.691657252 +0000 UTC m=+132.101283563" watchObservedRunningTime="2026-05-27 20:12:59.693397601 +0000 UTC m=+132.103023912" May 27 20:13:07.617677 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:13:07.617646 2571 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-6bc4d5d5c8-7lb24" May 27 20:13:07.853122 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:13:07.853057 2571 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-dccc57588-twsww" podUID="f5a33912-2fd4-4b0f-b2b6-0730dbd0d82d" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500" May 27 20:13:09.410852 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:13:09.410818 2571 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-console/console-968f55547-bbtxt"] May 27 20:13:09.414130 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:13:09.414092 2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-968f55547-bbtxt" May 27 20:13:09.416670 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:13:09.416649 2571 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" May 27 20:13:09.416798 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:13:09.416695 2571 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" May 27 20:13:09.417923 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:13:09.417896 2571 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" May 27 20:13:09.418032 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:13:09.417933 2571 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-h9gjg\"" May 27 20:13:09.418134 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:13:09.418089 2571 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" May 27 20:13:09.418355 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:13:09.418325 2571 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" May 27 20:13:09.418424 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:13:09.418398 2571 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" May 27 20:13:09.419270 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:13:09.419252 2571 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" May 27 20:13:09.423754 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:13:09.423731 2571 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" May 27 20:13:09.424183 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:13:09.424158 2571 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-968f55547-bbtxt"] May 27 20:13:09.449533 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:13:09.449510 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fhclb\" (UniqueName: \"kubernetes.io/projected/c23b3f5d-254f-468d-94ae-fe50871d35b3-kube-api-access-fhclb\") pod \"console-968f55547-bbtxt\" (UID: \"c23b3f5d-254f-468d-94ae-fe50871d35b3\") " pod="openshift-console/console-968f55547-bbtxt" May 27 20:13:09.449641 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:13:09.449553 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c23b3f5d-254f-468d-94ae-fe50871d35b3-console-oauth-config\") pod \"console-968f55547-bbtxt\" (UID: \"c23b3f5d-254f-468d-94ae-fe50871d35b3\") " pod="openshift-console/console-968f55547-bbtxt" May 27 20:13:09.449641 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:13:09.449603 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c23b3f5d-254f-468d-94ae-fe50871d35b3-service-ca\") pod \"console-968f55547-bbtxt\" (UID: \"c23b3f5d-254f-468d-94ae-fe50871d35b3\") " pod="openshift-console/console-968f55547-bbtxt" May 27 20:13:09.449757 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:13:09.449696 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c23b3f5d-254f-468d-94ae-fe50871d35b3-oauth-serving-cert\") pod \"console-968f55547-bbtxt\" (UID: \"c23b3f5d-254f-468d-94ae-fe50871d35b3\") " pod="openshift-console/console-968f55547-bbtxt" May 27 20:13:09.449757 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:13:09.449725 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c23b3f5d-254f-468d-94ae-fe50871d35b3-console-config\") pod \"console-968f55547-bbtxt\" (UID: \"c23b3f5d-254f-468d-94ae-fe50871d35b3\") " pod="openshift-console/console-968f55547-bbtxt" May 27 20:13:09.449842 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:13:09.449760 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c23b3f5d-254f-468d-94ae-fe50871d35b3-console-serving-cert\") pod \"console-968f55547-bbtxt\" (UID: \"c23b3f5d-254f-468d-94ae-fe50871d35b3\") " pod="openshift-console/console-968f55547-bbtxt" May 27 20:13:09.449842 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:13:09.449793 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c23b3f5d-254f-468d-94ae-fe50871d35b3-trusted-ca-bundle\") pod \"console-968f55547-bbtxt\" (UID: \"c23b3f5d-254f-468d-94ae-fe50871d35b3\") " pod="openshift-console/console-968f55547-bbtxt" May 27 20:13:09.550221 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:13:09.550198 2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c23b3f5d-254f-468d-94ae-fe50871d35b3-console-oauth-config\") pod \"console-968f55547-bbtxt\" (UID: \"c23b3f5d-254f-468d-94ae-fe50871d35b3\") " pod="openshift-console/console-968f55547-bbtxt" May 27 20:13:09.550309 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:13:09.550227 2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c23b3f5d-254f-468d-94ae-fe50871d35b3-service-ca\") pod \"console-968f55547-bbtxt\" (UID: \"c23b3f5d-254f-468d-94ae-fe50871d35b3\") " pod="openshift-console/console-968f55547-bbtxt" May 27 20:13:09.550309 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:13:09.550296 2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c23b3f5d-254f-468d-94ae-fe50871d35b3-oauth-serving-cert\") pod \"console-968f55547-bbtxt\" (UID: \"c23b3f5d-254f-468d-94ae-fe50871d35b3\") " pod="openshift-console/console-968f55547-bbtxt" May 27 20:13:09.550410 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:13:09.550320 2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c23b3f5d-254f-468d-94ae-fe50871d35b3-console-config\") pod \"console-968f55547-bbtxt\" (UID: \"c23b3f5d-254f-468d-94ae-fe50871d35b3\") " pod="openshift-console/console-968f55547-bbtxt" May 27 20:13:09.550460 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:13:09.550449 2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c23b3f5d-254f-468d-94ae-fe50871d35b3-console-serving-cert\") pod \"console-968f55547-bbtxt\" (UID: \"c23b3f5d-254f-468d-94ae-fe50871d35b3\") " pod="openshift-console/console-968f55547-bbtxt" May 27 20:13:09.550508 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:13:09.550481 2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c23b3f5d-254f-468d-94ae-fe50871d35b3-trusted-ca-bundle\") pod \"console-968f55547-bbtxt\" (UID: \"c23b3f5d-254f-468d-94ae-fe50871d35b3\") " pod="openshift-console/console-968f55547-bbtxt" May 27 20:13:09.550549 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:13:09.550505 2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-fhclb\" (UniqueName: \"kubernetes.io/projected/c23b3f5d-254f-468d-94ae-fe50871d35b3-kube-api-access-fhclb\") pod \"console-968f55547-bbtxt\" (UID: \"c23b3f5d-254f-468d-94ae-fe50871d35b3\") " pod="openshift-console/console-968f55547-bbtxt" May 27 20:13:09.551508 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:13:09.551469 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c23b3f5d-254f-468d-94ae-fe50871d35b3-oauth-serving-cert\") pod \"console-968f55547-bbtxt\" (UID: \"c23b3f5d-254f-468d-94ae-fe50871d35b3\") " pod="openshift-console/console-968f55547-bbtxt" May 27 20:13:09.551641 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:13:09.551509 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c23b3f5d-254f-468d-94ae-fe50871d35b3-service-ca\") pod \"console-968f55547-bbtxt\" (UID: \"c23b3f5d-254f-468d-94ae-fe50871d35b3\") " pod="openshift-console/console-968f55547-bbtxt" May 27 20:13:09.551749 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:13:09.551426 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c23b3f5d-254f-468d-94ae-fe50871d35b3-console-config\") pod \"console-968f55547-bbtxt\" (UID: \"c23b3f5d-254f-468d-94ae-fe50871d35b3\") " pod="openshift-console/console-968f55547-bbtxt" May 27 20:13:09.555719 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:13:09.551836 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c23b3f5d-254f-468d-94ae-fe50871d35b3-trusted-ca-bundle\") pod \"console-968f55547-bbtxt\" (UID: \"c23b3f5d-254f-468d-94ae-fe50871d35b3\") " pod="openshift-console/console-968f55547-bbtxt" May 27 20:13:09.555719 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:13:09.553718 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c23b3f5d-254f-468d-94ae-fe50871d35b3-console-oauth-config\") pod \"console-968f55547-bbtxt\" (UID: \"c23b3f5d-254f-468d-94ae-fe50871d35b3\") " pod="openshift-console/console-968f55547-bbtxt" May 27 20:13:09.555719 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:13:09.554276 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c23b3f5d-254f-468d-94ae-fe50871d35b3-console-serving-cert\") pod \"console-968f55547-bbtxt\" (UID: \"c23b3f5d-254f-468d-94ae-fe50871d35b3\") " pod="openshift-console/console-968f55547-bbtxt" May 27 20:13:09.558725 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:13:09.558708 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fhclb\" (UniqueName: \"kubernetes.io/projected/c23b3f5d-254f-468d-94ae-fe50871d35b3-kube-api-access-fhclb\") pod \"console-968f55547-bbtxt\" (UID: \"c23b3f5d-254f-468d-94ae-fe50871d35b3\") " pod="openshift-console/console-968f55547-bbtxt" May 27 20:13:09.723935 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:13:09.723852 2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-968f55547-bbtxt" May 27 20:13:09.845686 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:13:09.845643 2571 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-968f55547-bbtxt"] May 27 20:13:09.849406 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:13:09.849377 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc23b3f5d_254f_468d_94ae_fe50871d35b3.slice/crio-b0695fc654cdc609e0c150f7f937212b038c656617eadd3cd8f8df36ea2a5090 WatchSource:0}: Error finding container b0695fc654cdc609e0c150f7f937212b038c656617eadd3cd8f8df36ea2a5090: Status 404 returned error can't find the container with id b0695fc654cdc609e0c150f7f937212b038c656617eadd3cd8f8df36ea2a5090 May 27 20:13:10.680926 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:13:10.680883 2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-968f55547-bbtxt" event={"ID":"c23b3f5d-254f-468d-94ae-fe50871d35b3","Type":"ContainerStarted","Data":"b0695fc654cdc609e0c150f7f937212b038c656617eadd3cd8f8df36ea2a5090"} May 27 20:13:12.687279 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:13:12.687198 2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-968f55547-bbtxt" event={"ID":"c23b3f5d-254f-468d-94ae-fe50871d35b3","Type":"ContainerStarted","Data":"3378e735530dcfdc5611aada65f90b826e6b71743382568089ef5cec9db221c7"} May 27 20:13:12.707144 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:13:12.707084 2571 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-console/console-968f55547-bbtxt" podStartSLOduration=1.1469573770000001 podStartE2EDuration="3.707070963s" podCreationTimestamp="2026-05-27 20:13:09 +0000 UTC" firstStartedPulling="2026-05-27 20:13:09.851351289 +0000 UTC m=+142.260977585" lastFinishedPulling="2026-05-27 20:13:12.411464879 +0000 UTC m=+144.821091171" observedRunningTime="2026-05-27 20:13:12.705209685 +0000 UTC m=+145.114835996" watchObservedRunningTime="2026-05-27 20:13:12.707070963 +0000 UTC m=+145.116697274" May 27 20:13:17.852496 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:13:17.852453 2571 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-dccc57588-twsww" podUID="f5a33912-2fd4-4b0f-b2b6-0730dbd0d82d" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500" May 27 20:13:19.488263 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:13:19.488227 2571 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-console/console-968f55547-bbtxt"] May 27 20:13:19.724290 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:13:19.724251 2571 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-968f55547-bbtxt" May 27 20:13:25.515263 ip-10-0-142-251 kubenswrapper[2571]: E0527 20:13:25.515224 2571 pod_workers.go:1301] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-58qkv" podUID="cf585445-0773-47ca-ba45-3bad1a2d5b88" May 27 20:13:25.530891 ip-10-0-142-251 kubenswrapper[2571]: E0527 20:13:25.530866 2571 pod_workers.go:1301] "Error syncing pod, skipping" err="unmounted volumes=[cert], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-ingress-canary/ingress-canary-jxjsb" podUID="d6fb3e28-cbfd-47a3-8104-bbfa86006ad6" May 27 20:13:25.719197 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:13:25.719167 2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-58qkv" May 27 20:13:27.852670 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:13:27.852629 2571 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-dccc57588-twsww" podUID="f5a33912-2fd4-4b0f-b2b6-0730dbd0d82d" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500" May 27 20:13:27.853152 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:13:27.852704 2571 kubelet.go:2658] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-dccc57588-twsww" May 27 20:13:27.853375 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:13:27.853340 2571 kuberuntime_manager.go:1107] "Message for Container of pod" containerName="service-proxy" containerStatusID={"Type":"cri-o","ID":"4e633d4e18fad441fce3c40369a9601f1863e46d03201fabf3e190dc7b0ced97"} pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-dccc57588-twsww" containerMessage="Container service-proxy failed liveness probe, will be restarted" May 27 20:13:27.853458 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:13:27.853401 2571 kuberuntime_container.go:864] "Killing container with a grace period" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-dccc57588-twsww" podUID="f5a33912-2fd4-4b0f-b2b6-0730dbd0d82d" containerName="service-proxy" containerID="cri-o://4e633d4e18fad441fce3c40369a9601f1863e46d03201fabf3e190dc7b0ced97" gracePeriod=30 May 27 20:13:28.727698 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:13:28.727659 2571 generic.go:358] "Generic (PLEG): container finished" podID="f5a33912-2fd4-4b0f-b2b6-0730dbd0d82d" containerID="4e633d4e18fad441fce3c40369a9601f1863e46d03201fabf3e190dc7b0ced97" exitCode=2 May 27 20:13:28.727865 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:13:28.727709 2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-dccc57588-twsww" event={"ID":"f5a33912-2fd4-4b0f-b2b6-0730dbd0d82d","Type":"ContainerDied","Data":"4e633d4e18fad441fce3c40369a9601f1863e46d03201fabf3e190dc7b0ced97"} May 27 20:13:28.727865 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:13:28.727734 2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-dccc57588-twsww" event={"ID":"f5a33912-2fd4-4b0f-b2b6-0730dbd0d82d","Type":"ContainerStarted","Data":"7b9e6477824431d92b8bc7ee6e4b8016b05f0e38a5806ad7a7a91f6e146a2d20"} May 27 20:13:30.415394 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:13:30.415351 2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/cf585445-0773-47ca-ba45-3bad1a2d5b88-metrics-tls\") pod \"dns-default-58qkv\" (UID: \"cf585445-0773-47ca-ba45-3bad1a2d5b88\") " pod="openshift-dns/dns-default-58qkv" May 27 20:13:30.415748 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:13:30.415407 2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d6fb3e28-cbfd-47a3-8104-bbfa86006ad6-cert\") pod \"ingress-canary-jxjsb\" (UID: \"d6fb3e28-cbfd-47a3-8104-bbfa86006ad6\") " pod="openshift-ingress-canary/ingress-canary-jxjsb" May 27 20:13:30.417775 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:13:30.417741 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/cf585445-0773-47ca-ba45-3bad1a2d5b88-metrics-tls\") pod \"dns-default-58qkv\" (UID: \"cf585445-0773-47ca-ba45-3bad1a2d5b88\") " pod="openshift-dns/dns-default-58qkv" May 27 20:13:30.417889 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:13:30.417815 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d6fb3e28-cbfd-47a3-8104-bbfa86006ad6-cert\") pod \"ingress-canary-jxjsb\" (UID: \"d6fb3e28-cbfd-47a3-8104-bbfa86006ad6\") " pod="openshift-ingress-canary/ingress-canary-jxjsb" May 27 20:13:30.522211 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:13:30.522181 2571 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-jcg7q\"" May 27 20:13:30.530518 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:13:30.530499 2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-58qkv" May 27 20:13:30.653659 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:13:30.653627 2571 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-58qkv"] May 27 20:13:30.656138 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:13:30.656091 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcf585445_0773_47ca_ba45_3bad1a2d5b88.slice/crio-59059375be168e58927bad2f3d9208b0b9096b150aa807cbf6a911359578a494 WatchSource:0}: Error finding container 59059375be168e58927bad2f3d9208b0b9096b150aa807cbf6a911359578a494: Status 404 returned error can't find the container with id 59059375be168e58927bad2f3d9208b0b9096b150aa807cbf6a911359578a494 May 27 20:13:30.734129 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:13:30.734030 2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-58qkv" event={"ID":"cf585445-0773-47ca-ba45-3bad1a2d5b88","Type":"ContainerStarted","Data":"59059375be168e58927bad2f3d9208b0b9096b150aa807cbf6a911359578a494"} May 27 20:13:31.804605 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:13:31.804522 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-n4brm_4937a11c-a8df-40eb-9a62-0b9fa6ea34ef/dns-node-resolver/0.log" May 27 20:13:32.741570 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:13:32.741533 2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-58qkv" event={"ID":"cf585445-0773-47ca-ba45-3bad1a2d5b88","Type":"ContainerStarted","Data":"f58d13b3346eb5db3473ff3258cb9e21fa65a2ec0bfe8ba7d15cc939628f0cc9"} May 27 20:13:32.741746 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:13:32.741577 2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-58qkv" event={"ID":"cf585445-0773-47ca-ba45-3bad1a2d5b88","Type":"ContainerStarted","Data":"52d93de5f1c211df07a034e36a0ca2f486e293299e3cbb694bc9405f1f6a21cf"} May 27 20:13:32.741746 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:13:32.741682 2571 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-58qkv" May 27 20:13:32.761095 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:13:32.761046 2571 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-dns/dns-default-58qkv" podStartSLOduration=129.407338906 podStartE2EDuration="2m10.76103464s" podCreationTimestamp="2026-05-27 20:11:22 +0000 UTC" firstStartedPulling="2026-05-27 20:13:30.657928728 +0000 UTC m=+163.067555017" lastFinishedPulling="2026-05-27 20:13:32.011624459 +0000 UTC m=+164.421250751" observedRunningTime="2026-05-27 20:13:32.760803021 +0000 UTC m=+165.170429332" watchObservedRunningTime="2026-05-27 20:13:32.76103464 +0000 UTC m=+165.170660950" May 27 20:13:40.224451 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:13:40.224414 2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-jxjsb" May 27 20:13:40.227345 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:13:40.227321 2571 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-c8wsm\"" May 27 20:13:40.234877 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:13:40.234857 2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-jxjsb" May 27 20:13:40.354773 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:13:40.354740 2571 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-jxjsb"] May 27 20:13:40.358366 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:13:40.358338 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd6fb3e28_cbfd_47a3_8104_bbfa86006ad6.slice/crio-4a2d13be0bc196ae2dec50b4dbdc17ec7062712925bc275922d829a00205c15d WatchSource:0}: Error finding container 4a2d13be0bc196ae2dec50b4dbdc17ec7062712925bc275922d829a00205c15d: Status 404 returned error can't find the container with id 4a2d13be0bc196ae2dec50b4dbdc17ec7062712925bc275922d829a00205c15d May 27 20:13:40.765160 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:13:40.765119 2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-jxjsb" event={"ID":"d6fb3e28-cbfd-47a3-8104-bbfa86006ad6","Type":"ContainerStarted","Data":"4a2d13be0bc196ae2dec50b4dbdc17ec7062712925bc275922d829a00205c15d"} May 27 20:13:42.747124 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:13:42.747071 2571 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-58qkv" May 27 20:13:42.771405 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:13:42.771364 2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-jxjsb" event={"ID":"d6fb3e28-cbfd-47a3-8104-bbfa86006ad6","Type":"ContainerStarted","Data":"609a3532e6581a74f5359596bbd3ac8d6b400629ef12c6a932c489819da59a6c"} May 27 20:13:42.789335 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:13:42.789166 2571 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-jxjsb" podStartSLOduration=139.334148174 podStartE2EDuration="2m20.789148478s" podCreationTimestamp="2026-05-27 20:11:22 +0000 UTC" firstStartedPulling="2026-05-27 20:13:40.360146696 +0000 UTC m=+172.769772984" lastFinishedPulling="2026-05-27 20:13:41.815146991 +0000 UTC m=+174.224773288" observedRunningTime="2026-05-27 20:13:42.78859089 +0000 UTC m=+175.198217202" watchObservedRunningTime="2026-05-27 20:13:42.789148478 +0000 UTC m=+175.198774791" May 27 20:13:44.507353 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:13:44.507315 2571 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-console/console-968f55547-bbtxt" podUID="c23b3f5d-254f-468d-94ae-fe50871d35b3" containerName="console" containerID="cri-o://3378e735530dcfdc5611aada65f90b826e6b71743382568089ef5cec9db221c7" gracePeriod=15 May 27 20:13:44.747435 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:13:44.747414 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-968f55547-bbtxt_c23b3f5d-254f-468d-94ae-fe50871d35b3/console/0.log" May 27 20:13:44.747529 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:13:44.747489 2571 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-968f55547-bbtxt" May 27 20:13:44.780261 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:13:44.780240 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-968f55547-bbtxt_c23b3f5d-254f-468d-94ae-fe50871d35b3/console/0.log" May 27 20:13:44.780370 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:13:44.780277 2571 generic.go:358] "Generic (PLEG): container finished" podID="c23b3f5d-254f-468d-94ae-fe50871d35b3" containerID="3378e735530dcfdc5611aada65f90b826e6b71743382568089ef5cec9db221c7" exitCode=2 May 27 20:13:44.780370 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:13:44.780338 2571 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-968f55547-bbtxt" May 27 20:13:44.780460 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:13:44.780343 2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-968f55547-bbtxt" event={"ID":"c23b3f5d-254f-468d-94ae-fe50871d35b3","Type":"ContainerDied","Data":"3378e735530dcfdc5611aada65f90b826e6b71743382568089ef5cec9db221c7"} May 27 20:13:44.780460 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:13:44.780431 2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-968f55547-bbtxt" event={"ID":"c23b3f5d-254f-468d-94ae-fe50871d35b3","Type":"ContainerDied","Data":"b0695fc654cdc609e0c150f7f937212b038c656617eadd3cd8f8df36ea2a5090"} May 27 20:13:44.780460 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:13:44.780444 2571 scope.go:117] "RemoveContainer" containerID="3378e735530dcfdc5611aada65f90b826e6b71743382568089ef5cec9db221c7" May 27 20:13:44.790709 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:13:44.790684 2571 scope.go:117] "RemoveContainer" containerID="3378e735530dcfdc5611aada65f90b826e6b71743382568089ef5cec9db221c7" May 27 20:13:44.790976 ip-10-0-142-251 kubenswrapper[2571]: E0527 20:13:44.790954 2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3378e735530dcfdc5611aada65f90b826e6b71743382568089ef5cec9db221c7\": container with ID starting with 3378e735530dcfdc5611aada65f90b826e6b71743382568089ef5cec9db221c7 not found: ID does not exist" containerID="3378e735530dcfdc5611aada65f90b826e6b71743382568089ef5cec9db221c7" May 27 20:13:44.791019 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:13:44.790985 2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3378e735530dcfdc5611aada65f90b826e6b71743382568089ef5cec9db221c7"} err="failed to get container status \"3378e735530dcfdc5611aada65f90b826e6b71743382568089ef5cec9db221c7\": rpc error: code = NotFound desc = could not find container \"3378e735530dcfdc5611aada65f90b826e6b71743382568089ef5cec9db221c7\": container with ID starting with 3378e735530dcfdc5611aada65f90b826e6b71743382568089ef5cec9db221c7 not found: ID does not exist" May 27 20:13:44.822244 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:13:44.822222 2571 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c23b3f5d-254f-468d-94ae-fe50871d35b3-trusted-ca-bundle\") pod \"c23b3f5d-254f-468d-94ae-fe50871d35b3\" (UID: \"c23b3f5d-254f-468d-94ae-fe50871d35b3\") " May 27 20:13:44.822332 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:13:44.822252 2571 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c23b3f5d-254f-468d-94ae-fe50871d35b3-console-config\") pod \"c23b3f5d-254f-468d-94ae-fe50871d35b3\" (UID: \"c23b3f5d-254f-468d-94ae-fe50871d35b3\") " May 27 20:13:44.822332 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:13:44.822278 2571 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c23b3f5d-254f-468d-94ae-fe50871d35b3-console-serving-cert\") pod \"c23b3f5d-254f-468d-94ae-fe50871d35b3\" (UID: \"c23b3f5d-254f-468d-94ae-fe50871d35b3\") " May 27 20:13:44.822332 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:13:44.822315 2571 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c23b3f5d-254f-468d-94ae-fe50871d35b3-service-ca\") pod \"c23b3f5d-254f-468d-94ae-fe50871d35b3\" (UID: \"c23b3f5d-254f-468d-94ae-fe50871d35b3\") " May 27 20:13:44.822484 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:13:44.822462 2571 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c23b3f5d-254f-468d-94ae-fe50871d35b3-oauth-serving-cert\") pod \"c23b3f5d-254f-468d-94ae-fe50871d35b3\" (UID: \"c23b3f5d-254f-468d-94ae-fe50871d35b3\") " May 27 20:13:44.822553 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:13:44.822539 2571 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fhclb\" (UniqueName: \"kubernetes.io/projected/c23b3f5d-254f-468d-94ae-fe50871d35b3-kube-api-access-fhclb\") pod \"c23b3f5d-254f-468d-94ae-fe50871d35b3\" (UID: \"c23b3f5d-254f-468d-94ae-fe50871d35b3\") " May 27 20:13:44.822612 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:13:44.822581 2571 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c23b3f5d-254f-468d-94ae-fe50871d35b3-console-oauth-config\") pod \"c23b3f5d-254f-468d-94ae-fe50871d35b3\" (UID: \"c23b3f5d-254f-468d-94ae-fe50871d35b3\") " May 27 20:13:44.823048 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:13:44.822746 2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c23b3f5d-254f-468d-94ae-fe50871d35b3-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "c23b3f5d-254f-468d-94ae-fe50871d35b3" (UID: "c23b3f5d-254f-468d-94ae-fe50871d35b3"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" May 27 20:13:44.823048 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:13:44.822758 2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c23b3f5d-254f-468d-94ae-fe50871d35b3-service-ca" (OuterVolumeSpecName: "service-ca") pod "c23b3f5d-254f-468d-94ae-fe50871d35b3" (UID: "c23b3f5d-254f-468d-94ae-fe50871d35b3"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" May 27 20:13:44.823048 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:13:44.822791 2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c23b3f5d-254f-468d-94ae-fe50871d35b3-console-config" (OuterVolumeSpecName: "console-config") pod "c23b3f5d-254f-468d-94ae-fe50871d35b3" (UID: "c23b3f5d-254f-468d-94ae-fe50871d35b3"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" May 27 20:13:44.823048 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:13:44.822868 2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c23b3f5d-254f-468d-94ae-fe50871d35b3-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "c23b3f5d-254f-468d-94ae-fe50871d35b3" (UID: "c23b3f5d-254f-468d-94ae-fe50871d35b3"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" May 27 20:13:44.824710 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:13:44.824685 2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c23b3f5d-254f-468d-94ae-fe50871d35b3-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "c23b3f5d-254f-468d-94ae-fe50871d35b3" (UID: "c23b3f5d-254f-468d-94ae-fe50871d35b3"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" May 27 20:13:44.824804 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:13:44.824710 2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c23b3f5d-254f-468d-94ae-fe50871d35b3-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "c23b3f5d-254f-468d-94ae-fe50871d35b3" (UID: "c23b3f5d-254f-468d-94ae-fe50871d35b3"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" May 27 20:13:44.824804 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:13:44.824735 2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c23b3f5d-254f-468d-94ae-fe50871d35b3-kube-api-access-fhclb" (OuterVolumeSpecName: "kube-api-access-fhclb") pod "c23b3f5d-254f-468d-94ae-fe50871d35b3" (UID: "c23b3f5d-254f-468d-94ae-fe50871d35b3"). InnerVolumeSpecName "kube-api-access-fhclb". PluginName "kubernetes.io/projected", VolumeGIDValue "" May 27 20:13:44.923173 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:13:44.923151 2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fhclb\" (UniqueName: \"kubernetes.io/projected/c23b3f5d-254f-468d-94ae-fe50871d35b3-kube-api-access-fhclb\") on node \"ip-10-0-142-251.ec2.internal\" DevicePath \"\"" May 27 20:13:44.923173 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:13:44.923170 2571 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c23b3f5d-254f-468d-94ae-fe50871d35b3-console-oauth-config\") on node \"ip-10-0-142-251.ec2.internal\" DevicePath \"\"" May 27 20:13:44.923289 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:13:44.923180 2571 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c23b3f5d-254f-468d-94ae-fe50871d35b3-trusted-ca-bundle\") on node \"ip-10-0-142-251.ec2.internal\" DevicePath \"\"" May 27 20:13:44.923289 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:13:44.923217 2571 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c23b3f5d-254f-468d-94ae-fe50871d35b3-console-config\") on node \"ip-10-0-142-251.ec2.internal\" DevicePath \"\"" May 27 20:13:44.923289 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:13:44.923231 2571 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c23b3f5d-254f-468d-94ae-fe50871d35b3-console-serving-cert\") on node \"ip-10-0-142-251.ec2.internal\" DevicePath \"\"" May 27 20:13:44.923289 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:13:44.923245 2571 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c23b3f5d-254f-468d-94ae-fe50871d35b3-service-ca\") on node \"ip-10-0-142-251.ec2.internal\" DevicePath \"\"" May 27 20:13:44.923289 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:13:44.923256 2571 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c23b3f5d-254f-468d-94ae-fe50871d35b3-oauth-serving-cert\") on node \"ip-10-0-142-251.ec2.internal\" DevicePath \"\"" May 27 20:13:45.101624 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:13:45.101596 2571 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-console/console-968f55547-bbtxt"] May 27 20:13:45.105144 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:13:45.105126 2571 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-968f55547-bbtxt"] May 27 20:13:46.228172 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:13:46.228140 2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c23b3f5d-254f-468d-94ae-fe50871d35b3" path="/var/lib/kubelet/pods/c23b3f5d-254f-468d-94ae-fe50871d35b3/volumes" May 27 20:15:48.105402 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:15:48.105371 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rcvbq_92933172-b100-42e6-8018-dcb430647ae2/ovn-acl-logging/0.log" May 27 20:15:48.105904 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:15:48.105520 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rcvbq_92933172-b100-42e6-8018-dcb430647ae2/ovn-acl-logging/0.log" May 27 20:15:48.111317 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:15:48.111290 2571 kubelet.go:1628] "Image garbage collection succeeded" May 27 20:15:59.716334 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:15:59.716297 2571 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-bfc7f696d-mlpld"] May 27 20:15:59.718716 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:15:59.716562 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c23b3f5d-254f-468d-94ae-fe50871d35b3" containerName="console" May 27 20:15:59.718716 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:15:59.716574 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="c23b3f5d-254f-468d-94ae-fe50871d35b3" containerName="console" May 27 20:15:59.718716 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:15:59.716616 2571 memory_manager.go:356] "RemoveStaleState removing state" podUID="c23b3f5d-254f-468d-94ae-fe50871d35b3" containerName="console" May 27 20:15:59.719586 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:15:59.719567 2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-bfc7f696d-mlpld" May 27 20:15:59.722176 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:15:59.722152 2571 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" May 27 20:15:59.723388 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:15:59.723368 2571 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" May 27 20:15:59.723451 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:15:59.723369 2571 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-kj6kp\"" May 27 20:15:59.727852 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:15:59.727819 2571 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-bfc7f696d-mlpld"] May 27 20:15:59.855147 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:15:59.855081 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r2h56\" (UniqueName: \"kubernetes.io/projected/15aecb97-de79-4d5b-a5e3-4780bede9b7e-kube-api-access-r2h56\") pod \"openshift-lws-operator-bfc7f696d-mlpld\" (UID: \"15aecb97-de79-4d5b-a5e3-4780bede9b7e\") " pod="openshift-lws-operator/openshift-lws-operator-bfc7f696d-mlpld" May 27 20:15:59.855335 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:15:59.855225 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/15aecb97-de79-4d5b-a5e3-4780bede9b7e-tmp\") pod \"openshift-lws-operator-bfc7f696d-mlpld\" (UID: \"15aecb97-de79-4d5b-a5e3-4780bede9b7e\") " pod="openshift-lws-operator/openshift-lws-operator-bfc7f696d-mlpld" May 27 20:15:59.955596 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:15:59.955558 2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/15aecb97-de79-4d5b-a5e3-4780bede9b7e-tmp\") pod \"openshift-lws-operator-bfc7f696d-mlpld\" (UID: \"15aecb97-de79-4d5b-a5e3-4780bede9b7e\") " pod="openshift-lws-operator/openshift-lws-operator-bfc7f696d-mlpld" May 27 20:15:59.955728 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:15:59.955612 2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-r2h56\" (UniqueName: \"kubernetes.io/projected/15aecb97-de79-4d5b-a5e3-4780bede9b7e-kube-api-access-r2h56\") pod \"openshift-lws-operator-bfc7f696d-mlpld\" (UID: \"15aecb97-de79-4d5b-a5e3-4780bede9b7e\") " pod="openshift-lws-operator/openshift-lws-operator-bfc7f696d-mlpld" May 27 20:15:59.955937 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:15:59.955915 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/15aecb97-de79-4d5b-a5e3-4780bede9b7e-tmp\") pod \"openshift-lws-operator-bfc7f696d-mlpld\" (UID: \"15aecb97-de79-4d5b-a5e3-4780bede9b7e\") " pod="openshift-lws-operator/openshift-lws-operator-bfc7f696d-mlpld" May 27 20:15:59.964307 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:15:59.964268 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r2h56\" (UniqueName: \"kubernetes.io/projected/15aecb97-de79-4d5b-a5e3-4780bede9b7e-kube-api-access-r2h56\") pod \"openshift-lws-operator-bfc7f696d-mlpld\" (UID: \"15aecb97-de79-4d5b-a5e3-4780bede9b7e\") " pod="openshift-lws-operator/openshift-lws-operator-bfc7f696d-mlpld" May 27 20:16:00.029188 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:16:00.029154 2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-bfc7f696d-mlpld" May 27 20:16:00.166634 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:16:00.166586 2571 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-bfc7f696d-mlpld"] May 27 20:16:00.170809 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:16:00.170780 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod15aecb97_de79_4d5b_a5e3_4780bede9b7e.slice/crio-22d42f3f86b25ec11d704f5977135f2d7553d0830783824cbe410ffd8e709801 WatchSource:0}: Error finding container 22d42f3f86b25ec11d704f5977135f2d7553d0830783824cbe410ffd8e709801: Status 404 returned error can't find the container with id 22d42f3f86b25ec11d704f5977135f2d7553d0830783824cbe410ffd8e709801 May 27 20:16:00.172655 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:16:00.172633 2571 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider May 27 20:16:01.124932 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:16:01.124898 2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-bfc7f696d-mlpld" event={"ID":"15aecb97-de79-4d5b-a5e3-4780bede9b7e","Type":"ContainerStarted","Data":"22d42f3f86b25ec11d704f5977135f2d7553d0830783824cbe410ffd8e709801"} May 27 20:16:05.137247 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:16:05.137201 2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-bfc7f696d-mlpld" event={"ID":"15aecb97-de79-4d5b-a5e3-4780bede9b7e","Type":"ContainerStarted","Data":"3db212ade393df4e7b3ee7f348f27e4214b3310072deccdd73d011d038892860"} May 27 20:16:05.156029 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:16:05.155973 2571 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-bfc7f696d-mlpld" podStartSLOduration=1.988880357 podStartE2EDuration="6.155959028s" podCreationTimestamp="2026-05-27 20:15:59 +0000 UTC" firstStartedPulling="2026-05-27 20:16:00.172758781 +0000 UTC m=+312.582385070" lastFinishedPulling="2026-05-27 20:16:04.339837452 +0000 UTC m=+316.749463741" observedRunningTime="2026-05-27 20:16:05.15390309 +0000 UTC m=+317.563529427" watchObservedRunningTime="2026-05-27 20:16:05.155959028 +0000 UTC m=+317.565585339" May 27 20:16:22.193782 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:16:22.193743 2571 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-b59bfd74-q82qr"] May 27 20:16:22.202804 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:16:22.202769 2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-b59bfd74-q82qr" May 27 20:16:22.205689 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:16:22.205666 2571 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-52xz7\"" May 27 20:16:22.205836 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:16:22.205674 2571 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" May 27 20:16:22.206002 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:16:22.205983 2571 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" May 27 20:16:22.206249 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:16:22.206235 2571 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" May 27 20:16:22.206299 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:16:22.206251 2571 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" May 27 20:16:22.218052 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:16:22.218021 2571 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-b59bfd74-q82qr"] May 27 20:16:22.338291 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:16:22.338259 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/4976f387-5d72-4a0d-befe-78cd65e79e58-webhook-cert\") pod \"opendatahub-operator-controller-manager-b59bfd74-q82qr\" (UID: \"4976f387-5d72-4a0d-befe-78cd65e79e58\") " pod="opendatahub/opendatahub-operator-controller-manager-b59bfd74-q82qr" May 27 20:16:22.338472 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:16:22.338319 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4cssv\" (UniqueName: \"kubernetes.io/projected/4976f387-5d72-4a0d-befe-78cd65e79e58-kube-api-access-4cssv\") pod \"opendatahub-operator-controller-manager-b59bfd74-q82qr\" (UID: \"4976f387-5d72-4a0d-befe-78cd65e79e58\") " pod="opendatahub/opendatahub-operator-controller-manager-b59bfd74-q82qr" May 27 20:16:22.338472 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:16:22.338376 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/4976f387-5d72-4a0d-befe-78cd65e79e58-apiservice-cert\") pod \"opendatahub-operator-controller-manager-b59bfd74-q82qr\" (UID: \"4976f387-5d72-4a0d-befe-78cd65e79e58\") " pod="opendatahub/opendatahub-operator-controller-manager-b59bfd74-q82qr" May 27 20:16:22.439413 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:16:22.439371 2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/4976f387-5d72-4a0d-befe-78cd65e79e58-apiservice-cert\") pod \"opendatahub-operator-controller-manager-b59bfd74-q82qr\" (UID: \"4976f387-5d72-4a0d-befe-78cd65e79e58\") " pod="opendatahub/opendatahub-operator-controller-manager-b59bfd74-q82qr" May 27 20:16:22.439581 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:16:22.439429 2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/4976f387-5d72-4a0d-befe-78cd65e79e58-webhook-cert\") pod \"opendatahub-operator-controller-manager-b59bfd74-q82qr\" (UID: \"4976f387-5d72-4a0d-befe-78cd65e79e58\") " pod="opendatahub/opendatahub-operator-controller-manager-b59bfd74-q82qr" May 27 20:16:22.439581 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:16:22.439485 2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-4cssv\" (UniqueName: \"kubernetes.io/projected/4976f387-5d72-4a0d-befe-78cd65e79e58-kube-api-access-4cssv\") pod \"opendatahub-operator-controller-manager-b59bfd74-q82qr\" (UID: \"4976f387-5d72-4a0d-befe-78cd65e79e58\") " pod="opendatahub/opendatahub-operator-controller-manager-b59bfd74-q82qr" May 27 20:16:22.442165 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:16:22.442137 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/4976f387-5d72-4a0d-befe-78cd65e79e58-apiservice-cert\") pod \"opendatahub-operator-controller-manager-b59bfd74-q82qr\" (UID: \"4976f387-5d72-4a0d-befe-78cd65e79e58\") " pod="opendatahub/opendatahub-operator-controller-manager-b59bfd74-q82qr" May 27 20:16:22.442287 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:16:22.442193 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/4976f387-5d72-4a0d-befe-78cd65e79e58-webhook-cert\") pod \"opendatahub-operator-controller-manager-b59bfd74-q82qr\" (UID: \"4976f387-5d72-4a0d-befe-78cd65e79e58\") " pod="opendatahub/opendatahub-operator-controller-manager-b59bfd74-q82qr" May 27 20:16:22.457541 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:16:22.457473 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4cssv\" (UniqueName: \"kubernetes.io/projected/4976f387-5d72-4a0d-befe-78cd65e79e58-kube-api-access-4cssv\") pod \"opendatahub-operator-controller-manager-b59bfd74-q82qr\" (UID: \"4976f387-5d72-4a0d-befe-78cd65e79e58\") " pod="opendatahub/opendatahub-operator-controller-manager-b59bfd74-q82qr" May 27 20:16:22.513327 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:16:22.513285 2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-b59bfd74-q82qr" May 27 20:16:22.647377 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:16:22.647346 2571 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-b59bfd74-q82qr"] May 27 20:16:22.650618 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:16:22.650588 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4976f387_5d72_4a0d_befe_78cd65e79e58.slice/crio-fc3dde99e6fc93637d3d97e5f5c9511bbb41317c3b182fbf97e840575a9a5ec8 WatchSource:0}: Error finding container fc3dde99e6fc93637d3d97e5f5c9511bbb41317c3b182fbf97e840575a9a5ec8: Status 404 returned error can't find the container with id fc3dde99e6fc93637d3d97e5f5c9511bbb41317c3b182fbf97e840575a9a5ec8 May 27 20:16:23.186614 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:16:23.186571 2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-b59bfd74-q82qr" event={"ID":"4976f387-5d72-4a0d-befe-78cd65e79e58","Type":"ContainerStarted","Data":"fc3dde99e6fc93637d3d97e5f5c9511bbb41317c3b182fbf97e840575a9a5ec8"} May 27 20:16:26.195541 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:16:26.195499 2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-b59bfd74-q82qr" event={"ID":"4976f387-5d72-4a0d-befe-78cd65e79e58","Type":"ContainerStarted","Data":"d713b1cbcc47570349815c2254658c7f1996b28e39d1c8178fcb4ba1b1c9fc1f"} May 27 20:16:26.195943 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:16:26.195701 2571 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-b59bfd74-q82qr" May 27 20:16:26.229825 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:16:26.229757 2571 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-b59bfd74-q82qr" podStartSLOduration=1.719640113 podStartE2EDuration="4.229717226s" podCreationTimestamp="2026-05-27 20:16:22 +0000 UTC" firstStartedPulling="2026-05-27 20:16:22.652361785 +0000 UTC m=+335.061988074" lastFinishedPulling="2026-05-27 20:16:25.162438895 +0000 UTC m=+337.572065187" observedRunningTime="2026-05-27 20:16:26.227581348 +0000 UTC m=+338.637207660" watchObservedRunningTime="2026-05-27 20:16:26.229717226 +0000 UTC m=+338.639343543" May 27 20:16:37.201160 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:16:37.201053 2571 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-b59bfd74-q82qr" May 27 20:16:40.630924 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:16:40.630888 2571 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-85db9dc568-8vxjw"] May 27 20:16:40.633978 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:16:40.633958 2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-85db9dc568-8vxjw" May 27 20:16:40.637885 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:16:40.637862 2571 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" May 27 20:16:40.638019 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:16:40.637863 2571 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-9h42v\"" May 27 20:16:40.638019 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:16:40.637922 2571 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" May 27 20:16:40.638019 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:16:40.637873 2571 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" May 27 20:16:40.638019 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:16:40.637864 2571 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" May 27 20:16:40.644364 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:16:40.644332 2571 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-85db9dc568-8vxjw"] May 27 20:16:40.673462 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:16:40.673419 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4f200e9c-bdab-48c5-a609-654bb82a823d-tls-certs\") pod \"kube-auth-proxy-85db9dc568-8vxjw\" (UID: \"4f200e9c-bdab-48c5-a609-654bb82a823d\") " pod="openshift-ingress/kube-auth-proxy-85db9dc568-8vxjw" May 27 20:16:40.673653 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:16:40.673497 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4f200e9c-bdab-48c5-a609-654bb82a823d-tmp\") pod \"kube-auth-proxy-85db9dc568-8vxjw\" (UID: \"4f200e9c-bdab-48c5-a609-654bb82a823d\") " pod="openshift-ingress/kube-auth-proxy-85db9dc568-8vxjw" May 27 20:16:40.673653 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:16:40.673532 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mmmw5\" (UniqueName: \"kubernetes.io/projected/4f200e9c-bdab-48c5-a609-654bb82a823d-kube-api-access-mmmw5\") pod \"kube-auth-proxy-85db9dc568-8vxjw\" (UID: \"4f200e9c-bdab-48c5-a609-654bb82a823d\") " pod="openshift-ingress/kube-auth-proxy-85db9dc568-8vxjw" May 27 20:16:40.774767 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:16:40.774725 2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4f200e9c-bdab-48c5-a609-654bb82a823d-tmp\") pod \"kube-auth-proxy-85db9dc568-8vxjw\" (UID: \"4f200e9c-bdab-48c5-a609-654bb82a823d\") " pod="openshift-ingress/kube-auth-proxy-85db9dc568-8vxjw" May 27 20:16:40.774907 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:16:40.774779 2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-mmmw5\" (UniqueName: \"kubernetes.io/projected/4f200e9c-bdab-48c5-a609-654bb82a823d-kube-api-access-mmmw5\") pod \"kube-auth-proxy-85db9dc568-8vxjw\" (UID: \"4f200e9c-bdab-48c5-a609-654bb82a823d\") " pod="openshift-ingress/kube-auth-proxy-85db9dc568-8vxjw" May 27 20:16:40.774950 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:16:40.774904 2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4f200e9c-bdab-48c5-a609-654bb82a823d-tls-certs\") pod \"kube-auth-proxy-85db9dc568-8vxjw\" (UID: \"4f200e9c-bdab-48c5-a609-654bb82a823d\") " pod="openshift-ingress/kube-auth-proxy-85db9dc568-8vxjw" May 27 20:16:40.777327 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:16:40.777296 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4f200e9c-bdab-48c5-a609-654bb82a823d-tmp\") pod \"kube-auth-proxy-85db9dc568-8vxjw\" (UID: \"4f200e9c-bdab-48c5-a609-654bb82a823d\") " pod="openshift-ingress/kube-auth-proxy-85db9dc568-8vxjw" May 27 20:16:40.777479 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:16:40.777463 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4f200e9c-bdab-48c5-a609-654bb82a823d-tls-certs\") pod \"kube-auth-proxy-85db9dc568-8vxjw\" (UID: \"4f200e9c-bdab-48c5-a609-654bb82a823d\") " pod="openshift-ingress/kube-auth-proxy-85db9dc568-8vxjw" May 27 20:16:40.783154 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:16:40.783125 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mmmw5\" (UniqueName: \"kubernetes.io/projected/4f200e9c-bdab-48c5-a609-654bb82a823d-kube-api-access-mmmw5\") pod \"kube-auth-proxy-85db9dc568-8vxjw\" (UID: \"4f200e9c-bdab-48c5-a609-654bb82a823d\") " pod="openshift-ingress/kube-auth-proxy-85db9dc568-8vxjw" May 27 20:16:40.943931 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:16:40.943831 2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-85db9dc568-8vxjw" May 27 20:16:41.074394 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:16:41.074360 2571 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-85db9dc568-8vxjw"] May 27 20:16:41.078662 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:16:41.078621 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4f200e9c_bdab_48c5_a609_654bb82a823d.slice/crio-1d294bf39a0bc6c347216f88e3cd81fcc523634d8cd67fe7668353d6af1db000 WatchSource:0}: Error finding container 1d294bf39a0bc6c347216f88e3cd81fcc523634d8cd67fe7668353d6af1db000: Status 404 returned error can't find the container with id 1d294bf39a0bc6c347216f88e3cd81fcc523634d8cd67fe7668353d6af1db000 May 27 20:16:41.236628 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:16:41.236531 2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-85db9dc568-8vxjw" event={"ID":"4f200e9c-bdab-48c5-a609-654bb82a823d","Type":"ContainerStarted","Data":"1d294bf39a0bc6c347216f88e3cd81fcc523634d8cd67fe7668353d6af1db000"} May 27 20:16:42.755731 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:16:42.755686 2571 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-858dbf95b8-sbtjh"] May 27 20:16:42.766426 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:16:42.766390 2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-858dbf95b8-sbtjh" May 27 20:16:42.770880 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:16:42.770569 2571 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-k7br2\"" May 27 20:16:42.770880 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:16:42.770569 2571 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\"" May 27 20:16:42.771292 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:16:42.771251 2571 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-858dbf95b8-sbtjh"] May 27 20:16:42.794237 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:16:42.794198 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8jnw5\" (UniqueName: \"kubernetes.io/projected/123bc11b-e985-42a1-b0be-3023ea71b7aa-kube-api-access-8jnw5\") pod \"odh-model-controller-858dbf95b8-sbtjh\" (UID: \"123bc11b-e985-42a1-b0be-3023ea71b7aa\") " pod="opendatahub/odh-model-controller-858dbf95b8-sbtjh" May 27 20:16:42.794424 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:16:42.794316 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/123bc11b-e985-42a1-b0be-3023ea71b7aa-cert\") pod \"odh-model-controller-858dbf95b8-sbtjh\" (UID: \"123bc11b-e985-42a1-b0be-3023ea71b7aa\") " pod="opendatahub/odh-model-controller-858dbf95b8-sbtjh" May 27 20:16:42.894989 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:16:42.894946 2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/123bc11b-e985-42a1-b0be-3023ea71b7aa-cert\") pod \"odh-model-controller-858dbf95b8-sbtjh\" (UID: \"123bc11b-e985-42a1-b0be-3023ea71b7aa\") " pod="opendatahub/odh-model-controller-858dbf95b8-sbtjh" May 27 20:16:42.895191 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:16:42.895043 2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-8jnw5\" (UniqueName: \"kubernetes.io/projected/123bc11b-e985-42a1-b0be-3023ea71b7aa-kube-api-access-8jnw5\") pod \"odh-model-controller-858dbf95b8-sbtjh\" (UID: \"123bc11b-e985-42a1-b0be-3023ea71b7aa\") " pod="opendatahub/odh-model-controller-858dbf95b8-sbtjh" May 27 20:16:42.895191 ip-10-0-142-251 kubenswrapper[2571]: E0527 20:16:42.895151 2571 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found May 27 20:16:42.895325 ip-10-0-142-251 kubenswrapper[2571]: E0527 20:16:42.895225 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/123bc11b-e985-42a1-b0be-3023ea71b7aa-cert podName:123bc11b-e985-42a1-b0be-3023ea71b7aa nodeName:}" failed. No retries permitted until 2026-05-27 20:16:43.395208888 +0000 UTC m=+355.804835197 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/123bc11b-e985-42a1-b0be-3023ea71b7aa-cert") pod "odh-model-controller-858dbf95b8-sbtjh" (UID: "123bc11b-e985-42a1-b0be-3023ea71b7aa") : secret "odh-model-controller-webhook-cert" not found May 27 20:16:42.908824 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:16:42.908753 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8jnw5\" (UniqueName: \"kubernetes.io/projected/123bc11b-e985-42a1-b0be-3023ea71b7aa-kube-api-access-8jnw5\") pod \"odh-model-controller-858dbf95b8-sbtjh\" (UID: \"123bc11b-e985-42a1-b0be-3023ea71b7aa\") " pod="opendatahub/odh-model-controller-858dbf95b8-sbtjh" May 27 20:16:43.400052 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:16:43.400008 2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/123bc11b-e985-42a1-b0be-3023ea71b7aa-cert\") pod \"odh-model-controller-858dbf95b8-sbtjh\" (UID: \"123bc11b-e985-42a1-b0be-3023ea71b7aa\") " pod="opendatahub/odh-model-controller-858dbf95b8-sbtjh" May 27 20:16:43.400248 ip-10-0-142-251 kubenswrapper[2571]: E0527 20:16:43.400155 2571 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found May 27 20:16:43.400248 ip-10-0-142-251 kubenswrapper[2571]: E0527 20:16:43.400219 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/123bc11b-e985-42a1-b0be-3023ea71b7aa-cert podName:123bc11b-e985-42a1-b0be-3023ea71b7aa nodeName:}" failed. No retries permitted until 2026-05-27 20:16:44.400204536 +0000 UTC m=+356.809830825 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/123bc11b-e985-42a1-b0be-3023ea71b7aa-cert") pod "odh-model-controller-858dbf95b8-sbtjh" (UID: "123bc11b-e985-42a1-b0be-3023ea71b7aa") : secret "odh-model-controller-webhook-cert" not found May 27 20:16:44.408001 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:16:44.407957 2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/123bc11b-e985-42a1-b0be-3023ea71b7aa-cert\") pod \"odh-model-controller-858dbf95b8-sbtjh\" (UID: \"123bc11b-e985-42a1-b0be-3023ea71b7aa\") " pod="opendatahub/odh-model-controller-858dbf95b8-sbtjh" May 27 20:16:44.411372 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:16:44.411340 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/123bc11b-e985-42a1-b0be-3023ea71b7aa-cert\") pod \"odh-model-controller-858dbf95b8-sbtjh\" (UID: \"123bc11b-e985-42a1-b0be-3023ea71b7aa\") " pod="opendatahub/odh-model-controller-858dbf95b8-sbtjh" May 27 20:16:44.580542 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:16:44.580501 2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-858dbf95b8-sbtjh" May 27 20:16:44.866965 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:16:44.866933 2571 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-858dbf95b8-sbtjh"] May 27 20:16:44.869172 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:16:44.869145 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod123bc11b_e985_42a1_b0be_3023ea71b7aa.slice/crio-3f12994f19f401bd98491aed309234e1305960a0461c110e65f8e1693738e5ab WatchSource:0}: Error finding container 3f12994f19f401bd98491aed309234e1305960a0461c110e65f8e1693738e5ab: Status 404 returned error can't find the container with id 3f12994f19f401bd98491aed309234e1305960a0461c110e65f8e1693738e5ab May 27 20:16:45.249339 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:16:45.249297 2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-85db9dc568-8vxjw" event={"ID":"4f200e9c-bdab-48c5-a609-654bb82a823d","Type":"ContainerStarted","Data":"45ff8f3846b64d63814d90096d9726fd7fbe45f90b5a1255a0921e559d611458"} May 27 20:16:45.250366 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:16:45.250340 2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-858dbf95b8-sbtjh" event={"ID":"123bc11b-e985-42a1-b0be-3023ea71b7aa","Type":"ContainerStarted","Data":"3f12994f19f401bd98491aed309234e1305960a0461c110e65f8e1693738e5ab"} May 27 20:16:45.269560 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:16:45.269511 2571 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-85db9dc568-8vxjw" podStartSLOduration=1.550565575 podStartE2EDuration="5.269497794s" podCreationTimestamp="2026-05-27 20:16:40 +0000 UTC" firstStartedPulling="2026-05-27 20:16:41.080376208 +0000 UTC m=+353.490002499" lastFinishedPulling="2026-05-27 20:16:44.799308414 +0000 UTC m=+357.208934718" observedRunningTime="2026-05-27 20:16:45.267733461 +0000 UTC m=+357.677359774" watchObservedRunningTime="2026-05-27 20:16:45.269497794 +0000 UTC m=+357.679124104" May 27 20:16:49.040615 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:16:49.040576 2571 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-856948b99f-kxfwr"] May 27 20:16:49.043872 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:16:49.043851 2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-856948b99f-kxfwr" May 27 20:16:49.047304 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:16:49.047280 2571 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\"" May 27 20:16:49.047458 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:16:49.047358 2571 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-2jgch\"" May 27 20:16:49.057336 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:16:49.057304 2571 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-856948b99f-kxfwr"] May 27 20:16:49.146897 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:16:49.146845 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-chwj6\" (UniqueName: \"kubernetes.io/projected/ad931d17-fee2-4654-af65-c270e04d9fba-kube-api-access-chwj6\") pod \"kserve-controller-manager-856948b99f-kxfwr\" (UID: \"ad931d17-fee2-4654-af65-c270e04d9fba\") " pod="opendatahub/kserve-controller-manager-856948b99f-kxfwr" May 27 20:16:49.146897 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:16:49.146903 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ad931d17-fee2-4654-af65-c270e04d9fba-cert\") pod \"kserve-controller-manager-856948b99f-kxfwr\" (UID: \"ad931d17-fee2-4654-af65-c270e04d9fba\") " pod="opendatahub/kserve-controller-manager-856948b99f-kxfwr" May 27 20:16:49.247387 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:16:49.247347 2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-chwj6\" (UniqueName: \"kubernetes.io/projected/ad931d17-fee2-4654-af65-c270e04d9fba-kube-api-access-chwj6\") pod \"kserve-controller-manager-856948b99f-kxfwr\" (UID: \"ad931d17-fee2-4654-af65-c270e04d9fba\") " pod="opendatahub/kserve-controller-manager-856948b99f-kxfwr" May 27 20:16:49.247555 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:16:49.247401 2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ad931d17-fee2-4654-af65-c270e04d9fba-cert\") pod \"kserve-controller-manager-856948b99f-kxfwr\" (UID: \"ad931d17-fee2-4654-af65-c270e04d9fba\") " pod="opendatahub/kserve-controller-manager-856948b99f-kxfwr" May 27 20:16:49.247555 ip-10-0-142-251 kubenswrapper[2571]: E0527 20:16:49.247518 2571 secret.go:189] Couldn't get secret opendatahub/kserve-webhook-server-cert: secret "kserve-webhook-server-cert" not found May 27 20:16:49.247692 ip-10-0-142-251 kubenswrapper[2571]: E0527 20:16:49.247580 2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ad931d17-fee2-4654-af65-c270e04d9fba-cert podName:ad931d17-fee2-4654-af65-c270e04d9fba nodeName:}" failed. No retries permitted until 2026-05-27 20:16:49.747558396 +0000 UTC m=+362.157184685 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/ad931d17-fee2-4654-af65-c270e04d9fba-cert") pod "kserve-controller-manager-856948b99f-kxfwr" (UID: "ad931d17-fee2-4654-af65-c270e04d9fba") : secret "kserve-webhook-server-cert" not found May 27 20:16:49.258131 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:16:49.258076 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-chwj6\" (UniqueName: \"kubernetes.io/projected/ad931d17-fee2-4654-af65-c270e04d9fba-kube-api-access-chwj6\") pod \"kserve-controller-manager-856948b99f-kxfwr\" (UID: \"ad931d17-fee2-4654-af65-c270e04d9fba\") " pod="opendatahub/kserve-controller-manager-856948b99f-kxfwr" May 27 20:16:49.263790 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:16:49.263754 2571 generic.go:358] "Generic (PLEG): container finished" podID="123bc11b-e985-42a1-b0be-3023ea71b7aa" containerID="8224dc76b5ac02a1c66d97984c65e83a868cdceff3144ad3e0fcf3a616a623fe" exitCode=1 May 27 20:16:49.264005 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:16:49.263844 2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-858dbf95b8-sbtjh" event={"ID":"123bc11b-e985-42a1-b0be-3023ea71b7aa","Type":"ContainerDied","Data":"8224dc76b5ac02a1c66d97984c65e83a868cdceff3144ad3e0fcf3a616a623fe"} May 27 20:16:49.264119 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:16:49.264090 2571 scope.go:117] "RemoveContainer" containerID="8224dc76b5ac02a1c66d97984c65e83a868cdceff3144ad3e0fcf3a616a623fe" May 27 20:16:49.752354 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:16:49.752301 2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ad931d17-fee2-4654-af65-c270e04d9fba-cert\") pod \"kserve-controller-manager-856948b99f-kxfwr\" (UID: \"ad931d17-fee2-4654-af65-c270e04d9fba\") " pod="opendatahub/kserve-controller-manager-856948b99f-kxfwr" May 27 20:16:49.754885 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:16:49.754848 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ad931d17-fee2-4654-af65-c270e04d9fba-cert\") pod \"kserve-controller-manager-856948b99f-kxfwr\" (UID: \"ad931d17-fee2-4654-af65-c270e04d9fba\") " pod="opendatahub/kserve-controller-manager-856948b99f-kxfwr" May 27 20:16:49.954367 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:16:49.954261 2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-856948b99f-kxfwr" May 27 20:16:50.119827 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:16:50.119787 2571 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-856948b99f-kxfwr"] May 27 20:16:50.122823 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:16:50.122789 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podad931d17_fee2_4654_af65_c270e04d9fba.slice/crio-43ef4805c61a097e8d1038adb2130775f54499850cdc681a0c50f2db71fae70f WatchSource:0}: Error finding container 43ef4805c61a097e8d1038adb2130775f54499850cdc681a0c50f2db71fae70f: Status 404 returned error can't find the container with id 43ef4805c61a097e8d1038adb2130775f54499850cdc681a0c50f2db71fae70f May 27 20:16:50.268938 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:16:50.268887 2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-858dbf95b8-sbtjh" event={"ID":"123bc11b-e985-42a1-b0be-3023ea71b7aa","Type":"ContainerStarted","Data":"19d6fb369ed6ddae71b383e9f49e927d56a7b04b44c9a86e0a41a0a1ae9cdb7b"} May 27 20:16:50.269161 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:16:50.269133 2571 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-858dbf95b8-sbtjh" May 27 20:16:50.269933 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:16:50.269896 2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-856948b99f-kxfwr" event={"ID":"ad931d17-fee2-4654-af65-c270e04d9fba","Type":"ContainerStarted","Data":"43ef4805c61a097e8d1038adb2130775f54499850cdc681a0c50f2db71fae70f"} May 27 20:16:50.287691 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:16:50.287611 2571 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="opendatahub/odh-model-controller-858dbf95b8-sbtjh" podStartSLOduration=3.594695531 podStartE2EDuration="8.287590088s" podCreationTimestamp="2026-05-27 20:16:42 +0000 UTC" firstStartedPulling="2026-05-27 20:16:44.870977888 +0000 UTC m=+357.280604177" lastFinishedPulling="2026-05-27 20:16:49.563872428 +0000 UTC m=+361.973498734" observedRunningTime="2026-05-27 20:16:50.286920443 +0000 UTC m=+362.696546755" watchObservedRunningTime="2026-05-27 20:16:50.287590088 +0000 UTC m=+362.697216399" May 27 20:16:53.801079 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:16:53.801037 2571 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-operators/servicemesh-operator3-55f49c5f94-rl94q"] May 27 20:16:53.804561 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:16:53.804534 2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-operators/servicemesh-operator3-55f49c5f94-rl94q" May 27 20:16:53.807739 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:16:53.807717 2571 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-operators\"/\"openshift-service-ca.crt\"" May 27 20:16:53.807858 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:16:53.807717 2571 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-operators\"/\"servicemesh-operator3-dockercfg-lpgmx\"" May 27 20:16:53.807974 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:16:53.807951 2571 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-operators\"/\"kube-root-ca.crt\"" May 27 20:16:53.819936 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:16:53.819911 2571 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-operators/servicemesh-operator3-55f49c5f94-rl94q"] May 27 20:16:53.887384 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:16:53.887338 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-config\" (UniqueName: \"kubernetes.io/downward-api/61a69cb3-ba20-4e85-9a9c-c92292cca1d5-operator-config\") pod \"servicemesh-operator3-55f49c5f94-rl94q\" (UID: \"61a69cb3-ba20-4e85-9a9c-c92292cca1d5\") " pod="openshift-operators/servicemesh-operator3-55f49c5f94-rl94q" May 27 20:16:53.887565 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:16:53.887389 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dx8k2\" (UniqueName: \"kubernetes.io/projected/61a69cb3-ba20-4e85-9a9c-c92292cca1d5-kube-api-access-dx8k2\") pod \"servicemesh-operator3-55f49c5f94-rl94q\" (UID: \"61a69cb3-ba20-4e85-9a9c-c92292cca1d5\") " pod="openshift-operators/servicemesh-operator3-55f49c5f94-rl94q" May 27 20:16:53.988855 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:16:53.988808 2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"operator-config\" (UniqueName: \"kubernetes.io/downward-api/61a69cb3-ba20-4e85-9a9c-c92292cca1d5-operator-config\") pod \"servicemesh-operator3-55f49c5f94-rl94q\" (UID: \"61a69cb3-ba20-4e85-9a9c-c92292cca1d5\") " pod="openshift-operators/servicemesh-operator3-55f49c5f94-rl94q" May 27 20:16:53.989059 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:16:53.988864 2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-dx8k2\" (UniqueName: \"kubernetes.io/projected/61a69cb3-ba20-4e85-9a9c-c92292cca1d5-kube-api-access-dx8k2\") pod \"servicemesh-operator3-55f49c5f94-rl94q\" (UID: \"61a69cb3-ba20-4e85-9a9c-c92292cca1d5\") " pod="openshift-operators/servicemesh-operator3-55f49c5f94-rl94q" May 27 20:16:53.991978 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:16:53.991946 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"operator-config\" (UniqueName: \"kubernetes.io/downward-api/61a69cb3-ba20-4e85-9a9c-c92292cca1d5-operator-config\") pod \"servicemesh-operator3-55f49c5f94-rl94q\" (UID: \"61a69cb3-ba20-4e85-9a9c-c92292cca1d5\") " pod="openshift-operators/servicemesh-operator3-55f49c5f94-rl94q" May 27 20:16:53.998802 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:16:53.998778 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dx8k2\" (UniqueName: \"kubernetes.io/projected/61a69cb3-ba20-4e85-9a9c-c92292cca1d5-kube-api-access-dx8k2\") pod \"servicemesh-operator3-55f49c5f94-rl94q\" (UID: \"61a69cb3-ba20-4e85-9a9c-c92292cca1d5\") " pod="openshift-operators/servicemesh-operator3-55f49c5f94-rl94q" May 27 20:16:54.116347 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:16:54.116253 2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-operators/servicemesh-operator3-55f49c5f94-rl94q" May 27 20:16:54.486646 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:16:54.486583 2571 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-operators/servicemesh-operator3-55f49c5f94-rl94q"] May 27 20:16:54.490305 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:16:54.490262 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod61a69cb3_ba20_4e85_9a9c_c92292cca1d5.slice/crio-27ee3bc4f9be819f53df0c09effa9866a1aa8a2364596eca0cc4f791ad0d7bc9 WatchSource:0}: Error finding container 27ee3bc4f9be819f53df0c09effa9866a1aa8a2364596eca0cc4f791ad0d7bc9: Status 404 returned error can't find the container with id 27ee3bc4f9be819f53df0c09effa9866a1aa8a2364596eca0cc4f791ad0d7bc9 May 27 20:16:55.286669 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:16:55.286630 2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-856948b99f-kxfwr" event={"ID":"ad931d17-fee2-4654-af65-c270e04d9fba","Type":"ContainerStarted","Data":"31f6663b811c5313ef7b3ae7ed3811b83748b96bdd1a74b0c7775d03e67bff97"} May 27 20:16:55.287176 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:16:55.286726 2571 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-856948b99f-kxfwr" May 27 20:16:55.287836 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:16:55.287817 2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-operators/servicemesh-operator3-55f49c5f94-rl94q" event={"ID":"61a69cb3-ba20-4e85-9a9c-c92292cca1d5","Type":"ContainerStarted","Data":"27ee3bc4f9be819f53df0c09effa9866a1aa8a2364596eca0cc4f791ad0d7bc9"} May 27 20:16:55.308333 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:16:55.308279 2571 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-856948b99f-kxfwr" podStartSLOduration=3.038969209 podStartE2EDuration="7.308263626s" podCreationTimestamp="2026-05-27 20:16:48 +0000 UTC" firstStartedPulling="2026-05-27 20:16:50.124308848 +0000 UTC m=+362.533935137" lastFinishedPulling="2026-05-27 20:16:54.39360325 +0000 UTC m=+366.803229554" observedRunningTime="2026-05-27 20:16:55.307127641 +0000 UTC m=+367.716753947" watchObservedRunningTime="2026-05-27 20:16:55.308263626 +0000 UTC m=+367.717889937" May 27 20:16:58.299691 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:16:58.299649 2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-operators/servicemesh-operator3-55f49c5f94-rl94q" event={"ID":"61a69cb3-ba20-4e85-9a9c-c92292cca1d5","Type":"ContainerStarted","Data":"018bfe69987cfc3f53068e2628e83b5d7c4b091bdfd075f70e3efe96335e0958"} May 27 20:16:58.300127 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:16:58.299824 2571 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-operators/servicemesh-operator3-55f49c5f94-rl94q" May 27 20:16:58.320772 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:16:58.320724 2571 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-operators/servicemesh-operator3-55f49c5f94-rl94q" podStartSLOduration=2.47884896 podStartE2EDuration="5.320709574s" podCreationTimestamp="2026-05-27 20:16:53 +0000 UTC" firstStartedPulling="2026-05-27 20:16:54.492825407 +0000 UTC m=+366.902451696" lastFinishedPulling="2026-05-27 20:16:57.334686018 +0000 UTC m=+369.744312310" observedRunningTime="2026-05-27 20:16:58.318344164 +0000 UTC m=+370.727970492" watchObservedRunningTime="2026-05-27 20:16:58.320709574 +0000 UTC m=+370.730335884" May 27 20:17:01.275691 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:17:01.275659 2571 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-858dbf95b8-sbtjh" May 27 20:17:04.198520 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:17:04.198484 2571 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-798958bb55-f9dxg"] May 27 20:17:04.200790 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:17:04.200765 2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-798958bb55-f9dxg" May 27 20:17:04.203415 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:17:04.203392 2571 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" May 27 20:17:04.203415 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:17:04.203403 2571 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" May 27 20:17:04.203628 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:17:04.203601 2571 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" May 27 20:17:04.203746 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:17:04.203611 2571 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" May 27 20:17:04.203746 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:17:04.203659 2571 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-4wgdb\"" May 27 20:17:04.212248 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:17:04.212226 2571 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-798958bb55-f9dxg"] May 27 20:17:04.281517 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:17:04.281474 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/9ecca3e6-bc39-4fa9-b197-c9af65680d01-cacerts\") pod \"istiod-openshift-gateway-798958bb55-f9dxg\" (UID: \"9ecca3e6-bc39-4fa9-b197-c9af65680d01\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-f9dxg" May 27 20:17:04.281684 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:17:04.281554 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/9ecca3e6-bc39-4fa9-b197-c9af65680d01-istio-kubeconfig\") pod \"istiod-openshift-gateway-798958bb55-f9dxg\" (UID: \"9ecca3e6-bc39-4fa9-b197-c9af65680d01\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-f9dxg" May 27 20:17:04.281684 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:17:04.281586 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/9ecca3e6-bc39-4fa9-b197-c9af65680d01-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-798958bb55-f9dxg\" (UID: \"9ecca3e6-bc39-4fa9-b197-c9af65680d01\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-f9dxg" May 27 20:17:04.281684 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:17:04.281615 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/9ecca3e6-bc39-4fa9-b197-c9af65680d01-local-certs\") pod \"istiod-openshift-gateway-798958bb55-f9dxg\" (UID: \"9ecca3e6-bc39-4fa9-b197-c9af65680d01\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-f9dxg" May 27 20:17:04.281798 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:17:04.281686 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f7jjk\" (UniqueName: \"kubernetes.io/projected/9ecca3e6-bc39-4fa9-b197-c9af65680d01-kube-api-access-f7jjk\") pod \"istiod-openshift-gateway-798958bb55-f9dxg\" (UID: \"9ecca3e6-bc39-4fa9-b197-c9af65680d01\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-f9dxg" May 27 20:17:04.281798 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:17:04.281733 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/9ecca3e6-bc39-4fa9-b197-c9af65680d01-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-798958bb55-f9dxg\" (UID: \"9ecca3e6-bc39-4fa9-b197-c9af65680d01\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-f9dxg" May 27 20:17:04.281798 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:17:04.281786 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9ecca3e6-bc39-4fa9-b197-c9af65680d01-istio-token\") pod \"istiod-openshift-gateway-798958bb55-f9dxg\" (UID: \"9ecca3e6-bc39-4fa9-b197-c9af65680d01\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-f9dxg" May 27 20:17:04.382188 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:17:04.382145 2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/9ecca3e6-bc39-4fa9-b197-c9af65680d01-istio-kubeconfig\") pod \"istiod-openshift-gateway-798958bb55-f9dxg\" (UID: \"9ecca3e6-bc39-4fa9-b197-c9af65680d01\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-f9dxg" May 27 20:17:04.382384 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:17:04.382207 2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/9ecca3e6-bc39-4fa9-b197-c9af65680d01-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-798958bb55-f9dxg\" (UID: \"9ecca3e6-bc39-4fa9-b197-c9af65680d01\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-f9dxg" May 27 20:17:04.382384 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:17:04.382238 2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/9ecca3e6-bc39-4fa9-b197-c9af65680d01-local-certs\") pod \"istiod-openshift-gateway-798958bb55-f9dxg\" (UID: \"9ecca3e6-bc39-4fa9-b197-c9af65680d01\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-f9dxg" May 27 20:17:04.382384 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:17:04.382272 2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-f7jjk\" (UniqueName: \"kubernetes.io/projected/9ecca3e6-bc39-4fa9-b197-c9af65680d01-kube-api-access-f7jjk\") pod \"istiod-openshift-gateway-798958bb55-f9dxg\" (UID: \"9ecca3e6-bc39-4fa9-b197-c9af65680d01\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-f9dxg" May 27 20:17:04.382384 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:17:04.382306 2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/9ecca3e6-bc39-4fa9-b197-c9af65680d01-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-798958bb55-f9dxg\" (UID: \"9ecca3e6-bc39-4fa9-b197-c9af65680d01\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-f9dxg" May 27 20:17:04.382384 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:17:04.382359 2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9ecca3e6-bc39-4fa9-b197-c9af65680d01-istio-token\") pod \"istiod-openshift-gateway-798958bb55-f9dxg\" (UID: \"9ecca3e6-bc39-4fa9-b197-c9af65680d01\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-f9dxg" May 27 20:17:04.382384 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:17:04.382384 2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/9ecca3e6-bc39-4fa9-b197-c9af65680d01-cacerts\") pod \"istiod-openshift-gateway-798958bb55-f9dxg\" (UID: \"9ecca3e6-bc39-4fa9-b197-c9af65680d01\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-f9dxg" May 27 20:17:04.383273 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:17:04.383244 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/9ecca3e6-bc39-4fa9-b197-c9af65680d01-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-798958bb55-f9dxg\" (UID: \"9ecca3e6-bc39-4fa9-b197-c9af65680d01\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-f9dxg" May 27 20:17:04.385020 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:17:04.384992 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/9ecca3e6-bc39-4fa9-b197-c9af65680d01-cacerts\") pod \"istiod-openshift-gateway-798958bb55-f9dxg\" (UID: \"9ecca3e6-bc39-4fa9-b197-c9af65680d01\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-f9dxg" May 27 20:17:04.385243 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:17:04.385222 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/9ecca3e6-bc39-4fa9-b197-c9af65680d01-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-798958bb55-f9dxg\" (UID: \"9ecca3e6-bc39-4fa9-b197-c9af65680d01\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-f9dxg" May 27 20:17:04.385313 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:17:04.385287 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/9ecca3e6-bc39-4fa9-b197-c9af65680d01-istio-kubeconfig\") pod \"istiod-openshift-gateway-798958bb55-f9dxg\" (UID: \"9ecca3e6-bc39-4fa9-b197-c9af65680d01\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-f9dxg" May 27 20:17:04.385471 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:17:04.385452 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/9ecca3e6-bc39-4fa9-b197-c9af65680d01-local-certs\") pod \"istiod-openshift-gateway-798958bb55-f9dxg\" (UID: \"9ecca3e6-bc39-4fa9-b197-c9af65680d01\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-f9dxg" May 27 20:17:04.390359 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:17:04.390330 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9ecca3e6-bc39-4fa9-b197-c9af65680d01-istio-token\") pod \"istiod-openshift-gateway-798958bb55-f9dxg\" (UID: \"9ecca3e6-bc39-4fa9-b197-c9af65680d01\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-f9dxg" May 27 20:17:04.391518 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:17:04.391499 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-f7jjk\" (UniqueName: \"kubernetes.io/projected/9ecca3e6-bc39-4fa9-b197-c9af65680d01-kube-api-access-f7jjk\") pod \"istiod-openshift-gateway-798958bb55-f9dxg\" (UID: \"9ecca3e6-bc39-4fa9-b197-c9af65680d01\") " pod="openshift-ingress/istiod-openshift-gateway-798958bb55-f9dxg" May 27 20:17:04.510471 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:17:04.510344 2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-798958bb55-f9dxg" May 27 20:17:04.656620 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:17:04.656588 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9ecca3e6_bc39_4fa9_b197_c9af65680d01.slice/crio-106d138f6b305f1c33b59e96f2fb6a5732e13bc4649d21a91634b19d8a417f99 WatchSource:0}: Error finding container 106d138f6b305f1c33b59e96f2fb6a5732e13bc4649d21a91634b19d8a417f99: Status 404 returned error can't find the container with id 106d138f6b305f1c33b59e96f2fb6a5732e13bc4649d21a91634b19d8a417f99 May 27 20:17:04.670717 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:17:04.670685 2571 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-798958bb55-f9dxg"] May 27 20:17:05.322334 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:17:05.322297 2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-798958bb55-f9dxg" event={"ID":"9ecca3e6-bc39-4fa9-b197-c9af65680d01","Type":"ContainerStarted","Data":"106d138f6b305f1c33b59e96f2fb6a5732e13bc4649d21a91634b19d8a417f99"} May 27 20:17:07.338478 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:17:07.338428 2571 kubelet_resources.go:45] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} May 27 20:17:07.338804 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:17:07.338520 2571 kubelet_resources.go:45] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} May 27 20:17:08.335937 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:17:08.335893 2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-798958bb55-f9dxg" event={"ID":"9ecca3e6-bc39-4fa9-b197-c9af65680d01","Type":"ContainerStarted","Data":"28d9dd286edae386f9eb740afc0c61d258d92856ad303a4c3bcd552a5ef83efb"} May 27 20:17:08.336603 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:17:08.336564 2571 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-798958bb55-f9dxg" May 27 20:17:08.338194 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:17:08.338138 2571 patch_prober.go:28] interesting pod/istiod-openshift-gateway-798958bb55-f9dxg container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= May 27 20:17:08.338331 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:17:08.338201 2571 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-798958bb55-f9dxg" podUID="9ecca3e6-bc39-4fa9-b197-c9af65680d01" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503" May 27 20:17:08.361694 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:17:08.361618 2571 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-798958bb55-f9dxg" podStartSLOduration=1.681982487 podStartE2EDuration="4.361596344s" podCreationTimestamp="2026-05-27 20:17:04 +0000 UTC" firstStartedPulling="2026-05-27 20:17:04.658555093 +0000 UTC m=+377.068181382" lastFinishedPulling="2026-05-27 20:17:07.338168945 +0000 UTC m=+379.747795239" observedRunningTime="2026-05-27 20:17:08.360345912 +0000 UTC m=+380.769972223" watchObservedRunningTime="2026-05-27 20:17:08.361596344 +0000 UTC m=+380.771222656" May 27 20:17:09.305178 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:17:09.305147 2571 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-operators/servicemesh-operator3-55f49c5f94-rl94q" May 27 20:17:09.340237 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:17:09.340210 2571 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-798958bb55-f9dxg" May 27 20:17:26.298020 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:17:26.297990 2571 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-856948b99f-kxfwr" May 27 20:18:20.753488 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:18:20.753406 2571 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-657f44b778-hs4xt"] May 27 20:18:20.756607 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:18:20.756588 2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-657f44b778-hs4xt" May 27 20:18:20.759354 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:18:20.759326 2571 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" May 27 20:18:20.759479 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:18:20.759399 2571 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" May 27 20:18:20.759548 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:18:20.759482 2571 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-hkqxh\"" May 27 20:18:20.765217 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:18:20.765195 2571 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-657f44b778-hs4xt"] May 27 20:18:20.883439 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:18:20.883382 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-649kj\" (UniqueName: \"kubernetes.io/projected/cf66d312-e590-40f7-946e-2773bbfaf4d1-kube-api-access-649kj\") pod \"authorino-operator-657f44b778-hs4xt\" (UID: \"cf66d312-e590-40f7-946e-2773bbfaf4d1\") " pod="kuadrant-system/authorino-operator-657f44b778-hs4xt" May 27 20:18:20.984338 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:18:20.984294 2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-649kj\" (UniqueName: \"kubernetes.io/projected/cf66d312-e590-40f7-946e-2773bbfaf4d1-kube-api-access-649kj\") pod \"authorino-operator-657f44b778-hs4xt\" (UID: \"cf66d312-e590-40f7-946e-2773bbfaf4d1\") " pod="kuadrant-system/authorino-operator-657f44b778-hs4xt" May 27 20:18:20.999036 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:18:20.999008 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-649kj\" (UniqueName: \"kubernetes.io/projected/cf66d312-e590-40f7-946e-2773bbfaf4d1-kube-api-access-649kj\") pod \"authorino-operator-657f44b778-hs4xt\" (UID: \"cf66d312-e590-40f7-946e-2773bbfaf4d1\") " pod="kuadrant-system/authorino-operator-657f44b778-hs4xt" May 27 20:18:21.067291 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:18:21.067250 2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-657f44b778-hs4xt" May 27 20:18:21.215277 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:18:21.215243 2571 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-657f44b778-hs4xt"] May 27 20:18:21.217523 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:18:21.217486 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcf66d312_e590_40f7_946e_2773bbfaf4d1.slice/crio-433c9585948f0ae5b4aae1dc9365815786674179dcdd26a19f81a7e308120417 WatchSource:0}: Error finding container 433c9585948f0ae5b4aae1dc9365815786674179dcdd26a19f81a7e308120417: Status 404 returned error can't find the container with id 433c9585948f0ae5b4aae1dc9365815786674179dcdd26a19f81a7e308120417 May 27 20:18:21.564940 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:18:21.564896 2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-657f44b778-hs4xt" event={"ID":"cf66d312-e590-40f7-946e-2773bbfaf4d1","Type":"ContainerStarted","Data":"433c9585948f0ae5b4aae1dc9365815786674179dcdd26a19f81a7e308120417"} May 27 20:18:22.982269 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:18:22.982235 2571 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-console/console-76556fbcb9-zf7g2"] May 27 20:18:22.986172 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:18:22.986142 2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-76556fbcb9-zf7g2" May 27 20:18:22.989543 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:18:22.989515 2571 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" May 27 20:18:22.989841 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:18:22.989821 2571 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" May 27 20:18:22.991154 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:18:22.991133 2571 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" May 27 20:18:22.991227 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:18:22.991137 2571 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" May 27 20:18:22.991227 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:18:22.991168 2571 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" May 27 20:18:22.991505 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:18:22.991486 2571 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" May 27 20:18:22.991594 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:18:22.991551 2571 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-h9gjg\"" May 27 20:18:22.991594 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:18:22.991489 2571 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" May 27 20:18:22.996985 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:18:22.996964 2571 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" May 27 20:18:22.997290 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:18:22.997274 2571 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-76556fbcb9-zf7g2"] May 27 20:18:23.103188 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:18:23.103157 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/06b9f345-7df7-4d77-9e37-a53e47a54abd-console-config\") pod \"console-76556fbcb9-zf7g2\" (UID: \"06b9f345-7df7-4d77-9e37-a53e47a54abd\") " pod="openshift-console/console-76556fbcb9-zf7g2" May 27 20:18:23.103188 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:18:23.103188 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/06b9f345-7df7-4d77-9e37-a53e47a54abd-console-serving-cert\") pod \"console-76556fbcb9-zf7g2\" (UID: \"06b9f345-7df7-4d77-9e37-a53e47a54abd\") " pod="openshift-console/console-76556fbcb9-zf7g2" May 27 20:18:23.103401 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:18:23.103217 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/06b9f345-7df7-4d77-9e37-a53e47a54abd-service-ca\") pod \"console-76556fbcb9-zf7g2\" (UID: \"06b9f345-7df7-4d77-9e37-a53e47a54abd\") " pod="openshift-console/console-76556fbcb9-zf7g2" May 27 20:18:23.103401 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:18:23.103258 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-chhvj\" (UniqueName: \"kubernetes.io/projected/06b9f345-7df7-4d77-9e37-a53e47a54abd-kube-api-access-chhvj\") pod \"console-76556fbcb9-zf7g2\" (UID: \"06b9f345-7df7-4d77-9e37-a53e47a54abd\") " pod="openshift-console/console-76556fbcb9-zf7g2" May 27 20:18:23.103401 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:18:23.103285 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/06b9f345-7df7-4d77-9e37-a53e47a54abd-console-oauth-config\") pod \"console-76556fbcb9-zf7g2\" (UID: \"06b9f345-7df7-4d77-9e37-a53e47a54abd\") " pod="openshift-console/console-76556fbcb9-zf7g2" May 27 20:18:23.103401 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:18:23.103300 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/06b9f345-7df7-4d77-9e37-a53e47a54abd-trusted-ca-bundle\") pod \"console-76556fbcb9-zf7g2\" (UID: \"06b9f345-7df7-4d77-9e37-a53e47a54abd\") " pod="openshift-console/console-76556fbcb9-zf7g2" May 27 20:18:23.103401 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:18:23.103344 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/06b9f345-7df7-4d77-9e37-a53e47a54abd-oauth-serving-cert\") pod \"console-76556fbcb9-zf7g2\" (UID: \"06b9f345-7df7-4d77-9e37-a53e47a54abd\") " pod="openshift-console/console-76556fbcb9-zf7g2" May 27 20:18:23.204596 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:18:23.204568 2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/06b9f345-7df7-4d77-9e37-a53e47a54abd-console-config\") pod \"console-76556fbcb9-zf7g2\" (UID: \"06b9f345-7df7-4d77-9e37-a53e47a54abd\") " pod="openshift-console/console-76556fbcb9-zf7g2" May 27 20:18:23.204596 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:18:23.204596 2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/06b9f345-7df7-4d77-9e37-a53e47a54abd-console-serving-cert\") pod \"console-76556fbcb9-zf7g2\" (UID: \"06b9f345-7df7-4d77-9e37-a53e47a54abd\") " pod="openshift-console/console-76556fbcb9-zf7g2" May 27 20:18:23.204810 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:18:23.204624 2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/06b9f345-7df7-4d77-9e37-a53e47a54abd-service-ca\") pod \"console-76556fbcb9-zf7g2\" (UID: \"06b9f345-7df7-4d77-9e37-a53e47a54abd\") " pod="openshift-console/console-76556fbcb9-zf7g2" May 27 20:18:23.204810 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:18:23.204649 2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-chhvj\" (UniqueName: \"kubernetes.io/projected/06b9f345-7df7-4d77-9e37-a53e47a54abd-kube-api-access-chhvj\") pod \"console-76556fbcb9-zf7g2\" (UID: \"06b9f345-7df7-4d77-9e37-a53e47a54abd\") " pod="openshift-console/console-76556fbcb9-zf7g2" May 27 20:18:23.204810 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:18:23.204690 2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/06b9f345-7df7-4d77-9e37-a53e47a54abd-console-oauth-config\") pod \"console-76556fbcb9-zf7g2\" (UID: \"06b9f345-7df7-4d77-9e37-a53e47a54abd\") " pod="openshift-console/console-76556fbcb9-zf7g2" May 27 20:18:23.204810 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:18:23.204711 2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/06b9f345-7df7-4d77-9e37-a53e47a54abd-trusted-ca-bundle\") pod \"console-76556fbcb9-zf7g2\" (UID: \"06b9f345-7df7-4d77-9e37-a53e47a54abd\") " pod="openshift-console/console-76556fbcb9-zf7g2" May 27 20:18:23.204810 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:18:23.204740 2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/06b9f345-7df7-4d77-9e37-a53e47a54abd-oauth-serving-cert\") pod \"console-76556fbcb9-zf7g2\" (UID: \"06b9f345-7df7-4d77-9e37-a53e47a54abd\") " pod="openshift-console/console-76556fbcb9-zf7g2" May 27 20:18:23.205377 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:18:23.205343 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/06b9f345-7df7-4d77-9e37-a53e47a54abd-console-config\") pod \"console-76556fbcb9-zf7g2\" (UID: \"06b9f345-7df7-4d77-9e37-a53e47a54abd\") " pod="openshift-console/console-76556fbcb9-zf7g2" May 27 20:18:23.205490 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:18:23.205466 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/06b9f345-7df7-4d77-9e37-a53e47a54abd-service-ca\") pod \"console-76556fbcb9-zf7g2\" (UID: \"06b9f345-7df7-4d77-9e37-a53e47a54abd\") " pod="openshift-console/console-76556fbcb9-zf7g2" May 27 20:18:23.205628 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:18:23.205611 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/06b9f345-7df7-4d77-9e37-a53e47a54abd-trusted-ca-bundle\") pod \"console-76556fbcb9-zf7g2\" (UID: \"06b9f345-7df7-4d77-9e37-a53e47a54abd\") " pod="openshift-console/console-76556fbcb9-zf7g2" May 27 20:18:23.205744 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:18:23.205713 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/06b9f345-7df7-4d77-9e37-a53e47a54abd-oauth-serving-cert\") pod \"console-76556fbcb9-zf7g2\" (UID: \"06b9f345-7df7-4d77-9e37-a53e47a54abd\") " pod="openshift-console/console-76556fbcb9-zf7g2" May 27 20:18:23.207233 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:18:23.207212 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/06b9f345-7df7-4d77-9e37-a53e47a54abd-console-serving-cert\") pod \"console-76556fbcb9-zf7g2\" (UID: \"06b9f345-7df7-4d77-9e37-a53e47a54abd\") " pod="openshift-console/console-76556fbcb9-zf7g2" May 27 20:18:23.207304 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:18:23.207219 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/06b9f345-7df7-4d77-9e37-a53e47a54abd-console-oauth-config\") pod \"console-76556fbcb9-zf7g2\" (UID: \"06b9f345-7df7-4d77-9e37-a53e47a54abd\") " pod="openshift-console/console-76556fbcb9-zf7g2" May 27 20:18:23.212586 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:18:23.212563 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-chhvj\" (UniqueName: \"kubernetes.io/projected/06b9f345-7df7-4d77-9e37-a53e47a54abd-kube-api-access-chhvj\") pod \"console-76556fbcb9-zf7g2\" (UID: \"06b9f345-7df7-4d77-9e37-a53e47a54abd\") " pod="openshift-console/console-76556fbcb9-zf7g2" May 27 20:18:23.298022 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:18:23.297995 2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-76556fbcb9-zf7g2" May 27 20:18:23.433748 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:18:23.433700 2571 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-76556fbcb9-zf7g2"] May 27 20:18:23.611608 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:18:23.611533 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod06b9f345_7df7_4d77_9e37_a53e47a54abd.slice/crio-e7a48a87e7bbffe8d25c2740f00123b3796bfe37e716fc6fd024d08212df4d6c WatchSource:0}: Error finding container e7a48a87e7bbffe8d25c2740f00123b3796bfe37e716fc6fd024d08212df4d6c: Status 404 returned error can't find the container with id e7a48a87e7bbffe8d25c2740f00123b3796bfe37e716fc6fd024d08212df4d6c May 27 20:18:24.576556 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:18:24.576502 2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-76556fbcb9-zf7g2" event={"ID":"06b9f345-7df7-4d77-9e37-a53e47a54abd","Type":"ContainerStarted","Data":"35aaccd1cb3087a4408614a128a5ed4464a433b9cd257e7eb46082f93a117b79"} May 27 20:18:24.577052 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:18:24.576564 2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-76556fbcb9-zf7g2" event={"ID":"06b9f345-7df7-4d77-9e37-a53e47a54abd","Type":"ContainerStarted","Data":"e7a48a87e7bbffe8d25c2740f00123b3796bfe37e716fc6fd024d08212df4d6c"} May 27 20:18:24.577847 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:18:24.577823 2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-657f44b778-hs4xt" event={"ID":"cf66d312-e590-40f7-946e-2773bbfaf4d1","Type":"ContainerStarted","Data":"691c5ccfdf040f1c7a3faefe453adf85b172b57176de6b74f80f1b3deec2ce0e"} May 27 20:18:24.577965 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:18:24.577954 2571 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-657f44b778-hs4xt" May 27 20:18:24.596009 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:18:24.595951 2571 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-console/console-76556fbcb9-zf7g2" podStartSLOduration=2.595936167 podStartE2EDuration="2.595936167s" podCreationTimestamp="2026-05-27 20:18:22 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-27 20:18:24.593272617 +0000 UTC m=+457.002898927" watchObservedRunningTime="2026-05-27 20:18:24.595936167 +0000 UTC m=+457.005562477" May 27 20:18:24.609799 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:18:24.609748 2571 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-657f44b778-hs4xt" podStartSLOduration=2.170165718 podStartE2EDuration="4.609734042s" podCreationTimestamp="2026-05-27 20:18:20 +0000 UTC" firstStartedPulling="2026-05-27 20:18:21.219503935 +0000 UTC m=+453.629130224" lastFinishedPulling="2026-05-27 20:18:23.659072259 +0000 UTC m=+456.068698548" observedRunningTime="2026-05-27 20:18:24.608993183 +0000 UTC m=+457.018619496" watchObservedRunningTime="2026-05-27 20:18:24.609734042 +0000 UTC m=+457.019360330" May 27 20:18:33.299124 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:18:33.299070 2571 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-76556fbcb9-zf7g2" May 27 20:18:33.299710 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:18:33.299147 2571 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-76556fbcb9-zf7g2" May 27 20:18:33.303752 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:18:33.303730 2571 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-76556fbcb9-zf7g2" May 27 20:18:33.614940 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:18:33.614863 2571 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-76556fbcb9-zf7g2" May 27 20:18:35.583236 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:18:35.583204 2571 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-657f44b778-hs4xt" May 27 20:18:36.322826 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:18:36.322788 2571 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-84b657d985-cg2mr"] May 27 20:18:36.325974 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:18:36.325956 2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-84b657d985-cg2mr" May 27 20:18:36.330001 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:18:36.329977 2571 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-hpcdj\"" May 27 20:18:36.340826 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:18:36.340804 2571 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-84b657d985-cg2mr"] May 27 20:18:36.411179 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:18:36.411097 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/aeb5f7be-5c79-4d63-9b83-1ed9684d41d5-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-84b657d985-cg2mr\" (UID: \"aeb5f7be-5c79-4d63-9b83-1ed9684d41d5\") " pod="kuadrant-system/kuadrant-operator-controller-manager-84b657d985-cg2mr" May 27 20:18:36.411357 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:18:36.411192 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-blhzz\" (UniqueName: \"kubernetes.io/projected/aeb5f7be-5c79-4d63-9b83-1ed9684d41d5-kube-api-access-blhzz\") pod \"kuadrant-operator-controller-manager-84b657d985-cg2mr\" (UID: \"aeb5f7be-5c79-4d63-9b83-1ed9684d41d5\") " pod="kuadrant-system/kuadrant-operator-controller-manager-84b657d985-cg2mr" May 27 20:18:36.512217 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:18:36.512176 2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/aeb5f7be-5c79-4d63-9b83-1ed9684d41d5-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-84b657d985-cg2mr\" (UID: \"aeb5f7be-5c79-4d63-9b83-1ed9684d41d5\") " pod="kuadrant-system/kuadrant-operator-controller-manager-84b657d985-cg2mr" May 27 20:18:36.512217 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:18:36.512217 2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-blhzz\" (UniqueName: \"kubernetes.io/projected/aeb5f7be-5c79-4d63-9b83-1ed9684d41d5-kube-api-access-blhzz\") pod \"kuadrant-operator-controller-manager-84b657d985-cg2mr\" (UID: \"aeb5f7be-5c79-4d63-9b83-1ed9684d41d5\") " pod="kuadrant-system/kuadrant-operator-controller-manager-84b657d985-cg2mr" May 27 20:18:36.512624 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:18:36.512598 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/aeb5f7be-5c79-4d63-9b83-1ed9684d41d5-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-84b657d985-cg2mr\" (UID: \"aeb5f7be-5c79-4d63-9b83-1ed9684d41d5\") " pod="kuadrant-system/kuadrant-operator-controller-manager-84b657d985-cg2mr" May 27 20:18:36.524121 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:18:36.524076 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-blhzz\" (UniqueName: \"kubernetes.io/projected/aeb5f7be-5c79-4d63-9b83-1ed9684d41d5-kube-api-access-blhzz\") pod \"kuadrant-operator-controller-manager-84b657d985-cg2mr\" (UID: \"aeb5f7be-5c79-4d63-9b83-1ed9684d41d5\") " pod="kuadrant-system/kuadrant-operator-controller-manager-84b657d985-cg2mr" May 27 20:18:36.635882 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:18:36.635789 2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-84b657d985-cg2mr" May 27 20:18:36.719186 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:18:36.719152 2571 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-84b657d985-cg2mr"] May 27 20:18:36.730844 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:18:36.730567 2571 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-84b657d985-cg2mr"] May 27 20:18:36.748756 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:18:36.748728 2571 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-84b657d985-nqj72"] May 27 20:18:36.752633 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:18:36.752612 2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-84b657d985-nqj72" May 27 20:18:36.770974 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:18:36.770945 2571 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-84b657d985-nqj72"] May 27 20:18:36.815915 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:18:36.815882 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hm6lw\" (UniqueName: \"kubernetes.io/projected/7b3d9942-7d8b-48a4-814d-550c55b6a28e-kube-api-access-hm6lw\") pod \"kuadrant-operator-controller-manager-84b657d985-nqj72\" (UID: \"7b3d9942-7d8b-48a4-814d-550c55b6a28e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-84b657d985-nqj72" May 27 20:18:36.816137 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:18:36.815956 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7b3d9942-7d8b-48a4-814d-550c55b6a28e-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-84b657d985-nqj72\" (UID: \"7b3d9942-7d8b-48a4-814d-550c55b6a28e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-84b657d985-nqj72" May 27 20:18:36.902456 ip-10-0-142-251 kubenswrapper[2571]: E0527 20:18:36.902420 2571 log.go:32] "RunPodSandbox from runtime service failed" err=< May 27 20:18:36.902456 ip-10-0-142-251 kubenswrapper[2571]: rpc error: code = Unknown desc = failed to create pod network sandbox k8s_kuadrant-operator-controller-manager-84b657d985-cg2mr_kuadrant-system_aeb5f7be-5c79-4d63-9b83-1ed9684d41d5_0(046e390e29055f4506255613a7d4d46b170960cf14b23381246bca635e4a8c89): error adding pod kuadrant-system_kuadrant-operator-controller-manager-84b657d985-cg2mr to CNI network "multus-cni-network": plugin type="multus-shim" name="multus-cni-network" failed (add): CmdAdd (shim): CNI request failed with status 400: 'ContainerID:"046e390e29055f4506255613a7d4d46b170960cf14b23381246bca635e4a8c89" Netns:"/var/run/netns/27346e32-c3bf-4685-ad4d-257ce1ffd30c" IfName:"eth0" Args:"IgnoreUnknown=1;K8S_POD_NAMESPACE=kuadrant-system;K8S_POD_NAME=kuadrant-operator-controller-manager-84b657d985-cg2mr;K8S_POD_INFRA_CONTAINER_ID=046e390e29055f4506255613a7d4d46b170960cf14b23381246bca635e4a8c89;K8S_POD_UID=aeb5f7be-5c79-4d63-9b83-1ed9684d41d5" Path:"" ERRORED: error configuring pod [kuadrant-system/kuadrant-operator-controller-manager-84b657d985-cg2mr] networking: [kuadrant-system/kuadrant-operator-controller-manager-84b657d985-cg2mr/aeb5f7be-5c79-4d63-9b83-1ed9684d41d5:ovn-kubernetes]: error adding container to network "ovn-kubernetes": CNI request failed with status 400: '[kuadrant-system/kuadrant-operator-controller-manager-84b657d985-cg2mr 046e390e29055f4506255613a7d4d46b170960cf14b23381246bca635e4a8c89 network default NAD default] [kuadrant-system/kuadrant-operator-controller-manager-84b657d985-cg2mr 046e390e29055f4506255613a7d4d46b170960cf14b23381246bca635e4a8c89 network default NAD default] failed to configure pod interface: pod deleted waiting for OVS port binding for 0a:58:0a:85:00:17 [10.133.0.23/23] May 27 20:18:36.902456 ip-10-0-142-251 kubenswrapper[2571]: ' May 27 20:18:36.902456 ip-10-0-142-251 kubenswrapper[2571]: ': StdinData: {"auxiliaryCNIChainName":"vendor-cni-chain","binDir":"/var/lib/cni/bin","clusterNetwork":"/host/run/multus/cni/net.d/10-ovn-kubernetes.conf","cniVersion":"0.3.1","daemonSocketDir":"/run/multus/socket","globalNamespaces":"default,openshift-multus,openshift-sriov-network-operator,openshift-cnv","logLevel":"verbose","logToStderr":true,"name":"multus-cni-network","namespaceIsolation":true,"type":"multus-shim"} May 27 20:18:36.902456 ip-10-0-142-251 kubenswrapper[2571]: > May 27 20:18:36.902744 ip-10-0-142-251 kubenswrapper[2571]: E0527 20:18:36.902482 2571 kuberuntime_sandbox.go:70] "Failed to create sandbox for pod" err=< May 27 20:18:36.902744 ip-10-0-142-251 kubenswrapper[2571]: rpc error: code = Unknown desc = failed to create pod network sandbox k8s_kuadrant-operator-controller-manager-84b657d985-cg2mr_kuadrant-system_aeb5f7be-5c79-4d63-9b83-1ed9684d41d5_0(046e390e29055f4506255613a7d4d46b170960cf14b23381246bca635e4a8c89): error adding pod kuadrant-system_kuadrant-operator-controller-manager-84b657d985-cg2mr to CNI network "multus-cni-network": plugin type="multus-shim" name="multus-cni-network" failed (add): CmdAdd (shim): CNI request failed with status 400: 'ContainerID:"046e390e29055f4506255613a7d4d46b170960cf14b23381246bca635e4a8c89" Netns:"/var/run/netns/27346e32-c3bf-4685-ad4d-257ce1ffd30c" IfName:"eth0" Args:"IgnoreUnknown=1;K8S_POD_NAMESPACE=kuadrant-system;K8S_POD_NAME=kuadrant-operator-controller-manager-84b657d985-cg2mr;K8S_POD_INFRA_CONTAINER_ID=046e390e29055f4506255613a7d4d46b170960cf14b23381246bca635e4a8c89;K8S_POD_UID=aeb5f7be-5c79-4d63-9b83-1ed9684d41d5" Path:"" ERRORED: error configuring pod [kuadrant-system/kuadrant-operator-controller-manager-84b657d985-cg2mr] networking: [kuadrant-system/kuadrant-operator-controller-manager-84b657d985-cg2mr/aeb5f7be-5c79-4d63-9b83-1ed9684d41d5:ovn-kubernetes]: error adding container to network "ovn-kubernetes": CNI request failed with status 400: '[kuadrant-system/kuadrant-operator-controller-manager-84b657d985-cg2mr 046e390e29055f4506255613a7d4d46b170960cf14b23381246bca635e4a8c89 network default NAD default] [kuadrant-system/kuadrant-operator-controller-manager-84b657d985-cg2mr 046e390e29055f4506255613a7d4d46b170960cf14b23381246bca635e4a8c89 network default NAD default] failed to configure pod interface: pod deleted waiting for OVS port binding for 0a:58:0a:85:00:17 [10.133.0.23/23] May 27 20:18:36.902744 ip-10-0-142-251 kubenswrapper[2571]: ' May 27 20:18:36.902744 ip-10-0-142-251 kubenswrapper[2571]: ': StdinData: {"auxiliaryCNIChainName":"vendor-cni-chain","binDir":"/var/lib/cni/bin","clusterNetwork":"/host/run/multus/cni/net.d/10-ovn-kubernetes.conf","cniVersion":"0.3.1","daemonSocketDir":"/run/multus/socket","globalNamespaces":"default,openshift-multus,openshift-sriov-network-operator,openshift-cnv","logLevel":"verbose","logToStderr":true,"name":"multus-cni-network","namespaceIsolation":true,"type":"multus-shim"} May 27 20:18:36.902744 ip-10-0-142-251 kubenswrapper[2571]: > pod="kuadrant-system/kuadrant-operator-controller-manager-84b657d985-cg2mr" May 27 20:18:36.917080 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:18:36.917045 2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7b3d9942-7d8b-48a4-814d-550c55b6a28e-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-84b657d985-nqj72\" (UID: \"7b3d9942-7d8b-48a4-814d-550c55b6a28e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-84b657d985-nqj72" May 27 20:18:36.917207 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:18:36.917096 2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-hm6lw\" (UniqueName: \"kubernetes.io/projected/7b3d9942-7d8b-48a4-814d-550c55b6a28e-kube-api-access-hm6lw\") pod \"kuadrant-operator-controller-manager-84b657d985-nqj72\" (UID: \"7b3d9942-7d8b-48a4-814d-550c55b6a28e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-84b657d985-nqj72" May 27 20:18:36.917457 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:18:36.917436 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7b3d9942-7d8b-48a4-814d-550c55b6a28e-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-84b657d985-nqj72\" (UID: \"7b3d9942-7d8b-48a4-814d-550c55b6a28e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-84b657d985-nqj72" May 27 20:18:36.931505 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:18:36.931475 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hm6lw\" (UniqueName: \"kubernetes.io/projected/7b3d9942-7d8b-48a4-814d-550c55b6a28e-kube-api-access-hm6lw\") pod \"kuadrant-operator-controller-manager-84b657d985-nqj72\" (UID: \"7b3d9942-7d8b-48a4-814d-550c55b6a28e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-84b657d985-nqj72" May 27 20:18:37.066429 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:18:37.066396 2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-84b657d985-nqj72" May 27 20:18:37.190941 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:18:37.190916 2571 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-84b657d985-nqj72"] May 27 20:18:37.193256 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:18:37.193226 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7b3d9942_7d8b_48a4_814d_550c55b6a28e.slice/crio-30575ba96c843e0caf3e7a6a25450e7431bec393c694c7c8ae96d7c56513cf1d WatchSource:0}: Error finding container 30575ba96c843e0caf3e7a6a25450e7431bec393c694c7c8ae96d7c56513cf1d: Status 404 returned error can't find the container with id 30575ba96c843e0caf3e7a6a25450e7431bec393c694c7c8ae96d7c56513cf1d May 27 20:18:37.624773 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:18:37.624739 2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-84b657d985-nqj72" event={"ID":"7b3d9942-7d8b-48a4-814d-550c55b6a28e","Type":"ContainerStarted","Data":"30575ba96c843e0caf3e7a6a25450e7431bec393c694c7c8ae96d7c56513cf1d"} May 27 20:18:37.624773 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:18:37.624762 2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-84b657d985-cg2mr" May 27 20:18:37.627255 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:18:37.627217 2571 status_manager.go:895] "Failed to get status for pod" podUID="aeb5f7be-5c79-4d63-9b83-1ed9684d41d5" pod="kuadrant-system/kuadrant-operator-controller-manager-84b657d985-cg2mr" err="pods \"kuadrant-operator-controller-manager-84b657d985-cg2mr\" is forbidden: User \"system:node:ip-10-0-142-251.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-251.ec2.internal' and this object" May 27 20:18:37.629619 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:18:37.629601 2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-84b657d985-cg2mr" May 27 20:18:37.631991 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:18:37.631967 2571 status_manager.go:895] "Failed to get status for pod" podUID="aeb5f7be-5c79-4d63-9b83-1ed9684d41d5" pod="kuadrant-system/kuadrant-operator-controller-manager-84b657d985-cg2mr" err="pods \"kuadrant-operator-controller-manager-84b657d985-cg2mr\" is forbidden: User \"system:node:ip-10-0-142-251.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-251.ec2.internal' and this object" May 27 20:18:37.723468 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:18:37.723432 2571 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/aeb5f7be-5c79-4d63-9b83-1ed9684d41d5-extensions-socket-volume\") pod \"aeb5f7be-5c79-4d63-9b83-1ed9684d41d5\" (UID: \"aeb5f7be-5c79-4d63-9b83-1ed9684d41d5\") " May 27 20:18:37.723878 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:18:37.723497 2571 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-blhzz\" (UniqueName: \"kubernetes.io/projected/aeb5f7be-5c79-4d63-9b83-1ed9684d41d5-kube-api-access-blhzz\") pod \"aeb5f7be-5c79-4d63-9b83-1ed9684d41d5\" (UID: \"aeb5f7be-5c79-4d63-9b83-1ed9684d41d5\") " May 27 20:18:37.723878 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:18:37.723712 2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aeb5f7be-5c79-4d63-9b83-1ed9684d41d5-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "aeb5f7be-5c79-4d63-9b83-1ed9684d41d5" (UID: "aeb5f7be-5c79-4d63-9b83-1ed9684d41d5"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" May 27 20:18:37.725743 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:18:37.725715 2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aeb5f7be-5c79-4d63-9b83-1ed9684d41d5-kube-api-access-blhzz" (OuterVolumeSpecName: "kube-api-access-blhzz") pod "aeb5f7be-5c79-4d63-9b83-1ed9684d41d5" (UID: "aeb5f7be-5c79-4d63-9b83-1ed9684d41d5"). InnerVolumeSpecName "kube-api-access-blhzz". PluginName "kubernetes.io/projected", VolumeGIDValue "" May 27 20:18:37.824997 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:18:37.824962 2571 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/aeb5f7be-5c79-4d63-9b83-1ed9684d41d5-extensions-socket-volume\") on node \"ip-10-0-142-251.ec2.internal\" DevicePath \"\"" May 27 20:18:37.824997 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:18:37.824994 2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-blhzz\" (UniqueName: \"kubernetes.io/projected/aeb5f7be-5c79-4d63-9b83-1ed9684d41d5-kube-api-access-blhzz\") on node \"ip-10-0-142-251.ec2.internal\" DevicePath \"\"" May 27 20:18:38.229487 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:18:38.229443 2571 status_manager.go:895] "Failed to get status for pod" podUID="aeb5f7be-5c79-4d63-9b83-1ed9684d41d5" pod="kuadrant-system/kuadrant-operator-controller-manager-84b657d985-cg2mr" err="pods \"kuadrant-operator-controller-manager-84b657d985-cg2mr\" is forbidden: User \"system:node:ip-10-0-142-251.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-251.ec2.internal' and this object" May 27 20:18:38.230196 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:18:38.230160 2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="aeb5f7be-5c79-4d63-9b83-1ed9684d41d5" path="/var/lib/kubelet/pods/aeb5f7be-5c79-4d63-9b83-1ed9684d41d5/volumes" May 27 20:18:38.628387 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:18:38.628359 2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-84b657d985-cg2mr" May 27 20:18:38.633196 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:18:38.633166 2571 status_manager.go:895] "Failed to get status for pod" podUID="aeb5f7be-5c79-4d63-9b83-1ed9684d41d5" pod="kuadrant-system/kuadrant-operator-controller-manager-84b657d985-cg2mr" err="pods \"kuadrant-operator-controller-manager-84b657d985-cg2mr\" is forbidden: User \"system:node:ip-10-0-142-251.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-251.ec2.internal' and this object" May 27 20:18:43.652222 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:18:43.652179 2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-84b657d985-nqj72" event={"ID":"7b3d9942-7d8b-48a4-814d-550c55b6a28e","Type":"ContainerStarted","Data":"68a0f5ff526d3b1456f760f59207834f1cab41b130026d4ad7be10e624c84910"} May 27 20:18:43.652586 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:18:43.652466 2571 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-84b657d985-nqj72" May 27 20:18:43.674441 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:18:43.674390 2571 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-84b657d985-nqj72" podStartSLOduration=2.236173546 podStartE2EDuration="7.674375469s" podCreationTimestamp="2026-05-27 20:18:36 +0000 UTC" firstStartedPulling="2026-05-27 20:18:37.197813459 +0000 UTC m=+469.607439749" lastFinishedPulling="2026-05-27 20:18:42.636015379 +0000 UTC m=+475.045641672" observedRunningTime="2026-05-27 20:18:43.672317463 +0000 UTC m=+476.081943785" watchObservedRunningTime="2026-05-27 20:18:43.674375469 +0000 UTC m=+476.084001781" May 27 20:18:54.657664 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:18:54.657628 2571 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-84b657d985-nqj72" May 27 20:19:07.022184 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:19:07.022150 2571 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-84b657d985-nqj72"] May 27 20:19:07.022673 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:19:07.022393 2571 kuberuntime_container.go:864] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-84b657d985-nqj72" podUID="7b3d9942-7d8b-48a4-814d-550c55b6a28e" containerName="manager" containerID="cri-o://68a0f5ff526d3b1456f760f59207834f1cab41b130026d4ad7be10e624c84910" gracePeriod=10 May 27 20:19:07.263053 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:19:07.263029 2571 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-84b657d985-nqj72" May 27 20:19:07.371826 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:19:07.371714 2571 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hm6lw\" (UniqueName: \"kubernetes.io/projected/7b3d9942-7d8b-48a4-814d-550c55b6a28e-kube-api-access-hm6lw\") pod \"7b3d9942-7d8b-48a4-814d-550c55b6a28e\" (UID: \"7b3d9942-7d8b-48a4-814d-550c55b6a28e\") " May 27 20:19:07.371826 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:19:07.371775 2571 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7b3d9942-7d8b-48a4-814d-550c55b6a28e-extensions-socket-volume\") pod \"7b3d9942-7d8b-48a4-814d-550c55b6a28e\" (UID: \"7b3d9942-7d8b-48a4-814d-550c55b6a28e\") " May 27 20:19:07.372206 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:19:07.372174 2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7b3d9942-7d8b-48a4-814d-550c55b6a28e-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "7b3d9942-7d8b-48a4-814d-550c55b6a28e" (UID: "7b3d9942-7d8b-48a4-814d-550c55b6a28e"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" May 27 20:19:07.374074 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:19:07.374054 2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7b3d9942-7d8b-48a4-814d-550c55b6a28e-kube-api-access-hm6lw" (OuterVolumeSpecName: "kube-api-access-hm6lw") pod "7b3d9942-7d8b-48a4-814d-550c55b6a28e" (UID: "7b3d9942-7d8b-48a4-814d-550c55b6a28e"). InnerVolumeSpecName "kube-api-access-hm6lw". PluginName "kubernetes.io/projected", VolumeGIDValue "" May 27 20:19:07.473010 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:19:07.472968 2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hm6lw\" (UniqueName: \"kubernetes.io/projected/7b3d9942-7d8b-48a4-814d-550c55b6a28e-kube-api-access-hm6lw\") on node \"ip-10-0-142-251.ec2.internal\" DevicePath \"\"" May 27 20:19:07.473010 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:19:07.473000 2571 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7b3d9942-7d8b-48a4-814d-550c55b6a28e-extensions-socket-volume\") on node \"ip-10-0-142-251.ec2.internal\" DevicePath \"\"" May 27 20:19:07.727843 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:19:07.727757 2571 generic.go:358] "Generic (PLEG): container finished" podID="7b3d9942-7d8b-48a4-814d-550c55b6a28e" containerID="68a0f5ff526d3b1456f760f59207834f1cab41b130026d4ad7be10e624c84910" exitCode=0 May 27 20:19:07.727843 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:19:07.727829 2571 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-84b657d985-nqj72" May 27 20:19:07.728053 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:19:07.727844 2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-84b657d985-nqj72" event={"ID":"7b3d9942-7d8b-48a4-814d-550c55b6a28e","Type":"ContainerDied","Data":"68a0f5ff526d3b1456f760f59207834f1cab41b130026d4ad7be10e624c84910"} May 27 20:19:07.728053 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:19:07.727894 2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-84b657d985-nqj72" event={"ID":"7b3d9942-7d8b-48a4-814d-550c55b6a28e","Type":"ContainerDied","Data":"30575ba96c843e0caf3e7a6a25450e7431bec393c694c7c8ae96d7c56513cf1d"} May 27 20:19:07.728053 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:19:07.727912 2571 scope.go:117] "RemoveContainer" containerID="68a0f5ff526d3b1456f760f59207834f1cab41b130026d4ad7be10e624c84910" May 27 20:19:07.736238 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:19:07.736217 2571 scope.go:117] "RemoveContainer" containerID="68a0f5ff526d3b1456f760f59207834f1cab41b130026d4ad7be10e624c84910" May 27 20:19:07.736507 ip-10-0-142-251 kubenswrapper[2571]: E0527 20:19:07.736489 2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"68a0f5ff526d3b1456f760f59207834f1cab41b130026d4ad7be10e624c84910\": container with ID starting with 68a0f5ff526d3b1456f760f59207834f1cab41b130026d4ad7be10e624c84910 not found: ID does not exist" containerID="68a0f5ff526d3b1456f760f59207834f1cab41b130026d4ad7be10e624c84910" May 27 20:19:07.736561 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:19:07.736517 2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"68a0f5ff526d3b1456f760f59207834f1cab41b130026d4ad7be10e624c84910"} err="failed to get container status \"68a0f5ff526d3b1456f760f59207834f1cab41b130026d4ad7be10e624c84910\": rpc error: code = NotFound desc = could not find container \"68a0f5ff526d3b1456f760f59207834f1cab41b130026d4ad7be10e624c84910\": container with ID starting with 68a0f5ff526d3b1456f760f59207834f1cab41b130026d4ad7be10e624c84910 not found: ID does not exist" May 27 20:19:07.751721 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:19:07.751692 2571 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-84b657d985-nqj72"] May 27 20:19:07.759122 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:19:07.759081 2571 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-84b657d985-nqj72"] May 27 20:19:08.229090 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:19:08.229054 2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7b3d9942-7d8b-48a4-814d-550c55b6a28e" path="/var/lib/kubelet/pods/7b3d9942-7d8b-48a4-814d-550c55b6a28e/volumes" May 27 20:19:15.153810 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:19:15.153771 2571 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-7d549b5b-v7bzz"] May 27 20:19:15.154291 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:19:15.154123 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7b3d9942-7d8b-48a4-814d-550c55b6a28e" containerName="manager" May 27 20:19:15.154291 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:19:15.154138 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="7b3d9942-7d8b-48a4-814d-550c55b6a28e" containerName="manager" May 27 20:19:15.154291 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:19:15.154187 2571 memory_manager.go:356] "RemoveStaleState removing state" podUID="7b3d9942-7d8b-48a4-814d-550c55b6a28e" containerName="manager" May 27 20:19:15.157868 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:19:15.157849 2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-7d549b5b-v7bzz" May 27 20:19:15.160859 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:19:15.160836 2571 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" May 27 20:19:15.160979 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:19:15.160895 2571 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-ctvqq\"" May 27 20:19:15.165959 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:19:15.165938 2571 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7d549b5b-v7bzz"] May 27 20:19:15.231964 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:19:15.231929 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/375dc40a-e16e-48c4-9e40-dc7344e0808f-config-file\") pod \"limitador-limitador-7d549b5b-v7bzz\" (UID: \"375dc40a-e16e-48c4-9e40-dc7344e0808f\") " pod="kuadrant-system/limitador-limitador-7d549b5b-v7bzz" May 27 20:19:15.232135 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:19:15.231972 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kh97v\" (UniqueName: \"kubernetes.io/projected/375dc40a-e16e-48c4-9e40-dc7344e0808f-kube-api-access-kh97v\") pod \"limitador-limitador-7d549b5b-v7bzz\" (UID: \"375dc40a-e16e-48c4-9e40-dc7344e0808f\") " pod="kuadrant-system/limitador-limitador-7d549b5b-v7bzz" May 27 20:19:15.262648 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:19:15.262620 2571 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7d549b5b-v7bzz"] May 27 20:19:15.333321 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:19:15.333293 2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/375dc40a-e16e-48c4-9e40-dc7344e0808f-config-file\") pod \"limitador-limitador-7d549b5b-v7bzz\" (UID: \"375dc40a-e16e-48c4-9e40-dc7344e0808f\") " pod="kuadrant-system/limitador-limitador-7d549b5b-v7bzz" May 27 20:19:15.333483 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:19:15.333328 2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-kh97v\" (UniqueName: \"kubernetes.io/projected/375dc40a-e16e-48c4-9e40-dc7344e0808f-kube-api-access-kh97v\") pod \"limitador-limitador-7d549b5b-v7bzz\" (UID: \"375dc40a-e16e-48c4-9e40-dc7344e0808f\") " pod="kuadrant-system/limitador-limitador-7d549b5b-v7bzz" May 27 20:19:15.333883 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:19:15.333863 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/375dc40a-e16e-48c4-9e40-dc7344e0808f-config-file\") pod \"limitador-limitador-7d549b5b-v7bzz\" (UID: \"375dc40a-e16e-48c4-9e40-dc7344e0808f\") " pod="kuadrant-system/limitador-limitador-7d549b5b-v7bzz" May 27 20:19:15.342865 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:19:15.342842 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kh97v\" (UniqueName: \"kubernetes.io/projected/375dc40a-e16e-48c4-9e40-dc7344e0808f-kube-api-access-kh97v\") pod \"limitador-limitador-7d549b5b-v7bzz\" (UID: \"375dc40a-e16e-48c4-9e40-dc7344e0808f\") " pod="kuadrant-system/limitador-limitador-7d549b5b-v7bzz" May 27 20:19:15.468868 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:19:15.468792 2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-7d549b5b-v7bzz" May 27 20:19:15.612958 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:19:15.612925 2571 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7d549b5b-v7bzz"] May 27 20:19:15.616398 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:19:15.616373 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod375dc40a_e16e_48c4_9e40_dc7344e0808f.slice/crio-f75c0eac3f7f049533dc9b6989ca9ade41b01832af5ce612294bc9d4393686eb WatchSource:0}: Error finding container f75c0eac3f7f049533dc9b6989ca9ade41b01832af5ce612294bc9d4393686eb: Status 404 returned error can't find the container with id f75c0eac3f7f049533dc9b6989ca9ade41b01832af5ce612294bc9d4393686eb May 27 20:19:15.755572 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:19:15.755499 2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-7d549b5b-v7bzz" event={"ID":"375dc40a-e16e-48c4-9e40-dc7344e0808f","Type":"ContainerStarted","Data":"f75c0eac3f7f049533dc9b6989ca9ade41b01832af5ce612294bc9d4393686eb"} May 27 20:19:18.766968 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:19:18.766935 2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-7d549b5b-v7bzz" event={"ID":"375dc40a-e16e-48c4-9e40-dc7344e0808f","Type":"ContainerStarted","Data":"c8ffd4f60ad999509a746cfd3e5cccb401643771c11f83c00030cf59795c0ec9"} May 27 20:19:18.767308 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:19:18.767059 2571 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-7d549b5b-v7bzz" May 27 20:19:18.803540 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:19:18.803493 2571 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-7d549b5b-v7bzz" podStartSLOduration=0.814319858 podStartE2EDuration="3.80347909s" podCreationTimestamp="2026-05-27 20:19:15 +0000 UTC" firstStartedPulling="2026-05-27 20:19:15.618014176 +0000 UTC m=+508.027640465" lastFinishedPulling="2026-05-27 20:19:18.607173408 +0000 UTC m=+511.016799697" observedRunningTime="2026-05-27 20:19:18.802392879 +0000 UTC m=+511.212019204" watchObservedRunningTime="2026-05-27 20:19:18.80347909 +0000 UTC m=+511.213105401" May 27 20:19:29.770590 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:19:29.770554 2571 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-7d549b5b-v7bzz" May 27 20:19:31.485444 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:19:31.485365 2571 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-7d549b5b-v7bzz"] May 27 20:19:31.485832 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:19:31.485580 2571 kuberuntime_container.go:864] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-7d549b5b-v7bzz" podUID="375dc40a-e16e-48c4-9e40-dc7344e0808f" containerName="limitador" containerID="cri-o://c8ffd4f60ad999509a746cfd3e5cccb401643771c11f83c00030cf59795c0ec9" gracePeriod=30 May 27 20:19:32.020715 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:19:32.020691 2571 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-7d549b5b-v7bzz" May 27 20:19:32.065324 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:19:32.065238 2571 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/375dc40a-e16e-48c4-9e40-dc7344e0808f-config-file\") pod \"375dc40a-e16e-48c4-9e40-dc7344e0808f\" (UID: \"375dc40a-e16e-48c4-9e40-dc7344e0808f\") " May 27 20:19:32.065324 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:19:32.065281 2571 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kh97v\" (UniqueName: \"kubernetes.io/projected/375dc40a-e16e-48c4-9e40-dc7344e0808f-kube-api-access-kh97v\") pod \"375dc40a-e16e-48c4-9e40-dc7344e0808f\" (UID: \"375dc40a-e16e-48c4-9e40-dc7344e0808f\") " May 27 20:19:32.065573 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:19:32.065545 2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/375dc40a-e16e-48c4-9e40-dc7344e0808f-config-file" (OuterVolumeSpecName: "config-file") pod "375dc40a-e16e-48c4-9e40-dc7344e0808f" (UID: "375dc40a-e16e-48c4-9e40-dc7344e0808f"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue "" May 27 20:19:32.067372 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:19:32.067343 2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/375dc40a-e16e-48c4-9e40-dc7344e0808f-kube-api-access-kh97v" (OuterVolumeSpecName: "kube-api-access-kh97v") pod "375dc40a-e16e-48c4-9e40-dc7344e0808f" (UID: "375dc40a-e16e-48c4-9e40-dc7344e0808f"). InnerVolumeSpecName "kube-api-access-kh97v". PluginName "kubernetes.io/projected", VolumeGIDValue "" May 27 20:19:32.166017 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:19:32.165979 2571 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/375dc40a-e16e-48c4-9e40-dc7344e0808f-config-file\") on node \"ip-10-0-142-251.ec2.internal\" DevicePath \"\"" May 27 20:19:32.166017 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:19:32.166012 2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kh97v\" (UniqueName: \"kubernetes.io/projected/375dc40a-e16e-48c4-9e40-dc7344e0808f-kube-api-access-kh97v\") on node \"ip-10-0-142-251.ec2.internal\" DevicePath \"\"" May 27 20:19:32.811572 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:19:32.811540 2571 generic.go:358] "Generic (PLEG): container finished" podID="375dc40a-e16e-48c4-9e40-dc7344e0808f" containerID="c8ffd4f60ad999509a746cfd3e5cccb401643771c11f83c00030cf59795c0ec9" exitCode=0 May 27 20:19:32.811993 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:19:32.811604 2571 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-7d549b5b-v7bzz" May 27 20:19:32.811993 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:19:32.811609 2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-7d549b5b-v7bzz" event={"ID":"375dc40a-e16e-48c4-9e40-dc7344e0808f","Type":"ContainerDied","Data":"c8ffd4f60ad999509a746cfd3e5cccb401643771c11f83c00030cf59795c0ec9"} May 27 20:19:32.811993 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:19:32.811698 2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-7d549b5b-v7bzz" event={"ID":"375dc40a-e16e-48c4-9e40-dc7344e0808f","Type":"ContainerDied","Data":"f75c0eac3f7f049533dc9b6989ca9ade41b01832af5ce612294bc9d4393686eb"} May 27 20:19:32.811993 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:19:32.811712 2571 scope.go:117] "RemoveContainer" containerID="c8ffd4f60ad999509a746cfd3e5cccb401643771c11f83c00030cf59795c0ec9" May 27 20:19:32.819562 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:19:32.819546 2571 scope.go:117] "RemoveContainer" containerID="c8ffd4f60ad999509a746cfd3e5cccb401643771c11f83c00030cf59795c0ec9" May 27 20:19:32.819820 ip-10-0-142-251 kubenswrapper[2571]: E0527 20:19:32.819794 2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c8ffd4f60ad999509a746cfd3e5cccb401643771c11f83c00030cf59795c0ec9\": container with ID starting with c8ffd4f60ad999509a746cfd3e5cccb401643771c11f83c00030cf59795c0ec9 not found: ID does not exist" containerID="c8ffd4f60ad999509a746cfd3e5cccb401643771c11f83c00030cf59795c0ec9" May 27 20:19:32.819882 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:19:32.819830 2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c8ffd4f60ad999509a746cfd3e5cccb401643771c11f83c00030cf59795c0ec9"} err="failed to get container status \"c8ffd4f60ad999509a746cfd3e5cccb401643771c11f83c00030cf59795c0ec9\": rpc error: code = NotFound desc = could not find container \"c8ffd4f60ad999509a746cfd3e5cccb401643771c11f83c00030cf59795c0ec9\": container with ID starting with c8ffd4f60ad999509a746cfd3e5cccb401643771c11f83c00030cf59795c0ec9 not found: ID does not exist" May 27 20:19:32.831804 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:19:32.831775 2571 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-7d549b5b-v7bzz"] May 27 20:19:32.840761 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:19:32.840740 2571 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-7d549b5b-v7bzz"] May 27 20:19:34.228633 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:19:34.228602 2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="375dc40a-e16e-48c4-9e40-dc7344e0808f" path="/var/lib/kubelet/pods/375dc40a-e16e-48c4-9e40-dc7344e0808f/volumes" May 27 20:19:36.403796 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:19:36.403760 2571 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["opendatahub/postgres-68c4fbbd6f-s7hf9"] May 27 20:19:36.404357 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:19:36.404191 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="375dc40a-e16e-48c4-9e40-dc7344e0808f" containerName="limitador" May 27 20:19:36.404357 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:19:36.404208 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="375dc40a-e16e-48c4-9e40-dc7344e0808f" containerName="limitador" May 27 20:19:36.404357 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:19:36.404295 2571 memory_manager.go:356] "RemoveStaleState removing state" podUID="375dc40a-e16e-48c4-9e40-dc7344e0808f" containerName="limitador" May 27 20:19:36.407300 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:19:36.407280 2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/postgres-68c4fbbd6f-s7hf9" May 27 20:19:36.409933 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:19:36.409911 2571 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"postgres-creds\"" May 27 20:19:36.410036 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:19:36.409912 2571 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-7k9sc\"" May 27 20:19:36.417284 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:19:36.417260 2571 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/postgres-68c4fbbd6f-s7hf9"] May 27 20:19:36.502435 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:19:36.502394 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/b1d3f9e4-5e89-470f-be70-9f4d65f1bd3e-data\") pod \"postgres-68c4fbbd6f-s7hf9\" (UID: \"b1d3f9e4-5e89-470f-be70-9f4d65f1bd3e\") " pod="opendatahub/postgres-68c4fbbd6f-s7hf9" May 27 20:19:36.502571 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:19:36.502450 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cl8d2\" (UniqueName: \"kubernetes.io/projected/b1d3f9e4-5e89-470f-be70-9f4d65f1bd3e-kube-api-access-cl8d2\") pod \"postgres-68c4fbbd6f-s7hf9\" (UID: \"b1d3f9e4-5e89-470f-be70-9f4d65f1bd3e\") " pod="opendatahub/postgres-68c4fbbd6f-s7hf9" May 27 20:19:36.603091 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:19:36.603054 2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-cl8d2\" (UniqueName: \"kubernetes.io/projected/b1d3f9e4-5e89-470f-be70-9f4d65f1bd3e-kube-api-access-cl8d2\") pod \"postgres-68c4fbbd6f-s7hf9\" (UID: \"b1d3f9e4-5e89-470f-be70-9f4d65f1bd3e\") " pod="opendatahub/postgres-68c4fbbd6f-s7hf9" May 27 20:19:36.603227 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:19:36.603150 2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/b1d3f9e4-5e89-470f-be70-9f4d65f1bd3e-data\") pod \"postgres-68c4fbbd6f-s7hf9\" (UID: \"b1d3f9e4-5e89-470f-be70-9f4d65f1bd3e\") " pod="opendatahub/postgres-68c4fbbd6f-s7hf9" May 27 20:19:36.603443 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:19:36.603428 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/b1d3f9e4-5e89-470f-be70-9f4d65f1bd3e-data\") pod \"postgres-68c4fbbd6f-s7hf9\" (UID: \"b1d3f9e4-5e89-470f-be70-9f4d65f1bd3e\") " pod="opendatahub/postgres-68c4fbbd6f-s7hf9" May 27 20:19:36.613094 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:19:36.613070 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cl8d2\" (UniqueName: \"kubernetes.io/projected/b1d3f9e4-5e89-470f-be70-9f4d65f1bd3e-kube-api-access-cl8d2\") pod \"postgres-68c4fbbd6f-s7hf9\" (UID: \"b1d3f9e4-5e89-470f-be70-9f4d65f1bd3e\") " pod="opendatahub/postgres-68c4fbbd6f-s7hf9" May 27 20:19:36.717698 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:19:36.717641 2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/postgres-68c4fbbd6f-s7hf9" May 27 20:19:36.841313 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:19:36.841280 2571 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/postgres-68c4fbbd6f-s7hf9"] May 27 20:19:36.845040 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:19:36.845016 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb1d3f9e4_5e89_470f_be70_9f4d65f1bd3e.slice/crio-2379e555c3e5ee92be27900df96d562cf9128303f04818891d50e45d249cc64a WatchSource:0}: Error finding container 2379e555c3e5ee92be27900df96d562cf9128303f04818891d50e45d249cc64a: Status 404 returned error can't find the container with id 2379e555c3e5ee92be27900df96d562cf9128303f04818891d50e45d249cc64a May 27 20:19:37.829419 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:19:37.829384 2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/postgres-68c4fbbd6f-s7hf9" event={"ID":"b1d3f9e4-5e89-470f-be70-9f4d65f1bd3e","Type":"ContainerStarted","Data":"2379e555c3e5ee92be27900df96d562cf9128303f04818891d50e45d249cc64a"} May 27 20:19:42.850770 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:19:42.850733 2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/postgres-68c4fbbd6f-s7hf9" event={"ID":"b1d3f9e4-5e89-470f-be70-9f4d65f1bd3e","Type":"ContainerStarted","Data":"9475dce4bf087e8e7e9fb88aebe9088486411102c8ab35e5f83c84f24b53bf23"} May 27 20:19:42.851140 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:19:42.850862 2571 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/postgres-68c4fbbd6f-s7hf9" May 27 20:19:42.869812 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:19:42.869771 2571 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="opendatahub/postgres-68c4fbbd6f-s7hf9" podStartSLOduration=1.7299713730000001 podStartE2EDuration="6.869760074s" podCreationTimestamp="2026-05-27 20:19:36 +0000 UTC" firstStartedPulling="2026-05-27 20:19:36.846186272 +0000 UTC m=+529.255812561" lastFinishedPulling="2026-05-27 20:19:41.985974954 +0000 UTC m=+534.395601262" observedRunningTime="2026-05-27 20:19:42.868419676 +0000 UTC m=+535.278045987" watchObservedRunningTime="2026-05-27 20:19:42.869760074 +0000 UTC m=+535.279386385" May 27 20:19:48.883820 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:19:48.883792 2571 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/postgres-68c4fbbd6f-s7hf9" May 27 20:19:54.799219 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:19:54.799184 2571 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-7fc5b4db85-gtg5b"] May 27 20:19:54.827706 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:19:54.827675 2571 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7fc5b4db85-gtg5b"] May 27 20:19:54.827859 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:19:54.827782 2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7fc5b4db85-gtg5b" May 27 20:19:54.830555 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:19:54.830530 2571 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" May 27 20:19:54.830698 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:19:54.830562 2571 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" May 27 20:19:54.830698 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:19:54.830571 2571 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-jrjjn\"" May 27 20:19:54.949958 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:19:54.949922 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5lfp5\" (UniqueName: \"kubernetes.io/projected/0ac11f45-aa16-4ac7-9582-2574ba2908ea-kube-api-access-5lfp5\") pod \"maas-controller-7fc5b4db85-gtg5b\" (UID: \"0ac11f45-aa16-4ac7-9582-2574ba2908ea\") " pod="opendatahub/maas-controller-7fc5b4db85-gtg5b" May 27 20:19:54.950182 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:19:54.949972 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0ac11f45-aa16-4ac7-9582-2574ba2908ea-webhook-cert\") pod \"maas-controller-7fc5b4db85-gtg5b\" (UID: \"0ac11f45-aa16-4ac7-9582-2574ba2908ea\") " pod="opendatahub/maas-controller-7fc5b4db85-gtg5b" May 27 20:19:54.974951 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:19:54.974917 2571 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-67678bdf-wnxjw"] May 27 20:19:54.978247 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:19:54.978211 2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-67678bdf-wnxjw" May 27 20:19:54.991589 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:19:54.991565 2571 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-67678bdf-wnxjw"] May 27 20:19:55.051168 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:19:55.051076 2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-5lfp5\" (UniqueName: \"kubernetes.io/projected/0ac11f45-aa16-4ac7-9582-2574ba2908ea-kube-api-access-5lfp5\") pod \"maas-controller-7fc5b4db85-gtg5b\" (UID: \"0ac11f45-aa16-4ac7-9582-2574ba2908ea\") " pod="opendatahub/maas-controller-7fc5b4db85-gtg5b" May 27 20:19:55.051168 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:19:55.051138 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/97a920f6-dbe1-40bf-a722-c1936c3ce760-webhook-cert\") pod \"maas-controller-67678bdf-wnxjw\" (UID: \"97a920f6-dbe1-40bf-a722-c1936c3ce760\") " pod="opendatahub/maas-controller-67678bdf-wnxjw" May 27 20:19:55.051168 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:19:55.051161 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hknmd\" (UniqueName: \"kubernetes.io/projected/97a920f6-dbe1-40bf-a722-c1936c3ce760-kube-api-access-hknmd\") pod \"maas-controller-67678bdf-wnxjw\" (UID: \"97a920f6-dbe1-40bf-a722-c1936c3ce760\") " pod="opendatahub/maas-controller-67678bdf-wnxjw" May 27 20:19:55.051411 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:19:55.051182 2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0ac11f45-aa16-4ac7-9582-2574ba2908ea-webhook-cert\") pod \"maas-controller-7fc5b4db85-gtg5b\" (UID: \"0ac11f45-aa16-4ac7-9582-2574ba2908ea\") " pod="opendatahub/maas-controller-7fc5b4db85-gtg5b" May 27 20:19:55.053607 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:19:55.053585 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0ac11f45-aa16-4ac7-9582-2574ba2908ea-webhook-cert\") pod \"maas-controller-7fc5b4db85-gtg5b\" (UID: \"0ac11f45-aa16-4ac7-9582-2574ba2908ea\") " pod="opendatahub/maas-controller-7fc5b4db85-gtg5b" May 27 20:19:55.058509 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:19:55.058485 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5lfp5\" (UniqueName: \"kubernetes.io/projected/0ac11f45-aa16-4ac7-9582-2574ba2908ea-kube-api-access-5lfp5\") pod \"maas-controller-7fc5b4db85-gtg5b\" (UID: \"0ac11f45-aa16-4ac7-9582-2574ba2908ea\") " pod="opendatahub/maas-controller-7fc5b4db85-gtg5b" May 27 20:19:55.139543 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:19:55.139519 2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7fc5b4db85-gtg5b" May 27 20:19:55.152526 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:19:55.152497 2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/97a920f6-dbe1-40bf-a722-c1936c3ce760-webhook-cert\") pod \"maas-controller-67678bdf-wnxjw\" (UID: \"97a920f6-dbe1-40bf-a722-c1936c3ce760\") " pod="opendatahub/maas-controller-67678bdf-wnxjw" May 27 20:19:55.152627 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:19:55.152548 2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-hknmd\" (UniqueName: \"kubernetes.io/projected/97a920f6-dbe1-40bf-a722-c1936c3ce760-kube-api-access-hknmd\") pod \"maas-controller-67678bdf-wnxjw\" (UID: \"97a920f6-dbe1-40bf-a722-c1936c3ce760\") " pod="opendatahub/maas-controller-67678bdf-wnxjw" May 27 20:19:55.154709 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:19:55.154684 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/97a920f6-dbe1-40bf-a722-c1936c3ce760-webhook-cert\") pod \"maas-controller-67678bdf-wnxjw\" (UID: \"97a920f6-dbe1-40bf-a722-c1936c3ce760\") " pod="opendatahub/maas-controller-67678bdf-wnxjw" May 27 20:19:55.161147 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:19:55.161123 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hknmd\" (UniqueName: \"kubernetes.io/projected/97a920f6-dbe1-40bf-a722-c1936c3ce760-kube-api-access-hknmd\") pod \"maas-controller-67678bdf-wnxjw\" (UID: \"97a920f6-dbe1-40bf-a722-c1936c3ce760\") " pod="opendatahub/maas-controller-67678bdf-wnxjw" May 27 20:19:55.267706 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:19:55.267681 2571 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7fc5b4db85-gtg5b"] May 27 20:19:55.269900 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:19:55.269873 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0ac11f45_aa16_4ac7_9582_2574ba2908ea.slice/crio-30725ab32bd23b02a5d6578fcb91951ef9a6b38905eadb16d89badefc2904ebc WatchSource:0}: Error finding container 30725ab32bd23b02a5d6578fcb91951ef9a6b38905eadb16d89badefc2904ebc: Status 404 returned error can't find the container with id 30725ab32bd23b02a5d6578fcb91951ef9a6b38905eadb16d89badefc2904ebc May 27 20:19:55.287909 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:19:55.287885 2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-67678bdf-wnxjw" May 27 20:19:55.411833 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:19:55.411659 2571 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-67678bdf-wnxjw"] May 27 20:19:55.414285 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:19:55.414258 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod97a920f6_dbe1_40bf_a722_c1936c3ce760.slice/crio-ce081e3d8b32bdbc588e140a1dd952163e37b4d8f9e76bb5b565abcaece087d6 WatchSource:0}: Error finding container ce081e3d8b32bdbc588e140a1dd952163e37b4d8f9e76bb5b565abcaece087d6: Status 404 returned error can't find the container with id ce081e3d8b32bdbc588e140a1dd952163e37b4d8f9e76bb5b565abcaece087d6 May 27 20:19:55.897369 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:19:55.897327 2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7fc5b4db85-gtg5b" event={"ID":"0ac11f45-aa16-4ac7-9582-2574ba2908ea","Type":"ContainerStarted","Data":"30725ab32bd23b02a5d6578fcb91951ef9a6b38905eadb16d89badefc2904ebc"} May 27 20:19:55.900443 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:19:55.900410 2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-67678bdf-wnxjw" event={"ID":"97a920f6-dbe1-40bf-a722-c1936c3ce760","Type":"ContainerStarted","Data":"ce081e3d8b32bdbc588e140a1dd952163e37b4d8f9e76bb5b565abcaece087d6"} May 27 20:19:58.911124 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:19:58.911062 2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-67678bdf-wnxjw" event={"ID":"97a920f6-dbe1-40bf-a722-c1936c3ce760","Type":"ContainerStarted","Data":"7679fe1a1bfa07eaae9328710eec89e014a75ccb53403ecc365aa29e5d2b8463"} May 27 20:19:58.911563 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:19:58.911245 2571 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-67678bdf-wnxjw" May 27 20:19:58.912450 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:19:58.912422 2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7fc5b4db85-gtg5b" event={"ID":"0ac11f45-aa16-4ac7-9582-2574ba2908ea","Type":"ContainerStarted","Data":"2084c7c51ca23c92f778440d2aa23f63dfb7385a4f4fffda258ce5faf105b126"} May 27 20:19:58.912597 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:19:58.912585 2571 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-7fc5b4db85-gtg5b" May 27 20:19:58.936751 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:19:58.936705 2571 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="opendatahub/maas-controller-67678bdf-wnxjw" podStartSLOduration=2.407863178 podStartE2EDuration="4.936693352s" podCreationTimestamp="2026-05-27 20:19:54 +0000 UTC" firstStartedPulling="2026-05-27 20:19:55.415609941 +0000 UTC m=+547.825236229" lastFinishedPulling="2026-05-27 20:19:57.944440088 +0000 UTC m=+550.354066403" observedRunningTime="2026-05-27 20:19:58.935980417 +0000 UTC m=+551.345606727" watchObservedRunningTime="2026-05-27 20:19:58.936693352 +0000 UTC m=+551.346319663" May 27 20:19:58.962542 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:19:58.962495 2571 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="opendatahub/maas-controller-7fc5b4db85-gtg5b" podStartSLOduration=2.298840081 podStartE2EDuration="4.962484068s" podCreationTimestamp="2026-05-27 20:19:54 +0000 UTC" firstStartedPulling="2026-05-27 20:19:55.271210954 +0000 UTC m=+547.680837243" lastFinishedPulling="2026-05-27 20:19:57.934854938 +0000 UTC m=+550.344481230" observedRunningTime="2026-05-27 20:19:58.961905376 +0000 UTC m=+551.371531688" watchObservedRunningTime="2026-05-27 20:19:58.962484068 +0000 UTC m=+551.372110378" May 27 20:20:09.921398 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:20:09.921366 2571 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-67678bdf-wnxjw" May 27 20:20:09.921773 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:20:09.921417 2571 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-7fc5b4db85-gtg5b" May 27 20:20:09.981323 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:20:09.981292 2571 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-7fc5b4db85-gtg5b"] May 27 20:20:09.981535 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:20:09.981514 2571 kuberuntime_container.go:864] "Killing container with a grace period" pod="opendatahub/maas-controller-7fc5b4db85-gtg5b" podUID="0ac11f45-aa16-4ac7-9582-2574ba2908ea" containerName="manager" containerID="cri-o://2084c7c51ca23c92f778440d2aa23f63dfb7385a4f4fffda258ce5faf105b126" gracePeriod=30 May 27 20:20:10.219374 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:20:10.219350 2571 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7fc5b4db85-gtg5b" May 27 20:20:10.372206 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:20:10.372170 2571 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5lfp5\" (UniqueName: \"kubernetes.io/projected/0ac11f45-aa16-4ac7-9582-2574ba2908ea-kube-api-access-5lfp5\") pod \"0ac11f45-aa16-4ac7-9582-2574ba2908ea\" (UID: \"0ac11f45-aa16-4ac7-9582-2574ba2908ea\") " May 27 20:20:10.372206 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:20:10.372204 2571 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0ac11f45-aa16-4ac7-9582-2574ba2908ea-webhook-cert\") pod \"0ac11f45-aa16-4ac7-9582-2574ba2908ea\" (UID: \"0ac11f45-aa16-4ac7-9582-2574ba2908ea\") " May 27 20:20:10.374510 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:20:10.374483 2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0ac11f45-aa16-4ac7-9582-2574ba2908ea-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "0ac11f45-aa16-4ac7-9582-2574ba2908ea" (UID: "0ac11f45-aa16-4ac7-9582-2574ba2908ea"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" May 27 20:20:10.374652 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:20:10.374604 2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0ac11f45-aa16-4ac7-9582-2574ba2908ea-kube-api-access-5lfp5" (OuterVolumeSpecName: "kube-api-access-5lfp5") pod "0ac11f45-aa16-4ac7-9582-2574ba2908ea" (UID: "0ac11f45-aa16-4ac7-9582-2574ba2908ea"). InnerVolumeSpecName "kube-api-access-5lfp5". PluginName "kubernetes.io/projected", VolumeGIDValue "" May 27 20:20:10.472784 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:20:10.472701 2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5lfp5\" (UniqueName: \"kubernetes.io/projected/0ac11f45-aa16-4ac7-9582-2574ba2908ea-kube-api-access-5lfp5\") on node \"ip-10-0-142-251.ec2.internal\" DevicePath \"\"" May 27 20:20:10.472784 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:20:10.472731 2571 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0ac11f45-aa16-4ac7-9582-2574ba2908ea-webhook-cert\") on node \"ip-10-0-142-251.ec2.internal\" DevicePath \"\"" May 27 20:20:10.951999 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:20:10.951962 2571 generic.go:358] "Generic (PLEG): container finished" podID="0ac11f45-aa16-4ac7-9582-2574ba2908ea" containerID="2084c7c51ca23c92f778440d2aa23f63dfb7385a4f4fffda258ce5faf105b126" exitCode=0 May 27 20:20:10.952440 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:20:10.952028 2571 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7fc5b4db85-gtg5b" May 27 20:20:10.952440 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:20:10.952038 2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7fc5b4db85-gtg5b" event={"ID":"0ac11f45-aa16-4ac7-9582-2574ba2908ea","Type":"ContainerDied","Data":"2084c7c51ca23c92f778440d2aa23f63dfb7385a4f4fffda258ce5faf105b126"} May 27 20:20:10.952440 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:20:10.952076 2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7fc5b4db85-gtg5b" event={"ID":"0ac11f45-aa16-4ac7-9582-2574ba2908ea","Type":"ContainerDied","Data":"30725ab32bd23b02a5d6578fcb91951ef9a6b38905eadb16d89badefc2904ebc"} May 27 20:20:10.952440 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:20:10.952095 2571 scope.go:117] "RemoveContainer" containerID="2084c7c51ca23c92f778440d2aa23f63dfb7385a4f4fffda258ce5faf105b126" May 27 20:20:10.959938 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:20:10.959921 2571 scope.go:117] "RemoveContainer" containerID="2084c7c51ca23c92f778440d2aa23f63dfb7385a4f4fffda258ce5faf105b126" May 27 20:20:10.960223 ip-10-0-142-251 kubenswrapper[2571]: E0527 20:20:10.960205 2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2084c7c51ca23c92f778440d2aa23f63dfb7385a4f4fffda258ce5faf105b126\": container with ID starting with 2084c7c51ca23c92f778440d2aa23f63dfb7385a4f4fffda258ce5faf105b126 not found: ID does not exist" containerID="2084c7c51ca23c92f778440d2aa23f63dfb7385a4f4fffda258ce5faf105b126" May 27 20:20:10.960294 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:20:10.960230 2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2084c7c51ca23c92f778440d2aa23f63dfb7385a4f4fffda258ce5faf105b126"} err="failed to get container status \"2084c7c51ca23c92f778440d2aa23f63dfb7385a4f4fffda258ce5faf105b126\": rpc error: code = NotFound desc = could not find container \"2084c7c51ca23c92f778440d2aa23f63dfb7385a4f4fffda258ce5faf105b126\": container with ID starting with 2084c7c51ca23c92f778440d2aa23f63dfb7385a4f4fffda258ce5faf105b126 not found: ID does not exist" May 27 20:20:10.972378 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:20:10.972348 2571 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-7fc5b4db85-gtg5b"] May 27 20:20:10.976623 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:20:10.976602 2571 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-7fc5b4db85-gtg5b"] May 27 20:20:12.233621 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:20:12.233579 2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0ac11f45-aa16-4ac7-9582-2574ba2908ea" path="/var/lib/kubelet/pods/0ac11f45-aa16-4ac7-9582-2574ba2908ea/volumes" May 27 20:20:26.861916 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:20:26.861880 2571 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-6776cdf886-wcqkx"] May 27 20:20:26.862408 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:20:26.862228 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0ac11f45-aa16-4ac7-9582-2574ba2908ea" containerName="manager" May 27 20:20:26.862408 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:20:26.862242 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="0ac11f45-aa16-4ac7-9582-2574ba2908ea" containerName="manager" May 27 20:20:26.862408 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:20:26.862300 2571 memory_manager.go:356] "RemoveStaleState removing state" podUID="0ac11f45-aa16-4ac7-9582-2574ba2908ea" containerName="manager" May 27 20:20:26.866113 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:20:26.866079 2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-6776cdf886-wcqkx" May 27 20:20:26.870211 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:20:26.870172 2571 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-serving-cert\"" May 27 20:20:26.870211 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:20:26.870172 2571 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-bt7f8\"" May 27 20:20:26.876714 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:20:26.876688 2571 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-6776cdf886-wcqkx"] May 27 20:20:26.917957 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:20:26.917925 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ls7sf\" (UniqueName: \"kubernetes.io/projected/06e664ca-e53f-470b-8b2c-0abebf376de6-kube-api-access-ls7sf\") pod \"maas-api-6776cdf886-wcqkx\" (UID: \"06e664ca-e53f-470b-8b2c-0abebf376de6\") " pod="opendatahub/maas-api-6776cdf886-wcqkx" May 27 20:20:26.918167 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:20:26.917965 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/06e664ca-e53f-470b-8b2c-0abebf376de6-maas-api-tls\") pod \"maas-api-6776cdf886-wcqkx\" (UID: \"06e664ca-e53f-470b-8b2c-0abebf376de6\") " pod="opendatahub/maas-api-6776cdf886-wcqkx" May 27 20:20:27.018540 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:20:27.018498 2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-ls7sf\" (UniqueName: \"kubernetes.io/projected/06e664ca-e53f-470b-8b2c-0abebf376de6-kube-api-access-ls7sf\") pod \"maas-api-6776cdf886-wcqkx\" (UID: \"06e664ca-e53f-470b-8b2c-0abebf376de6\") " pod="opendatahub/maas-api-6776cdf886-wcqkx" May 27 20:20:27.018540 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:20:27.018545 2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/06e664ca-e53f-470b-8b2c-0abebf376de6-maas-api-tls\") pod \"maas-api-6776cdf886-wcqkx\" (UID: \"06e664ca-e53f-470b-8b2c-0abebf376de6\") " pod="opendatahub/maas-api-6776cdf886-wcqkx" May 27 20:20:27.021456 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:20:27.021427 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/06e664ca-e53f-470b-8b2c-0abebf376de6-maas-api-tls\") pod \"maas-api-6776cdf886-wcqkx\" (UID: \"06e664ca-e53f-470b-8b2c-0abebf376de6\") " pod="opendatahub/maas-api-6776cdf886-wcqkx" May 27 20:20:27.027617 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:20:27.027596 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ls7sf\" (UniqueName: \"kubernetes.io/projected/06e664ca-e53f-470b-8b2c-0abebf376de6-kube-api-access-ls7sf\") pod \"maas-api-6776cdf886-wcqkx\" (UID: \"06e664ca-e53f-470b-8b2c-0abebf376de6\") " pod="opendatahub/maas-api-6776cdf886-wcqkx" May 27 20:20:27.176783 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:20:27.176688 2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-6776cdf886-wcqkx" May 27 20:20:27.315329 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:20:27.315305 2571 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-6776cdf886-wcqkx"] May 27 20:20:27.317473 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:20:27.317442 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod06e664ca_e53f_470b_8b2c_0abebf376de6.slice/crio-cc36bc30d2cd1e16bb45c88c854d4c7917c1fe56173118a42055128f12aae9ec WatchSource:0}: Error finding container cc36bc30d2cd1e16bb45c88c854d4c7917c1fe56173118a42055128f12aae9ec: Status 404 returned error can't find the container with id cc36bc30d2cd1e16bb45c88c854d4c7917c1fe56173118a42055128f12aae9ec May 27 20:20:27.331519 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:20:27.331482 2571 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-58b6c8fdc7-mfs2j"] May 27 20:20:27.336155 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:20:27.336134 2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-58b6c8fdc7-mfs2j" May 27 20:20:27.338785 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:20:27.338763 2571 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\"" May 27 20:20:27.338982 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:20:27.338963 2571 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-zpsws\"" May 27 20:20:27.342986 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:20:27.342965 2571 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-58b6c8fdc7-mfs2j"] May 27 20:20:27.421274 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:20:27.421238 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l5cls\" (UniqueName: \"kubernetes.io/projected/39a39b37-93cb-4b21-97a6-8f3d2350f1e4-kube-api-access-l5cls\") pod \"payload-processing-58b6c8fdc7-mfs2j\" (UID: \"39a39b37-93cb-4b21-97a6-8f3d2350f1e4\") " pod="openshift-ingress/payload-processing-58b6c8fdc7-mfs2j" May 27 20:20:27.521796 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:20:27.521762 2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-l5cls\" (UniqueName: \"kubernetes.io/projected/39a39b37-93cb-4b21-97a6-8f3d2350f1e4-kube-api-access-l5cls\") pod \"payload-processing-58b6c8fdc7-mfs2j\" (UID: \"39a39b37-93cb-4b21-97a6-8f3d2350f1e4\") " pod="openshift-ingress/payload-processing-58b6c8fdc7-mfs2j" May 27 20:20:27.530399 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:20:27.530374 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-l5cls\" (UniqueName: \"kubernetes.io/projected/39a39b37-93cb-4b21-97a6-8f3d2350f1e4-kube-api-access-l5cls\") pod \"payload-processing-58b6c8fdc7-mfs2j\" (UID: \"39a39b37-93cb-4b21-97a6-8f3d2350f1e4\") " pod="openshift-ingress/payload-processing-58b6c8fdc7-mfs2j" May 27 20:20:27.649887 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:20:27.649857 2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-58b6c8fdc7-mfs2j" May 27 20:20:27.804692 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:20:27.804666 2571 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-58b6c8fdc7-mfs2j"] May 27 20:20:27.806899 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:20:27.806877 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod39a39b37_93cb_4b21_97a6_8f3d2350f1e4.slice/crio-61ff6b52448469f19f7c3688aa186c273ff7f15b0cb8e778d17df6bf6a9e156d WatchSource:0}: Error finding container 61ff6b52448469f19f7c3688aa186c273ff7f15b0cb8e778d17df6bf6a9e156d: Status 404 returned error can't find the container with id 61ff6b52448469f19f7c3688aa186c273ff7f15b0cb8e778d17df6bf6a9e156d May 27 20:20:28.012656 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:20:28.012624 2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-6776cdf886-wcqkx" event={"ID":"06e664ca-e53f-470b-8b2c-0abebf376de6","Type":"ContainerStarted","Data":"cc36bc30d2cd1e16bb45c88c854d4c7917c1fe56173118a42055128f12aae9ec"} May 27 20:20:28.013690 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:20:28.013665 2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-58b6c8fdc7-mfs2j" event={"ID":"39a39b37-93cb-4b21-97a6-8f3d2350f1e4","Type":"ContainerStarted","Data":"61ff6b52448469f19f7c3688aa186c273ff7f15b0cb8e778d17df6bf6a9e156d"} May 27 20:20:30.022813 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:20:30.022774 2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-6776cdf886-wcqkx" event={"ID":"06e664ca-e53f-470b-8b2c-0abebf376de6","Type":"ContainerStarted","Data":"fa5b127b5bf2dbe5fc092613aa40e4f869340d861e237e7a1c8c7af63b44a222"} May 27 20:20:30.023260 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:20:30.022860 2571 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-6776cdf886-wcqkx" May 27 20:20:30.024136 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:20:30.024089 2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-58b6c8fdc7-mfs2j" event={"ID":"39a39b37-93cb-4b21-97a6-8f3d2350f1e4","Type":"ContainerStarted","Data":"2339165b3bdd875f38f914f219ccd6bd8a2a277d9acce94e97ad0fa7477015de"} May 27 20:20:30.024243 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:20:30.024225 2571 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-58b6c8fdc7-mfs2j" May 27 20:20:30.039613 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:20:30.039569 2571 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="opendatahub/maas-api-6776cdf886-wcqkx" podStartSLOduration=2.063372797 podStartE2EDuration="4.03955599s" podCreationTimestamp="2026-05-27 20:20:26 +0000 UTC" firstStartedPulling="2026-05-27 20:20:27.318890305 +0000 UTC m=+579.728516593" lastFinishedPulling="2026-05-27 20:20:29.295073494 +0000 UTC m=+581.704699786" observedRunningTime="2026-05-27 20:20:30.037459617 +0000 UTC m=+582.447085941" watchObservedRunningTime="2026-05-27 20:20:30.03955599 +0000 UTC m=+582.449182301" May 27 20:20:30.053025 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:20:30.052950 2571 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-ingress/payload-processing-58b6c8fdc7-mfs2j" podStartSLOduration=1.5643632109999999 podStartE2EDuration="3.052935636s" podCreationTimestamp="2026-05-27 20:20:27 +0000 UTC" firstStartedPulling="2026-05-27 20:20:27.808647531 +0000 UTC m=+580.218273823" lastFinishedPulling="2026-05-27 20:20:29.297219956 +0000 UTC m=+581.706846248" observedRunningTime="2026-05-27 20:20:30.051959514 +0000 UTC m=+582.461585825" watchObservedRunningTime="2026-05-27 20:20:30.052935636 +0000 UTC m=+582.462561950" May 27 20:20:36.033271 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:20:36.033241 2571 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-6776cdf886-wcqkx" May 27 20:20:41.029284 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:20:41.029246 2571 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-58b6c8fdc7-mfs2j" May 27 20:20:44.430950 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:20:44.430913 2571 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-stt85"] May 27 20:20:44.434483 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:20:44.434461 2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-stt85" May 27 20:20:44.437036 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:20:44.437017 2571 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" May 27 20:20:44.438187 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:20:44.438167 2571 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-wg78k\"" May 27 20:20:44.438279 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:20:44.438198 2571 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\"" May 27 20:20:44.438279 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:20:44.438245 2571 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" May 27 20:20:44.443435 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:20:44.443415 2571 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-stt85"] May 27 20:20:44.547851 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:20:44.547815 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/1fac8988-9553-4589-8f90-552bd024c343-kserve-provision-location\") pod \"facebook-opt-125m-simulated-kserve-8f8dc67b7-stt85\" (UID: \"1fac8988-9553-4589-8f90-552bd024c343\") " pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-stt85" May 27 20:20:44.547994 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:20:44.547865 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1fac8988-9553-4589-8f90-552bd024c343-home\") pod \"facebook-opt-125m-simulated-kserve-8f8dc67b7-stt85\" (UID: \"1fac8988-9553-4589-8f90-552bd024c343\") " pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-stt85" May 27 20:20:44.547994 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:20:44.547918 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1fac8988-9553-4589-8f90-552bd024c343-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-8f8dc67b7-stt85\" (UID: \"1fac8988-9553-4589-8f90-552bd024c343\") " pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-stt85" May 27 20:20:44.547994 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:20:44.547955 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4tccz\" (UniqueName: \"kubernetes.io/projected/1fac8988-9553-4589-8f90-552bd024c343-kube-api-access-4tccz\") pod \"facebook-opt-125m-simulated-kserve-8f8dc67b7-stt85\" (UID: \"1fac8988-9553-4589-8f90-552bd024c343\") " pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-stt85" May 27 20:20:44.547994 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:20:44.547988 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1fac8988-9553-4589-8f90-552bd024c343-dshm\") pod \"facebook-opt-125m-simulated-kserve-8f8dc67b7-stt85\" (UID: \"1fac8988-9553-4589-8f90-552bd024c343\") " pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-stt85" May 27 20:20:44.548149 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:20:44.548030 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1fac8988-9553-4589-8f90-552bd024c343-model-cache\") pod \"facebook-opt-125m-simulated-kserve-8f8dc67b7-stt85\" (UID: \"1fac8988-9553-4589-8f90-552bd024c343\") " pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-stt85" May 27 20:20:44.649094 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:20:44.649056 2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1fac8988-9553-4589-8f90-552bd024c343-model-cache\") pod \"facebook-opt-125m-simulated-kserve-8f8dc67b7-stt85\" (UID: \"1fac8988-9553-4589-8f90-552bd024c343\") " pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-stt85" May 27 20:20:44.649276 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:20:44.649128 2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/1fac8988-9553-4589-8f90-552bd024c343-kserve-provision-location\") pod \"facebook-opt-125m-simulated-kserve-8f8dc67b7-stt85\" (UID: \"1fac8988-9553-4589-8f90-552bd024c343\") " pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-stt85" May 27 20:20:44.649276 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:20:44.649172 2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1fac8988-9553-4589-8f90-552bd024c343-home\") pod \"facebook-opt-125m-simulated-kserve-8f8dc67b7-stt85\" (UID: \"1fac8988-9553-4589-8f90-552bd024c343\") " pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-stt85" May 27 20:20:44.649276 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:20:44.649200 2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1fac8988-9553-4589-8f90-552bd024c343-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-8f8dc67b7-stt85\" (UID: \"1fac8988-9553-4589-8f90-552bd024c343\") " pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-stt85" May 27 20:20:44.649276 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:20:44.649248 2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-4tccz\" (UniqueName: \"kubernetes.io/projected/1fac8988-9553-4589-8f90-552bd024c343-kube-api-access-4tccz\") pod \"facebook-opt-125m-simulated-kserve-8f8dc67b7-stt85\" (UID: \"1fac8988-9553-4589-8f90-552bd024c343\") " pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-stt85" May 27 20:20:44.649483 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:20:44.649294 2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1fac8988-9553-4589-8f90-552bd024c343-dshm\") pod \"facebook-opt-125m-simulated-kserve-8f8dc67b7-stt85\" (UID: \"1fac8988-9553-4589-8f90-552bd024c343\") " pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-stt85" May 27 20:20:44.649483 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:20:44.649433 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1fac8988-9553-4589-8f90-552bd024c343-model-cache\") pod \"facebook-opt-125m-simulated-kserve-8f8dc67b7-stt85\" (UID: \"1fac8988-9553-4589-8f90-552bd024c343\") " pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-stt85" May 27 20:20:44.649584 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:20:44.649513 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1fac8988-9553-4589-8f90-552bd024c343-home\") pod \"facebook-opt-125m-simulated-kserve-8f8dc67b7-stt85\" (UID: \"1fac8988-9553-4589-8f90-552bd024c343\") " pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-stt85" May 27 20:20:44.649584 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:20:44.649532 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/1fac8988-9553-4589-8f90-552bd024c343-kserve-provision-location\") pod \"facebook-opt-125m-simulated-kserve-8f8dc67b7-stt85\" (UID: \"1fac8988-9553-4589-8f90-552bd024c343\") " pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-stt85" May 27 20:20:44.651669 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:20:44.651645 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1fac8988-9553-4589-8f90-552bd024c343-dshm\") pod \"facebook-opt-125m-simulated-kserve-8f8dc67b7-stt85\" (UID: \"1fac8988-9553-4589-8f90-552bd024c343\") " pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-stt85" May 27 20:20:44.651784 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:20:44.651769 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1fac8988-9553-4589-8f90-552bd024c343-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-8f8dc67b7-stt85\" (UID: \"1fac8988-9553-4589-8f90-552bd024c343\") " pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-stt85" May 27 20:20:44.657246 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:20:44.657227 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4tccz\" (UniqueName: \"kubernetes.io/projected/1fac8988-9553-4589-8f90-552bd024c343-kube-api-access-4tccz\") pod \"facebook-opt-125m-simulated-kserve-8f8dc67b7-stt85\" (UID: \"1fac8988-9553-4589-8f90-552bd024c343\") " pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-stt85" May 27 20:20:44.745081 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:20:44.745021 2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-stt85" May 27 20:20:44.880052 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:20:44.880027 2571 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-stt85"] May 27 20:20:44.888863 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:20:44.888813 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1fac8988_9553_4589_8f90_552bd024c343.slice/crio-8beb3831d7b6f929945003e08c62faddb6fc30309cd6342bdfd9e0e1d4d0464c WatchSource:0}: Error finding container 8beb3831d7b6f929945003e08c62faddb6fc30309cd6342bdfd9e0e1d4d0464c: Status 404 returned error can't find the container with id 8beb3831d7b6f929945003e08c62faddb6fc30309cd6342bdfd9e0e1d4d0464c May 27 20:20:45.072013 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:20:45.071980 2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-stt85" event={"ID":"1fac8988-9553-4589-8f90-552bd024c343","Type":"ContainerStarted","Data":"8beb3831d7b6f929945003e08c62faddb6fc30309cd6342bdfd9e0e1d4d0464c"} May 27 20:20:48.132384 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:20:48.132345 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rcvbq_92933172-b100-42e6-8018-dcb430647ae2/ovn-acl-logging/0.log" May 27 20:20:48.133541 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:20:48.133512 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rcvbq_92933172-b100-42e6-8018-dcb430647ae2/ovn-acl-logging/0.log" May 27 20:20:52.103054 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:20:52.102962 2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-stt85" event={"ID":"1fac8988-9553-4589-8f90-552bd024c343","Type":"ContainerStarted","Data":"b2bb710613aa526cc098ef9ab2fe0c0b2b92c9b64215f0d55a75214e0c264862"} May 27 20:20:58.125736 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:20:58.125692 2571 generic.go:358] "Generic (PLEG): container finished" podID="1fac8988-9553-4589-8f90-552bd024c343" containerID="b2bb710613aa526cc098ef9ab2fe0c0b2b92c9b64215f0d55a75214e0c264862" exitCode=0 May 27 20:20:58.126279 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:20:58.125757 2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-stt85" event={"ID":"1fac8988-9553-4589-8f90-552bd024c343","Type":"ContainerDied","Data":"b2bb710613aa526cc098ef9ab2fe0c0b2b92c9b64215f0d55a75214e0c264862"} May 27 20:21:00.134278 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:21:00.134242 2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-stt85" event={"ID":"1fac8988-9553-4589-8f90-552bd024c343","Type":"ContainerStarted","Data":"aab202c53f723b0fc0d086f4f060a0b70e91d7eb40478e17dc782f898bae6233"} May 27 20:21:00.136547 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:21:00.134453 2571 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-stt85" May 27 20:21:00.153891 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:21:00.153845 2571 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-stt85" podStartSLOduration=1.5696365860000001 podStartE2EDuration="16.153833619s" podCreationTimestamp="2026-05-27 20:20:44 +0000 UTC" firstStartedPulling="2026-05-27 20:20:44.891452475 +0000 UTC m=+597.301078764" lastFinishedPulling="2026-05-27 20:20:59.475649508 +0000 UTC m=+611.885275797" observedRunningTime="2026-05-27 20:21:00.151375341 +0000 UTC m=+612.561001651" watchObservedRunningTime="2026-05-27 20:21:00.153833619 +0000 UTC m=+612.563459929" May 27 20:21:03.829326 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:21:03.829238 2571 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-84db68679b-l978n"] May 27 20:21:03.832874 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:21:03.832848 2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-84db68679b-l978n" May 27 20:21:03.835782 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:21:03.835760 2571 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\"" May 27 20:21:03.843654 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:21:03.843617 2571 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-84db68679b-l978n"] May 27 20:21:03.915156 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:21:03.915034 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/89e74934-672f-486b-b0ca-d0ea888cbadb-home\") pod \"e2e-trlp-test-simulated-kserve-84db68679b-l978n\" (UID: \"89e74934-672f-486b-b0ca-d0ea888cbadb\") " pod="llm/e2e-trlp-test-simulated-kserve-84db68679b-l978n" May 27 20:21:03.915156 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:21:03.915094 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/89e74934-672f-486b-b0ca-d0ea888cbadb-dshm\") pod \"e2e-trlp-test-simulated-kserve-84db68679b-l978n\" (UID: \"89e74934-672f-486b-b0ca-d0ea888cbadb\") " pod="llm/e2e-trlp-test-simulated-kserve-84db68679b-l978n" May 27 20:21:03.915390 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:21:03.915217 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/89e74934-672f-486b-b0ca-d0ea888cbadb-kserve-provision-location\") pod \"e2e-trlp-test-simulated-kserve-84db68679b-l978n\" (UID: \"89e74934-672f-486b-b0ca-d0ea888cbadb\") " pod="llm/e2e-trlp-test-simulated-kserve-84db68679b-l978n" May 27 20:21:03.915390 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:21:03.915254 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l6bgs\" (UniqueName: \"kubernetes.io/projected/89e74934-672f-486b-b0ca-d0ea888cbadb-kube-api-access-l6bgs\") pod \"e2e-trlp-test-simulated-kserve-84db68679b-l978n\" (UID: \"89e74934-672f-486b-b0ca-d0ea888cbadb\") " pod="llm/e2e-trlp-test-simulated-kserve-84db68679b-l978n" May 27 20:21:03.915500 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:21:03.915419 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/89e74934-672f-486b-b0ca-d0ea888cbadb-model-cache\") pod \"e2e-trlp-test-simulated-kserve-84db68679b-l978n\" (UID: \"89e74934-672f-486b-b0ca-d0ea888cbadb\") " pod="llm/e2e-trlp-test-simulated-kserve-84db68679b-l978n" May 27 20:21:03.915500 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:21:03.915455 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/89e74934-672f-486b-b0ca-d0ea888cbadb-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-84db68679b-l978n\" (UID: \"89e74934-672f-486b-b0ca-d0ea888cbadb\") " pod="llm/e2e-trlp-test-simulated-kserve-84db68679b-l978n" May 27 20:21:04.016082 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:21:04.016046 2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/89e74934-672f-486b-b0ca-d0ea888cbadb-model-cache\") pod \"e2e-trlp-test-simulated-kserve-84db68679b-l978n\" (UID: \"89e74934-672f-486b-b0ca-d0ea888cbadb\") " pod="llm/e2e-trlp-test-simulated-kserve-84db68679b-l978n" May 27 20:21:04.016082 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:21:04.016084 2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/89e74934-672f-486b-b0ca-d0ea888cbadb-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-84db68679b-l978n\" (UID: \"89e74934-672f-486b-b0ca-d0ea888cbadb\") " pod="llm/e2e-trlp-test-simulated-kserve-84db68679b-l978n" May 27 20:21:04.016335 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:21:04.016149 2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/89e74934-672f-486b-b0ca-d0ea888cbadb-home\") pod \"e2e-trlp-test-simulated-kserve-84db68679b-l978n\" (UID: \"89e74934-672f-486b-b0ca-d0ea888cbadb\") " pod="llm/e2e-trlp-test-simulated-kserve-84db68679b-l978n" May 27 20:21:04.016335 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:21:04.016168 2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/89e74934-672f-486b-b0ca-d0ea888cbadb-dshm\") pod \"e2e-trlp-test-simulated-kserve-84db68679b-l978n\" (UID: \"89e74934-672f-486b-b0ca-d0ea888cbadb\") " pod="llm/e2e-trlp-test-simulated-kserve-84db68679b-l978n" May 27 20:21:04.016335 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:21:04.016187 2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/89e74934-672f-486b-b0ca-d0ea888cbadb-kserve-provision-location\") pod \"e2e-trlp-test-simulated-kserve-84db68679b-l978n\" (UID: \"89e74934-672f-486b-b0ca-d0ea888cbadb\") " pod="llm/e2e-trlp-test-simulated-kserve-84db68679b-l978n" May 27 20:21:04.016335 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:21:04.016207 2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-l6bgs\" (UniqueName: \"kubernetes.io/projected/89e74934-672f-486b-b0ca-d0ea888cbadb-kube-api-access-l6bgs\") pod \"e2e-trlp-test-simulated-kserve-84db68679b-l978n\" (UID: \"89e74934-672f-486b-b0ca-d0ea888cbadb\") " pod="llm/e2e-trlp-test-simulated-kserve-84db68679b-l978n" May 27 20:21:04.016561 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:21:04.016536 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/89e74934-672f-486b-b0ca-d0ea888cbadb-model-cache\") pod \"e2e-trlp-test-simulated-kserve-84db68679b-l978n\" (UID: \"89e74934-672f-486b-b0ca-d0ea888cbadb\") " pod="llm/e2e-trlp-test-simulated-kserve-84db68679b-l978n" May 27 20:21:04.016623 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:21:04.016570 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/89e74934-672f-486b-b0ca-d0ea888cbadb-home\") pod \"e2e-trlp-test-simulated-kserve-84db68679b-l978n\" (UID: \"89e74934-672f-486b-b0ca-d0ea888cbadb\") " pod="llm/e2e-trlp-test-simulated-kserve-84db68679b-l978n" May 27 20:21:04.016743 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:21:04.016721 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/89e74934-672f-486b-b0ca-d0ea888cbadb-kserve-provision-location\") pod \"e2e-trlp-test-simulated-kserve-84db68679b-l978n\" (UID: \"89e74934-672f-486b-b0ca-d0ea888cbadb\") " pod="llm/e2e-trlp-test-simulated-kserve-84db68679b-l978n" May 27 20:21:04.018851 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:21:04.018818 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/89e74934-672f-486b-b0ca-d0ea888cbadb-dshm\") pod \"e2e-trlp-test-simulated-kserve-84db68679b-l978n\" (UID: \"89e74934-672f-486b-b0ca-d0ea888cbadb\") " pod="llm/e2e-trlp-test-simulated-kserve-84db68679b-l978n" May 27 20:21:04.019160 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:21:04.019142 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/89e74934-672f-486b-b0ca-d0ea888cbadb-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-84db68679b-l978n\" (UID: \"89e74934-672f-486b-b0ca-d0ea888cbadb\") " pod="llm/e2e-trlp-test-simulated-kserve-84db68679b-l978n" May 27 20:21:04.024513 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:21:04.024478 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-l6bgs\" (UniqueName: \"kubernetes.io/projected/89e74934-672f-486b-b0ca-d0ea888cbadb-kube-api-access-l6bgs\") pod \"e2e-trlp-test-simulated-kserve-84db68679b-l978n\" (UID: \"89e74934-672f-486b-b0ca-d0ea888cbadb\") " pod="llm/e2e-trlp-test-simulated-kserve-84db68679b-l978n" May 27 20:21:04.145412 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:21:04.145319 2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-84db68679b-l978n" May 27 20:21:04.280269 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:21:04.280245 2571 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-84db68679b-l978n"] May 27 20:21:04.293570 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:21:04.293549 2571 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider May 27 20:21:05.153548 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:21:05.153504 2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-84db68679b-l978n" event={"ID":"89e74934-672f-486b-b0ca-d0ea888cbadb","Type":"ContainerStarted","Data":"24a1f170b6e3cc5c3c7c82d00bc47e7f0b7930956df61c39770f4f79b9a8a729"} May 27 20:21:05.153548 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:21:05.153546 2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-84db68679b-l978n" event={"ID":"89e74934-672f-486b-b0ca-d0ea888cbadb","Type":"ContainerStarted","Data":"080d845699b855e4ab15609b72b81f69fb979360f30e677c2086c70fb4c7364c"} May 27 20:21:10.171996 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:21:10.171961 2571 generic.go:358] "Generic (PLEG): container finished" podID="89e74934-672f-486b-b0ca-d0ea888cbadb" containerID="24a1f170b6e3cc5c3c7c82d00bc47e7f0b7930956df61c39770f4f79b9a8a729" exitCode=0 May 27 20:21:10.172385 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:21:10.172035 2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-84db68679b-l978n" event={"ID":"89e74934-672f-486b-b0ca-d0ea888cbadb","Type":"ContainerDied","Data":"24a1f170b6e3cc5c3c7c82d00bc47e7f0b7930956df61c39770f4f79b9a8a729"} May 27 20:21:11.152498 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:21:11.152467 2571 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-stt85" May 27 20:21:11.177245 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:21:11.177209 2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-84db68679b-l978n" event={"ID":"89e74934-672f-486b-b0ca-d0ea888cbadb","Type":"ContainerStarted","Data":"d10a55837077f7f67ea855e92db25d9339493a308908aa322fa974056b49ae2a"} May 27 20:21:11.177631 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:21:11.177406 2571 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-84db68679b-l978n" May 27 20:21:11.195952 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:21:11.195906 2571 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-84db68679b-l978n" podStartSLOduration=7.924276315 podStartE2EDuration="8.195893618s" podCreationTimestamp="2026-05-27 20:21:03 +0000 UTC" firstStartedPulling="2026-05-27 20:21:10.172628027 +0000 UTC m=+622.582254316" lastFinishedPulling="2026-05-27 20:21:10.444245327 +0000 UTC m=+622.853871619" observedRunningTime="2026-05-27 20:21:11.194868425 +0000 UTC m=+623.604494735" watchObservedRunningTime="2026-05-27 20:21:11.195893618 +0000 UTC m=+623.605519929" May 27 20:21:22.197874 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:21:22.197840 2571 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-84db68679b-l978n" May 27 20:23:32.208273 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:23:32.208237 2571 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-67678bdf-wnxjw"] May 27 20:23:32.209018 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:23:32.208471 2571 kuberuntime_container.go:864] "Killing container with a grace period" pod="opendatahub/maas-controller-67678bdf-wnxjw" podUID="97a920f6-dbe1-40bf-a722-c1936c3ce760" containerName="manager" containerID="cri-o://7679fe1a1bfa07eaae9328710eec89e014a75ccb53403ecc365aa29e5d2b8463" gracePeriod=30 May 27 20:23:32.451613 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:23:32.451590 2571 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-67678bdf-wnxjw" May 27 20:23:32.536033 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:23:32.536003 2571 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/97a920f6-dbe1-40bf-a722-c1936c3ce760-webhook-cert\") pod \"97a920f6-dbe1-40bf-a722-c1936c3ce760\" (UID: \"97a920f6-dbe1-40bf-a722-c1936c3ce760\") " May 27 20:23:32.536207 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:23:32.536065 2571 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hknmd\" (UniqueName: \"kubernetes.io/projected/97a920f6-dbe1-40bf-a722-c1936c3ce760-kube-api-access-hknmd\") pod \"97a920f6-dbe1-40bf-a722-c1936c3ce760\" (UID: \"97a920f6-dbe1-40bf-a722-c1936c3ce760\") " May 27 20:23:32.538125 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:23:32.538070 2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/97a920f6-dbe1-40bf-a722-c1936c3ce760-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "97a920f6-dbe1-40bf-a722-c1936c3ce760" (UID: "97a920f6-dbe1-40bf-a722-c1936c3ce760"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" May 27 20:23:32.538234 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:23:32.538142 2571 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/97a920f6-dbe1-40bf-a722-c1936c3ce760-kube-api-access-hknmd" (OuterVolumeSpecName: "kube-api-access-hknmd") pod "97a920f6-dbe1-40bf-a722-c1936c3ce760" (UID: "97a920f6-dbe1-40bf-a722-c1936c3ce760"). InnerVolumeSpecName "kube-api-access-hknmd". PluginName "kubernetes.io/projected", VolumeGIDValue "" May 27 20:23:32.636566 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:23:32.636526 2571 generic.go:358] "Generic (PLEG): container finished" podID="97a920f6-dbe1-40bf-a722-c1936c3ce760" containerID="7679fe1a1bfa07eaae9328710eec89e014a75ccb53403ecc365aa29e5d2b8463" exitCode=0 May 27 20:23:32.636726 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:23:32.636594 2571 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-67678bdf-wnxjw" May 27 20:23:32.636726 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:23:32.636622 2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hknmd\" (UniqueName: \"kubernetes.io/projected/97a920f6-dbe1-40bf-a722-c1936c3ce760-kube-api-access-hknmd\") on node \"ip-10-0-142-251.ec2.internal\" DevicePath \"\"" May 27 20:23:32.636726 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:23:32.636624 2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-67678bdf-wnxjw" event={"ID":"97a920f6-dbe1-40bf-a722-c1936c3ce760","Type":"ContainerDied","Data":"7679fe1a1bfa07eaae9328710eec89e014a75ccb53403ecc365aa29e5d2b8463"} May 27 20:23:32.636726 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:23:32.636641 2571 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/97a920f6-dbe1-40bf-a722-c1936c3ce760-webhook-cert\") on node \"ip-10-0-142-251.ec2.internal\" DevicePath \"\"" May 27 20:23:32.636726 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:23:32.636663 2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-67678bdf-wnxjw" event={"ID":"97a920f6-dbe1-40bf-a722-c1936c3ce760","Type":"ContainerDied","Data":"ce081e3d8b32bdbc588e140a1dd952163e37b4d8f9e76bb5b565abcaece087d6"} May 27 20:23:32.636726 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:23:32.636681 2571 scope.go:117] "RemoveContainer" containerID="7679fe1a1bfa07eaae9328710eec89e014a75ccb53403ecc365aa29e5d2b8463" May 27 20:23:32.645257 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:23:32.645234 2571 scope.go:117] "RemoveContainer" containerID="7679fe1a1bfa07eaae9328710eec89e014a75ccb53403ecc365aa29e5d2b8463" May 27 20:23:32.645486 ip-10-0-142-251 kubenswrapper[2571]: E0527 20:23:32.645469 2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7679fe1a1bfa07eaae9328710eec89e014a75ccb53403ecc365aa29e5d2b8463\": container with ID starting with 7679fe1a1bfa07eaae9328710eec89e014a75ccb53403ecc365aa29e5d2b8463 not found: ID does not exist" containerID="7679fe1a1bfa07eaae9328710eec89e014a75ccb53403ecc365aa29e5d2b8463" May 27 20:23:32.645546 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:23:32.645493 2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7679fe1a1bfa07eaae9328710eec89e014a75ccb53403ecc365aa29e5d2b8463"} err="failed to get container status \"7679fe1a1bfa07eaae9328710eec89e014a75ccb53403ecc365aa29e5d2b8463\": rpc error: code = NotFound desc = could not find container \"7679fe1a1bfa07eaae9328710eec89e014a75ccb53403ecc365aa29e5d2b8463\": container with ID starting with 7679fe1a1bfa07eaae9328710eec89e014a75ccb53403ecc365aa29e5d2b8463 not found: ID does not exist" May 27 20:23:32.657380 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:23:32.657355 2571 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-67678bdf-wnxjw"] May 27 20:23:32.660684 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:23:32.660662 2571 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-67678bdf-wnxjw"] May 27 20:23:33.137159 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:23:33.137121 2571 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-67678bdf-kpmt6"] May 27 20:23:33.137448 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:23:33.137436 2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="97a920f6-dbe1-40bf-a722-c1936c3ce760" containerName="manager" May 27 20:23:33.137448 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:23:33.137449 2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="97a920f6-dbe1-40bf-a722-c1936c3ce760" containerName="manager" May 27 20:23:33.137534 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:23:33.137521 2571 memory_manager.go:356] "RemoveStaleState removing state" podUID="97a920f6-dbe1-40bf-a722-c1936c3ce760" containerName="manager" May 27 20:23:33.141737 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:23:33.141705 2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-67678bdf-kpmt6" May 27 20:23:33.146337 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:23:33.146315 2571 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" May 27 20:23:33.146435 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:23:33.146348 2571 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" May 27 20:23:33.146435 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:23:33.146315 2571 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-jrjjn\"" May 27 20:23:33.150071 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:23:33.150046 2571 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-67678bdf-kpmt6"] May 27 20:23:33.241333 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:23:33.241306 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/485cd20e-b78a-4d6f-afe7-ca5ae64387f8-webhook-cert\") pod \"maas-controller-67678bdf-kpmt6\" (UID: \"485cd20e-b78a-4d6f-afe7-ca5ae64387f8\") " pod="opendatahub/maas-controller-67678bdf-kpmt6" May 27 20:23:33.241654 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:23:33.241364 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p7vqk\" (UniqueName: \"kubernetes.io/projected/485cd20e-b78a-4d6f-afe7-ca5ae64387f8-kube-api-access-p7vqk\") pod \"maas-controller-67678bdf-kpmt6\" (UID: \"485cd20e-b78a-4d6f-afe7-ca5ae64387f8\") " pod="opendatahub/maas-controller-67678bdf-kpmt6" May 27 20:23:33.342142 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:23:33.342090 2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-p7vqk\" (UniqueName: \"kubernetes.io/projected/485cd20e-b78a-4d6f-afe7-ca5ae64387f8-kube-api-access-p7vqk\") pod \"maas-controller-67678bdf-kpmt6\" (UID: \"485cd20e-b78a-4d6f-afe7-ca5ae64387f8\") " pod="opendatahub/maas-controller-67678bdf-kpmt6" May 27 20:23:33.342263 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:23:33.342180 2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/485cd20e-b78a-4d6f-afe7-ca5ae64387f8-webhook-cert\") pod \"maas-controller-67678bdf-kpmt6\" (UID: \"485cd20e-b78a-4d6f-afe7-ca5ae64387f8\") " pod="opendatahub/maas-controller-67678bdf-kpmt6" May 27 20:23:33.344650 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:23:33.344630 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/485cd20e-b78a-4d6f-afe7-ca5ae64387f8-webhook-cert\") pod \"maas-controller-67678bdf-kpmt6\" (UID: \"485cd20e-b78a-4d6f-afe7-ca5ae64387f8\") " pod="opendatahub/maas-controller-67678bdf-kpmt6" May 27 20:23:33.352460 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:23:33.352427 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p7vqk\" (UniqueName: \"kubernetes.io/projected/485cd20e-b78a-4d6f-afe7-ca5ae64387f8-kube-api-access-p7vqk\") pod \"maas-controller-67678bdf-kpmt6\" (UID: \"485cd20e-b78a-4d6f-afe7-ca5ae64387f8\") " pod="opendatahub/maas-controller-67678bdf-kpmt6" May 27 20:23:33.452612 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:23:33.452521 2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-67678bdf-kpmt6" May 27 20:23:33.583614 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:23:33.583547 2571 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-67678bdf-kpmt6"] May 27 20:23:33.586403 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:23:33.586372 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod485cd20e_b78a_4d6f_afe7_ca5ae64387f8.slice/crio-99fdf5aa1a36b982169baf2dc00a15536a1a5c2c2dd9e14cac8c0121402082f1 WatchSource:0}: Error finding container 99fdf5aa1a36b982169baf2dc00a15536a1a5c2c2dd9e14cac8c0121402082f1: Status 404 returned error can't find the container with id 99fdf5aa1a36b982169baf2dc00a15536a1a5c2c2dd9e14cac8c0121402082f1 May 27 20:23:33.641755 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:23:33.641722 2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-67678bdf-kpmt6" event={"ID":"485cd20e-b78a-4d6f-afe7-ca5ae64387f8","Type":"ContainerStarted","Data":"99fdf5aa1a36b982169baf2dc00a15536a1a5c2c2dd9e14cac8c0121402082f1"} May 27 20:23:34.229687 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:23:34.229654 2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="97a920f6-dbe1-40bf-a722-c1936c3ce760" path="/var/lib/kubelet/pods/97a920f6-dbe1-40bf-a722-c1936c3ce760/volumes" May 27 20:23:34.647087 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:23:34.647048 2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-67678bdf-kpmt6" event={"ID":"485cd20e-b78a-4d6f-afe7-ca5ae64387f8","Type":"ContainerStarted","Data":"85eac1d1a726fb49ee6d08cb23b9e9ceca59c5719c7e68a362e38696dbbbeb96"} May 27 20:23:34.647479 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:23:34.647129 2571 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-67678bdf-kpmt6" May 27 20:23:34.664899 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:23:34.664847 2571 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="opendatahub/maas-controller-67678bdf-kpmt6" podStartSLOduration=1.250147958 podStartE2EDuration="1.664834329s" podCreationTimestamp="2026-05-27 20:23:33 +0000 UTC" firstStartedPulling="2026-05-27 20:23:33.5876999 +0000 UTC m=+765.997326202" lastFinishedPulling="2026-05-27 20:23:34.002386281 +0000 UTC m=+766.412012573" observedRunningTime="2026-05-27 20:23:34.663571066 +0000 UTC m=+767.073197374" watchObservedRunningTime="2026-05-27 20:23:34.664834329 +0000 UTC m=+767.074460645" May 27 20:23:45.656286 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:23:45.656250 2571 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-67678bdf-kpmt6" May 27 20:25:48.157858 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:25:48.157830 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rcvbq_92933172-b100-42e6-8018-dcb430647ae2/ovn-acl-logging/0.log" May 27 20:25:48.158327 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:25:48.157909 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rcvbq_92933172-b100-42e6-8018-dcb430647ae2/ovn-acl-logging/0.log" May 27 20:30:48.184322 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:30:48.184288 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rcvbq_92933172-b100-42e6-8018-dcb430647ae2/ovn-acl-logging/0.log" May 27 20:30:48.184841 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:30:48.184408 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rcvbq_92933172-b100-42e6-8018-dcb430647ae2/ovn-acl-logging/0.log" May 27 20:35:48.206186 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:35:48.206150 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rcvbq_92933172-b100-42e6-8018-dcb430647ae2/ovn-acl-logging/0.log" May 27 20:35:48.207507 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:35:48.207485 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rcvbq_92933172-b100-42e6-8018-dcb430647ae2/ovn-acl-logging/0.log" May 27 20:40:48.234785 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:40:48.234756 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rcvbq_92933172-b100-42e6-8018-dcb430647ae2/ovn-acl-logging/0.log" May 27 20:40:48.236648 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:40:48.236621 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rcvbq_92933172-b100-42e6-8018-dcb430647ae2/ovn-acl-logging/0.log" May 27 20:44:18.860164 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:44:18.860136 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-856948b99f-kxfwr_ad931d17-fee2-4654-af65-c270e04d9fba/manager/0.log" May 27 20:44:18.987683 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:44:18.987657 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-6776cdf886-wcqkx_06e664ca-e53f-470b-8b2c-0abebf376de6/maas-api/0.log" May 27 20:44:19.258790 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:44:19.258715 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-67678bdf-kpmt6_485cd20e-b78a-4d6f-afe7-ca5ae64387f8/manager/0.log" May 27 20:44:19.378182 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:44:19.378149 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-858dbf95b8-sbtjh_123bc11b-e985-42a1-b0be-3023ea71b7aa/manager/1.log" May 27 20:44:19.778038 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:44:19.778005 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-b59bfd74-q82qr_4976f387-5d72-4a0d-befe-78cd65e79e58/manager/0.log" May 27 20:44:19.890489 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:44:19.890457 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_postgres-68c4fbbd6f-s7hf9_b1d3f9e4-5e89-470f-be70-9f4d65f1bd3e/postgres/0.log" May 27 20:44:21.374970 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:44:21.374943 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-657f44b778-hs4xt_cf66d312-e590-40f7-946e-2773bbfaf4d1/manager/0.log" May 27 20:44:22.663459 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:44:22.663429 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-798958bb55-f9dxg_9ecca3e6-bc39-4fa9-b197-c9af65680d01/discovery/0.log" May 27 20:44:22.775811 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:44:22.775782 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-85db9dc568-8vxjw_4f200e9c-bdab-48c5-a609-654bb82a823d/kube-auth-proxy/0.log" May 27 20:44:23.122054 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:44:23.122024 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58b6c8fdc7-mfs2j_39a39b37-93cb-4b21-97a6-8f3d2350f1e4/payload-processing/0.log" May 27 20:44:23.854377 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:44:23.854348 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-84db68679b-l978n_89e74934-672f-486b-b0ca-d0ea888cbadb/storage-initializer/0.log" May 27 20:44:23.861173 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:44:23.861149 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-84db68679b-l978n_89e74934-672f-486b-b0ca-d0ea888cbadb/main/0.log" May 27 20:44:24.125749 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:44:24.125665 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-8f8dc67b7-stt85_1fac8988-9553-4589-8f90-552bd024c343/storage-initializer/0.log" May 27 20:44:24.132867 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:44:24.132843 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-8f8dc67b7-stt85_1fac8988-9553-4589-8f90-552bd024c343/main/0.log" May 27 20:44:31.377811 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:44:31.377767 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-9lgmk_87d3dcf4-4db8-4d23-a9be-dac49b48ab6a/global-pull-secret-syncer/0.log" May 27 20:44:31.499436 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:44:31.499407 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-j46mn_27613ab0-7fd5-4d50-a40c-58aa4b58ff6a/konnectivity-agent/0.log" May 27 20:44:31.604733 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:44:31.604698 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-142-251.ec2.internal_e8c054cb9a59ce3874e2199aacfd5039/haproxy/0.log" May 27 20:44:36.038261 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:44:36.038234 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-657f44b778-hs4xt_cf66d312-e590-40f7-946e-2773bbfaf4d1/manager/0.log" May 27 20:44:38.132399 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:44:38.132369 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-r9grk_bb42b85b-4372-4700-ba7f-d5e91b80e3f1/node-exporter/0.log" May 27 20:44:38.154732 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:44:38.154709 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-r9grk_bb42b85b-4372-4700-ba7f-d5e91b80e3f1/kube-rbac-proxy/0.log" May 27 20:44:38.176738 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:44:38.176720 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-r9grk_bb42b85b-4372-4700-ba7f-d5e91b80e3f1/init-textfile/0.log" May 27 20:44:38.287943 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:44:38.287925 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-9c959559f-wgtzh_5710019c-8d7a-446e-b750-85706ec7bf14/kube-rbac-proxy-main/0.log" May 27 20:44:38.308583 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:44:38.308560 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-9c959559f-wgtzh_5710019c-8d7a-446e-b750-85706ec7bf14/kube-rbac-proxy-self/0.log" May 27 20:44:38.332782 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:44:38.332761 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-9c959559f-wgtzh_5710019c-8d7a-446e-b750-85706ec7bf14/openshift-state-metrics/0.log" May 27 20:44:39.911879 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:44:39.911842 2571 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-must-gather-lv5mm/perf-node-gather-daemonset-sj5rv"] May 27 20:44:39.915436 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:44:39.915416 2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-lv5mm/perf-node-gather-daemonset-sj5rv" May 27 20:44:39.918096 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:44:39.918072 2571 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-lv5mm\"/\"default-dockercfg-pl5xv\"" May 27 20:44:39.918229 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:44:39.918124 2571 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-lv5mm\"/\"kube-root-ca.crt\"" May 27 20:44:39.919326 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:44:39.919305 2571 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-lv5mm\"/\"openshift-service-ca.crt\"" May 27 20:44:39.925545 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:44:39.925524 2571 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-lv5mm/perf-node-gather-daemonset-sj5rv"] May 27 20:44:40.091484 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:44:40.091454 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/be86ef4a-b78b-43ea-b627-563115b10cd5-sys\") pod \"perf-node-gather-daemonset-sj5rv\" (UID: \"be86ef4a-b78b-43ea-b627-563115b10cd5\") " pod="openshift-must-gather-lv5mm/perf-node-gather-daemonset-sj5rv" May 27 20:44:40.091630 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:44:40.091499 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/be86ef4a-b78b-43ea-b627-563115b10cd5-proc\") pod \"perf-node-gather-daemonset-sj5rv\" (UID: \"be86ef4a-b78b-43ea-b627-563115b10cd5\") " pod="openshift-must-gather-lv5mm/perf-node-gather-daemonset-sj5rv" May 27 20:44:40.091630 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:44:40.091520 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9l2sq\" (UniqueName: \"kubernetes.io/projected/be86ef4a-b78b-43ea-b627-563115b10cd5-kube-api-access-9l2sq\") pod \"perf-node-gather-daemonset-sj5rv\" (UID: \"be86ef4a-b78b-43ea-b627-563115b10cd5\") " pod="openshift-must-gather-lv5mm/perf-node-gather-daemonset-sj5rv" May 27 20:44:40.091630 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:44:40.091570 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/be86ef4a-b78b-43ea-b627-563115b10cd5-podres\") pod \"perf-node-gather-daemonset-sj5rv\" (UID: \"be86ef4a-b78b-43ea-b627-563115b10cd5\") " pod="openshift-must-gather-lv5mm/perf-node-gather-daemonset-sj5rv" May 27 20:44:40.091630 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:44:40.091590 2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/be86ef4a-b78b-43ea-b627-563115b10cd5-lib-modules\") pod \"perf-node-gather-daemonset-sj5rv\" (UID: \"be86ef4a-b78b-43ea-b627-563115b10cd5\") " pod="openshift-must-gather-lv5mm/perf-node-gather-daemonset-sj5rv" May 27 20:44:40.192880 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:44:40.192815 2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/be86ef4a-b78b-43ea-b627-563115b10cd5-podres\") pod \"perf-node-gather-daemonset-sj5rv\" (UID: \"be86ef4a-b78b-43ea-b627-563115b10cd5\") " pod="openshift-must-gather-lv5mm/perf-node-gather-daemonset-sj5rv" May 27 20:44:40.192880 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:44:40.192856 2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/be86ef4a-b78b-43ea-b627-563115b10cd5-lib-modules\") pod \"perf-node-gather-daemonset-sj5rv\" (UID: \"be86ef4a-b78b-43ea-b627-563115b10cd5\") " pod="openshift-must-gather-lv5mm/perf-node-gather-daemonset-sj5rv" May 27 20:44:40.193067 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:44:40.192891 2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/be86ef4a-b78b-43ea-b627-563115b10cd5-sys\") pod \"perf-node-gather-daemonset-sj5rv\" (UID: \"be86ef4a-b78b-43ea-b627-563115b10cd5\") " pod="openshift-must-gather-lv5mm/perf-node-gather-daemonset-sj5rv" May 27 20:44:40.193067 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:44:40.192930 2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/be86ef4a-b78b-43ea-b627-563115b10cd5-proc\") pod \"perf-node-gather-daemonset-sj5rv\" (UID: \"be86ef4a-b78b-43ea-b627-563115b10cd5\") " pod="openshift-must-gather-lv5mm/perf-node-gather-daemonset-sj5rv" May 27 20:44:40.193067 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:44:40.192954 2571 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-9l2sq\" (UniqueName: \"kubernetes.io/projected/be86ef4a-b78b-43ea-b627-563115b10cd5-kube-api-access-9l2sq\") pod \"perf-node-gather-daemonset-sj5rv\" (UID: \"be86ef4a-b78b-43ea-b627-563115b10cd5\") " pod="openshift-must-gather-lv5mm/perf-node-gather-daemonset-sj5rv" May 27 20:44:40.193067 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:44:40.192977 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/be86ef4a-b78b-43ea-b627-563115b10cd5-podres\") pod \"perf-node-gather-daemonset-sj5rv\" (UID: \"be86ef4a-b78b-43ea-b627-563115b10cd5\") " pod="openshift-must-gather-lv5mm/perf-node-gather-daemonset-sj5rv" May 27 20:44:40.193067 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:44:40.193019 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/be86ef4a-b78b-43ea-b627-563115b10cd5-sys\") pod \"perf-node-gather-daemonset-sj5rv\" (UID: \"be86ef4a-b78b-43ea-b627-563115b10cd5\") " pod="openshift-must-gather-lv5mm/perf-node-gather-daemonset-sj5rv" May 27 20:44:40.193067 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:44:40.193028 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/be86ef4a-b78b-43ea-b627-563115b10cd5-proc\") pod \"perf-node-gather-daemonset-sj5rv\" (UID: \"be86ef4a-b78b-43ea-b627-563115b10cd5\") " pod="openshift-must-gather-lv5mm/perf-node-gather-daemonset-sj5rv" May 27 20:44:40.193067 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:44:40.193044 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/be86ef4a-b78b-43ea-b627-563115b10cd5-lib-modules\") pod \"perf-node-gather-daemonset-sj5rv\" (UID: \"be86ef4a-b78b-43ea-b627-563115b10cd5\") " pod="openshift-must-gather-lv5mm/perf-node-gather-daemonset-sj5rv" May 27 20:44:40.202236 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:44:40.202217 2571 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9l2sq\" (UniqueName: \"kubernetes.io/projected/be86ef4a-b78b-43ea-b627-563115b10cd5-kube-api-access-9l2sq\") pod \"perf-node-gather-daemonset-sj5rv\" (UID: \"be86ef4a-b78b-43ea-b627-563115b10cd5\") " pod="openshift-must-gather-lv5mm/perf-node-gather-daemonset-sj5rv" May 27 20:44:40.225911 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:44:40.225890 2571 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-lv5mm/perf-node-gather-daemonset-sj5rv" May 27 20:44:40.352154 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:44:40.352128 2571 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-lv5mm/perf-node-gather-daemonset-sj5rv"] May 27 20:44:40.354808 ip-10-0-142-251 kubenswrapper[2571]: W0527 20:44:40.354780 2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podbe86ef4a_b78b_43ea_b627_563115b10cd5.slice/crio-f65f8fd8a138ebac5d1e7a739feaaf3e5718297f479cd46b96143e37555d8635 WatchSource:0}: Error finding container f65f8fd8a138ebac5d1e7a739feaaf3e5718297f479cd46b96143e37555d8635: Status 404 returned error can't find the container with id f65f8fd8a138ebac5d1e7a739feaaf3e5718297f479cd46b96143e37555d8635 May 27 20:44:40.356687 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:44:40.356672 2571 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider May 27 20:44:40.868135 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:44:40.868090 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-76556fbcb9-zf7g2_06b9f345-7df7-4d77-9e37-a53e47a54abd/console/0.log" May 27 20:44:40.942544 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:44:40.942499 2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-lv5mm/perf-node-gather-daemonset-sj5rv" event={"ID":"be86ef4a-b78b-43ea-b627-563115b10cd5","Type":"ContainerStarted","Data":"7a7cf9a6c8c3bdbd4881e310b9e6fdba2bc6be8a92ef2afa7b608928a7c52a4b"} May 27 20:44:40.942544 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:44:40.942542 2571 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-lv5mm/perf-node-gather-daemonset-sj5rv" event={"ID":"be86ef4a-b78b-43ea-b627-563115b10cd5","Type":"ContainerStarted","Data":"f65f8fd8a138ebac5d1e7a739feaaf3e5718297f479cd46b96143e37555d8635"} May 27 20:44:40.942924 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:44:40.942611 2571 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-lv5mm/perf-node-gather-daemonset-sj5rv" May 27 20:44:40.973628 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:44:40.973577 2571 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-must-gather-lv5mm/perf-node-gather-daemonset-sj5rv" podStartSLOduration=1.973565842 podStartE2EDuration="1.973565842s" podCreationTimestamp="2026-05-27 20:44:39 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-27 20:44:40.97153933 +0000 UTC m=+2033.381165641" watchObservedRunningTime="2026-05-27 20:44:40.973565842 +0000 UTC m=+2033.383192153" May 27 20:44:42.202482 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:44:42.202450 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-58qkv_cf585445-0773-47ca-ba45-3bad1a2d5b88/dns/0.log" May 27 20:44:42.232250 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:44:42.232214 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-58qkv_cf585445-0773-47ca-ba45-3bad1a2d5b88/kube-rbac-proxy/0.log" May 27 20:44:42.377392 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:44:42.377361 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-n4brm_4937a11c-a8df-40eb-9a62-0b9fa6ea34ef/dns-node-resolver/0.log" May 27 20:44:42.850061 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:44:42.850030 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-6bc4d5d5c8-7lb24_d292858c-6349-4ea3-8019-fa4bbf14d6a9/registry/0.log" May 27 20:44:42.890641 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:44:42.890613 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-f4p5s_cdf5a20d-c5e1-4d30-a6ce-85c91afd58e2/node-ca/0.log" May 27 20:44:43.779060 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:44:43.779033 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-798958bb55-f9dxg_9ecca3e6-bc39-4fa9-b197-c9af65680d01/discovery/0.log" May 27 20:44:43.799977 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:44:43.799941 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-85db9dc568-8vxjw_4f200e9c-bdab-48c5-a609-654bb82a823d/kube-auth-proxy/0.log" May 27 20:44:43.928855 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:44:43.928818 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58b6c8fdc7-mfs2j_39a39b37-93cb-4b21-97a6-8f3d2350f1e4/payload-processing/0.log" May 27 20:44:44.500547 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:44:44.500515 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-jxjsb_d6fb3e28-cbfd-47a3-8104-bbfa86006ad6/serve-healthcheck-canary/0.log" May 27 20:44:45.065841 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:44:45.065815 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-625bj_66ac589d-841c-4b3d-87f5-e57a079b95e3/kube-rbac-proxy/0.log" May 27 20:44:45.086635 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:44:45.086608 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-625bj_66ac589d-841c-4b3d-87f5-e57a079b95e3/exporter/0.log" May 27 20:44:45.107037 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:44:45.107014 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-625bj_66ac589d-841c-4b3d-87f5-e57a079b95e3/extractor/0.log" May 27 20:44:46.955949 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:44:46.955917 2571 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-lv5mm/perf-node-gather-daemonset-sj5rv" May 27 20:44:47.119281 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:44:47.119254 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-856948b99f-kxfwr_ad931d17-fee2-4654-af65-c270e04d9fba/manager/0.log" May 27 20:44:47.194286 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:44:47.194253 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-6776cdf886-wcqkx_06e664ca-e53f-470b-8b2c-0abebf376de6/maas-api/0.log" May 27 20:44:47.307917 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:44:47.307882 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-67678bdf-kpmt6_485cd20e-b78a-4d6f-afe7-ca5ae64387f8/manager/0.log" May 27 20:44:47.336291 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:44:47.336253 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-858dbf95b8-sbtjh_123bc11b-e985-42a1-b0be-3023ea71b7aa/manager/0.log" May 27 20:44:47.346428 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:44:47.346398 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-858dbf95b8-sbtjh_123bc11b-e985-42a1-b0be-3023ea71b7aa/manager/1.log" May 27 20:44:47.431366 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:44:47.431341 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-b59bfd74-q82qr_4976f387-5d72-4a0d-befe-78cd65e79e58/manager/0.log" May 27 20:44:47.454509 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:44:47.454482 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_postgres-68c4fbbd6f-s7hf9_b1d3f9e4-5e89-470f-be70-9f4d65f1bd3e/postgres/0.log" May 27 20:44:48.700021 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:44:48.699996 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-bfc7f696d-mlpld_15aecb97-de79-4d5b-a5e3-4780bede9b7e/openshift-lws-operator/0.log" May 27 20:44:54.471697 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:44:54.471666 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-265pt_2efad6a2-ff87-479a-894d-d2397edf22d9/kube-multus/0.log" May 27 20:44:54.799649 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:44:54.799624 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-zhh65_d4f71add-cfe6-4636-a5d9-fb823d22a61d/kube-multus-additional-cni-plugins/0.log" May 27 20:44:54.822648 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:44:54.822622 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-zhh65_d4f71add-cfe6-4636-a5d9-fb823d22a61d/egress-router-binary-copy/0.log" May 27 20:44:54.843300 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:44:54.843283 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-zhh65_d4f71add-cfe6-4636-a5d9-fb823d22a61d/cni-plugins/0.log" May 27 20:44:54.865597 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:44:54.865576 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-zhh65_d4f71add-cfe6-4636-a5d9-fb823d22a61d/bond-cni-plugin/0.log" May 27 20:44:54.885972 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:44:54.885955 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-zhh65_d4f71add-cfe6-4636-a5d9-fb823d22a61d/routeoverride-cni/0.log" May 27 20:44:54.906419 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:44:54.906400 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-zhh65_d4f71add-cfe6-4636-a5d9-fb823d22a61d/whereabouts-cni-bincopy/0.log" May 27 20:44:54.927728 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:44:54.927710 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-zhh65_d4f71add-cfe6-4636-a5d9-fb823d22a61d/whereabouts-cni/0.log" May 27 20:44:55.097546 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:44:55.097481 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-zsx5p_96207b98-716e-4c63-847f-fc976f5563f2/network-metrics-daemon/0.log" May 27 20:44:55.118144 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:44:55.118123 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-zsx5p_96207b98-716e-4c63-847f-fc976f5563f2/kube-rbac-proxy/0.log" May 27 20:44:56.158847 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:44:56.158815 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rcvbq_92933172-b100-42e6-8018-dcb430647ae2/ovn-controller/0.log" May 27 20:44:56.176966 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:44:56.176939 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rcvbq_92933172-b100-42e6-8018-dcb430647ae2/ovn-acl-logging/0.log" May 27 20:44:56.185133 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:44:56.185115 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rcvbq_92933172-b100-42e6-8018-dcb430647ae2/ovn-acl-logging/1.log" May 27 20:44:56.203077 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:44:56.203057 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rcvbq_92933172-b100-42e6-8018-dcb430647ae2/kube-rbac-proxy-node/0.log" May 27 20:44:56.225041 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:44:56.225017 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rcvbq_92933172-b100-42e6-8018-dcb430647ae2/kube-rbac-proxy-ovn-metrics/0.log" May 27 20:44:56.244591 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:44:56.244568 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rcvbq_92933172-b100-42e6-8018-dcb430647ae2/northd/0.log" May 27 20:44:56.268507 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:44:56.268479 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rcvbq_92933172-b100-42e6-8018-dcb430647ae2/nbdb/0.log" May 27 20:44:56.290354 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:44:56.290334 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rcvbq_92933172-b100-42e6-8018-dcb430647ae2/sbdb/0.log" May 27 20:44:56.380615 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:44:56.380595 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rcvbq_92933172-b100-42e6-8018-dcb430647ae2/ovnkube-controller/0.log" May 27 20:44:57.808328 ip-10-0-142-251 kubenswrapper[2571]: I0527 20:44:57.808295 2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-diagnostics_network-check-target-jdzbr_de3a4bee-c44a-4da9-833f-7e04d40c1f82/network-check-target-container/0.log"