Jun 01 21:40:40.184895 ip-10-0-138-241 systemd[1]: Starting Kubernetes Kubelet... Jun 01 21:40:40.522484 ip-10-0-138-241 kubenswrapper[2573]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jun 01 21:40:40.522484 ip-10-0-138-241 kubenswrapper[2573]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jun 01 21:40:40.522484 ip-10-0-138-241 kubenswrapper[2573]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jun 01 21:40:40.522484 ip-10-0-138-241 kubenswrapper[2573]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jun 01 21:40:40.522484 ip-10-0-138-241 kubenswrapper[2573]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jun 01 21:40:40.525506 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:40.525417 2573 server.go:212] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jun 01 21:40:40.529443 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.529426 2573 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 01 21:40:40.529443 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.529443 2573 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 01 21:40:40.529510 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.529448 2573 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 01 21:40:40.529510 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.529453 2573 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 01 21:40:40.529510 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.529457 2573 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 01 21:40:40.529510 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.529460 2573 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 01 21:40:40.529510 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.529462 2573 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 01 21:40:40.529510 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.529465 2573 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 01 21:40:40.529510 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.529468 2573 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 01 21:40:40.529510 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.529470 2573 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 01 21:40:40.529510 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.529473 2573 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 01 21:40:40.529510 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.529476 2573 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 01 21:40:40.529510 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.529478 2573 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 01 21:40:40.529510 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.529481 2573 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 01 21:40:40.529510 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.529489 2573 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 01 21:40:40.529510 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.529492 2573 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 01 21:40:40.529510 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.529495 2573 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 01 21:40:40.529510 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.529497 2573 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 01 21:40:40.529510 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.529500 2573 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 01 21:40:40.529510 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.529502 2573 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 01 21:40:40.529510 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.529505 2573 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 01 21:40:40.529977 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.529508 2573 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 01 21:40:40.529977 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.529510 2573 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 01 21:40:40.529977 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.529513 2573 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 01 21:40:40.529977 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.529516 2573 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 01 21:40:40.529977 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.529519 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 01 21:40:40.529977 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.529522 2573 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI Jun 01 21:40:40.529977 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.529525 2573 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations Jun 01 21:40:40.529977 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.529527 2573 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 01 21:40:40.529977 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.529530 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 01 21:40:40.529977 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.529544 2573 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 01 21:40:40.529977 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.529547 2573 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 01 21:40:40.529977 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.529550 2573 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 01 21:40:40.529977 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.529552 2573 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 01 21:40:40.529977 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.529555 2573 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 01 21:40:40.529977 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.529557 2573 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 01 21:40:40.529977 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.529560 2573 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 01 21:40:40.529977 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.529562 2573 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 01 21:40:40.529977 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.529565 2573 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 01 21:40:40.529977 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.529570 2573 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 01 21:40:40.530679 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.529573 2573 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 01 21:40:40.530679 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.529576 2573 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController Jun 01 21:40:40.530679 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.529579 2573 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 01 21:40:40.530679 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.529581 2573 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 01 21:40:40.530679 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.529584 2573 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 01 21:40:40.530679 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.529587 2573 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 01 21:40:40.530679 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.529589 2573 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 01 21:40:40.530679 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.529592 2573 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 01 21:40:40.530679 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.529596 2573 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 01 21:40:40.530679 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.529599 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 01 21:40:40.530679 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.529602 2573 feature_gate.go:328] unrecognized feature gate: Example2 Jun 01 21:40:40.530679 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.529604 2573 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 01 21:40:40.530679 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.529607 2573 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 01 21:40:40.530679 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.529610 2573 feature_gate.go:328] unrecognized feature gate: Example Jun 01 21:40:40.530679 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.529613 2573 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 01 21:40:40.530679 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.529616 2573 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 01 21:40:40.530679 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.529618 2573 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 01 21:40:40.530679 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.529621 2573 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 01 21:40:40.530679 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.529623 2573 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 01 21:40:40.531406 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.529626 2573 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 01 21:40:40.531406 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.529629 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 01 21:40:40.531406 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.529631 2573 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 01 21:40:40.531406 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.529634 2573 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 01 21:40:40.531406 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.529636 2573 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 01 21:40:40.531406 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.529639 2573 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 01 21:40:40.531406 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.529641 2573 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 01 21:40:40.531406 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.529644 2573 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS Jun 01 21:40:40.531406 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.529647 2573 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 01 21:40:40.531406 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.529650 2573 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 01 21:40:40.531406 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.529652 2573 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 01 21:40:40.531406 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.529655 2573 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 01 21:40:40.531406 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.529658 2573 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 01 21:40:40.531406 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.529660 2573 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 01 21:40:40.531406 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.529663 2573 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 01 21:40:40.531406 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.529665 2573 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 01 21:40:40.531406 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.529668 2573 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 01 21:40:40.531406 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.529670 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 01 21:40:40.531406 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.529672 2573 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 01 21:40:40.531406 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.529675 2573 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 01 21:40:40.532235 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.529678 2573 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 01 21:40:40.532235 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.529681 2573 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 01 21:40:40.532235 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.529684 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 01 21:40:40.532235 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.529686 2573 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 01 21:40:40.532235 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.529689 2573 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 01 21:40:40.532235 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.529692 2573 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 01 21:40:40.532235 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.529694 2573 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 01 21:40:40.532235 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.530858 2573 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 01 21:40:40.532235 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.530893 2573 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 01 21:40:40.532235 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.530898 2573 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 01 21:40:40.532235 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.530910 2573 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 01 21:40:40.532235 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.530914 2573 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 01 21:40:40.532235 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.530919 2573 feature_gate.go:328] unrecognized feature gate: Example2 Jun 01 21:40:40.532235 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.530923 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 01 21:40:40.532235 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.530928 2573 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI Jun 01 21:40:40.532235 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.530932 2573 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 01 21:40:40.532235 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.530937 2573 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 01 21:40:40.532235 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.530941 2573 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 01 21:40:40.532235 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.530948 2573 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 01 21:40:40.532235 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.530954 2573 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 01 21:40:40.533118 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.530958 2573 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 01 21:40:40.533118 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.530962 2573 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 01 21:40:40.533118 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.530971 2573 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 01 21:40:40.533118 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.530975 2573 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 01 21:40:40.533118 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.530979 2573 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 01 21:40:40.533118 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.530983 2573 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 01 21:40:40.533118 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.530988 2573 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 01 21:40:40.533118 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.530992 2573 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 01 21:40:40.533118 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.530997 2573 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 01 21:40:40.533118 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.531001 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 01 21:40:40.533118 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.531006 2573 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 01 21:40:40.533118 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.531009 2573 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 01 21:40:40.533118 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.531013 2573 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 01 21:40:40.533118 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.531045 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 01 21:40:40.533118 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.531050 2573 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 01 21:40:40.533118 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.531060 2573 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 01 21:40:40.533118 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.531064 2573 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 01 21:40:40.533118 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.531068 2573 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 01 21:40:40.533118 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.531072 2573 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 01 21:40:40.533118 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.531078 2573 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 01 21:40:40.533771 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.531082 2573 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 01 21:40:40.533771 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.531086 2573 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 01 21:40:40.533771 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.531090 2573 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 01 21:40:40.533771 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.531094 2573 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 01 21:40:40.533771 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.531098 2573 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 01 21:40:40.533771 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.531101 2573 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 01 21:40:40.533771 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.531105 2573 feature_gate.go:328] unrecognized feature gate: Example Jun 01 21:40:40.533771 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.531211 2573 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 01 21:40:40.533771 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.531223 2573 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 01 21:40:40.533771 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.531228 2573 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 01 21:40:40.533771 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.531235 2573 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 01 21:40:40.533771 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.531240 2573 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 01 21:40:40.533771 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.531245 2573 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 01 21:40:40.533771 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.531250 2573 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 01 21:40:40.533771 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.531258 2573 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 01 21:40:40.533771 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.531264 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 01 21:40:40.533771 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.531301 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 01 21:40:40.533771 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.531324 2573 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 01 21:40:40.533771 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.531328 2573 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 01 21:40:40.533771 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.531343 2573 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 01 21:40:40.534348 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.531499 2573 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 01 21:40:40.534348 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.531524 2573 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 01 21:40:40.534348 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.531547 2573 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 01 21:40:40.534348 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.531553 2573 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 01 21:40:40.534348 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.531558 2573 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 01 21:40:40.534348 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.531563 2573 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 01 21:40:40.534348 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.531568 2573 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS Jun 01 21:40:40.534348 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.531572 2573 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 01 21:40:40.534348 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.531577 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 01 21:40:40.534348 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.531581 2573 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 01 21:40:40.534348 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.531586 2573 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 01 21:40:40.534348 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.531591 2573 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 01 21:40:40.534348 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.531595 2573 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 01 21:40:40.534348 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.531599 2573 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 01 21:40:40.534348 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.531603 2573 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 01 21:40:40.534348 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.531607 2573 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 01 21:40:40.534348 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.531611 2573 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations Jun 01 21:40:40.534348 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.531616 2573 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 01 21:40:40.534348 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.531621 2573 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 01 21:40:40.534348 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.531625 2573 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 01 21:40:40.534895 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.531630 2573 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 01 21:40:40.534895 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.531635 2573 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 01 21:40:40.534895 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.531640 2573 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 01 21:40:40.534895 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.531644 2573 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController Jun 01 21:40:40.534895 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.531648 2573 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 01 21:40:40.534895 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.531653 2573 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 01 21:40:40.534895 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.531656 2573 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 01 21:40:40.534895 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.531661 2573 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 01 21:40:40.534895 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.531669 2573 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 01 21:40:40.534895 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.531673 2573 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 01 21:40:40.534895 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.531678 2573 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 01 21:40:40.534895 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.531682 2573 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 01 21:40:40.534895 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.531687 2573 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 01 21:40:40.534895 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:40.531806 2573 flags.go:64] FLAG: --address="0.0.0.0" Jun 01 21:40:40.534895 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:40.531818 2573 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jun 01 21:40:40.534895 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:40.531828 2573 flags.go:64] FLAG: --anonymous-auth="true" Jun 01 21:40:40.534895 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:40.531835 2573 flags.go:64] FLAG: --application-metrics-count-limit="100" Jun 01 21:40:40.534895 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:40.531843 2573 flags.go:64] FLAG: --authentication-token-webhook="false" Jun 01 21:40:40.534895 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:40.531848 2573 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jun 01 21:40:40.534895 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:40.531855 2573 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jun 01 21:40:40.534895 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:40.531861 2573 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jun 01 21:40:40.535461 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:40.531866 2573 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jun 01 21:40:40.535461 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:40.531871 2573 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jun 01 21:40:40.535461 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:40.531878 2573 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jun 01 21:40:40.535461 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:40.531884 2573 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jun 01 21:40:40.535461 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:40.531889 2573 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jun 01 21:40:40.535461 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:40.531894 2573 flags.go:64] FLAG: --cgroup-root="" Jun 01 21:40:40.535461 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:40.531899 2573 flags.go:64] FLAG: --cgroups-per-qos="true" Jun 01 21:40:40.535461 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:40.531904 2573 flags.go:64] FLAG: --client-ca-file="" Jun 01 21:40:40.535461 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:40.531909 2573 flags.go:64] FLAG: --cloud-config="" Jun 01 21:40:40.535461 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:40.531913 2573 flags.go:64] FLAG: --cloud-provider="external" Jun 01 21:40:40.535461 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:40.531919 2573 flags.go:64] FLAG: --cluster-dns="[]" Jun 01 21:40:40.535461 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:40.531926 2573 flags.go:64] FLAG: --cluster-domain="" Jun 01 21:40:40.535461 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:40.531930 2573 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jun 01 21:40:40.535461 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:40.531935 2573 flags.go:64] FLAG: --config-dir="" Jun 01 21:40:40.535461 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:40.531940 2573 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jun 01 21:40:40.535461 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:40.531946 2573 flags.go:64] FLAG: --container-log-max-files="5" Jun 01 21:40:40.535461 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:40.531952 2573 flags.go:64] FLAG: --container-log-max-size="10Mi" Jun 01 21:40:40.535461 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:40.531957 2573 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jun 01 21:40:40.535461 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:40.531962 2573 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jun 01 21:40:40.535461 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:40.531968 2573 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jun 01 21:40:40.535461 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:40.531973 2573 flags.go:64] FLAG: --contention-profiling="false" Jun 01 21:40:40.535461 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:40.531978 2573 flags.go:64] FLAG: --cpu-cfs-quota="true" Jun 01 21:40:40.535461 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:40.531983 2573 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jun 01 21:40:40.535461 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:40.531988 2573 flags.go:64] FLAG: --cpu-manager-policy="none" Jun 01 21:40:40.535461 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:40.531994 2573 flags.go:64] FLAG: --cpu-manager-policy-options="" Jun 01 21:40:40.536275 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:40.532001 2573 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jun 01 21:40:40.536275 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:40.532006 2573 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jun 01 21:40:40.536275 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:40.532011 2573 flags.go:64] FLAG: --enable-debugging-handlers="true" Jun 01 21:40:40.536275 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:40.532016 2573 flags.go:64] FLAG: --enable-load-reader="false" Jun 01 21:40:40.536275 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:40.532022 2573 flags.go:64] FLAG: --enable-server="true" Jun 01 21:40:40.536275 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:40.532027 2573 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jun 01 21:40:40.536275 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:40.532034 2573 flags.go:64] FLAG: --event-burst="100" Jun 01 21:40:40.536275 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:40.532040 2573 flags.go:64] FLAG: --event-qps="50" Jun 01 21:40:40.536275 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:40.532044 2573 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jun 01 21:40:40.536275 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:40.532049 2573 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jun 01 21:40:40.536275 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:40.532054 2573 flags.go:64] FLAG: --eviction-hard="" Jun 01 21:40:40.536275 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:40.532060 2573 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jun 01 21:40:40.536275 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:40.532065 2573 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jun 01 21:40:40.536275 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:40.532070 2573 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jun 01 21:40:40.536275 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:40.532075 2573 flags.go:64] FLAG: --eviction-soft="" Jun 01 21:40:40.536275 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:40.532079 2573 flags.go:64] FLAG: --eviction-soft-grace-period="" Jun 01 21:40:40.536275 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:40.532084 2573 flags.go:64] FLAG: --exit-on-lock-contention="false" Jun 01 21:40:40.536275 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:40.532089 2573 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jun 01 21:40:40.536275 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:40.532094 2573 flags.go:64] FLAG: --experimental-mounter-path="" Jun 01 21:40:40.536275 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:40.532099 2573 flags.go:64] FLAG: --fail-cgroupv1="false" Jun 01 21:40:40.536275 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:40.532104 2573 flags.go:64] FLAG: --fail-swap-on="true" Jun 01 21:40:40.536275 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:40.532109 2573 flags.go:64] FLAG: --feature-gates="" Jun 01 21:40:40.536275 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:40.532115 2573 flags.go:64] FLAG: --file-check-frequency="20s" Jun 01 21:40:40.536275 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:40.532120 2573 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jun 01 21:40:40.536275 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:40.532126 2573 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jun 01 21:40:40.536954 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:40.532132 2573 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jun 01 21:40:40.536954 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:40.532137 2573 flags.go:64] FLAG: --healthz-port="10248" Jun 01 21:40:40.536954 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:40.532142 2573 flags.go:64] FLAG: --help="false" Jun 01 21:40:40.536954 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:40.532147 2573 flags.go:64] FLAG: --hostname-override="ip-10-0-138-241.ec2.internal" Jun 01 21:40:40.536954 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:40.532152 2573 flags.go:64] FLAG: --housekeeping-interval="10s" Jun 01 21:40:40.536954 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:40.532157 2573 flags.go:64] FLAG: --http-check-frequency="20s" Jun 01 21:40:40.536954 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:40.532165 2573 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jun 01 21:40:40.536954 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:40.532171 2573 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jun 01 21:40:40.536954 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:40.532178 2573 flags.go:64] FLAG: --image-gc-high-threshold="85" Jun 01 21:40:40.536954 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:40.532183 2573 flags.go:64] FLAG: --image-gc-low-threshold="80" Jun 01 21:40:40.536954 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:40.532188 2573 flags.go:64] FLAG: --image-service-endpoint="" Jun 01 21:40:40.536954 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:40.532192 2573 flags.go:64] FLAG: --kernel-memcg-notification="false" Jun 01 21:40:40.536954 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:40.532197 2573 flags.go:64] FLAG: --kube-api-burst="100" Jun 01 21:40:40.536954 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:40.532202 2573 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jun 01 21:40:40.536954 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:40.532207 2573 flags.go:64] FLAG: --kube-api-qps="50" Jun 01 21:40:40.536954 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:40.532212 2573 flags.go:64] FLAG: --kube-reserved="" Jun 01 21:40:40.536954 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:40.532218 2573 flags.go:64] FLAG: --kube-reserved-cgroup="" Jun 01 21:40:40.536954 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:40.532222 2573 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jun 01 21:40:40.536954 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:40.532228 2573 flags.go:64] FLAG: --kubelet-cgroups="" Jun 01 21:40:40.536954 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:40.532232 2573 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jun 01 21:40:40.536954 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:40.532237 2573 flags.go:64] FLAG: --lock-file="" Jun 01 21:40:40.536954 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:40.532241 2573 flags.go:64] FLAG: --log-cadvisor-usage="false" Jun 01 21:40:40.536954 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:40.532245 2573 flags.go:64] FLAG: --log-flush-frequency="5s" Jun 01 21:40:40.536954 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:40.532250 2573 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jun 01 21:40:40.537617 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:40.532258 2573 flags.go:64] FLAG: --log-json-split-stream="false" Jun 01 21:40:40.537617 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:40.532263 2573 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jun 01 21:40:40.537617 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:40.532268 2573 flags.go:64] FLAG: --log-text-split-stream="false" Jun 01 21:40:40.537617 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:40.532273 2573 flags.go:64] FLAG: --logging-format="text" Jun 01 21:40:40.537617 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:40.532278 2573 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jun 01 21:40:40.537617 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:40.532284 2573 flags.go:64] FLAG: --make-iptables-util-chains="true" Jun 01 21:40:40.537617 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:40.532289 2573 flags.go:64] FLAG: --manifest-url="" Jun 01 21:40:40.537617 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:40.532293 2573 flags.go:64] FLAG: --manifest-url-header="" Jun 01 21:40:40.537617 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:40.532300 2573 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jun 01 21:40:40.537617 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:40.532315 2573 flags.go:64] FLAG: --max-open-files="1000000" Jun 01 21:40:40.537617 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:40.532323 2573 flags.go:64] FLAG: --max-pods="110" Jun 01 21:40:40.537617 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:40.532328 2573 flags.go:64] FLAG: --maximum-dead-containers="-1" Jun 01 21:40:40.537617 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:40.532333 2573 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jun 01 21:40:40.537617 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:40.532338 2573 flags.go:64] FLAG: --memory-manager-policy="None" Jun 01 21:40:40.537617 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:40.532345 2573 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jun 01 21:40:40.537617 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:40.532349 2573 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jun 01 21:40:40.537617 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:40.532354 2573 flags.go:64] FLAG: --node-ip="0.0.0.0" Jun 01 21:40:40.537617 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:40.532359 2573 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jun 01 21:40:40.537617 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:40.532371 2573 flags.go:64] FLAG: --node-status-max-images="50" Jun 01 21:40:40.537617 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:40.532376 2573 flags.go:64] FLAG: --node-status-update-frequency="10s" Jun 01 21:40:40.537617 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:40.532381 2573 flags.go:64] FLAG: --oom-score-adj="-999" Jun 01 21:40:40.537617 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:40.532385 2573 flags.go:64] FLAG: --pod-cidr="" Jun 01 21:40:40.537617 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:40.532390 2573 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:f1083dc6abd9805f79797fa11c9accacc4a38caa073c94cb99097faffce05983" Jun 01 21:40:40.538196 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:40.532398 2573 flags.go:64] FLAG: --pod-manifest-path="" Jun 01 21:40:40.538196 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:40.532403 2573 flags.go:64] FLAG: --pod-max-pids="-1" Jun 01 21:40:40.538196 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:40.532408 2573 flags.go:64] FLAG: --pods-per-core="0" Jun 01 21:40:40.538196 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:40.532413 2573 flags.go:64] FLAG: --port="10250" Jun 01 21:40:40.538196 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:40.532418 2573 flags.go:64] FLAG: --protect-kernel-defaults="false" Jun 01 21:40:40.538196 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:40.532422 2573 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0bc88623c08662e48" Jun 01 21:40:40.538196 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:40.532427 2573 flags.go:64] FLAG: --qos-reserved="" Jun 01 21:40:40.538196 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:40.532432 2573 flags.go:64] FLAG: --read-only-port="10255" Jun 01 21:40:40.538196 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:40.532437 2573 flags.go:64] FLAG: --register-node="true" Jun 01 21:40:40.538196 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:40.532442 2573 flags.go:64] FLAG: --register-schedulable="true" Jun 01 21:40:40.538196 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:40.532447 2573 flags.go:64] FLAG: --register-with-taints="" Jun 01 21:40:40.538196 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:40.532453 2573 flags.go:64] FLAG: --registry-burst="10" Jun 01 21:40:40.538196 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:40.532457 2573 flags.go:64] FLAG: --registry-qps="5" Jun 01 21:40:40.538196 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:40.532462 2573 flags.go:64] FLAG: --reserved-cpus="" Jun 01 21:40:40.538196 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:40.532467 2573 flags.go:64] FLAG: --reserved-memory="" Jun 01 21:40:40.538196 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:40.532473 2573 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jun 01 21:40:40.538196 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:40.532478 2573 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jun 01 21:40:40.538196 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:40.532483 2573 flags.go:64] FLAG: --rotate-certificates="false" Jun 01 21:40:40.538196 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:40.532488 2573 flags.go:64] FLAG: --rotate-server-certificates="false" Jun 01 21:40:40.538196 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:40.532500 2573 flags.go:64] FLAG: --runonce="false" Jun 01 21:40:40.538196 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:40.532505 2573 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jun 01 21:40:40.538196 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:40.532510 2573 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jun 01 21:40:40.538196 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:40.532515 2573 flags.go:64] FLAG: --seccomp-default="false" Jun 01 21:40:40.538196 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:40.532521 2573 flags.go:64] FLAG: --serialize-image-pulls="true" Jun 01 21:40:40.538196 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:40.532526 2573 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jun 01 21:40:40.538196 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:40.532550 2573 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jun 01 21:40:40.538847 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:40.532556 2573 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jun 01 21:40:40.538847 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:40.532561 2573 flags.go:64] FLAG: --storage-driver-password="root" Jun 01 21:40:40.538847 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:40.532565 2573 flags.go:64] FLAG: --storage-driver-secure="false" Jun 01 21:40:40.538847 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:40.532570 2573 flags.go:64] FLAG: --storage-driver-table="stats" Jun 01 21:40:40.538847 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:40.532575 2573 flags.go:64] FLAG: --storage-driver-user="root" Jun 01 21:40:40.538847 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:40.532579 2573 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jun 01 21:40:40.538847 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:40.532584 2573 flags.go:64] FLAG: --sync-frequency="1m0s" Jun 01 21:40:40.538847 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:40.532589 2573 flags.go:64] FLAG: --system-cgroups="" Jun 01 21:40:40.538847 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:40.532594 2573 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jun 01 21:40:40.538847 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:40.532604 2573 flags.go:64] FLAG: --system-reserved-cgroup="" Jun 01 21:40:40.538847 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:40.532608 2573 flags.go:64] FLAG: --tls-cert-file="" Jun 01 21:40:40.538847 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:40.532613 2573 flags.go:64] FLAG: --tls-cipher-suites="[]" Jun 01 21:40:40.538847 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:40.532620 2573 flags.go:64] FLAG: --tls-min-version="" Jun 01 21:40:40.538847 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:40.532625 2573 flags.go:64] FLAG: --tls-private-key-file="" Jun 01 21:40:40.538847 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:40.532629 2573 flags.go:64] FLAG: --topology-manager-policy="none" Jun 01 21:40:40.538847 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:40.532634 2573 flags.go:64] FLAG: --topology-manager-policy-options="" Jun 01 21:40:40.538847 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:40.532639 2573 flags.go:64] FLAG: --topology-manager-scope="container" Jun 01 21:40:40.538847 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:40.532644 2573 flags.go:64] FLAG: --v="2" Jun 01 21:40:40.538847 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:40.532651 2573 flags.go:64] FLAG: --version="false" Jun 01 21:40:40.538847 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:40.532657 2573 flags.go:64] FLAG: --vmodule="" Jun 01 21:40:40.538847 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:40.532663 2573 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jun 01 21:40:40.538847 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:40.532669 2573 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jun 01 21:40:40.538847 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.532840 2573 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 01 21:40:40.538847 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.532847 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 01 21:40:40.539420 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.532852 2573 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 01 21:40:40.539420 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.532857 2573 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 01 21:40:40.539420 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.532864 2573 feature_gate.go:328] unrecognized feature gate: Example Jun 01 21:40:40.539420 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.532869 2573 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations Jun 01 21:40:40.539420 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.532873 2573 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 01 21:40:40.539420 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.532881 2573 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 01 21:40:40.539420 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.532885 2573 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 01 21:40:40.539420 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.532889 2573 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 01 21:40:40.539420 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.532894 2573 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 01 21:40:40.539420 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.532898 2573 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 01 21:40:40.539420 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.532902 2573 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 01 21:40:40.539420 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.532906 2573 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS Jun 01 21:40:40.539420 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.532910 2573 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 01 21:40:40.539420 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.532915 2573 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 01 21:40:40.539420 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.532918 2573 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 01 21:40:40.539420 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.532923 2573 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 01 21:40:40.539420 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.532927 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 01 21:40:40.539420 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.532931 2573 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 01 21:40:40.539420 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.532935 2573 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 01 21:40:40.539420 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.532939 2573 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 01 21:40:40.540040 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.532943 2573 feature_gate.go:328] unrecognized feature gate: Example2 Jun 01 21:40:40.540040 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.532948 2573 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 01 21:40:40.540040 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.532951 2573 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 01 21:40:40.540040 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.532955 2573 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 01 21:40:40.540040 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.532960 2573 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 01 21:40:40.540040 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.532964 2573 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 01 21:40:40.540040 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.532968 2573 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 01 21:40:40.540040 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.532973 2573 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 01 21:40:40.540040 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.532976 2573 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 01 21:40:40.540040 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.532980 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 01 21:40:40.540040 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.532984 2573 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 01 21:40:40.540040 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.532988 2573 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 01 21:40:40.540040 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.532993 2573 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController Jun 01 21:40:40.540040 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.532996 2573 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 01 21:40:40.540040 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.533001 2573 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 01 21:40:40.540040 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.533006 2573 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 01 21:40:40.540040 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.533010 2573 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 01 21:40:40.540040 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.533016 2573 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 01 21:40:40.540040 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.533020 2573 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 01 21:40:40.540506 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.533024 2573 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 01 21:40:40.540506 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.533028 2573 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 01 21:40:40.540506 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.533035 2573 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 01 21:40:40.540506 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.533041 2573 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI Jun 01 21:40:40.540506 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.533046 2573 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 01 21:40:40.540506 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.533050 2573 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 01 21:40:40.540506 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.533056 2573 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 01 21:40:40.540506 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.533062 2573 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 01 21:40:40.540506 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.533068 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 01 21:40:40.540506 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.533072 2573 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 01 21:40:40.540506 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.533076 2573 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 01 21:40:40.540506 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.533080 2573 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 01 21:40:40.540506 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.533085 2573 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 01 21:40:40.540506 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.533089 2573 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 01 21:40:40.540506 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.533094 2573 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 01 21:40:40.540506 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.533097 2573 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 01 21:40:40.540506 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.533101 2573 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 01 21:40:40.540506 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.533106 2573 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 01 21:40:40.540506 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.533110 2573 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 01 21:40:40.541002 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.533114 2573 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 01 21:40:40.541002 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.533118 2573 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 01 21:40:40.541002 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.533121 2573 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 01 21:40:40.541002 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.533125 2573 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 01 21:40:40.541002 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.533129 2573 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 01 21:40:40.541002 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.533133 2573 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 01 21:40:40.541002 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.533137 2573 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 01 21:40:40.541002 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.533141 2573 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 01 21:40:40.541002 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.533149 2573 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 01 21:40:40.541002 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.533154 2573 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 01 21:40:40.541002 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.533159 2573 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 01 21:40:40.541002 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.533165 2573 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 01 21:40:40.541002 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.533170 2573 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 01 21:40:40.541002 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.533174 2573 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 01 21:40:40.541002 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.533178 2573 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 01 21:40:40.541002 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.533182 2573 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 01 21:40:40.541002 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.533187 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 01 21:40:40.541002 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.533191 2573 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 01 21:40:40.541002 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.533196 2573 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 01 21:40:40.541002 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.533200 2573 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 01 21:40:40.541494 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.533205 2573 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 01 21:40:40.541494 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.533209 2573 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 01 21:40:40.541494 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.533213 2573 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 01 21:40:40.541494 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.533217 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 01 21:40:40.541494 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.533222 2573 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 01 21:40:40.541494 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.533226 2573 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 01 21:40:40.541494 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:40.533234 2573 feature_gate.go:384] feature gates: {map[DynamicResourceAllocation:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutatingAdmissionPolicy:false NodeSwap:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:false]} Jun 01 21:40:40.541494 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:40.541202 2573 server.go:530] "Kubelet version" kubeletVersion="v1.33.12" Jun 01 21:40:40.541494 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:40.541220 2573 server.go:532] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jun 01 21:40:40.541494 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.541267 2573 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 01 21:40:40.541494 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.541272 2573 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 01 21:40:40.541494 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.541276 2573 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations Jun 01 21:40:40.541494 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.541279 2573 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController Jun 01 21:40:40.541494 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.541282 2573 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 01 21:40:40.541494 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.541285 2573 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 01 21:40:40.541494 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.541288 2573 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 01 21:40:40.542009 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.541290 2573 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 01 21:40:40.542009 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.541293 2573 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 01 21:40:40.542009 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.541296 2573 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 01 21:40:40.542009 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.541298 2573 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 01 21:40:40.542009 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.541301 2573 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 01 21:40:40.542009 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.541303 2573 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 01 21:40:40.542009 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.541306 2573 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 01 21:40:40.542009 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.541309 2573 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 01 21:40:40.542009 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.541312 2573 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 01 21:40:40.542009 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.541315 2573 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 01 21:40:40.542009 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.541317 2573 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 01 21:40:40.542009 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.541320 2573 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 01 21:40:40.542009 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.541322 2573 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 01 21:40:40.542009 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.541325 2573 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 01 21:40:40.542009 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.541328 2573 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 01 21:40:40.542009 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.541331 2573 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 01 21:40:40.542009 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.541334 2573 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 01 21:40:40.542009 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.541336 2573 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 01 21:40:40.542009 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.541339 2573 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 01 21:40:40.542009 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.541341 2573 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 01 21:40:40.542508 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.541344 2573 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 01 21:40:40.542508 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.541346 2573 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 01 21:40:40.542508 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.541349 2573 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 01 21:40:40.542508 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.541352 2573 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 01 21:40:40.542508 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.541356 2573 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 01 21:40:40.542508 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.541359 2573 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 01 21:40:40.542508 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.541362 2573 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 01 21:40:40.542508 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.541364 2573 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 01 21:40:40.542508 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.541367 2573 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 01 21:40:40.542508 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.541370 2573 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 01 21:40:40.542508 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.541372 2573 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 01 21:40:40.542508 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.541375 2573 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 01 21:40:40.542508 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.541378 2573 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 01 21:40:40.542508 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.541380 2573 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 01 21:40:40.542508 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.541383 2573 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 01 21:40:40.542508 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.541385 2573 feature_gate.go:328] unrecognized feature gate: Example Jun 01 21:40:40.542508 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.541388 2573 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 01 21:40:40.542508 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.541391 2573 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 01 21:40:40.542508 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.541393 2573 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 01 21:40:40.542508 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.541396 2573 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 01 21:40:40.543021 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.541399 2573 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 01 21:40:40.543021 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.541401 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 01 21:40:40.543021 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.541405 2573 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 01 21:40:40.543021 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.541410 2573 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 01 21:40:40.543021 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.541413 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 01 21:40:40.543021 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.541416 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 01 21:40:40.543021 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.541419 2573 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 01 21:40:40.543021 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.541421 2573 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 01 21:40:40.543021 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.541424 2573 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 01 21:40:40.543021 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.541427 2573 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 01 21:40:40.543021 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.541430 2573 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 01 21:40:40.543021 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.541433 2573 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 01 21:40:40.543021 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.541435 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 01 21:40:40.543021 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.541438 2573 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 01 21:40:40.543021 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.541440 2573 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 01 21:40:40.543021 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.541445 2573 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 01 21:40:40.543021 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.541449 2573 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 01 21:40:40.543021 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.541453 2573 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 01 21:40:40.543021 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.541456 2573 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 01 21:40:40.543487 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.541458 2573 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 01 21:40:40.543487 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.541461 2573 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 01 21:40:40.543487 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.541464 2573 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 01 21:40:40.543487 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.541466 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 01 21:40:40.543487 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.541469 2573 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 01 21:40:40.543487 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.541472 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 01 21:40:40.543487 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.541474 2573 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 01 21:40:40.543487 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.541477 2573 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 01 21:40:40.543487 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.541479 2573 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 01 21:40:40.543487 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.541482 2573 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 01 21:40:40.543487 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.541484 2573 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 01 21:40:40.543487 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.541487 2573 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 01 21:40:40.543487 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.541489 2573 feature_gate.go:328] unrecognized feature gate: Example2 Jun 01 21:40:40.543487 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.541492 2573 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 01 21:40:40.543487 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.541495 2573 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 01 21:40:40.543487 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.541498 2573 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS Jun 01 21:40:40.543487 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.541501 2573 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 01 21:40:40.543487 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.541504 2573 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 01 21:40:40.543487 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.541506 2573 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI Jun 01 21:40:40.543487 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.541509 2573 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 01 21:40:40.544018 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:40.541514 2573 feature_gate.go:384] feature gates: {map[DynamicResourceAllocation:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutatingAdmissionPolicy:false NodeSwap:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:false]} Jun 01 21:40:40.544018 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.541625 2573 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 01 21:40:40.544018 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.541629 2573 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 01 21:40:40.544018 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.541632 2573 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 01 21:40:40.544018 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.541636 2573 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 01 21:40:40.544018 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.541638 2573 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 01 21:40:40.544018 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.541642 2573 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 01 21:40:40.544018 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.541645 2573 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 01 21:40:40.544018 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.541647 2573 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 01 21:40:40.544018 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.541651 2573 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController Jun 01 21:40:40.544018 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.541654 2573 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 01 21:40:40.544018 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.541658 2573 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 01 21:40:40.544018 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.541661 2573 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 01 21:40:40.544018 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.541664 2573 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 01 21:40:40.544018 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.541667 2573 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 01 21:40:40.544018 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.541670 2573 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 01 21:40:40.544428 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.541673 2573 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 01 21:40:40.544428 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.541676 2573 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 01 21:40:40.544428 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.541679 2573 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 01 21:40:40.544428 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.541682 2573 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 01 21:40:40.544428 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.541685 2573 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 01 21:40:40.544428 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.541688 2573 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 01 21:40:40.544428 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.541691 2573 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 01 21:40:40.544428 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.541693 2573 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 01 21:40:40.544428 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.541697 2573 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 01 21:40:40.544428 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.541700 2573 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 01 21:40:40.544428 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.541703 2573 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 01 21:40:40.544428 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.541706 2573 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 01 21:40:40.544428 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.541709 2573 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 01 21:40:40.544428 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.541712 2573 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 01 21:40:40.544428 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.541714 2573 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 01 21:40:40.544428 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.541717 2573 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 01 21:40:40.544428 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.541720 2573 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 01 21:40:40.544428 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.541724 2573 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 01 21:40:40.544428 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.541726 2573 feature_gate.go:328] unrecognized feature gate: Example2 Jun 01 21:40:40.544929 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.541730 2573 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 01 21:40:40.544929 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.541734 2573 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 01 21:40:40.544929 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.541737 2573 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 01 21:40:40.544929 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.541740 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 01 21:40:40.544929 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.541743 2573 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 01 21:40:40.544929 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.541746 2573 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 01 21:40:40.544929 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.541749 2573 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 01 21:40:40.544929 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.541752 2573 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 01 21:40:40.544929 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.541755 2573 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 01 21:40:40.544929 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.541759 2573 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 01 21:40:40.544929 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.541761 2573 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 01 21:40:40.544929 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.541764 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 01 21:40:40.544929 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.541766 2573 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 01 21:40:40.544929 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.541769 2573 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 01 21:40:40.544929 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.541772 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 01 21:40:40.544929 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.541774 2573 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 01 21:40:40.544929 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.541777 2573 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 01 21:40:40.544929 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.541779 2573 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 01 21:40:40.544929 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.541782 2573 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 01 21:40:40.545405 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.541784 2573 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI Jun 01 21:40:40.545405 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.541787 2573 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations Jun 01 21:40:40.545405 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.541790 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 01 21:40:40.545405 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.541792 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 01 21:40:40.545405 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.541795 2573 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 01 21:40:40.545405 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.541798 2573 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 01 21:40:40.545405 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.541800 2573 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 01 21:40:40.545405 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.541803 2573 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 01 21:40:40.545405 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.541806 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 01 21:40:40.545405 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.541808 2573 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 01 21:40:40.545405 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.541811 2573 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 01 21:40:40.545405 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.541814 2573 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 01 21:40:40.545405 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.541816 2573 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 01 21:40:40.545405 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.541819 2573 feature_gate.go:328] unrecognized feature gate: Example Jun 01 21:40:40.545405 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.541821 2573 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 01 21:40:40.545405 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.541824 2573 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 01 21:40:40.545405 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.541827 2573 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 01 21:40:40.545405 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.541829 2573 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 01 21:40:40.545405 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.541832 2573 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 01 21:40:40.545405 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.541834 2573 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 01 21:40:40.545912 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.541837 2573 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 01 21:40:40.545912 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.541840 2573 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 01 21:40:40.545912 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.541842 2573 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 01 21:40:40.545912 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.541845 2573 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 01 21:40:40.545912 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.541848 2573 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 01 21:40:40.545912 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.541851 2573 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS Jun 01 21:40:40.545912 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.541853 2573 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 01 21:40:40.545912 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.541856 2573 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 01 21:40:40.545912 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.541858 2573 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 01 21:40:40.545912 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.541861 2573 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 01 21:40:40.545912 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.541863 2573 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 01 21:40:40.545912 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.541866 2573 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 01 21:40:40.545912 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:40.541868 2573 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 01 21:40:40.545912 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:40.541873 2573 feature_gate.go:384] feature gates: {map[DynamicResourceAllocation:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutatingAdmissionPolicy:false NodeSwap:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:false]} Jun 01 21:40:40.545912 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:40.542512 2573 server.go:962] "Client rotation is on, will bootstrap in background" Jun 01 21:40:40.546278 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:40.544456 2573 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jun 01 21:40:40.546278 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:40.545254 2573 server.go:1019] "Starting client certificate rotation" Jun 01 21:40:40.546278 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:40.545361 2573 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jun 01 21:40:40.546278 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:40.545401 2573 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jun 01 21:40:40.564006 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:40.563985 2573 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jun 01 21:40:40.565928 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:40.565911 2573 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jun 01 21:40:40.575172 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:40.575141 2573 log.go:25] "Validated CRI v1 runtime API" Jun 01 21:40:40.582308 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:40.582286 2573 log.go:25] "Validated CRI v1 image API" Jun 01 21:40:40.583436 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:40.583421 2573 server.go:1452] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jun 01 21:40:40.586056 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:40.586032 2573 fs.go:135] Filesystem UUIDs: map[2b640368-2b39-4a21-9382-00bd79d76865:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 991bd01d-76f2-44cb-8919-469745bf10a5:/dev/nvme0n1p3] Jun 01 21:40:40.586134 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:40.586054 2573 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jun 01 21:40:40.590525 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:40.590415 2573 manager.go:217] Machine: {Timestamp:2026-06-01 21:40:40.589898396 +0000 UTC m=+0.309950331 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3104555 MemoryCapacity:33164496896 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec228037a19f056ce299f7bee1bf0057 SystemUUID:ec228037-a19f-056c-e299-f7bee1bf0057 BootID:a4a50c89-ab8c-47d3-a37d-1cfbcb19b608 Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6103040 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:09:a0:37:16:e7 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:09:a0:37:16:e7 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:8a:70:94:cf:fe:ca Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164496896 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jun 01 21:40:40.590911 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:40.590889 2573 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jun 01 21:40:40.591001 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:40.590982 2573 manager.go:233] Version: {KernelVersion:5.14.0-570.117.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260521-1 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jun 01 21:40:40.593568 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:40.593542 2573 container_manager_linux.go:270] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jun 01 21:40:40.593722 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:40.593571 2573 container_manager_linux.go:275] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-138-241.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jun 01 21:40:40.593774 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:40.593729 2573 topology_manager.go:138] "Creating topology manager with none policy" Jun 01 21:40:40.593774 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:40.593737 2573 container_manager_linux.go:306] "Creating device plugin manager" Jun 01 21:40:40.593774 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:40.593745 2573 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jun 01 21:40:40.594433 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:40.594423 2573 server.go:72] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jun 01 21:40:40.595150 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:40.595140 2573 state_mem.go:36] "Initialized new in-memory state store" Jun 01 21:40:40.595260 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:40.595251 2573 server.go:1267] "Using root directory" path="/var/lib/kubelet" Jun 01 21:40:40.595960 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:40.595944 2573 reflector.go:430] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jun 01 21:40:40.597040 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:40.597029 2573 kubelet.go:491] "Attempting to sync node with API server" Jun 01 21:40:40.597089 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:40.597045 2573 kubelet.go:386] "Adding static pod path" path="/etc/kubernetes/manifests" Jun 01 21:40:40.597089 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:40.597057 2573 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jun 01 21:40:40.597089 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:40.597068 2573 kubelet.go:397] "Adding apiserver pod source" Jun 01 21:40:40.597089 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:40.597076 2573 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jun 01 21:40:40.598024 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:40.598011 2573 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jun 01 21:40:40.598079 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:40.598030 2573 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jun 01 21:40:40.600694 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:40.600667 2573 kuberuntime_manager.go:279] "Container runtime initialized" containerRuntime="cri-o" version="1.33.12-3.rhaos4.20.git64c6a00.el9" apiVersion="v1" Jun 01 21:40:40.602240 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:40.602227 2573 kubelet.go:953] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jun 01 21:40:40.603953 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:40.603942 2573 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jun 01 21:40:40.603988 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:40.603959 2573 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jun 01 21:40:40.603988 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:40.603965 2573 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jun 01 21:40:40.603988 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:40.603970 2573 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jun 01 21:40:40.603988 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:40.603984 2573 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jun 01 21:40:40.604136 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:40.603994 2573 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jun 01 21:40:40.604136 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:40.604003 2573 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jun 01 21:40:40.604136 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:40.604008 2573 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jun 01 21:40:40.604136 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:40.604015 2573 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jun 01 21:40:40.604136 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:40.604021 2573 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jun 01 21:40:40.604136 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:40.604029 2573 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jun 01 21:40:40.604136 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:40.604035 2573 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jun 01 21:40:40.605454 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:40.605442 2573 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jun 01 21:40:40.605454 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:40.605454 2573 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/image" Jun 01 21:40:40.608704 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:40.608691 2573 watchdog_linux.go:99] "Systemd watchdog is not enabled" Jun 01 21:40:40.608748 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:40.608731 2573 server.go:1295] "Started kubelet" Jun 01 21:40:40.608885 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:40.608836 2573 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jun 01 21:40:40.608983 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:40.608827 2573 ratelimit.go:55] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jun 01 21:40:40.608983 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:40.608979 2573 server_v1.go:47] "podresources" method="list" useActivePods=true Jun 01 21:40:40.609583 ip-10-0-138-241 systemd[1]: Started Kubernetes Kubelet. Jun 01 21:40:40.609953 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:40.609935 2573 server.go:255] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jun 01 21:40:40.610791 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:40.610779 2573 server.go:317] "Adding debug handlers to kubelet server" Jun 01 21:40:40.611062 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:40.611041 2573 csi_plugin.go:988] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-138-241.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jun 01 21:40:40.611062 ip-10-0-138-241 kubenswrapper[2573]: E0601 21:40:40.611054 2573 reflector.go:200] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-138-241.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jun 01 21:40:40.611186 ip-10-0-138-241 kubenswrapper[2573]: E0601 21:40:40.611060 2573 reflector.go:200] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jun 01 21:40:40.614310 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:40.614295 2573 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jun 01 21:40:40.614397 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:40.614355 2573 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jun 01 21:40:40.614397 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:40.614382 2573 volume_manager.go:295] "The desired_state_of_world populator starts" Jun 01 21:40:40.614397 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:40.614396 2573 volume_manager.go:297] "Starting Kubelet Volume Manager" Jun 01 21:40:40.614612 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:40.614597 2573 desired_state_of_world_populator.go:150] "Desired state populator starts to run" Jun 01 21:40:40.614688 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:40.614666 2573 reconstruct.go:97] "Volume reconstruction finished" Jun 01 21:40:40.614688 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:40.614673 2573 reconciler.go:26] "Reconciler: start to sync state" Jun 01 21:40:40.615775 ip-10-0-138-241 kubenswrapper[2573]: E0601 21:40:40.614879 2573 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-138-241.ec2.internal\" not found" Jun 01 21:40:40.619688 ip-10-0-138-241 kubenswrapper[2573]: E0601 21:40:40.619663 2573 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-138-241.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jun 01 21:40:40.619917 ip-10-0-138-241 kubenswrapper[2573]: E0601 21:40:40.619893 2573 kubelet.go:1618] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jun 01 21:40:40.620350 ip-10-0-138-241 kubenswrapper[2573]: E0601 21:40:40.620220 2573 reflector.go:200] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jun 01 21:40:40.621354 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:40.621334 2573 factory.go:55] Registering systemd factory Jun 01 21:40:40.621354 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:40.621353 2573 factory.go:223] Registration of the systemd container factory successfully Jun 01 21:40:40.621592 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:40.621577 2573 factory.go:153] Registering CRI-O factory Jun 01 21:40:40.621660 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:40.621597 2573 factory.go:223] Registration of the crio container factory successfully Jun 01 21:40:40.621722 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:40.621700 2573 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jun 01 21:40:40.621771 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:40.621722 2573 factory.go:103] Registering Raw factory Jun 01 21:40:40.621771 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:40.621744 2573 manager.go:1196] Started watching for new ooms in manager Jun 01 21:40:40.622715 ip-10-0-138-241 kubenswrapper[2573]: E0601 21:40:40.619890 2573 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-138-241.ec2.internal.18b512ede2558545 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-138-241.ec2.internal,UID:ip-10-0-138-241.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-138-241.ec2.internal,},FirstTimestamp:2026-06-01 21:40:40.608703813 +0000 UTC m=+0.328755748,LastTimestamp:2026-06-01 21:40:40.608703813 +0000 UTC m=+0.328755748,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-138-241.ec2.internal,}" Jun 01 21:40:40.622823 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:40.622781 2573 manager.go:319] Starting recovery of all containers Jun 01 21:40:40.631921 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:40.631777 2573 manager.go:324] Recovery completed Jun 01 21:40:40.636227 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:40.636213 2573 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach" Jun 01 21:40:40.637064 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:40.637043 2573 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-mdz4x" Jun 01 21:40:40.638320 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:40.638305 2573 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-138-241.ec2.internal" event="NodeHasSufficientMemory" Jun 01 21:40:40.638387 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:40.638334 2573 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-138-241.ec2.internal" event="NodeHasNoDiskPressure" Jun 01 21:40:40.638387 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:40.638347 2573 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-138-241.ec2.internal" event="NodeHasSufficientPID" Jun 01 21:40:40.638896 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:40.638882 2573 cpu_manager.go:222] "Starting CPU manager" policy="none" Jun 01 21:40:40.638896 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:40.638896 2573 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jun 01 21:40:40.638983 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:40.638941 2573 state_mem.go:36] "Initialized new in-memory state store" Jun 01 21:40:40.640166 ip-10-0-138-241 kubenswrapper[2573]: E0601 21:40:40.640097 2573 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-138-241.ec2.internal.18b512ede4197030 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-138-241.ec2.internal,UID:ip-10-0-138-241.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-138-241.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-138-241.ec2.internal,},FirstTimestamp:2026-06-01 21:40:40.638320688 +0000 UTC m=+0.358372623,LastTimestamp:2026-06-01 21:40:40.638320688 +0000 UTC m=+0.358372623,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-138-241.ec2.internal,}" Jun 01 21:40:40.640814 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:40.640803 2573 policy_none.go:49] "None policy: Start" Jun 01 21:40:40.640868 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:40.640826 2573 memory_manager.go:186] "Starting memorymanager" policy="None" Jun 01 21:40:40.640868 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:40.640837 2573 state_mem.go:35] "Initializing new in-memory state store" Jun 01 21:40:40.644669 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:40.644652 2573 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-mdz4x" Jun 01 21:40:40.671943 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:40.671916 2573 kubelet_network_linux.go:49] "Initialized iptables rules." protocol="IPv4" Jun 01 21:40:40.673178 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:40.673156 2573 kubelet_network_linux.go:49] "Initialized iptables rules." protocol="IPv6" Jun 01 21:40:40.673178 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:40.673179 2573 status_manager.go:230] "Starting to sync pod status with apiserver" Jun 01 21:40:40.673327 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:40.673196 2573 watchdog_linux.go:127] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jun 01 21:40:40.673327 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:40.673202 2573 kubelet.go:2451] "Starting kubelet main sync loop" Jun 01 21:40:40.673327 ip-10-0-138-241 kubenswrapper[2573]: E0601 21:40:40.673238 2573 kubelet.go:2475] "Skipping pod synchronization" err="[container runtime status check may not have completed yet, PLEG is not healthy: pleg has yet to be successful]" Jun 01 21:40:40.675205 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:40.675185 2573 reflector.go:430] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jun 01 21:40:40.683214 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:40.683200 2573 manager.go:341] "Starting Device Plugin manager" Jun 01 21:40:40.683315 ip-10-0-138-241 kubenswrapper[2573]: E0601 21:40:40.683239 2573 manager.go:517] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jun 01 21:40:40.683315 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:40.683250 2573 server.go:85] "Starting device plugin registration server" Jun 01 21:40:40.683506 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:40.683495 2573 eviction_manager.go:189] "Eviction manager: starting control loop" Jun 01 21:40:40.683557 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:40.683507 2573 container_log_manager.go:189] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jun 01 21:40:40.683654 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:40.683634 2573 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jun 01 21:40:40.683758 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:40.683711 2573 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jun 01 21:40:40.683758 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:40.683721 2573 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jun 01 21:40:40.684210 ip-10-0-138-241 kubenswrapper[2573]: E0601 21:40:40.684191 2573 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jun 01 21:40:40.684272 ip-10-0-138-241 kubenswrapper[2573]: E0601 21:40:40.684235 2573 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-138-241.ec2.internal\" not found" Jun 01 21:40:40.773431 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:40.773323 2573 kubelet.go:2537] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-241.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-138-241.ec2.internal"] Jun 01 21:40:40.773431 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:40.773422 2573 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach" Jun 01 21:40:40.774617 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:40.774596 2573 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-138-241.ec2.internal" event="NodeHasSufficientMemory" Jun 01 21:40:40.774728 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:40.774627 2573 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-138-241.ec2.internal" event="NodeHasNoDiskPressure" Jun 01 21:40:40.774728 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:40.774638 2573 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-138-241.ec2.internal" event="NodeHasSufficientPID" Jun 01 21:40:40.775692 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:40.775680 2573 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach" Jun 01 21:40:40.775857 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:40.775842 2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-241.ec2.internal" Jun 01 21:40:40.775902 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:40.775882 2573 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach" Jun 01 21:40:40.776458 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:40.776441 2573 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-138-241.ec2.internal" event="NodeHasSufficientMemory" Jun 01 21:40:40.776458 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:40.776453 2573 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-138-241.ec2.internal" event="NodeHasSufficientMemory" Jun 01 21:40:40.776604 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:40.776470 2573 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-138-241.ec2.internal" event="NodeHasNoDiskPressure" Jun 01 21:40:40.776604 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:40.776474 2573 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-138-241.ec2.internal" event="NodeHasNoDiskPressure" Jun 01 21:40:40.776604 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:40.776483 2573 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-138-241.ec2.internal" event="NodeHasSufficientPID" Jun 01 21:40:40.776604 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:40.776484 2573 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-138-241.ec2.internal" event="NodeHasSufficientPID" Jun 01 21:40:40.777473 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:40.777459 2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-241.ec2.internal" Jun 01 21:40:40.777514 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:40.777490 2573 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach" Jun 01 21:40:40.778191 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:40.778176 2573 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-138-241.ec2.internal" event="NodeHasSufficientMemory" Jun 01 21:40:40.778267 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:40.778203 2573 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-138-241.ec2.internal" event="NodeHasNoDiskPressure" Jun 01 21:40:40.778267 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:40.778219 2573 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-138-241.ec2.internal" event="NodeHasSufficientPID" Jun 01 21:40:40.784369 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:40.784356 2573 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach" Jun 01 21:40:40.785012 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:40.784997 2573 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-138-241.ec2.internal" event="NodeHasSufficientMemory" Jun 01 21:40:40.785102 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:40.785021 2573 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-138-241.ec2.internal" event="NodeHasNoDiskPressure" Jun 01 21:40:40.785102 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:40.785034 2573 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-138-241.ec2.internal" event="NodeHasSufficientPID" Jun 01 21:40:40.785102 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:40.785067 2573 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-138-241.ec2.internal" Jun 01 21:40:40.794655 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:40.794636 2573 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-138-241.ec2.internal" Jun 01 21:40:40.794738 ip-10-0-138-241 kubenswrapper[2573]: E0601 21:40:40.794662 2573 kubelet_node_status.go:597] "Error updating node status, will retry" err="error getting node \"ip-10-0-138-241.ec2.internal\": node \"ip-10-0-138-241.ec2.internal\" not found" Jun 01 21:40:40.797344 ip-10-0-138-241 kubenswrapper[2573]: E0601 21:40:40.797319 2573 kubelet.go:3336] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-138-241.ec2.internal\" not found" node="ip-10-0-138-241.ec2.internal" Jun 01 21:40:40.801342 ip-10-0-138-241 kubenswrapper[2573]: E0601 21:40:40.801322 2573 kubelet.go:3336] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-138-241.ec2.internal\" not found" node="ip-10-0-138-241.ec2.internal" Jun 01 21:40:40.809380 ip-10-0-138-241 kubenswrapper[2573]: E0601 21:40:40.809359 2573 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-138-241.ec2.internal\" not found" Jun 01 21:40:40.909743 ip-10-0-138-241 kubenswrapper[2573]: E0601 21:40:40.909705 2573 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-138-241.ec2.internal\" not found" Jun 01 21:40:40.916067 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:40.916040 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/79c6196cbb2336b7ddc61a969d8bd0f9-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-138-241.ec2.internal\" (UID: \"79c6196cbb2336b7ddc61a969d8bd0f9\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-241.ec2.internal" Jun 01 21:40:40.916206 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:40.916082 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/79c6196cbb2336b7ddc61a969d8bd0f9-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-138-241.ec2.internal\" (UID: \"79c6196cbb2336b7ddc61a969d8bd0f9\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-241.ec2.internal" Jun 01 21:40:40.916206 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:40.916102 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/bbf2997371dfbc83d81e28332ffdac56-config\") pod \"kube-apiserver-proxy-ip-10-0-138-241.ec2.internal\" (UID: \"bbf2997371dfbc83d81e28332ffdac56\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-138-241.ec2.internal" Jun 01 21:40:41.010196 ip-10-0-138-241 kubenswrapper[2573]: E0601 21:40:41.010160 2573 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-138-241.ec2.internal\" not found" Jun 01 21:40:41.016498 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:41.016471 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/bbf2997371dfbc83d81e28332ffdac56-config\") pod \"kube-apiserver-proxy-ip-10-0-138-241.ec2.internal\" (UID: \"bbf2997371dfbc83d81e28332ffdac56\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-138-241.ec2.internal" Jun 01 21:40:41.016582 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:41.016504 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/79c6196cbb2336b7ddc61a969d8bd0f9-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-138-241.ec2.internal\" (UID: \"79c6196cbb2336b7ddc61a969d8bd0f9\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-241.ec2.internal" Jun 01 21:40:41.016582 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:41.016525 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/79c6196cbb2336b7ddc61a969d8bd0f9-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-138-241.ec2.internal\" (UID: \"79c6196cbb2336b7ddc61a969d8bd0f9\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-241.ec2.internal" Jun 01 21:40:41.016651 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:41.016590 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/79c6196cbb2336b7ddc61a969d8bd0f9-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-138-241.ec2.internal\" (UID: \"79c6196cbb2336b7ddc61a969d8bd0f9\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-241.ec2.internal" Jun 01 21:40:41.016651 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:41.016596 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/bbf2997371dfbc83d81e28332ffdac56-config\") pod \"kube-apiserver-proxy-ip-10-0-138-241.ec2.internal\" (UID: \"bbf2997371dfbc83d81e28332ffdac56\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-138-241.ec2.internal" Jun 01 21:40:41.016651 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:41.016606 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/79c6196cbb2336b7ddc61a969d8bd0f9-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-138-241.ec2.internal\" (UID: \"79c6196cbb2336b7ddc61a969d8bd0f9\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-241.ec2.internal" Jun 01 21:40:41.099844 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:41.099771 2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-241.ec2.internal" Jun 01 21:40:41.104346 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:41.104329 2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-241.ec2.internal" Jun 01 21:40:41.110796 ip-10-0-138-241 kubenswrapper[2573]: E0601 21:40:41.110774 2573 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-138-241.ec2.internal\" not found" Jun 01 21:40:41.211341 ip-10-0-138-241 kubenswrapper[2573]: E0601 21:40:41.211286 2573 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-138-241.ec2.internal\" not found" Jun 01 21:40:41.311777 ip-10-0-138-241 kubenswrapper[2573]: E0601 21:40:41.311749 2573 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-138-241.ec2.internal\" not found" Jun 01 21:40:41.412283 ip-10-0-138-241 kubenswrapper[2573]: E0601 21:40:41.412217 2573 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-138-241.ec2.internal\" not found" Jun 01 21:40:41.512778 ip-10-0-138-241 kubenswrapper[2573]: E0601 21:40:41.512729 2573 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-138-241.ec2.internal\" not found" Jun 01 21:40:41.545203 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:41.545174 2573 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jun 01 21:40:41.545737 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:41.545331 2573 reflector.go:556] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jun 01 21:40:41.613470 ip-10-0-138-241 kubenswrapper[2573]: E0601 21:40:41.613431 2573 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-138-241.ec2.internal\" not found" Jun 01 21:40:41.615874 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:41.615851 2573 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jun 01 21:40:41.633549 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:41.633513 2573 reflector.go:430] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jun 01 21:40:41.645876 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:41.645843 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod79c6196cbb2336b7ddc61a969d8bd0f9.slice/crio-bffd34d9909864d01cf9b4d3c01a371c06b8a8a523fea8b78ec90949c87c5f95 WatchSource:0}: Error finding container bffd34d9909864d01cf9b4d3c01a371c06b8a8a523fea8b78ec90949c87c5f95: Status 404 returned error can't find the container with id bffd34d9909864d01cf9b4d3c01a371c06b8a8a523fea8b78ec90949c87c5f95 Jun 01 21:40:41.646014 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:41.645963 2573 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-05-31 21:35:40 +0000 UTC" deadline="2027-12-24 20:08:37.795918716 +0000 UTC" Jun 01 21:40:41.646014 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:41.645986 2573 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13702h27m56.149935871s" Jun 01 21:40:41.646121 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:41.646085 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbbf2997371dfbc83d81e28332ffdac56.slice/crio-2498749c16a661afa218d5f60106b58ba8b146466ee38c1eeec52580e2d01e65 WatchSource:0}: Error finding container 2498749c16a661afa218d5f60106b58ba8b146466ee38c1eeec52580e2d01e65: Status 404 returned error can't find the container with id 2498749c16a661afa218d5f60106b58ba8b146466ee38c1eeec52580e2d01e65 Jun 01 21:40:41.650132 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:41.650048 2573 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-fnm72" Jun 01 21:40:41.650475 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:41.650458 2573 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jun 01 21:40:41.659364 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:41.659344 2573 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-fnm72" Jun 01 21:40:41.675933 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:41.675838 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-241.ec2.internal" event={"ID":"bbf2997371dfbc83d81e28332ffdac56","Type":"ContainerStarted","Data":"2498749c16a661afa218d5f60106b58ba8b146466ee38c1eeec52580e2d01e65"} Jun 01 21:40:41.676691 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:41.676671 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-241.ec2.internal" event={"ID":"79c6196cbb2336b7ddc61a969d8bd0f9","Type":"ContainerStarted","Data":"bffd34d9909864d01cf9b4d3c01a371c06b8a8a523fea8b78ec90949c87c5f95"} Jun 01 21:40:41.705775 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:41.705745 2573 reflector.go:430] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jun 01 21:40:41.714174 ip-10-0-138-241 kubenswrapper[2573]: E0601 21:40:41.714153 2573 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-138-241.ec2.internal\" not found" Jun 01 21:40:41.748866 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:41.748838 2573 reflector.go:430] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jun 01 21:40:41.814462 ip-10-0-138-241 kubenswrapper[2573]: E0601 21:40:41.814434 2573 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-138-241.ec2.internal\" not found" Jun 01 21:40:41.914880 ip-10-0-138-241 kubenswrapper[2573]: E0601 21:40:41.914838 2573 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-138-241.ec2.internal\" not found" Jun 01 21:40:41.997305 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:41.997225 2573 reflector.go:430] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jun 01 21:40:42.014891 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:42.014864 2573 kubelet.go:3340] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-241.ec2.internal" Jun 01 21:40:42.027566 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:42.027545 2573 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jun 01 21:40:42.028245 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:42.028233 2573 kubelet.go:3340] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-241.ec2.internal" Jun 01 21:40:42.037438 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:42.037410 2573 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jun 01 21:40:42.547881 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:42.547809 2573 reflector.go:430] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jun 01 21:40:42.598472 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:42.598441 2573 apiserver.go:52] "Watching apiserver" Jun 01 21:40:42.608251 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:42.608227 2573 reflector.go:430] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jun 01 21:40:42.608609 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:42.608588 2573 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-multus/multus-additional-cni-plugins-mhq4q","openshift-multus/network-metrics-daemon-w4d7s","openshift-network-diagnostics/network-check-target-z7k6x","kube-system/kube-apiserver-proxy-ip-10-0-138-241.ec2.internal","openshift-cluster-node-tuning-operator/tuned-bfz2l","openshift-dns/node-resolver-kfs2m","openshift-network-operator/iptables-alerter-c88gr","openshift-ovn-kubernetes/ovnkube-node-rcqkp","kube-system/konnectivity-agent-mwdhn","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c2qnd","openshift-image-registry/node-ca-qcs4d","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-241.ec2.internal","openshift-multus/multus-4vd5l"] Jun 01 21:40:42.610475 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:42.610454 2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-mwdhn" Jun 01 21:40:42.611390 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:42.611367 2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w4d7s" Jun 01 21:40:42.611501 ip-10-0-138-241 kubenswrapper[2573]: E0601 21:40:42.611428 2573 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-w4d7s" podUID="f99cc923-cc5c-4895-8b2b-6ba878ba343b" Jun 01 21:40:42.612229 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:42.612215 2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z7k6x" Jun 01 21:40:42.612274 ip-10-0-138-241 kubenswrapper[2573]: E0601 21:40:42.612260 2573 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z7k6x" podUID="e2e547dd-415d-46fe-9608-4a3410d73834" Jun 01 21:40:42.613149 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:42.613126 2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-qcs4d" Jun 01 21:40:42.613306 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:42.613260 2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-bfz2l" Jun 01 21:40:42.613739 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:42.613716 2573 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jun 01 21:40:42.614251 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:42.614229 2573 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jun 01 21:40:42.614337 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:42.614270 2573 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-knjbp\"" Jun 01 21:40:42.616603 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:42.616103 2573 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jun 01 21:40:42.616603 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:42.616261 2573 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jun 01 21:40:42.616603 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:42.616396 2573 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-sxqgv\"" Jun 01 21:40:42.616603 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:42.616503 2573 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jun 01 21:40:42.617050 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:42.617013 2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c2qnd" Jun 01 21:40:42.617149 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:42.617135 2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-c88gr" Jun 01 21:40:42.617208 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:42.617148 2573 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jun 01 21:40:42.618783 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:42.618764 2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-rcqkp" Jun 01 21:40:42.618867 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:42.618844 2573 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-j72rf\"" Jun 01 21:40:42.619392 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:42.619373 2573 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jun 01 21:40:42.619786 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:42.619766 2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-kfs2m" Jun 01 21:40:42.620030 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:42.620016 2573 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jun 01 21:40:42.620129 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:42.620104 2573 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jun 01 21:40:42.620177 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:42.620146 2573 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-2rk7v\"" Jun 01 21:40:42.620388 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:42.620372 2573 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jun 01 21:40:42.620758 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:42.620743 2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-mhq4q" Jun 01 21:40:42.621334 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:42.621318 2573 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jun 01 21:40:42.621590 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:42.621568 2573 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-x8v9k\"" Jun 01 21:40:42.621687 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:42.621597 2573 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jun 01 21:40:42.621687 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:42.621619 2573 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jun 01 21:40:42.621687 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:42.621683 2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-4vd5l" Jun 01 21:40:42.622041 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:42.622016 2573 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jun 01 21:40:42.622041 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:42.622023 2573 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jun 01 21:40:42.622171 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:42.622074 2573 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jun 01 21:40:42.623293 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:42.623275 2573 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jun 01 21:40:42.623354 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:42.623280 2573 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-xj59c\"" Jun 01 21:40:42.623388 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:42.623299 2573 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jun 01 21:40:42.623631 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:42.623594 2573 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jun 01 21:40:42.623719 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:42.623709 2573 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jun 01 21:40:42.623792 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:42.623782 2573 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-p8hrv\"" Jun 01 21:40:42.623849 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:42.623789 2573 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jun 01 21:40:42.623849 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:42.623819 2573 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-vfn77\"" Jun 01 21:40:42.623931 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:42.623874 2573 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jun 01 21:40:42.624560 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:42.624523 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7dqw7\" (UniqueName: \"kubernetes.io/projected/e2e547dd-415d-46fe-9608-4a3410d73834-kube-api-access-7dqw7\") pod \"network-check-target-z7k6x\" (UID: \"e2e547dd-415d-46fe-9608-4a3410d73834\") " pod="openshift-network-diagnostics/network-check-target-z7k6x" Jun 01 21:40:42.624655 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:42.624562 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/7ada00c1-dbeb-4df5-bfca-923ef619c2d1-host\") pod \"node-ca-qcs4d\" (UID: \"7ada00c1-dbeb-4df5-bfca-923ef619c2d1\") " pod="openshift-image-registry/node-ca-qcs4d" Jun 01 21:40:42.624655 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:42.624578 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c2491d2d-ab6b-422e-af1d-41cc31129778-tmp\") pod \"tuned-bfz2l\" (UID: \"c2491d2d-ab6b-422e-af1d-41cc31129778\") " pod="openshift-cluster-node-tuning-operator/tuned-bfz2l" Jun 01 21:40:42.624655 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:42.624595 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/bd935121-781a-4420-80da-4b28e731a602-kubelet-dir\") pod \"aws-ebs-csi-driver-node-c2qnd\" (UID: \"bd935121-781a-4420-80da-4b28e731a602\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c2qnd" Jun 01 21:40:42.624655 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:42.624610 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/bd935121-781a-4420-80da-4b28e731a602-socket-dir\") pod \"aws-ebs-csi-driver-node-c2qnd\" (UID: \"bd935121-781a-4420-80da-4b28e731a602\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c2qnd" Jun 01 21:40:42.624655 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:42.624626 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fthss\" (UniqueName: \"kubernetes.io/projected/f99cc923-cc5c-4895-8b2b-6ba878ba343b-kube-api-access-fthss\") pod \"network-metrics-daemon-w4d7s\" (UID: \"f99cc923-cc5c-4895-8b2b-6ba878ba343b\") " pod="openshift-multus/network-metrics-daemon-w4d7s" Jun 01 21:40:42.624655 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:42.624641 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qk9pm\" (UniqueName: \"kubernetes.io/projected/7ada00c1-dbeb-4df5-bfca-923ef619c2d1-kube-api-access-qk9pm\") pod \"node-ca-qcs4d\" (UID: \"7ada00c1-dbeb-4df5-bfca-923ef619c2d1\") " pod="openshift-image-registry/node-ca-qcs4d" Jun 01 21:40:42.624976 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:42.624674 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cbmtr\" (UniqueName: \"kubernetes.io/projected/c2491d2d-ab6b-422e-af1d-41cc31129778-kube-api-access-cbmtr\") pod \"tuned-bfz2l\" (UID: \"c2491d2d-ab6b-422e-af1d-41cc31129778\") " pod="openshift-cluster-node-tuning-operator/tuned-bfz2l" Jun 01 21:40:42.624976 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:42.624703 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/bd935121-781a-4420-80da-4b28e731a602-etc-selinux\") pod \"aws-ebs-csi-driver-node-c2qnd\" (UID: \"bd935121-781a-4420-80da-4b28e731a602\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c2qnd" Jun 01 21:40:42.624976 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:42.624753 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/bd935121-781a-4420-80da-4b28e731a602-sys-fs\") pod \"aws-ebs-csi-driver-node-c2qnd\" (UID: \"bd935121-781a-4420-80da-4b28e731a602\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c2qnd" Jun 01 21:40:42.624976 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:42.624777 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f99cc923-cc5c-4895-8b2b-6ba878ba343b-metrics-certs\") pod \"network-metrics-daemon-w4d7s\" (UID: \"f99cc923-cc5c-4895-8b2b-6ba878ba343b\") " pod="openshift-multus/network-metrics-daemon-w4d7s" Jun 01 21:40:42.624976 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:42.624793 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/c2491d2d-ab6b-422e-af1d-41cc31129778-etc-modprobe-d\") pod \"tuned-bfz2l\" (UID: \"c2491d2d-ab6b-422e-af1d-41cc31129778\") " pod="openshift-cluster-node-tuning-operator/tuned-bfz2l" Jun 01 21:40:42.624976 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:42.624809 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/c2491d2d-ab6b-422e-af1d-41cc31129778-etc-sysctl-d\") pod \"tuned-bfz2l\" (UID: \"c2491d2d-ab6b-422e-af1d-41cc31129778\") " pod="openshift-cluster-node-tuning-operator/tuned-bfz2l" Jun 01 21:40:42.624976 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:42.624837 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/c2491d2d-ab6b-422e-af1d-41cc31129778-run\") pod \"tuned-bfz2l\" (UID: \"c2491d2d-ab6b-422e-af1d-41cc31129778\") " pod="openshift-cluster-node-tuning-operator/tuned-bfz2l" Jun 01 21:40:42.624976 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:42.624889 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c2491d2d-ab6b-422e-af1d-41cc31129778-var-lib-kubelet\") pod \"tuned-bfz2l\" (UID: \"c2491d2d-ab6b-422e-af1d-41cc31129778\") " pod="openshift-cluster-node-tuning-operator/tuned-bfz2l" Jun 01 21:40:42.624976 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:42.624903 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c2491d2d-ab6b-422e-af1d-41cc31129778-host\") pod \"tuned-bfz2l\" (UID: \"c2491d2d-ab6b-422e-af1d-41cc31129778\") " pod="openshift-cluster-node-tuning-operator/tuned-bfz2l" Jun 01 21:40:42.624976 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:42.624919 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/fbd5149c-c44e-4b0d-959f-a8c158933a5c-iptables-alerter-script\") pod \"iptables-alerter-c88gr\" (UID: \"fbd5149c-c44e-4b0d-959f-a8c158933a5c\") " pod="openshift-network-operator/iptables-alerter-c88gr" Jun 01 21:40:42.624976 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:42.624946 2573 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jun 01 21:40:42.624976 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:42.624957 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c2491d2d-ab6b-422e-af1d-41cc31129778-etc-kubernetes\") pod \"tuned-bfz2l\" (UID: \"c2491d2d-ab6b-422e-af1d-41cc31129778\") " pod="openshift-cluster-node-tuning-operator/tuned-bfz2l" Jun 01 21:40:42.625614 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:42.624984 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/c2491d2d-ab6b-422e-af1d-41cc31129778-etc-sysctl-conf\") pod \"tuned-bfz2l\" (UID: \"c2491d2d-ab6b-422e-af1d-41cc31129778\") " pod="openshift-cluster-node-tuning-operator/tuned-bfz2l" Jun 01 21:40:42.625614 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:42.625009 2573 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jun 01 21:40:42.625614 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:42.625016 2573 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jun 01 21:40:42.625614 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:42.625007 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/c2491d2d-ab6b-422e-af1d-41cc31129778-etc-systemd\") pod \"tuned-bfz2l\" (UID: \"c2491d2d-ab6b-422e-af1d-41cc31129778\") " pod="openshift-cluster-node-tuning-operator/tuned-bfz2l" Jun 01 21:40:42.625614 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:42.625086 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c2491d2d-ab6b-422e-af1d-41cc31129778-lib-modules\") pod \"tuned-bfz2l\" (UID: \"c2491d2d-ab6b-422e-af1d-41cc31129778\") " pod="openshift-cluster-node-tuning-operator/tuned-bfz2l" Jun 01 21:40:42.625614 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:42.625099 2573 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jun 01 21:40:42.625614 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:42.625135 2573 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-kzlsv\"" Jun 01 21:40:42.625614 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:42.625137 2573 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jun 01 21:40:42.625614 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:42.625111 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rmtgv\" (UniqueName: \"kubernetes.io/projected/bd935121-781a-4420-80da-4b28e731a602-kube-api-access-rmtgv\") pod \"aws-ebs-csi-driver-node-c2qnd\" (UID: \"bd935121-781a-4420-80da-4b28e731a602\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c2qnd" Jun 01 21:40:42.625614 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:42.625191 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lqhv8\" (UniqueName: \"kubernetes.io/projected/fbd5149c-c44e-4b0d-959f-a8c158933a5c-kube-api-access-lqhv8\") pod \"iptables-alerter-c88gr\" (UID: \"fbd5149c-c44e-4b0d-959f-a8c158933a5c\") " pod="openshift-network-operator/iptables-alerter-c88gr" Jun 01 21:40:42.625614 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:42.625224 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/c2491d2d-ab6b-422e-af1d-41cc31129778-etc-sysconfig\") pod \"tuned-bfz2l\" (UID: \"c2491d2d-ab6b-422e-af1d-41cc31129778\") " pod="openshift-cluster-node-tuning-operator/tuned-bfz2l" Jun 01 21:40:42.625614 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:42.625249 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c2491d2d-ab6b-422e-af1d-41cc31129778-sys\") pod \"tuned-bfz2l\" (UID: \"c2491d2d-ab6b-422e-af1d-41cc31129778\") " pod="openshift-cluster-node-tuning-operator/tuned-bfz2l" Jun 01 21:40:42.625614 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:42.625272 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/c2491d2d-ab6b-422e-af1d-41cc31129778-etc-tuned\") pod \"tuned-bfz2l\" (UID: \"c2491d2d-ab6b-422e-af1d-41cc31129778\") " pod="openshift-cluster-node-tuning-operator/tuned-bfz2l" Jun 01 21:40:42.625614 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:42.625306 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/bd935121-781a-4420-80da-4b28e731a602-device-dir\") pod \"aws-ebs-csi-driver-node-c2qnd\" (UID: \"bd935121-781a-4420-80da-4b28e731a602\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c2qnd" Jun 01 21:40:42.625614 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:42.625326 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/9a3c0ed4-56c7-4f73-a903-7b8fe4652a41-konnectivity-ca\") pod \"konnectivity-agent-mwdhn\" (UID: \"9a3c0ed4-56c7-4f73-a903-7b8fe4652a41\") " pod="kube-system/konnectivity-agent-mwdhn" Jun 01 21:40:42.625614 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:42.625344 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/fbd5149c-c44e-4b0d-959f-a8c158933a5c-host-slash\") pod \"iptables-alerter-c88gr\" (UID: \"fbd5149c-c44e-4b0d-959f-a8c158933a5c\") " pod="openshift-network-operator/iptables-alerter-c88gr" Jun 01 21:40:42.625614 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:42.625364 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/bd935121-781a-4420-80da-4b28e731a602-registration-dir\") pod \"aws-ebs-csi-driver-node-c2qnd\" (UID: \"bd935121-781a-4420-80da-4b28e731a602\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c2qnd" Jun 01 21:40:42.625614 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:42.625409 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/7ada00c1-dbeb-4df5-bfca-923ef619c2d1-serviceca\") pod \"node-ca-qcs4d\" (UID: \"7ada00c1-dbeb-4df5-bfca-923ef619c2d1\") " pod="openshift-image-registry/node-ca-qcs4d" Jun 01 21:40:42.625614 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:42.625428 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/9a3c0ed4-56c7-4f73-a903-7b8fe4652a41-agent-certs\") pod \"konnectivity-agent-mwdhn\" (UID: \"9a3c0ed4-56c7-4f73-a903-7b8fe4652a41\") " pod="kube-system/konnectivity-agent-mwdhn" Jun 01 21:40:42.660106 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:42.660068 2573 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-05-31 21:35:41 +0000 UTC" deadline="2028-02-22 09:05:37.767784832 +0000 UTC" Jun 01 21:40:42.660106 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:42.660099 2573 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15131h24m55.107689732s" Jun 01 21:40:42.716420 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:42.716386 2573 desired_state_of_world_populator.go:158] "Finished populating initial desired state of world" Jun 01 21:40:42.726106 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:42.725673 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c2491d2d-ab6b-422e-af1d-41cc31129778-var-lib-kubelet\") pod \"tuned-bfz2l\" (UID: \"c2491d2d-ab6b-422e-af1d-41cc31129778\") " pod="openshift-cluster-node-tuning-operator/tuned-bfz2l" Jun 01 21:40:42.726106 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:42.725713 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c2491d2d-ab6b-422e-af1d-41cc31129778-host\") pod \"tuned-bfz2l\" (UID: \"c2491d2d-ab6b-422e-af1d-41cc31129778\") " pod="openshift-cluster-node-tuning-operator/tuned-bfz2l" Jun 01 21:40:42.726106 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:42.725738 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/fbd5149c-c44e-4b0d-959f-a8c158933a5c-iptables-alerter-script\") pod \"iptables-alerter-c88gr\" (UID: \"fbd5149c-c44e-4b0d-959f-a8c158933a5c\") " pod="openshift-network-operator/iptables-alerter-c88gr" Jun 01 21:40:42.726106 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:42.725768 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/0192780a-312c-4eae-a683-ca316fa48cb2-host-run-multus-certs\") pod \"multus-4vd5l\" (UID: \"0192780a-312c-4eae-a683-ca316fa48cb2\") " pod="openshift-multus/multus-4vd5l" Jun 01 21:40:42.726106 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:42.725794 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c2491d2d-ab6b-422e-af1d-41cc31129778-etc-kubernetes\") pod \"tuned-bfz2l\" (UID: \"c2491d2d-ab6b-422e-af1d-41cc31129778\") " pod="openshift-cluster-node-tuning-operator/tuned-bfz2l" Jun 01 21:40:42.726106 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:42.725817 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c2491d2d-ab6b-422e-af1d-41cc31129778-lib-modules\") pod \"tuned-bfz2l\" (UID: \"c2491d2d-ab6b-422e-af1d-41cc31129778\") " pod="openshift-cluster-node-tuning-operator/tuned-bfz2l" Jun 01 21:40:42.726106 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:42.725840 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c2491d2d-ab6b-422e-af1d-41cc31129778-host\") pod \"tuned-bfz2l\" (UID: \"c2491d2d-ab6b-422e-af1d-41cc31129778\") " pod="openshift-cluster-node-tuning-operator/tuned-bfz2l" Jun 01 21:40:42.726106 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:42.725843 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-lqhv8\" (UniqueName: \"kubernetes.io/projected/fbd5149c-c44e-4b0d-959f-a8c158933a5c-kube-api-access-lqhv8\") pod \"iptables-alerter-c88gr\" (UID: \"fbd5149c-c44e-4b0d-959f-a8c158933a5c\") " pod="openshift-network-operator/iptables-alerter-c88gr" Jun 01 21:40:42.726106 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:42.725889 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0192780a-312c-4eae-a683-ca316fa48cb2-multus-cni-dir\") pod \"multus-4vd5l\" (UID: \"0192780a-312c-4eae-a683-ca316fa48cb2\") " pod="openshift-multus/multus-4vd5l" Jun 01 21:40:42.726106 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:42.725892 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c2491d2d-ab6b-422e-af1d-41cc31129778-var-lib-kubelet\") pod \"tuned-bfz2l\" (UID: \"c2491d2d-ab6b-422e-af1d-41cc31129778\") " pod="openshift-cluster-node-tuning-operator/tuned-bfz2l" Jun 01 21:40:42.726106 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:42.725914 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/0192780a-312c-4eae-a683-ca316fa48cb2-os-release\") pod \"multus-4vd5l\" (UID: \"0192780a-312c-4eae-a683-ca316fa48cb2\") " pod="openshift-multus/multus-4vd5l" Jun 01 21:40:42.726106 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:42.725939 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sdn65\" (UniqueName: \"kubernetes.io/projected/bcc44458-cf91-4d2d-9b1b-e9da2db57098-kube-api-access-sdn65\") pod \"node-resolver-kfs2m\" (UID: \"bcc44458-cf91-4d2d-9b1b-e9da2db57098\") " pod="openshift-dns/node-resolver-kfs2m" Jun 01 21:40:42.726106 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:42.725961 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c2491d2d-ab6b-422e-af1d-41cc31129778-etc-kubernetes\") pod \"tuned-bfz2l\" (UID: \"c2491d2d-ab6b-422e-af1d-41cc31129778\") " pod="openshift-cluster-node-tuning-operator/tuned-bfz2l" Jun 01 21:40:42.726106 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:42.725980 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fcf0209a-2440-4267-834a-b176f4e337e4-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-rcqkp\" (UID: \"fcf0209a-2440-4267-834a-b176f4e337e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-rcqkp" Jun 01 21:40:42.726106 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:42.726012 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lw7gz\" (UniqueName: \"kubernetes.io/projected/fcf0209a-2440-4267-834a-b176f4e337e4-kube-api-access-lw7gz\") pod \"ovnkube-node-rcqkp\" (UID: \"fcf0209a-2440-4267-834a-b176f4e337e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-rcqkp" Jun 01 21:40:42.726106 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:42.726044 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/c2491d2d-ab6b-422e-af1d-41cc31129778-etc-sysconfig\") pod \"tuned-bfz2l\" (UID: \"c2491d2d-ab6b-422e-af1d-41cc31129778\") " pod="openshift-cluster-node-tuning-operator/tuned-bfz2l" Jun 01 21:40:42.726106 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:42.726067 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c2491d2d-ab6b-422e-af1d-41cc31129778-sys\") pod \"tuned-bfz2l\" (UID: \"c2491d2d-ab6b-422e-af1d-41cc31129778\") " pod="openshift-cluster-node-tuning-operator/tuned-bfz2l" Jun 01 21:40:42.727024 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:42.726090 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c2491d2d-ab6b-422e-af1d-41cc31129778-lib-modules\") pod \"tuned-bfz2l\" (UID: \"c2491d2d-ab6b-422e-af1d-41cc31129778\") " pod="openshift-cluster-node-tuning-operator/tuned-bfz2l" Jun 01 21:40:42.727024 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:42.726113 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/bd935121-781a-4420-80da-4b28e731a602-device-dir\") pod \"aws-ebs-csi-driver-node-c2qnd\" (UID: \"bd935121-781a-4420-80da-4b28e731a602\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c2qnd" Jun 01 21:40:42.727024 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:42.726137 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/fbd5149c-c44e-4b0d-959f-a8c158933a5c-host-slash\") pod \"iptables-alerter-c88gr\" (UID: \"fbd5149c-c44e-4b0d-959f-a8c158933a5c\") " pod="openshift-network-operator/iptables-alerter-c88gr" Jun 01 21:40:42.727024 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:42.726143 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/c2491d2d-ab6b-422e-af1d-41cc31129778-etc-sysconfig\") pod \"tuned-bfz2l\" (UID: \"c2491d2d-ab6b-422e-af1d-41cc31129778\") " pod="openshift-cluster-node-tuning-operator/tuned-bfz2l" Jun 01 21:40:42.727024 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:42.726168 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0192780a-312c-4eae-a683-ca316fa48cb2-etc-kubernetes\") pod \"multus-4vd5l\" (UID: \"0192780a-312c-4eae-a683-ca316fa48cb2\") " pod="openshift-multus/multus-4vd5l" Jun 01 21:40:42.727024 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:42.726186 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/bd935121-781a-4420-80da-4b28e731a602-device-dir\") pod \"aws-ebs-csi-driver-node-c2qnd\" (UID: \"bd935121-781a-4420-80da-4b28e731a602\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c2qnd" Jun 01 21:40:42.727024 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:42.726191 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/36aa25cf-725a-42f0-8f7b-938f5dbe52d4-cnibin\") pod \"multus-additional-cni-plugins-mhq4q\" (UID: \"36aa25cf-725a-42f0-8f7b-938f5dbe52d4\") " pod="openshift-multus/multus-additional-cni-plugins-mhq4q" Jun 01 21:40:42.727024 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:42.726197 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c2491d2d-ab6b-422e-af1d-41cc31129778-sys\") pod \"tuned-bfz2l\" (UID: \"c2491d2d-ab6b-422e-af1d-41cc31129778\") " pod="openshift-cluster-node-tuning-operator/tuned-bfz2l" Jun 01 21:40:42.727024 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:42.726232 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/fbd5149c-c44e-4b0d-959f-a8c158933a5c-host-slash\") pod \"iptables-alerter-c88gr\" (UID: \"fbd5149c-c44e-4b0d-959f-a8c158933a5c\") " pod="openshift-network-operator/iptables-alerter-c88gr" Jun 01 21:40:42.727024 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:42.726243 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/36aa25cf-725a-42f0-8f7b-938f5dbe52d4-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-mhq4q\" (UID: \"36aa25cf-725a-42f0-8f7b-938f5dbe52d4\") " pod="openshift-multus/multus-additional-cni-plugins-mhq4q" Jun 01 21:40:42.727024 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:42.726240 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/fbd5149c-c44e-4b0d-959f-a8c158933a5c-iptables-alerter-script\") pod \"iptables-alerter-c88gr\" (UID: \"fbd5149c-c44e-4b0d-959f-a8c158933a5c\") " pod="openshift-network-operator/iptables-alerter-c88gr" Jun 01 21:40:42.727024 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:42.726278 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/36aa25cf-725a-42f0-8f7b-938f5dbe52d4-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-mhq4q\" (UID: \"36aa25cf-725a-42f0-8f7b-938f5dbe52d4\") " pod="openshift-multus/multus-additional-cni-plugins-mhq4q" Jun 01 21:40:42.727024 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:42.726314 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jc4mz\" (UniqueName: \"kubernetes.io/projected/36aa25cf-725a-42f0-8f7b-938f5dbe52d4-kube-api-access-jc4mz\") pod \"multus-additional-cni-plugins-mhq4q\" (UID: \"36aa25cf-725a-42f0-8f7b-938f5dbe52d4\") " pod="openshift-multus/multus-additional-cni-plugins-mhq4q" Jun 01 21:40:42.727024 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:42.726341 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/fcf0209a-2440-4267-834a-b176f4e337e4-host-kubelet\") pod \"ovnkube-node-rcqkp\" (UID: \"fcf0209a-2440-4267-834a-b176f4e337e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-rcqkp" Jun 01 21:40:42.727024 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:42.726365 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/fcf0209a-2440-4267-834a-b176f4e337e4-host-slash\") pod \"ovnkube-node-rcqkp\" (UID: \"fcf0209a-2440-4267-834a-b176f4e337e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-rcqkp" Jun 01 21:40:42.727024 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:42.726403 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/fcf0209a-2440-4267-834a-b176f4e337e4-host-run-netns\") pod \"ovnkube-node-rcqkp\" (UID: \"fcf0209a-2440-4267-834a-b176f4e337e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-rcqkp" Jun 01 21:40:42.727024 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:42.726427 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/fcf0209a-2440-4267-834a-b176f4e337e4-run-systemd\") pod \"ovnkube-node-rcqkp\" (UID: \"fcf0209a-2440-4267-834a-b176f4e337e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-rcqkp" Jun 01 21:40:42.727773 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:42.726452 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/fcf0209a-2440-4267-834a-b176f4e337e4-var-lib-openvswitch\") pod \"ovnkube-node-rcqkp\" (UID: \"fcf0209a-2440-4267-834a-b176f4e337e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-rcqkp" Jun 01 21:40:42.727773 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:42.726474 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/fcf0209a-2440-4267-834a-b176f4e337e4-env-overrides\") pod \"ovnkube-node-rcqkp\" (UID: \"fcf0209a-2440-4267-834a-b176f4e337e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-rcqkp" Jun 01 21:40:42.727773 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:42.726489 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/fcf0209a-2440-4267-834a-b176f4e337e4-ovn-node-metrics-cert\") pod \"ovnkube-node-rcqkp\" (UID: \"fcf0209a-2440-4267-834a-b176f4e337e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-rcqkp" Jun 01 21:40:42.727773 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:42.726528 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/7ada00c1-dbeb-4df5-bfca-923ef619c2d1-serviceca\") pod \"node-ca-qcs4d\" (UID: \"7ada00c1-dbeb-4df5-bfca-923ef619c2d1\") " pod="openshift-image-registry/node-ca-qcs4d" Jun 01 21:40:42.727773 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:42.726572 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/0192780a-312c-4eae-a683-ca316fa48cb2-multus-socket-dir-parent\") pod \"multus-4vd5l\" (UID: \"0192780a-312c-4eae-a683-ca316fa48cb2\") " pod="openshift-multus/multus-4vd5l" Jun 01 21:40:42.727773 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:42.726589 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/bcc44458-cf91-4d2d-9b1b-e9da2db57098-hosts-file\") pod \"node-resolver-kfs2m\" (UID: \"bcc44458-cf91-4d2d-9b1b-e9da2db57098\") " pod="openshift-dns/node-resolver-kfs2m" Jun 01 21:40:42.727773 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:42.726626 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/fcf0209a-2440-4267-834a-b176f4e337e4-run-openvswitch\") pod \"ovnkube-node-rcqkp\" (UID: \"fcf0209a-2440-4267-834a-b176f4e337e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-rcqkp" Jun 01 21:40:42.727773 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:42.726644 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0192780a-312c-4eae-a683-ca316fa48cb2-system-cni-dir\") pod \"multus-4vd5l\" (UID: \"0192780a-312c-4eae-a683-ca316fa48cb2\") " pod="openshift-multus/multus-4vd5l" Jun 01 21:40:42.727773 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:42.726664 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/0192780a-312c-4eae-a683-ca316fa48cb2-host-run-k8s-cni-cncf-io\") pod \"multus-4vd5l\" (UID: \"0192780a-312c-4eae-a683-ca316fa48cb2\") " pod="openshift-multus/multus-4vd5l" Jun 01 21:40:42.727773 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:42.726813 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bcc44458-cf91-4d2d-9b1b-e9da2db57098-tmp-dir\") pod \"node-resolver-kfs2m\" (UID: \"bcc44458-cf91-4d2d-9b1b-e9da2db57098\") " pod="openshift-dns/node-resolver-kfs2m" Jun 01 21:40:42.727773 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:42.726853 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c2491d2d-ab6b-422e-af1d-41cc31129778-tmp\") pod \"tuned-bfz2l\" (UID: \"c2491d2d-ab6b-422e-af1d-41cc31129778\") " pod="openshift-cluster-node-tuning-operator/tuned-bfz2l" Jun 01 21:40:42.727773 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:42.726891 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/fcf0209a-2440-4267-834a-b176f4e337e4-host-cni-bin\") pod \"ovnkube-node-rcqkp\" (UID: \"fcf0209a-2440-4267-834a-b176f4e337e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-rcqkp" Jun 01 21:40:42.727773 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:42.726999 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/bd935121-781a-4420-80da-4b28e731a602-etc-selinux\") pod \"aws-ebs-csi-driver-node-c2qnd\" (UID: \"bd935121-781a-4420-80da-4b28e731a602\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c2qnd" Jun 01 21:40:42.727773 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:42.727045 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/0192780a-312c-4eae-a683-ca316fa48cb2-hostroot\") pod \"multus-4vd5l\" (UID: \"0192780a-312c-4eae-a683-ca316fa48cb2\") " pod="openshift-multus/multus-4vd5l" Jun 01 21:40:42.727773 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:42.727057 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/7ada00c1-dbeb-4df5-bfca-923ef619c2d1-serviceca\") pod \"node-ca-qcs4d\" (UID: \"7ada00c1-dbeb-4df5-bfca-923ef619c2d1\") " pod="openshift-image-registry/node-ca-qcs4d" Jun 01 21:40:42.727773 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:42.727071 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/0192780a-312c-4eae-a683-ca316fa48cb2-multus-conf-dir\") pod \"multus-4vd5l\" (UID: \"0192780a-312c-4eae-a683-ca316fa48cb2\") " pod="openshift-multus/multus-4vd5l" Jun 01 21:40:42.727773 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:42.727097 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/fcf0209a-2440-4267-834a-b176f4e337e4-run-ovn\") pod \"ovnkube-node-rcqkp\" (UID: \"fcf0209a-2440-4267-834a-b176f4e337e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-rcqkp" Jun 01 21:40:42.728423 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:42.727132 2573 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jun 01 21:40:42.728423 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:42.727146 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/c2491d2d-ab6b-422e-af1d-41cc31129778-etc-sysctl-conf\") pod \"tuned-bfz2l\" (UID: \"c2491d2d-ab6b-422e-af1d-41cc31129778\") " pod="openshift-cluster-node-tuning-operator/tuned-bfz2l" Jun 01 21:40:42.728423 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:42.727171 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/bd935121-781a-4420-80da-4b28e731a602-etc-selinux\") pod \"aws-ebs-csi-driver-node-c2qnd\" (UID: \"bd935121-781a-4420-80da-4b28e731a602\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c2qnd" Jun 01 21:40:42.728423 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:42.727177 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/c2491d2d-ab6b-422e-af1d-41cc31129778-etc-systemd\") pod \"tuned-bfz2l\" (UID: \"c2491d2d-ab6b-422e-af1d-41cc31129778\") " pod="openshift-cluster-node-tuning-operator/tuned-bfz2l" Jun 01 21:40:42.728423 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:42.727222 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/c2491d2d-ab6b-422e-af1d-41cc31129778-etc-systemd\") pod \"tuned-bfz2l\" (UID: \"c2491d2d-ab6b-422e-af1d-41cc31129778\") " pod="openshift-cluster-node-tuning-operator/tuned-bfz2l" Jun 01 21:40:42.728423 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:42.727280 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-rmtgv\" (UniqueName: \"kubernetes.io/projected/bd935121-781a-4420-80da-4b28e731a602-kube-api-access-rmtgv\") pod \"aws-ebs-csi-driver-node-c2qnd\" (UID: \"bd935121-781a-4420-80da-4b28e731a602\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c2qnd" Jun 01 21:40:42.728423 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:42.727324 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0192780a-312c-4eae-a683-ca316fa48cb2-host-var-lib-kubelet\") pod \"multus-4vd5l\" (UID: \"0192780a-312c-4eae-a683-ca316fa48cb2\") " pod="openshift-multus/multus-4vd5l" Jun 01 21:40:42.728423 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:42.727328 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/c2491d2d-ab6b-422e-af1d-41cc31129778-etc-sysctl-conf\") pod \"tuned-bfz2l\" (UID: \"c2491d2d-ab6b-422e-af1d-41cc31129778\") " pod="openshift-cluster-node-tuning-operator/tuned-bfz2l" Jun 01 21:40:42.728423 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:42.727359 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/36aa25cf-725a-42f0-8f7b-938f5dbe52d4-tuning-conf-dir\") pod \"multus-additional-cni-plugins-mhq4q\" (UID: \"36aa25cf-725a-42f0-8f7b-938f5dbe52d4\") " pod="openshift-multus/multus-additional-cni-plugins-mhq4q" Jun 01 21:40:42.728423 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:42.727391 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/c2491d2d-ab6b-422e-af1d-41cc31129778-etc-tuned\") pod \"tuned-bfz2l\" (UID: \"c2491d2d-ab6b-422e-af1d-41cc31129778\") " pod="openshift-cluster-node-tuning-operator/tuned-bfz2l" Jun 01 21:40:42.728423 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:42.727451 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/9a3c0ed4-56c7-4f73-a903-7b8fe4652a41-konnectivity-ca\") pod \"konnectivity-agent-mwdhn\" (UID: \"9a3c0ed4-56c7-4f73-a903-7b8fe4652a41\") " pod="kube-system/konnectivity-agent-mwdhn" Jun 01 21:40:42.728423 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:42.727484 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/0192780a-312c-4eae-a683-ca316fa48cb2-cnibin\") pod \"multus-4vd5l\" (UID: \"0192780a-312c-4eae-a683-ca316fa48cb2\") " pod="openshift-multus/multus-4vd5l" Jun 01 21:40:42.728423 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:42.727509 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/0192780a-312c-4eae-a683-ca316fa48cb2-host-var-lib-cni-bin\") pod \"multus-4vd5l\" (UID: \"0192780a-312c-4eae-a683-ca316fa48cb2\") " pod="openshift-multus/multus-4vd5l" Jun 01 21:40:42.728423 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:42.727583 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/0192780a-312c-4eae-a683-ca316fa48cb2-multus-daemon-config\") pod \"multus-4vd5l\" (UID: \"0192780a-312c-4eae-a683-ca316fa48cb2\") " pod="openshift-multus/multus-4vd5l" Jun 01 21:40:42.728423 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:42.727614 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fcf0209a-2440-4267-834a-b176f4e337e4-host-run-ovn-kubernetes\") pod \"ovnkube-node-rcqkp\" (UID: \"fcf0209a-2440-4267-834a-b176f4e337e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-rcqkp" Jun 01 21:40:42.728423 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:42.727655 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/fcf0209a-2440-4267-834a-b176f4e337e4-host-cni-netd\") pod \"ovnkube-node-rcqkp\" (UID: \"fcf0209a-2440-4267-834a-b176f4e337e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-rcqkp" Jun 01 21:40:42.728423 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:42.727682 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/fcf0209a-2440-4267-834a-b176f4e337e4-ovnkube-config\") pod \"ovnkube-node-rcqkp\" (UID: \"fcf0209a-2440-4267-834a-b176f4e337e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-rcqkp" Jun 01 21:40:42.729244 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:42.727710 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/bd935121-781a-4420-80da-4b28e731a602-registration-dir\") pod \"aws-ebs-csi-driver-node-c2qnd\" (UID: \"bd935121-781a-4420-80da-4b28e731a602\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c2qnd" Jun 01 21:40:42.729244 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:42.727768 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/36aa25cf-725a-42f0-8f7b-938f5dbe52d4-system-cni-dir\") pod \"multus-additional-cni-plugins-mhq4q\" (UID: \"36aa25cf-725a-42f0-8f7b-938f5dbe52d4\") " pod="openshift-multus/multus-additional-cni-plugins-mhq4q" Jun 01 21:40:42.729244 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:42.727796 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/9a3c0ed4-56c7-4f73-a903-7b8fe4652a41-agent-certs\") pod \"konnectivity-agent-mwdhn\" (UID: \"9a3c0ed4-56c7-4f73-a903-7b8fe4652a41\") " pod="kube-system/konnectivity-agent-mwdhn" Jun 01 21:40:42.729244 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:42.727822 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nzztm\" (UniqueName: \"kubernetes.io/projected/0192780a-312c-4eae-a683-ca316fa48cb2-kube-api-access-nzztm\") pod \"multus-4vd5l\" (UID: \"0192780a-312c-4eae-a683-ca316fa48cb2\") " pod="openshift-multus/multus-4vd5l" Jun 01 21:40:42.729244 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:42.727849 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/fcf0209a-2440-4267-834a-b176f4e337e4-log-socket\") pod \"ovnkube-node-rcqkp\" (UID: \"fcf0209a-2440-4267-834a-b176f4e337e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-rcqkp" Jun 01 21:40:42.729244 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:42.727873 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/fcf0209a-2440-4267-834a-b176f4e337e4-ovnkube-script-lib\") pod \"ovnkube-node-rcqkp\" (UID: \"fcf0209a-2440-4267-834a-b176f4e337e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-rcqkp" Jun 01 21:40:42.729244 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:42.727903 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-7dqw7\" (UniqueName: \"kubernetes.io/projected/e2e547dd-415d-46fe-9608-4a3410d73834-kube-api-access-7dqw7\") pod \"network-check-target-z7k6x\" (UID: \"e2e547dd-415d-46fe-9608-4a3410d73834\") " pod="openshift-network-diagnostics/network-check-target-z7k6x" Jun 01 21:40:42.729244 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:42.727928 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/7ada00c1-dbeb-4df5-bfca-923ef619c2d1-host\") pod \"node-ca-qcs4d\" (UID: \"7ada00c1-dbeb-4df5-bfca-923ef619c2d1\") " pod="openshift-image-registry/node-ca-qcs4d" Jun 01 21:40:42.729244 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:42.728009 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/bd935121-781a-4420-80da-4b28e731a602-kubelet-dir\") pod \"aws-ebs-csi-driver-node-c2qnd\" (UID: \"bd935121-781a-4420-80da-4b28e731a602\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c2qnd" Jun 01 21:40:42.729244 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:42.728037 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/bd935121-781a-4420-80da-4b28e731a602-socket-dir\") pod \"aws-ebs-csi-driver-node-c2qnd\" (UID: \"bd935121-781a-4420-80da-4b28e731a602\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c2qnd" Jun 01 21:40:42.729244 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:42.728061 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/9a3c0ed4-56c7-4f73-a903-7b8fe4652a41-konnectivity-ca\") pod \"konnectivity-agent-mwdhn\" (UID: \"9a3c0ed4-56c7-4f73-a903-7b8fe4652a41\") " pod="kube-system/konnectivity-agent-mwdhn" Jun 01 21:40:42.729244 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:42.728115 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/7ada00c1-dbeb-4df5-bfca-923ef619c2d1-host\") pod \"node-ca-qcs4d\" (UID: \"7ada00c1-dbeb-4df5-bfca-923ef619c2d1\") " pod="openshift-image-registry/node-ca-qcs4d" Jun 01 21:40:42.729244 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:42.728064 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/0192780a-312c-4eae-a683-ca316fa48cb2-host-run-netns\") pod \"multus-4vd5l\" (UID: \"0192780a-312c-4eae-a683-ca316fa48cb2\") " pod="openshift-multus/multus-4vd5l" Jun 01 21:40:42.729244 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:42.728160 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/0192780a-312c-4eae-a683-ca316fa48cb2-host-var-lib-cni-multus\") pod \"multus-4vd5l\" (UID: \"0192780a-312c-4eae-a683-ca316fa48cb2\") " pod="openshift-multus/multus-4vd5l" Jun 01 21:40:42.729244 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:42.728213 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/bd935121-781a-4420-80da-4b28e731a602-registration-dir\") pod \"aws-ebs-csi-driver-node-c2qnd\" (UID: \"bd935121-781a-4420-80da-4b28e731a602\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c2qnd" Jun 01 21:40:42.729244 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:42.728249 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/36aa25cf-725a-42f0-8f7b-938f5dbe52d4-os-release\") pod \"multus-additional-cni-plugins-mhq4q\" (UID: \"36aa25cf-725a-42f0-8f7b-938f5dbe52d4\") " pod="openshift-multus/multus-additional-cni-plugins-mhq4q" Jun 01 21:40:42.729244 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:42.728276 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/fcf0209a-2440-4267-834a-b176f4e337e4-node-log\") pod \"ovnkube-node-rcqkp\" (UID: \"fcf0209a-2440-4267-834a-b176f4e337e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-rcqkp" Jun 01 21:40:42.729980 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:42.728305 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-fthss\" (UniqueName: \"kubernetes.io/projected/f99cc923-cc5c-4895-8b2b-6ba878ba343b-kube-api-access-fthss\") pod \"network-metrics-daemon-w4d7s\" (UID: \"f99cc923-cc5c-4895-8b2b-6ba878ba343b\") " pod="openshift-multus/network-metrics-daemon-w4d7s" Jun 01 21:40:42.729980 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:42.728333 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-qk9pm\" (UniqueName: \"kubernetes.io/projected/7ada00c1-dbeb-4df5-bfca-923ef619c2d1-kube-api-access-qk9pm\") pod \"node-ca-qcs4d\" (UID: \"7ada00c1-dbeb-4df5-bfca-923ef619c2d1\") " pod="openshift-image-registry/node-ca-qcs4d" Jun 01 21:40:42.729980 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:42.728360 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-cbmtr\" (UniqueName: \"kubernetes.io/projected/c2491d2d-ab6b-422e-af1d-41cc31129778-kube-api-access-cbmtr\") pod \"tuned-bfz2l\" (UID: \"c2491d2d-ab6b-422e-af1d-41cc31129778\") " pod="openshift-cluster-node-tuning-operator/tuned-bfz2l" Jun 01 21:40:42.729980 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:42.728387 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/bd935121-781a-4420-80da-4b28e731a602-sys-fs\") pod \"aws-ebs-csi-driver-node-c2qnd\" (UID: \"bd935121-781a-4420-80da-4b28e731a602\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c2qnd" Jun 01 21:40:42.729980 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:42.728461 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/0192780a-312c-4eae-a683-ca316fa48cb2-cni-binary-copy\") pod \"multus-4vd5l\" (UID: \"0192780a-312c-4eae-a683-ca316fa48cb2\") " pod="openshift-multus/multus-4vd5l" Jun 01 21:40:42.729980 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:42.728491 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/36aa25cf-725a-42f0-8f7b-938f5dbe52d4-cni-binary-copy\") pod \"multus-additional-cni-plugins-mhq4q\" (UID: \"36aa25cf-725a-42f0-8f7b-938f5dbe52d4\") " pod="openshift-multus/multus-additional-cni-plugins-mhq4q" Jun 01 21:40:42.729980 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:42.728523 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/fcf0209a-2440-4267-834a-b176f4e337e4-systemd-units\") pod \"ovnkube-node-rcqkp\" (UID: \"fcf0209a-2440-4267-834a-b176f4e337e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-rcqkp" Jun 01 21:40:42.729980 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:42.728567 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/fcf0209a-2440-4267-834a-b176f4e337e4-etc-openvswitch\") pod \"ovnkube-node-rcqkp\" (UID: \"fcf0209a-2440-4267-834a-b176f4e337e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-rcqkp" Jun 01 21:40:42.729980 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:42.728605 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f99cc923-cc5c-4895-8b2b-6ba878ba343b-metrics-certs\") pod \"network-metrics-daemon-w4d7s\" (UID: \"f99cc923-cc5c-4895-8b2b-6ba878ba343b\") " pod="openshift-multus/network-metrics-daemon-w4d7s" Jun 01 21:40:42.729980 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:42.728631 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/c2491d2d-ab6b-422e-af1d-41cc31129778-etc-modprobe-d\") pod \"tuned-bfz2l\" (UID: \"c2491d2d-ab6b-422e-af1d-41cc31129778\") " pod="openshift-cluster-node-tuning-operator/tuned-bfz2l" Jun 01 21:40:42.729980 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:42.728656 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/c2491d2d-ab6b-422e-af1d-41cc31129778-etc-sysctl-d\") pod \"tuned-bfz2l\" (UID: \"c2491d2d-ab6b-422e-af1d-41cc31129778\") " pod="openshift-cluster-node-tuning-operator/tuned-bfz2l" Jun 01 21:40:42.729980 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:42.728680 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/c2491d2d-ab6b-422e-af1d-41cc31129778-run\") pod \"tuned-bfz2l\" (UID: \"c2491d2d-ab6b-422e-af1d-41cc31129778\") " pod="openshift-cluster-node-tuning-operator/tuned-bfz2l" Jun 01 21:40:42.729980 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:42.728767 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/c2491d2d-ab6b-422e-af1d-41cc31129778-run\") pod \"tuned-bfz2l\" (UID: \"c2491d2d-ab6b-422e-af1d-41cc31129778\") " pod="openshift-cluster-node-tuning-operator/tuned-bfz2l" Jun 01 21:40:42.729980 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:42.728831 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/bd935121-781a-4420-80da-4b28e731a602-sys-fs\") pod \"aws-ebs-csi-driver-node-c2qnd\" (UID: \"bd935121-781a-4420-80da-4b28e731a602\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c2qnd" Jun 01 21:40:42.729980 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:42.728944 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/bd935121-781a-4420-80da-4b28e731a602-kubelet-dir\") pod \"aws-ebs-csi-driver-node-c2qnd\" (UID: \"bd935121-781a-4420-80da-4b28e731a602\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c2qnd" Jun 01 21:40:42.729980 ip-10-0-138-241 kubenswrapper[2573]: E0601 21:40:42.729099 2573 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 01 21:40:42.729980 ip-10-0-138-241 kubenswrapper[2573]: E0601 21:40:42.729245 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f99cc923-cc5c-4895-8b2b-6ba878ba343b-metrics-certs podName:f99cc923-cc5c-4895-8b2b-6ba878ba343b nodeName:}" failed. No retries permitted until 2026-06-01 21:40:43.229192084 +0000 UTC m=+2.949244025 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f99cc923-cc5c-4895-8b2b-6ba878ba343b-metrics-certs") pod "network-metrics-daemon-w4d7s" (UID: "f99cc923-cc5c-4895-8b2b-6ba878ba343b") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 01 21:40:42.730711 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:42.729268 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/c2491d2d-ab6b-422e-af1d-41cc31129778-etc-sysctl-d\") pod \"tuned-bfz2l\" (UID: \"c2491d2d-ab6b-422e-af1d-41cc31129778\") " pod="openshift-cluster-node-tuning-operator/tuned-bfz2l" Jun 01 21:40:42.730711 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:42.729362 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/bd935121-781a-4420-80da-4b28e731a602-socket-dir\") pod \"aws-ebs-csi-driver-node-c2qnd\" (UID: \"bd935121-781a-4420-80da-4b28e731a602\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c2qnd" Jun 01 21:40:42.730711 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:42.729476 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/c2491d2d-ab6b-422e-af1d-41cc31129778-etc-modprobe-d\") pod \"tuned-bfz2l\" (UID: \"c2491d2d-ab6b-422e-af1d-41cc31129778\") " pod="openshift-cluster-node-tuning-operator/tuned-bfz2l" Jun 01 21:40:42.731319 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:42.731291 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c2491d2d-ab6b-422e-af1d-41cc31129778-tmp\") pod \"tuned-bfz2l\" (UID: \"c2491d2d-ab6b-422e-af1d-41cc31129778\") " pod="openshift-cluster-node-tuning-operator/tuned-bfz2l" Jun 01 21:40:42.733070 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:42.733025 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/c2491d2d-ab6b-422e-af1d-41cc31129778-etc-tuned\") pod \"tuned-bfz2l\" (UID: \"c2491d2d-ab6b-422e-af1d-41cc31129778\") " pod="openshift-cluster-node-tuning-operator/tuned-bfz2l" Jun 01 21:40:42.733180 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:42.733061 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/9a3c0ed4-56c7-4f73-a903-7b8fe4652a41-agent-certs\") pod \"konnectivity-agent-mwdhn\" (UID: \"9a3c0ed4-56c7-4f73-a903-7b8fe4652a41\") " pod="kube-system/konnectivity-agent-mwdhn" Jun 01 21:40:42.734842 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:42.734818 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lqhv8\" (UniqueName: \"kubernetes.io/projected/fbd5149c-c44e-4b0d-959f-a8c158933a5c-kube-api-access-lqhv8\") pod \"iptables-alerter-c88gr\" (UID: \"fbd5149c-c44e-4b0d-959f-a8c158933a5c\") " pod="openshift-network-operator/iptables-alerter-c88gr" Jun 01 21:40:42.735904 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:42.735872 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rmtgv\" (UniqueName: \"kubernetes.io/projected/bd935121-781a-4420-80da-4b28e731a602-kube-api-access-rmtgv\") pod \"aws-ebs-csi-driver-node-c2qnd\" (UID: \"bd935121-781a-4420-80da-4b28e731a602\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c2qnd" Jun 01 21:40:42.740034 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:42.739996 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fthss\" (UniqueName: \"kubernetes.io/projected/f99cc923-cc5c-4895-8b2b-6ba878ba343b-kube-api-access-fthss\") pod \"network-metrics-daemon-w4d7s\" (UID: \"f99cc923-cc5c-4895-8b2b-6ba878ba343b\") " pod="openshift-multus/network-metrics-daemon-w4d7s" Jun 01 21:40:42.740416 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:42.740397 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cbmtr\" (UniqueName: \"kubernetes.io/projected/c2491d2d-ab6b-422e-af1d-41cc31129778-kube-api-access-cbmtr\") pod \"tuned-bfz2l\" (UID: \"c2491d2d-ab6b-422e-af1d-41cc31129778\") " pod="openshift-cluster-node-tuning-operator/tuned-bfz2l" Jun 01 21:40:42.745515 ip-10-0-138-241 kubenswrapper[2573]: E0601 21:40:42.745497 2573 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 01 21:40:42.745640 ip-10-0-138-241 kubenswrapper[2573]: E0601 21:40:42.745519 2573 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 01 21:40:42.745640 ip-10-0-138-241 kubenswrapper[2573]: E0601 21:40:42.745546 2573 projected.go:194] Error preparing data for projected volume kube-api-access-7dqw7 for pod openshift-network-diagnostics/network-check-target-z7k6x: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 01 21:40:42.745640 ip-10-0-138-241 kubenswrapper[2573]: E0601 21:40:42.745605 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e2e547dd-415d-46fe-9608-4a3410d73834-kube-api-access-7dqw7 podName:e2e547dd-415d-46fe-9608-4a3410d73834 nodeName:}" failed. No retries permitted until 2026-06-01 21:40:43.245587627 +0000 UTC m=+2.965639548 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-7dqw7" (UniqueName: "kubernetes.io/projected/e2e547dd-415d-46fe-9608-4a3410d73834-kube-api-access-7dqw7") pod "network-check-target-z7k6x" (UID: "e2e547dd-415d-46fe-9608-4a3410d73834") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 01 21:40:42.747726 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:42.747706 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qk9pm\" (UniqueName: \"kubernetes.io/projected/7ada00c1-dbeb-4df5-bfca-923ef619c2d1-kube-api-access-qk9pm\") pod \"node-ca-qcs4d\" (UID: \"7ada00c1-dbeb-4df5-bfca-923ef619c2d1\") " pod="openshift-image-registry/node-ca-qcs4d" Jun 01 21:40:42.830054 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:42.829965 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/0192780a-312c-4eae-a683-ca316fa48cb2-hostroot\") pod \"multus-4vd5l\" (UID: \"0192780a-312c-4eae-a683-ca316fa48cb2\") " pod="openshift-multus/multus-4vd5l" Jun 01 21:40:42.830054 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:42.830006 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/0192780a-312c-4eae-a683-ca316fa48cb2-multus-conf-dir\") pod \"multus-4vd5l\" (UID: \"0192780a-312c-4eae-a683-ca316fa48cb2\") " pod="openshift-multus/multus-4vd5l" Jun 01 21:40:42.830054 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:42.830031 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/fcf0209a-2440-4267-834a-b176f4e337e4-run-ovn\") pod \"ovnkube-node-rcqkp\" (UID: \"fcf0209a-2440-4267-834a-b176f4e337e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-rcqkp" Jun 01 21:40:42.830054 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:42.830053 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0192780a-312c-4eae-a683-ca316fa48cb2-host-var-lib-kubelet\") pod \"multus-4vd5l\" (UID: \"0192780a-312c-4eae-a683-ca316fa48cb2\") " pod="openshift-multus/multus-4vd5l" Jun 01 21:40:42.830343 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:42.830069 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/36aa25cf-725a-42f0-8f7b-938f5dbe52d4-tuning-conf-dir\") pod \"multus-additional-cni-plugins-mhq4q\" (UID: \"36aa25cf-725a-42f0-8f7b-938f5dbe52d4\") " pod="openshift-multus/multus-additional-cni-plugins-mhq4q" Jun 01 21:40:42.830343 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:42.830086 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/0192780a-312c-4eae-a683-ca316fa48cb2-cnibin\") pod \"multus-4vd5l\" (UID: \"0192780a-312c-4eae-a683-ca316fa48cb2\") " pod="openshift-multus/multus-4vd5l" Jun 01 21:40:42.830343 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:42.830093 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/0192780a-312c-4eae-a683-ca316fa48cb2-hostroot\") pod \"multus-4vd5l\" (UID: \"0192780a-312c-4eae-a683-ca316fa48cb2\") " pod="openshift-multus/multus-4vd5l" Jun 01 21:40:42.830343 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:42.830107 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/0192780a-312c-4eae-a683-ca316fa48cb2-host-var-lib-cni-bin\") pod \"multus-4vd5l\" (UID: \"0192780a-312c-4eae-a683-ca316fa48cb2\") " pod="openshift-multus/multus-4vd5l" Jun 01 21:40:42.830343 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:42.830133 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/0192780a-312c-4eae-a683-ca316fa48cb2-multus-daemon-config\") pod \"multus-4vd5l\" (UID: \"0192780a-312c-4eae-a683-ca316fa48cb2\") " pod="openshift-multus/multus-4vd5l" Jun 01 21:40:42.830343 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:42.830155 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fcf0209a-2440-4267-834a-b176f4e337e4-host-run-ovn-kubernetes\") pod \"ovnkube-node-rcqkp\" (UID: \"fcf0209a-2440-4267-834a-b176f4e337e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-rcqkp" Jun 01 21:40:42.830343 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:42.830160 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0192780a-312c-4eae-a683-ca316fa48cb2-host-var-lib-kubelet\") pod \"multus-4vd5l\" (UID: \"0192780a-312c-4eae-a683-ca316fa48cb2\") " pod="openshift-multus/multus-4vd5l" Jun 01 21:40:42.830343 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:42.830176 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/fcf0209a-2440-4267-834a-b176f4e337e4-host-cni-netd\") pod \"ovnkube-node-rcqkp\" (UID: \"fcf0209a-2440-4267-834a-b176f4e337e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-rcqkp" Jun 01 21:40:42.830343 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:42.830199 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/fcf0209a-2440-4267-834a-b176f4e337e4-ovnkube-config\") pod \"ovnkube-node-rcqkp\" (UID: \"fcf0209a-2440-4267-834a-b176f4e337e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-rcqkp" Jun 01 21:40:42.830343 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:42.830201 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/0192780a-312c-4eae-a683-ca316fa48cb2-multus-conf-dir\") pod \"multus-4vd5l\" (UID: \"0192780a-312c-4eae-a683-ca316fa48cb2\") " pod="openshift-multus/multus-4vd5l" Jun 01 21:40:42.830343 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:42.830226 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/36aa25cf-725a-42f0-8f7b-938f5dbe52d4-system-cni-dir\") pod \"multus-additional-cni-plugins-mhq4q\" (UID: \"36aa25cf-725a-42f0-8f7b-938f5dbe52d4\") " pod="openshift-multus/multus-additional-cni-plugins-mhq4q" Jun 01 21:40:42.830343 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:42.830242 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/fcf0209a-2440-4267-834a-b176f4e337e4-run-ovn\") pod \"ovnkube-node-rcqkp\" (UID: \"fcf0209a-2440-4267-834a-b176f4e337e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-rcqkp" Jun 01 21:40:42.830343 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:42.830253 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-nzztm\" (UniqueName: \"kubernetes.io/projected/0192780a-312c-4eae-a683-ca316fa48cb2-kube-api-access-nzztm\") pod \"multus-4vd5l\" (UID: \"0192780a-312c-4eae-a683-ca316fa48cb2\") " pod="openshift-multus/multus-4vd5l" Jun 01 21:40:42.830343 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:42.830272 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/fcf0209a-2440-4267-834a-b176f4e337e4-log-socket\") pod \"ovnkube-node-rcqkp\" (UID: \"fcf0209a-2440-4267-834a-b176f4e337e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-rcqkp" Jun 01 21:40:42.830343 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:42.830297 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/fcf0209a-2440-4267-834a-b176f4e337e4-ovnkube-script-lib\") pod \"ovnkube-node-rcqkp\" (UID: \"fcf0209a-2440-4267-834a-b176f4e337e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-rcqkp" Jun 01 21:40:42.830343 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:42.830329 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/0192780a-312c-4eae-a683-ca316fa48cb2-host-run-netns\") pod \"multus-4vd5l\" (UID: \"0192780a-312c-4eae-a683-ca316fa48cb2\") " pod="openshift-multus/multus-4vd5l" Jun 01 21:40:42.831109 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:42.830362 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/0192780a-312c-4eae-a683-ca316fa48cb2-host-var-lib-cni-multus\") pod \"multus-4vd5l\" (UID: \"0192780a-312c-4eae-a683-ca316fa48cb2\") " pod="openshift-multus/multus-4vd5l" Jun 01 21:40:42.831109 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:42.830386 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/36aa25cf-725a-42f0-8f7b-938f5dbe52d4-os-release\") pod \"multus-additional-cni-plugins-mhq4q\" (UID: \"36aa25cf-725a-42f0-8f7b-938f5dbe52d4\") " pod="openshift-multus/multus-additional-cni-plugins-mhq4q" Jun 01 21:40:42.831109 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:42.830406 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/fcf0209a-2440-4267-834a-b176f4e337e4-node-log\") pod \"ovnkube-node-rcqkp\" (UID: \"fcf0209a-2440-4267-834a-b176f4e337e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-rcqkp" Jun 01 21:40:42.831109 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:42.830426 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/0192780a-312c-4eae-a683-ca316fa48cb2-cni-binary-copy\") pod \"multus-4vd5l\" (UID: \"0192780a-312c-4eae-a683-ca316fa48cb2\") " pod="openshift-multus/multus-4vd5l" Jun 01 21:40:42.831109 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:42.830440 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/36aa25cf-725a-42f0-8f7b-938f5dbe52d4-cni-binary-copy\") pod \"multus-additional-cni-plugins-mhq4q\" (UID: \"36aa25cf-725a-42f0-8f7b-938f5dbe52d4\") " pod="openshift-multus/multus-additional-cni-plugins-mhq4q" Jun 01 21:40:42.831109 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:42.830446 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/fcf0209a-2440-4267-834a-b176f4e337e4-host-cni-netd\") pod \"ovnkube-node-rcqkp\" (UID: \"fcf0209a-2440-4267-834a-b176f4e337e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-rcqkp" Jun 01 21:40:42.831109 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:42.830455 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/fcf0209a-2440-4267-834a-b176f4e337e4-systemd-units\") pod \"ovnkube-node-rcqkp\" (UID: \"fcf0209a-2440-4267-834a-b176f4e337e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-rcqkp" Jun 01 21:40:42.831109 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:42.830497 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/fcf0209a-2440-4267-834a-b176f4e337e4-systemd-units\") pod \"ovnkube-node-rcqkp\" (UID: \"fcf0209a-2440-4267-834a-b176f4e337e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-rcqkp" Jun 01 21:40:42.831109 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:42.830504 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/fcf0209a-2440-4267-834a-b176f4e337e4-etc-openvswitch\") pod \"ovnkube-node-rcqkp\" (UID: \"fcf0209a-2440-4267-834a-b176f4e337e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-rcqkp" Jun 01 21:40:42.831109 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:42.830557 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/fcf0209a-2440-4267-834a-b176f4e337e4-etc-openvswitch\") pod \"ovnkube-node-rcqkp\" (UID: \"fcf0209a-2440-4267-834a-b176f4e337e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-rcqkp" Jun 01 21:40:42.831109 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:42.830573 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/0192780a-312c-4eae-a683-ca316fa48cb2-host-run-multus-certs\") pod \"multus-4vd5l\" (UID: \"0192780a-312c-4eae-a683-ca316fa48cb2\") " pod="openshift-multus/multus-4vd5l" Jun 01 21:40:42.831109 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:42.830603 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0192780a-312c-4eae-a683-ca316fa48cb2-multus-cni-dir\") pod \"multus-4vd5l\" (UID: \"0192780a-312c-4eae-a683-ca316fa48cb2\") " pod="openshift-multus/multus-4vd5l" Jun 01 21:40:42.831109 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:42.830613 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/0192780a-312c-4eae-a683-ca316fa48cb2-cnibin\") pod \"multus-4vd5l\" (UID: \"0192780a-312c-4eae-a683-ca316fa48cb2\") " pod="openshift-multus/multus-4vd5l" Jun 01 21:40:42.831109 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:42.830630 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/0192780a-312c-4eae-a683-ca316fa48cb2-os-release\") pod \"multus-4vd5l\" (UID: \"0192780a-312c-4eae-a683-ca316fa48cb2\") " pod="openshift-multus/multus-4vd5l" Jun 01 21:40:42.831109 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:42.830650 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-sdn65\" (UniqueName: \"kubernetes.io/projected/bcc44458-cf91-4d2d-9b1b-e9da2db57098-kube-api-access-sdn65\") pod \"node-resolver-kfs2m\" (UID: \"bcc44458-cf91-4d2d-9b1b-e9da2db57098\") " pod="openshift-dns/node-resolver-kfs2m" Jun 01 21:40:42.831109 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:42.830667 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fcf0209a-2440-4267-834a-b176f4e337e4-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-rcqkp\" (UID: \"fcf0209a-2440-4267-834a-b176f4e337e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-rcqkp" Jun 01 21:40:42.831109 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:42.830689 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-lw7gz\" (UniqueName: \"kubernetes.io/projected/fcf0209a-2440-4267-834a-b176f4e337e4-kube-api-access-lw7gz\") pod \"ovnkube-node-rcqkp\" (UID: \"fcf0209a-2440-4267-834a-b176f4e337e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-rcqkp" Jun 01 21:40:42.831872 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:42.830718 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0192780a-312c-4eae-a683-ca316fa48cb2-etc-kubernetes\") pod \"multus-4vd5l\" (UID: \"0192780a-312c-4eae-a683-ca316fa48cb2\") " pod="openshift-multus/multus-4vd5l" Jun 01 21:40:42.831872 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:42.830744 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/36aa25cf-725a-42f0-8f7b-938f5dbe52d4-tuning-conf-dir\") pod \"multus-additional-cni-plugins-mhq4q\" (UID: \"36aa25cf-725a-42f0-8f7b-938f5dbe52d4\") " pod="openshift-multus/multus-additional-cni-plugins-mhq4q" Jun 01 21:40:42.831872 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:42.830749 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/36aa25cf-725a-42f0-8f7b-938f5dbe52d4-cnibin\") pod \"multus-additional-cni-plugins-mhq4q\" (UID: \"36aa25cf-725a-42f0-8f7b-938f5dbe52d4\") " pod="openshift-multus/multus-additional-cni-plugins-mhq4q" Jun 01 21:40:42.831872 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:42.830390 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fcf0209a-2440-4267-834a-b176f4e337e4-host-run-ovn-kubernetes\") pod \"ovnkube-node-rcqkp\" (UID: \"fcf0209a-2440-4267-834a-b176f4e337e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-rcqkp" Jun 01 21:40:42.831872 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:42.830775 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/36aa25cf-725a-42f0-8f7b-938f5dbe52d4-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-mhq4q\" (UID: \"36aa25cf-725a-42f0-8f7b-938f5dbe52d4\") " pod="openshift-multus/multus-additional-cni-plugins-mhq4q" Jun 01 21:40:42.831872 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:42.830877 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/36aa25cf-725a-42f0-8f7b-938f5dbe52d4-system-cni-dir\") pod \"multus-additional-cni-plugins-mhq4q\" (UID: \"36aa25cf-725a-42f0-8f7b-938f5dbe52d4\") " pod="openshift-multus/multus-additional-cni-plugins-mhq4q" Jun 01 21:40:42.831872 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:42.830890 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/fcf0209a-2440-4267-834a-b176f4e337e4-ovnkube-config\") pod \"ovnkube-node-rcqkp\" (UID: \"fcf0209a-2440-4267-834a-b176f4e337e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-rcqkp" Jun 01 21:40:42.831872 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:42.830897 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/0192780a-312c-4eae-a683-ca316fa48cb2-multus-daemon-config\") pod \"multus-4vd5l\" (UID: \"0192780a-312c-4eae-a683-ca316fa48cb2\") " pod="openshift-multus/multus-4vd5l" Jun 01 21:40:42.831872 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:42.830925 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/0192780a-312c-4eae-a683-ca316fa48cb2-host-run-multus-certs\") pod \"multus-4vd5l\" (UID: \"0192780a-312c-4eae-a683-ca316fa48cb2\") " pod="openshift-multus/multus-4vd5l" Jun 01 21:40:42.831872 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:42.830955 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/0192780a-312c-4eae-a683-ca316fa48cb2-host-var-lib-cni-multus\") pod \"multus-4vd5l\" (UID: \"0192780a-312c-4eae-a683-ca316fa48cb2\") " pod="openshift-multus/multus-4vd5l" Jun 01 21:40:42.831872 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:42.830994 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/0192780a-312c-4eae-a683-ca316fa48cb2-host-run-netns\") pod \"multus-4vd5l\" (UID: \"0192780a-312c-4eae-a683-ca316fa48cb2\") " pod="openshift-multus/multus-4vd5l" Jun 01 21:40:42.831872 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:42.831019 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/0192780a-312c-4eae-a683-ca316fa48cb2-host-var-lib-cni-bin\") pod \"multus-4vd5l\" (UID: \"0192780a-312c-4eae-a683-ca316fa48cb2\") " pod="openshift-multus/multus-4vd5l" Jun 01 21:40:42.831872 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:42.831054 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0192780a-312c-4eae-a683-ca316fa48cb2-multus-cni-dir\") pod \"multus-4vd5l\" (UID: \"0192780a-312c-4eae-a683-ca316fa48cb2\") " pod="openshift-multus/multus-4vd5l" Jun 01 21:40:42.831872 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:42.831075 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/fcf0209a-2440-4267-834a-b176f4e337e4-node-log\") pod \"ovnkube-node-rcqkp\" (UID: \"fcf0209a-2440-4267-834a-b176f4e337e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-rcqkp" Jun 01 21:40:42.831872 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:42.831124 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/0192780a-312c-4eae-a683-ca316fa48cb2-os-release\") pod \"multus-4vd5l\" (UID: \"0192780a-312c-4eae-a683-ca316fa48cb2\") " pod="openshift-multus/multus-4vd5l" Jun 01 21:40:42.831872 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:42.831150 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/36aa25cf-725a-42f0-8f7b-938f5dbe52d4-os-release\") pod \"multus-additional-cni-plugins-mhq4q\" (UID: \"36aa25cf-725a-42f0-8f7b-938f5dbe52d4\") " pod="openshift-multus/multus-additional-cni-plugins-mhq4q" Jun 01 21:40:42.831872 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:42.831194 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0192780a-312c-4eae-a683-ca316fa48cb2-etc-kubernetes\") pod \"multus-4vd5l\" (UID: \"0192780a-312c-4eae-a683-ca316fa48cb2\") " pod="openshift-multus/multus-4vd5l" Jun 01 21:40:42.831872 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:42.831228 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/36aa25cf-725a-42f0-8f7b-938f5dbe52d4-cnibin\") pod \"multus-additional-cni-plugins-mhq4q\" (UID: \"36aa25cf-725a-42f0-8f7b-938f5dbe52d4\") " pod="openshift-multus/multus-additional-cni-plugins-mhq4q" Jun 01 21:40:42.832646 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:42.831312 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/fcf0209a-2440-4267-834a-b176f4e337e4-log-socket\") pod \"ovnkube-node-rcqkp\" (UID: \"fcf0209a-2440-4267-834a-b176f4e337e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-rcqkp" Jun 01 21:40:42.832646 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:42.831337 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fcf0209a-2440-4267-834a-b176f4e337e4-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-rcqkp\" (UID: \"fcf0209a-2440-4267-834a-b176f4e337e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-rcqkp" Jun 01 21:40:42.832646 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:42.831363 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/36aa25cf-725a-42f0-8f7b-938f5dbe52d4-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-mhq4q\" (UID: \"36aa25cf-725a-42f0-8f7b-938f5dbe52d4\") " pod="openshift-multus/multus-additional-cni-plugins-mhq4q" Jun 01 21:40:42.832646 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:42.831381 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-jc4mz\" (UniqueName: \"kubernetes.io/projected/36aa25cf-725a-42f0-8f7b-938f5dbe52d4-kube-api-access-jc4mz\") pod \"multus-additional-cni-plugins-mhq4q\" (UID: \"36aa25cf-725a-42f0-8f7b-938f5dbe52d4\") " pod="openshift-multus/multus-additional-cni-plugins-mhq4q" Jun 01 21:40:42.832646 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:42.831397 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/36aa25cf-725a-42f0-8f7b-938f5dbe52d4-cni-binary-copy\") pod \"multus-additional-cni-plugins-mhq4q\" (UID: \"36aa25cf-725a-42f0-8f7b-938f5dbe52d4\") " pod="openshift-multus/multus-additional-cni-plugins-mhq4q" Jun 01 21:40:42.832646 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:42.831397 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/0192780a-312c-4eae-a683-ca316fa48cb2-cni-binary-copy\") pod \"multus-4vd5l\" (UID: \"0192780a-312c-4eae-a683-ca316fa48cb2\") " pod="openshift-multus/multus-4vd5l" Jun 01 21:40:42.832646 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:42.831431 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/fcf0209a-2440-4267-834a-b176f4e337e4-host-kubelet\") pod \"ovnkube-node-rcqkp\" (UID: \"fcf0209a-2440-4267-834a-b176f4e337e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-rcqkp" Jun 01 21:40:42.832646 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:42.831556 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/fcf0209a-2440-4267-834a-b176f4e337e4-host-slash\") pod \"ovnkube-node-rcqkp\" (UID: \"fcf0209a-2440-4267-834a-b176f4e337e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-rcqkp" Jun 01 21:40:42.832646 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:42.831589 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/fcf0209a-2440-4267-834a-b176f4e337e4-host-run-netns\") pod \"ovnkube-node-rcqkp\" (UID: \"fcf0209a-2440-4267-834a-b176f4e337e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-rcqkp" Jun 01 21:40:42.832646 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:42.831616 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/fcf0209a-2440-4267-834a-b176f4e337e4-run-systemd\") pod \"ovnkube-node-rcqkp\" (UID: \"fcf0209a-2440-4267-834a-b176f4e337e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-rcqkp" Jun 01 21:40:42.832646 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:42.831623 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/fcf0209a-2440-4267-834a-b176f4e337e4-host-kubelet\") pod \"ovnkube-node-rcqkp\" (UID: \"fcf0209a-2440-4267-834a-b176f4e337e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-rcqkp" Jun 01 21:40:42.832646 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:42.831655 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/fcf0209a-2440-4267-834a-b176f4e337e4-var-lib-openvswitch\") pod \"ovnkube-node-rcqkp\" (UID: \"fcf0209a-2440-4267-834a-b176f4e337e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-rcqkp" Jun 01 21:40:42.832646 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:42.831661 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/fcf0209a-2440-4267-834a-b176f4e337e4-host-slash\") pod \"ovnkube-node-rcqkp\" (UID: \"fcf0209a-2440-4267-834a-b176f4e337e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-rcqkp" Jun 01 21:40:42.832646 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:42.831691 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/fcf0209a-2440-4267-834a-b176f4e337e4-env-overrides\") pod \"ovnkube-node-rcqkp\" (UID: \"fcf0209a-2440-4267-834a-b176f4e337e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-rcqkp" Jun 01 21:40:42.832646 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:42.831698 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/fcf0209a-2440-4267-834a-b176f4e337e4-var-lib-openvswitch\") pod \"ovnkube-node-rcqkp\" (UID: \"fcf0209a-2440-4267-834a-b176f4e337e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-rcqkp" Jun 01 21:40:42.832646 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:42.831736 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/fcf0209a-2440-4267-834a-b176f4e337e4-ovn-node-metrics-cert\") pod \"ovnkube-node-rcqkp\" (UID: \"fcf0209a-2440-4267-834a-b176f4e337e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-rcqkp" Jun 01 21:40:42.832646 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:42.831767 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/0192780a-312c-4eae-a683-ca316fa48cb2-multus-socket-dir-parent\") pod \"multus-4vd5l\" (UID: \"0192780a-312c-4eae-a683-ca316fa48cb2\") " pod="openshift-multus/multus-4vd5l" Jun 01 21:40:42.833241 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:42.831814 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/bcc44458-cf91-4d2d-9b1b-e9da2db57098-hosts-file\") pod \"node-resolver-kfs2m\" (UID: \"bcc44458-cf91-4d2d-9b1b-e9da2db57098\") " pod="openshift-dns/node-resolver-kfs2m" Jun 01 21:40:42.833241 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:42.831819 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/fcf0209a-2440-4267-834a-b176f4e337e4-host-run-netns\") pod \"ovnkube-node-rcqkp\" (UID: \"fcf0209a-2440-4267-834a-b176f4e337e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-rcqkp" Jun 01 21:40:42.833241 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:42.831843 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/fcf0209a-2440-4267-834a-b176f4e337e4-run-openvswitch\") pod \"ovnkube-node-rcqkp\" (UID: \"fcf0209a-2440-4267-834a-b176f4e337e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-rcqkp" Jun 01 21:40:42.833241 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:42.831853 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/fcf0209a-2440-4267-834a-b176f4e337e4-run-systemd\") pod \"ovnkube-node-rcqkp\" (UID: \"fcf0209a-2440-4267-834a-b176f4e337e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-rcqkp" Jun 01 21:40:42.833241 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:42.831871 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0192780a-312c-4eae-a683-ca316fa48cb2-system-cni-dir\") pod \"multus-4vd5l\" (UID: \"0192780a-312c-4eae-a683-ca316fa48cb2\") " pod="openshift-multus/multus-4vd5l" Jun 01 21:40:42.833241 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:42.831889 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/36aa25cf-725a-42f0-8f7b-938f5dbe52d4-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-mhq4q\" (UID: \"36aa25cf-725a-42f0-8f7b-938f5dbe52d4\") " pod="openshift-multus/multus-additional-cni-plugins-mhq4q" Jun 01 21:40:42.833241 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:42.831904 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/fcf0209a-2440-4267-834a-b176f4e337e4-ovnkube-script-lib\") pod \"ovnkube-node-rcqkp\" (UID: \"fcf0209a-2440-4267-834a-b176f4e337e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-rcqkp" Jun 01 21:40:42.833241 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:42.831926 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/0192780a-312c-4eae-a683-ca316fa48cb2-multus-socket-dir-parent\") pod \"multus-4vd5l\" (UID: \"0192780a-312c-4eae-a683-ca316fa48cb2\") " pod="openshift-multus/multus-4vd5l" Jun 01 21:40:42.833241 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:42.831931 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/36aa25cf-725a-42f0-8f7b-938f5dbe52d4-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-mhq4q\" (UID: \"36aa25cf-725a-42f0-8f7b-938f5dbe52d4\") " pod="openshift-multus/multus-additional-cni-plugins-mhq4q" Jun 01 21:40:42.833241 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:42.831936 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0192780a-312c-4eae-a683-ca316fa48cb2-system-cni-dir\") pod \"multus-4vd5l\" (UID: \"0192780a-312c-4eae-a683-ca316fa48cb2\") " pod="openshift-multus/multus-4vd5l" Jun 01 21:40:42.833241 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:42.831970 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/bcc44458-cf91-4d2d-9b1b-e9da2db57098-hosts-file\") pod \"node-resolver-kfs2m\" (UID: \"bcc44458-cf91-4d2d-9b1b-e9da2db57098\") " pod="openshift-dns/node-resolver-kfs2m" Jun 01 21:40:42.833241 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:42.831979 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/0192780a-312c-4eae-a683-ca316fa48cb2-host-run-k8s-cni-cncf-io\") pod \"multus-4vd5l\" (UID: \"0192780a-312c-4eae-a683-ca316fa48cb2\") " pod="openshift-multus/multus-4vd5l" Jun 01 21:40:42.833241 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:42.831993 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/fcf0209a-2440-4267-834a-b176f4e337e4-run-openvswitch\") pod \"ovnkube-node-rcqkp\" (UID: \"fcf0209a-2440-4267-834a-b176f4e337e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-rcqkp" Jun 01 21:40:42.833241 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:42.832039 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bcc44458-cf91-4d2d-9b1b-e9da2db57098-tmp-dir\") pod \"node-resolver-kfs2m\" (UID: \"bcc44458-cf91-4d2d-9b1b-e9da2db57098\") " pod="openshift-dns/node-resolver-kfs2m" Jun 01 21:40:42.833241 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:42.832075 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/fcf0209a-2440-4267-834a-b176f4e337e4-host-cni-bin\") pod \"ovnkube-node-rcqkp\" (UID: \"fcf0209a-2440-4267-834a-b176f4e337e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-rcqkp" Jun 01 21:40:42.833241 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:42.832100 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/0192780a-312c-4eae-a683-ca316fa48cb2-host-run-k8s-cni-cncf-io\") pod \"multus-4vd5l\" (UID: \"0192780a-312c-4eae-a683-ca316fa48cb2\") " pod="openshift-multus/multus-4vd5l" Jun 01 21:40:42.833241 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:42.832172 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/fcf0209a-2440-4267-834a-b176f4e337e4-host-cni-bin\") pod \"ovnkube-node-rcqkp\" (UID: \"fcf0209a-2440-4267-834a-b176f4e337e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-rcqkp" Jun 01 21:40:42.833241 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:42.832233 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/fcf0209a-2440-4267-834a-b176f4e337e4-env-overrides\") pod \"ovnkube-node-rcqkp\" (UID: \"fcf0209a-2440-4267-834a-b176f4e337e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-rcqkp" Jun 01 21:40:42.833895 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:42.832397 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bcc44458-cf91-4d2d-9b1b-e9da2db57098-tmp-dir\") pod \"node-resolver-kfs2m\" (UID: \"bcc44458-cf91-4d2d-9b1b-e9da2db57098\") " pod="openshift-dns/node-resolver-kfs2m" Jun 01 21:40:42.834275 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:42.834248 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/fcf0209a-2440-4267-834a-b176f4e337e4-ovn-node-metrics-cert\") pod \"ovnkube-node-rcqkp\" (UID: \"fcf0209a-2440-4267-834a-b176f4e337e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-rcqkp" Jun 01 21:40:42.840070 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:42.840017 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nzztm\" (UniqueName: \"kubernetes.io/projected/0192780a-312c-4eae-a683-ca316fa48cb2-kube-api-access-nzztm\") pod \"multus-4vd5l\" (UID: \"0192780a-312c-4eae-a683-ca316fa48cb2\") " pod="openshift-multus/multus-4vd5l" Jun 01 21:40:42.840189 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:42.840105 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sdn65\" (UniqueName: \"kubernetes.io/projected/bcc44458-cf91-4d2d-9b1b-e9da2db57098-kube-api-access-sdn65\") pod \"node-resolver-kfs2m\" (UID: \"bcc44458-cf91-4d2d-9b1b-e9da2db57098\") " pod="openshift-dns/node-resolver-kfs2m" Jun 01 21:40:42.840257 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:42.840231 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lw7gz\" (UniqueName: \"kubernetes.io/projected/fcf0209a-2440-4267-834a-b176f4e337e4-kube-api-access-lw7gz\") pod \"ovnkube-node-rcqkp\" (UID: \"fcf0209a-2440-4267-834a-b176f4e337e4\") " pod="openshift-ovn-kubernetes/ovnkube-node-rcqkp" Jun 01 21:40:42.840301 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:42.840265 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jc4mz\" (UniqueName: \"kubernetes.io/projected/36aa25cf-725a-42f0-8f7b-938f5dbe52d4-kube-api-access-jc4mz\") pod \"multus-additional-cni-plugins-mhq4q\" (UID: \"36aa25cf-725a-42f0-8f7b-938f5dbe52d4\") " pod="openshift-multus/multus-additional-cni-plugins-mhq4q" Jun 01 21:40:42.922244 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:42.922207 2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-mwdhn" Jun 01 21:40:42.928073 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:42.928050 2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-qcs4d" Jun 01 21:40:42.935075 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:42.935055 2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-bfz2l" Jun 01 21:40:42.941617 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:42.941595 2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-c88gr" Jun 01 21:40:42.949237 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:42.949210 2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c2qnd" Jun 01 21:40:42.956961 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:42.956938 2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-rcqkp" Jun 01 21:40:42.962602 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:42.962578 2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-kfs2m" Jun 01 21:40:42.970230 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:42.970210 2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-mhq4q" Jun 01 21:40:42.979913 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:42.979885 2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-4vd5l" Jun 01 21:40:43.234735 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:43.234655 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f99cc923-cc5c-4895-8b2b-6ba878ba343b-metrics-certs\") pod \"network-metrics-daemon-w4d7s\" (UID: \"f99cc923-cc5c-4895-8b2b-6ba878ba343b\") " pod="openshift-multus/network-metrics-daemon-w4d7s" Jun 01 21:40:43.234898 ip-10-0-138-241 kubenswrapper[2573]: E0601 21:40:43.234781 2573 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 01 21:40:43.234898 ip-10-0-138-241 kubenswrapper[2573]: E0601 21:40:43.234833 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f99cc923-cc5c-4895-8b2b-6ba878ba343b-metrics-certs podName:f99cc923-cc5c-4895-8b2b-6ba878ba343b nodeName:}" failed. No retries permitted until 2026-06-01 21:40:44.234820424 +0000 UTC m=+3.954872344 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f99cc923-cc5c-4895-8b2b-6ba878ba343b-metrics-certs") pod "network-metrics-daemon-w4d7s" (UID: "f99cc923-cc5c-4895-8b2b-6ba878ba343b") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 01 21:40:43.335660 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:43.335528 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-7dqw7\" (UniqueName: \"kubernetes.io/projected/e2e547dd-415d-46fe-9608-4a3410d73834-kube-api-access-7dqw7\") pod \"network-check-target-z7k6x\" (UID: \"e2e547dd-415d-46fe-9608-4a3410d73834\") " pod="openshift-network-diagnostics/network-check-target-z7k6x" Jun 01 21:40:43.335833 ip-10-0-138-241 kubenswrapper[2573]: E0601 21:40:43.335675 2573 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 01 21:40:43.335833 ip-10-0-138-241 kubenswrapper[2573]: E0601 21:40:43.335695 2573 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 01 21:40:43.335833 ip-10-0-138-241 kubenswrapper[2573]: E0601 21:40:43.335707 2573 projected.go:194] Error preparing data for projected volume kube-api-access-7dqw7 for pod openshift-network-diagnostics/network-check-target-z7k6x: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 01 21:40:43.335833 ip-10-0-138-241 kubenswrapper[2573]: E0601 21:40:43.335769 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e2e547dd-415d-46fe-9608-4a3410d73834-kube-api-access-7dqw7 podName:e2e547dd-415d-46fe-9608-4a3410d73834 nodeName:}" failed. No retries permitted until 2026-06-01 21:40:44.335750405 +0000 UTC m=+4.055802345 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-7dqw7" (UniqueName: "kubernetes.io/projected/e2e547dd-415d-46fe-9608-4a3410d73834-kube-api-access-7dqw7") pod "network-check-target-z7k6x" (UID: "e2e547dd-415d-46fe-9608-4a3410d73834") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 01 21:40:43.475213 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:43.475185 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbcc44458_cf91_4d2d_9b1b_e9da2db57098.slice/crio-5f9a060aa177bd4dab116dfa27d8db5571d35200b774a921b5eae43c69ef6d7c WatchSource:0}: Error finding container 5f9a060aa177bd4dab116dfa27d8db5571d35200b774a921b5eae43c69ef6d7c: Status 404 returned error can't find the container with id 5f9a060aa177bd4dab116dfa27d8db5571d35200b774a921b5eae43c69ef6d7c Jun 01 21:40:43.490928 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:43.490903 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0192780a_312c_4eae_a683_ca316fa48cb2.slice/crio-774f1f374136df57aa7d7e8bb217a9cc1e51f21dfe5d8d9cc19b1021045e983f WatchSource:0}: Error finding container 774f1f374136df57aa7d7e8bb217a9cc1e51f21dfe5d8d9cc19b1021045e983f: Status 404 returned error can't find the container with id 774f1f374136df57aa7d7e8bb217a9cc1e51f21dfe5d8d9cc19b1021045e983f Jun 01 21:40:43.491562 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:43.491449 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7ada00c1_dbeb_4df5_bfca_923ef619c2d1.slice/crio-29fade27ba84821fa8fcfc40ce5e3793c35b0d3f7e9fb97a104e07289ed3afd4 WatchSource:0}: Error finding container 29fade27ba84821fa8fcfc40ce5e3793c35b0d3f7e9fb97a104e07289ed3afd4: Status 404 returned error can't find the container with id 29fade27ba84821fa8fcfc40ce5e3793c35b0d3f7e9fb97a104e07289ed3afd4 Jun 01 21:40:43.492386 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:43.492364 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfcf0209a_2440_4267_834a_b176f4e337e4.slice/crio-ce8abd0064c230c701a7afc5a63d7cfd5e3779e433292ce36c9ea5c9afbe2401 WatchSource:0}: Error finding container ce8abd0064c230c701a7afc5a63d7cfd5e3779e433292ce36c9ea5c9afbe2401: Status 404 returned error can't find the container with id ce8abd0064c230c701a7afc5a63d7cfd5e3779e433292ce36c9ea5c9afbe2401 Jun 01 21:40:43.493260 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:43.493187 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod36aa25cf_725a_42f0_8f7b_938f5dbe52d4.slice/crio-749dc9201c6117e956139f223ca5b731e49173f3cf6a9945c49a2676e034ca8d WatchSource:0}: Error finding container 749dc9201c6117e956139f223ca5b731e49173f3cf6a9945c49a2676e034ca8d: Status 404 returned error can't find the container with id 749dc9201c6117e956139f223ca5b731e49173f3cf6a9945c49a2676e034ca8d Jun 01 21:40:43.494229 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:43.494206 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfbd5149c_c44e_4b0d_959f_a8c158933a5c.slice/crio-c4eab81bfed0746b2f819c7e31007e8203e224881bef6624d69ed3a8e19a24cf WatchSource:0}: Error finding container c4eab81bfed0746b2f819c7e31007e8203e224881bef6624d69ed3a8e19a24cf: Status 404 returned error can't find the container with id c4eab81bfed0746b2f819c7e31007e8203e224881bef6624d69ed3a8e19a24cf Jun 01 21:40:43.497154 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:43.497132 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9a3c0ed4_56c7_4f73_a903_7b8fe4652a41.slice/crio-ef78b1b8c79d22f6227fc4e3ecb037c527888f837b9661130f8f8ab2dc4be131 WatchSource:0}: Error finding container ef78b1b8c79d22f6227fc4e3ecb037c527888f837b9661130f8f8ab2dc4be131: Status 404 returned error can't find the container with id ef78b1b8c79d22f6227fc4e3ecb037c527888f837b9661130f8f8ab2dc4be131 Jun 01 21:40:43.498017 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:43.497977 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc2491d2d_ab6b_422e_af1d_41cc31129778.slice/crio-574cc1940d1756d39cd57a852ba73bb1b872a89c8f16da0ff831a19e88b2b3ad WatchSource:0}: Error finding container 574cc1940d1756d39cd57a852ba73bb1b872a89c8f16da0ff831a19e88b2b3ad: Status 404 returned error can't find the container with id 574cc1940d1756d39cd57a852ba73bb1b872a89c8f16da0ff831a19e88b2b3ad Jun 01 21:40:43.499989 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:40:43.499967 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbd935121_781a_4420_80da_4b28e731a602.slice/crio-12af8a541f6aa18ab84da0cf9624e460da7ad6bf594710e69043957fa838a201 WatchSource:0}: Error finding container 12af8a541f6aa18ab84da0cf9624e460da7ad6bf594710e69043957fa838a201: Status 404 returned error can't find the container with id 12af8a541f6aa18ab84da0cf9624e460da7ad6bf594710e69043957fa838a201 Jun 01 21:40:43.661442 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:43.661254 2573 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-05-31 21:35:41 +0000 UTC" deadline="2028-02-22 05:52:43.302121164 +0000 UTC" Jun 01 21:40:43.661442 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:43.661436 2573 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15128h11m59.640687598s" Jun 01 21:40:43.683100 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:43.683066 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-241.ec2.internal" event={"ID":"bbf2997371dfbc83d81e28332ffdac56","Type":"ContainerStarted","Data":"ea03cfc49fab6a6c7049ce6a4a5ce3013794c34e8955b279cfe620b63984adc5"} Jun 01 21:40:43.684206 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:43.684176 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c2qnd" event={"ID":"bd935121-781a-4420-80da-4b28e731a602","Type":"ContainerStarted","Data":"12af8a541f6aa18ab84da0cf9624e460da7ad6bf594710e69043957fa838a201"} Jun 01 21:40:43.685205 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:43.685177 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-bfz2l" event={"ID":"c2491d2d-ab6b-422e-af1d-41cc31129778","Type":"ContainerStarted","Data":"574cc1940d1756d39cd57a852ba73bb1b872a89c8f16da0ff831a19e88b2b3ad"} Jun 01 21:40:43.686154 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:43.686127 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-mwdhn" event={"ID":"9a3c0ed4-56c7-4f73-a903-7b8fe4652a41","Type":"ContainerStarted","Data":"ef78b1b8c79d22f6227fc4e3ecb037c527888f837b9661130f8f8ab2dc4be131"} Jun 01 21:40:43.686958 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:43.686936 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-c88gr" event={"ID":"fbd5149c-c44e-4b0d-959f-a8c158933a5c","Type":"ContainerStarted","Data":"c4eab81bfed0746b2f819c7e31007e8203e224881bef6624d69ed3a8e19a24cf"} Jun 01 21:40:43.687958 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:43.687939 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-rcqkp" event={"ID":"fcf0209a-2440-4267-834a-b176f4e337e4","Type":"ContainerStarted","Data":"ce8abd0064c230c701a7afc5a63d7cfd5e3779e433292ce36c9ea5c9afbe2401"} Jun 01 21:40:43.689339 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:43.689309 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-4vd5l" event={"ID":"0192780a-312c-4eae-a683-ca316fa48cb2","Type":"ContainerStarted","Data":"774f1f374136df57aa7d7e8bb217a9cc1e51f21dfe5d8d9cc19b1021045e983f"} Jun 01 21:40:43.691603 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:43.691584 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-kfs2m" event={"ID":"bcc44458-cf91-4d2d-9b1b-e9da2db57098","Type":"ContainerStarted","Data":"5f9a060aa177bd4dab116dfa27d8db5571d35200b774a921b5eae43c69ef6d7c"} Jun 01 21:40:43.692421 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:43.692401 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mhq4q" event={"ID":"36aa25cf-725a-42f0-8f7b-938f5dbe52d4","Type":"ContainerStarted","Data":"749dc9201c6117e956139f223ca5b731e49173f3cf6a9945c49a2676e034ca8d"} Jun 01 21:40:43.693915 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:43.693898 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-qcs4d" event={"ID":"7ada00c1-dbeb-4df5-bfca-923ef619c2d1","Type":"ContainerStarted","Data":"29fade27ba84821fa8fcfc40ce5e3793c35b0d3f7e9fb97a104e07289ed3afd4"} Jun 01 21:40:44.242730 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:44.242694 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f99cc923-cc5c-4895-8b2b-6ba878ba343b-metrics-certs\") pod \"network-metrics-daemon-w4d7s\" (UID: \"f99cc923-cc5c-4895-8b2b-6ba878ba343b\") " pod="openshift-multus/network-metrics-daemon-w4d7s" Jun 01 21:40:44.242905 ip-10-0-138-241 kubenswrapper[2573]: E0601 21:40:44.242879 2573 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 01 21:40:44.242968 ip-10-0-138-241 kubenswrapper[2573]: E0601 21:40:44.242946 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f99cc923-cc5c-4895-8b2b-6ba878ba343b-metrics-certs podName:f99cc923-cc5c-4895-8b2b-6ba878ba343b nodeName:}" failed. No retries permitted until 2026-06-01 21:40:46.242927572 +0000 UTC m=+5.962979500 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f99cc923-cc5c-4895-8b2b-6ba878ba343b-metrics-certs") pod "network-metrics-daemon-w4d7s" (UID: "f99cc923-cc5c-4895-8b2b-6ba878ba343b") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 01 21:40:44.343657 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:44.343614 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-7dqw7\" (UniqueName: \"kubernetes.io/projected/e2e547dd-415d-46fe-9608-4a3410d73834-kube-api-access-7dqw7\") pod \"network-check-target-z7k6x\" (UID: \"e2e547dd-415d-46fe-9608-4a3410d73834\") " pod="openshift-network-diagnostics/network-check-target-z7k6x" Jun 01 21:40:44.343871 ip-10-0-138-241 kubenswrapper[2573]: E0601 21:40:44.343848 2573 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 01 21:40:44.343871 ip-10-0-138-241 kubenswrapper[2573]: E0601 21:40:44.343870 2573 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 01 21:40:44.343993 ip-10-0-138-241 kubenswrapper[2573]: E0601 21:40:44.343883 2573 projected.go:194] Error preparing data for projected volume kube-api-access-7dqw7 for pod openshift-network-diagnostics/network-check-target-z7k6x: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 01 21:40:44.343993 ip-10-0-138-241 kubenswrapper[2573]: E0601 21:40:44.343950 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e2e547dd-415d-46fe-9608-4a3410d73834-kube-api-access-7dqw7 podName:e2e547dd-415d-46fe-9608-4a3410d73834 nodeName:}" failed. No retries permitted until 2026-06-01 21:40:46.343931211 +0000 UTC m=+6.063983134 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-7dqw7" (UniqueName: "kubernetes.io/projected/e2e547dd-415d-46fe-9608-4a3410d73834-kube-api-access-7dqw7") pod "network-check-target-z7k6x" (UID: "e2e547dd-415d-46fe-9608-4a3410d73834") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 01 21:40:44.676600 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:44.673661 2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z7k6x" Jun 01 21:40:44.676600 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:44.673730 2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w4d7s" Jun 01 21:40:44.676600 ip-10-0-138-241 kubenswrapper[2573]: E0601 21:40:44.673841 2573 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-w4d7s" podUID="f99cc923-cc5c-4895-8b2b-6ba878ba343b" Jun 01 21:40:44.676600 ip-10-0-138-241 kubenswrapper[2573]: E0601 21:40:44.674265 2573 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z7k6x" podUID="e2e547dd-415d-46fe-9608-4a3410d73834" Jun 01 21:40:44.710330 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:44.710290 2573 generic.go:358] "Generic (PLEG): container finished" podID="79c6196cbb2336b7ddc61a969d8bd0f9" containerID="40fe66c004c846f91a72b171960b7b00daeac32382594f439ef6bb896689a203" exitCode=0 Jun 01 21:40:44.711320 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:44.711292 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-241.ec2.internal" event={"ID":"79c6196cbb2336b7ddc61a969d8bd0f9","Type":"ContainerDied","Data":"40fe66c004c846f91a72b171960b7b00daeac32382594f439ef6bb896689a203"} Jun 01 21:40:44.726932 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:44.725803 2573 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-241.ec2.internal" podStartSLOduration=2.725784642 podStartE2EDuration="2.725784642s" podCreationTimestamp="2026-06-01 21:40:42 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-06-01 21:40:43.698820286 +0000 UTC m=+3.418872229" watchObservedRunningTime="2026-06-01 21:40:44.725784642 +0000 UTC m=+4.445836586" Jun 01 21:40:45.728216 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:45.728175 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-241.ec2.internal" event={"ID":"79c6196cbb2336b7ddc61a969d8bd0f9","Type":"ContainerStarted","Data":"dd57b245a7b51501dc7622e153c5394c4db14cdd9361e13b5ea58b3315e5d229"} Jun 01 21:40:46.261643 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:46.261604 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f99cc923-cc5c-4895-8b2b-6ba878ba343b-metrics-certs\") pod \"network-metrics-daemon-w4d7s\" (UID: \"f99cc923-cc5c-4895-8b2b-6ba878ba343b\") " pod="openshift-multus/network-metrics-daemon-w4d7s" Jun 01 21:40:46.261840 ip-10-0-138-241 kubenswrapper[2573]: E0601 21:40:46.261768 2573 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 01 21:40:46.261840 ip-10-0-138-241 kubenswrapper[2573]: E0601 21:40:46.261829 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f99cc923-cc5c-4895-8b2b-6ba878ba343b-metrics-certs podName:f99cc923-cc5c-4895-8b2b-6ba878ba343b nodeName:}" failed. No retries permitted until 2026-06-01 21:40:50.261812027 +0000 UTC m=+9.981863962 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f99cc923-cc5c-4895-8b2b-6ba878ba343b-metrics-certs") pod "network-metrics-daemon-w4d7s" (UID: "f99cc923-cc5c-4895-8b2b-6ba878ba343b") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 01 21:40:46.363257 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:46.362601 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-7dqw7\" (UniqueName: \"kubernetes.io/projected/e2e547dd-415d-46fe-9608-4a3410d73834-kube-api-access-7dqw7\") pod \"network-check-target-z7k6x\" (UID: \"e2e547dd-415d-46fe-9608-4a3410d73834\") " pod="openshift-network-diagnostics/network-check-target-z7k6x" Jun 01 21:40:46.363257 ip-10-0-138-241 kubenswrapper[2573]: E0601 21:40:46.362821 2573 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 01 21:40:46.363257 ip-10-0-138-241 kubenswrapper[2573]: E0601 21:40:46.362842 2573 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 01 21:40:46.363257 ip-10-0-138-241 kubenswrapper[2573]: E0601 21:40:46.362855 2573 projected.go:194] Error preparing data for projected volume kube-api-access-7dqw7 for pod openshift-network-diagnostics/network-check-target-z7k6x: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 01 21:40:46.363257 ip-10-0-138-241 kubenswrapper[2573]: E0601 21:40:46.362912 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e2e547dd-415d-46fe-9608-4a3410d73834-kube-api-access-7dqw7 podName:e2e547dd-415d-46fe-9608-4a3410d73834 nodeName:}" failed. No retries permitted until 2026-06-01 21:40:50.362894073 +0000 UTC m=+10.082945996 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-7dqw7" (UniqueName: "kubernetes.io/projected/e2e547dd-415d-46fe-9608-4a3410d73834-kube-api-access-7dqw7") pod "network-check-target-z7k6x" (UID: "e2e547dd-415d-46fe-9608-4a3410d73834") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 01 21:40:46.673780 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:46.673411 2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w4d7s" Jun 01 21:40:46.673780 ip-10-0-138-241 kubenswrapper[2573]: E0601 21:40:46.673575 2573 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-w4d7s" podUID="f99cc923-cc5c-4895-8b2b-6ba878ba343b" Jun 01 21:40:46.674026 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:46.673980 2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z7k6x" Jun 01 21:40:46.674099 ip-10-0-138-241 kubenswrapper[2573]: E0601 21:40:46.674086 2573 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z7k6x" podUID="e2e547dd-415d-46fe-9608-4a3410d73834" Jun 01 21:40:48.674028 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:48.673810 2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w4d7s" Jun 01 21:40:48.674028 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:48.673844 2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z7k6x" Jun 01 21:40:48.674606 ip-10-0-138-241 kubenswrapper[2573]: E0601 21:40:48.674055 2573 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-w4d7s" podUID="f99cc923-cc5c-4895-8b2b-6ba878ba343b" Jun 01 21:40:48.674606 ip-10-0-138-241 kubenswrapper[2573]: E0601 21:40:48.674171 2573 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z7k6x" podUID="e2e547dd-415d-46fe-9608-4a3410d73834" Jun 01 21:40:50.292472 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:50.292433 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f99cc923-cc5c-4895-8b2b-6ba878ba343b-metrics-certs\") pod \"network-metrics-daemon-w4d7s\" (UID: \"f99cc923-cc5c-4895-8b2b-6ba878ba343b\") " pod="openshift-multus/network-metrics-daemon-w4d7s" Jun 01 21:40:50.292959 ip-10-0-138-241 kubenswrapper[2573]: E0601 21:40:50.292563 2573 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 01 21:40:50.292959 ip-10-0-138-241 kubenswrapper[2573]: E0601 21:40:50.292656 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f99cc923-cc5c-4895-8b2b-6ba878ba343b-metrics-certs podName:f99cc923-cc5c-4895-8b2b-6ba878ba343b nodeName:}" failed. No retries permitted until 2026-06-01 21:40:58.292634937 +0000 UTC m=+18.012686883 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f99cc923-cc5c-4895-8b2b-6ba878ba343b-metrics-certs") pod "network-metrics-daemon-w4d7s" (UID: "f99cc923-cc5c-4895-8b2b-6ba878ba343b") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 01 21:40:50.393216 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:50.393175 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-7dqw7\" (UniqueName: \"kubernetes.io/projected/e2e547dd-415d-46fe-9608-4a3410d73834-kube-api-access-7dqw7\") pod \"network-check-target-z7k6x\" (UID: \"e2e547dd-415d-46fe-9608-4a3410d73834\") " pod="openshift-network-diagnostics/network-check-target-z7k6x" Jun 01 21:40:50.393415 ip-10-0-138-241 kubenswrapper[2573]: E0601 21:40:50.393372 2573 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 01 21:40:50.393415 ip-10-0-138-241 kubenswrapper[2573]: E0601 21:40:50.393397 2573 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 01 21:40:50.393415 ip-10-0-138-241 kubenswrapper[2573]: E0601 21:40:50.393410 2573 projected.go:194] Error preparing data for projected volume kube-api-access-7dqw7 for pod openshift-network-diagnostics/network-check-target-z7k6x: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 01 21:40:50.393584 ip-10-0-138-241 kubenswrapper[2573]: E0601 21:40:50.393477 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e2e547dd-415d-46fe-9608-4a3410d73834-kube-api-access-7dqw7 podName:e2e547dd-415d-46fe-9608-4a3410d73834 nodeName:}" failed. No retries permitted until 2026-06-01 21:40:58.393458104 +0000 UTC m=+18.113510042 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-7dqw7" (UniqueName: "kubernetes.io/projected/e2e547dd-415d-46fe-9608-4a3410d73834-kube-api-access-7dqw7") pod "network-check-target-z7k6x" (UID: "e2e547dd-415d-46fe-9608-4a3410d73834") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 01 21:40:50.675350 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:50.675279 2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z7k6x" Jun 01 21:40:50.675350 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:50.675320 2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w4d7s" Jun 01 21:40:50.675570 ip-10-0-138-241 kubenswrapper[2573]: E0601 21:40:50.675393 2573 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z7k6x" podUID="e2e547dd-415d-46fe-9608-4a3410d73834" Jun 01 21:40:50.675570 ip-10-0-138-241 kubenswrapper[2573]: E0601 21:40:50.675547 2573 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-w4d7s" podUID="f99cc923-cc5c-4895-8b2b-6ba878ba343b" Jun 01 21:40:52.674921 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:52.674380 2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w4d7s" Jun 01 21:40:52.674921 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:52.674438 2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z7k6x" Jun 01 21:40:52.674921 ip-10-0-138-241 kubenswrapper[2573]: E0601 21:40:52.674552 2573 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-w4d7s" podUID="f99cc923-cc5c-4895-8b2b-6ba878ba343b" Jun 01 21:40:52.674921 ip-10-0-138-241 kubenswrapper[2573]: E0601 21:40:52.674635 2573 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z7k6x" podUID="e2e547dd-415d-46fe-9608-4a3410d73834" Jun 01 21:40:54.674640 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:54.674592 2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z7k6x" Jun 01 21:40:54.675197 ip-10-0-138-241 kubenswrapper[2573]: E0601 21:40:54.674757 2573 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z7k6x" podUID="e2e547dd-415d-46fe-9608-4a3410d73834" Jun 01 21:40:54.675197 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:54.674592 2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w4d7s" Jun 01 21:40:54.675197 ip-10-0-138-241 kubenswrapper[2573]: E0601 21:40:54.674978 2573 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-w4d7s" podUID="f99cc923-cc5c-4895-8b2b-6ba878ba343b" Jun 01 21:40:56.674319 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:56.674277 2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w4d7s" Jun 01 21:40:56.674319 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:56.674277 2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z7k6x" Jun 01 21:40:56.674849 ip-10-0-138-241 kubenswrapper[2573]: E0601 21:40:56.674413 2573 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-w4d7s" podUID="f99cc923-cc5c-4895-8b2b-6ba878ba343b" Jun 01 21:40:56.674849 ip-10-0-138-241 kubenswrapper[2573]: E0601 21:40:56.674517 2573 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z7k6x" podUID="e2e547dd-415d-46fe-9608-4a3410d73834" Jun 01 21:40:58.344340 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:58.344301 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f99cc923-cc5c-4895-8b2b-6ba878ba343b-metrics-certs\") pod \"network-metrics-daemon-w4d7s\" (UID: \"f99cc923-cc5c-4895-8b2b-6ba878ba343b\") " pod="openshift-multus/network-metrics-daemon-w4d7s" Jun 01 21:40:58.344758 ip-10-0-138-241 kubenswrapper[2573]: E0601 21:40:58.344434 2573 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 01 21:40:58.344758 ip-10-0-138-241 kubenswrapper[2573]: E0601 21:40:58.344497 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f99cc923-cc5c-4895-8b2b-6ba878ba343b-metrics-certs podName:f99cc923-cc5c-4895-8b2b-6ba878ba343b nodeName:}" failed. No retries permitted until 2026-06-01 21:41:14.344482428 +0000 UTC m=+34.064534349 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f99cc923-cc5c-4895-8b2b-6ba878ba343b-metrics-certs") pod "network-metrics-daemon-w4d7s" (UID: "f99cc923-cc5c-4895-8b2b-6ba878ba343b") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 01 21:40:58.444932 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:58.444887 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-7dqw7\" (UniqueName: \"kubernetes.io/projected/e2e547dd-415d-46fe-9608-4a3410d73834-kube-api-access-7dqw7\") pod \"network-check-target-z7k6x\" (UID: \"e2e547dd-415d-46fe-9608-4a3410d73834\") " pod="openshift-network-diagnostics/network-check-target-z7k6x" Jun 01 21:40:58.445125 ip-10-0-138-241 kubenswrapper[2573]: E0601 21:40:58.445089 2573 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 01 21:40:58.445125 ip-10-0-138-241 kubenswrapper[2573]: E0601 21:40:58.445113 2573 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 01 21:40:58.445125 ip-10-0-138-241 kubenswrapper[2573]: E0601 21:40:58.445127 2573 projected.go:194] Error preparing data for projected volume kube-api-access-7dqw7 for pod openshift-network-diagnostics/network-check-target-z7k6x: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 01 21:40:58.445272 ip-10-0-138-241 kubenswrapper[2573]: E0601 21:40:58.445194 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e2e547dd-415d-46fe-9608-4a3410d73834-kube-api-access-7dqw7 podName:e2e547dd-415d-46fe-9608-4a3410d73834 nodeName:}" failed. No retries permitted until 2026-06-01 21:41:14.445175477 +0000 UTC m=+34.165227401 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-7dqw7" (UniqueName: "kubernetes.io/projected/e2e547dd-415d-46fe-9608-4a3410d73834-kube-api-access-7dqw7") pod "network-check-target-z7k6x" (UID: "e2e547dd-415d-46fe-9608-4a3410d73834") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 01 21:40:58.673591 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:58.673364 2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w4d7s" Jun 01 21:40:58.673591 ip-10-0-138-241 kubenswrapper[2573]: E0601 21:40:58.673508 2573 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-w4d7s" podUID="f99cc923-cc5c-4895-8b2b-6ba878ba343b" Jun 01 21:40:58.673591 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:40:58.673567 2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z7k6x" Jun 01 21:40:58.673855 ip-10-0-138-241 kubenswrapper[2573]: E0601 21:40:58.673671 2573 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z7k6x" podUID="e2e547dd-415d-46fe-9608-4a3410d73834" Jun 01 21:41:00.677615 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:41:00.677579 2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w4d7s" Jun 01 21:41:00.678068 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:41:00.677592 2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z7k6x" Jun 01 21:41:00.678068 ip-10-0-138-241 kubenswrapper[2573]: E0601 21:41:00.677701 2573 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-w4d7s" podUID="f99cc923-cc5c-4895-8b2b-6ba878ba343b" Jun 01 21:41:00.678068 ip-10-0-138-241 kubenswrapper[2573]: E0601 21:41:00.677828 2573 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z7k6x" podUID="e2e547dd-415d-46fe-9608-4a3410d73834" Jun 01 21:41:02.673737 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:41:02.673391 2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w4d7s" Jun 01 21:41:02.674691 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:41:02.673445 2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z7k6x" Jun 01 21:41:02.674691 ip-10-0-138-241 kubenswrapper[2573]: E0601 21:41:02.673849 2573 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-w4d7s" podUID="f99cc923-cc5c-4895-8b2b-6ba878ba343b" Jun 01 21:41:02.674691 ip-10-0-138-241 kubenswrapper[2573]: E0601 21:41:02.673913 2573 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z7k6x" podUID="e2e547dd-415d-46fe-9608-4a3410d73834" Jun 01 21:41:02.759107 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:41:02.759069 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c2qnd" event={"ID":"bd935121-781a-4420-80da-4b28e731a602","Type":"ContainerStarted","Data":"3a701da19616a39d4458d021f7555a8b9a3679d40c19132e950a8e067b26a7f7"} Jun 01 21:41:02.760466 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:41:02.760427 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-bfz2l" event={"ID":"c2491d2d-ab6b-422e-af1d-41cc31129778","Type":"ContainerStarted","Data":"f6d7735dbb86fe1e07f152b9b4177ee9cd7ffb085d03a1ead39813b2c91e5683"} Jun 01 21:41:02.761837 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:41:02.761807 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-mwdhn" event={"ID":"9a3c0ed4-56c7-4f73-a903-7b8fe4652a41","Type":"ContainerStarted","Data":"650ddc4836c67cca833ddf0c1f0d058568f893c2e26c17fbf94dbb80986e6cc4"} Jun 01 21:41:02.764383 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:41:02.764364 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rcqkp_fcf0209a-2440-4267-834a-b176f4e337e4/ovn-acl-logging/0.log" Jun 01 21:41:02.764715 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:41:02.764693 2573 generic.go:358] "Generic (PLEG): container finished" podID="fcf0209a-2440-4267-834a-b176f4e337e4" containerID="be231aa51df163fd061be1066427213ead0e03360ae8c497ec218f19b815bb1b" exitCode=1 Jun 01 21:41:02.764807 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:41:02.764757 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-rcqkp" event={"ID":"fcf0209a-2440-4267-834a-b176f4e337e4","Type":"ContainerStarted","Data":"20855cdd30846e34c35c043067115935920714bfa62e7cbc1cd45b5c023a0947"} Jun 01 21:41:02.764807 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:41:02.764777 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-rcqkp" event={"ID":"fcf0209a-2440-4267-834a-b176f4e337e4","Type":"ContainerStarted","Data":"9d89a0216b4b2050170bb9438b1757be7ff7ac97a675a215b8281388f091daff"} Jun 01 21:41:02.764807 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:41:02.764790 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-rcqkp" event={"ID":"fcf0209a-2440-4267-834a-b176f4e337e4","Type":"ContainerStarted","Data":"e72322732971222e4b97b351f9aa24412770b8c06cff186ea7f3793a1a10e683"} Jun 01 21:41:02.764807 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:41:02.764802 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-rcqkp" event={"ID":"fcf0209a-2440-4267-834a-b176f4e337e4","Type":"ContainerStarted","Data":"414d00e8fdd4f52b8ed6106cf982ac91cf9d85b62f6b13b881cf7e19735ce573"} Jun 01 21:41:02.764985 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:41:02.764823 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-rcqkp" event={"ID":"fcf0209a-2440-4267-834a-b176f4e337e4","Type":"ContainerDied","Data":"be231aa51df163fd061be1066427213ead0e03360ae8c497ec218f19b815bb1b"} Jun 01 21:41:02.764985 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:41:02.764837 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-rcqkp" event={"ID":"fcf0209a-2440-4267-834a-b176f4e337e4","Type":"ContainerStarted","Data":"9d87d8ac3317a90f882348eb0a1be88d3e98487b4f6c7217967881d85667d4ac"} Jun 01 21:41:02.766319 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:41:02.766296 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-4vd5l" event={"ID":"0192780a-312c-4eae-a683-ca316fa48cb2","Type":"ContainerStarted","Data":"5c140548d7208a3d75cca52c1d88b166278ae9bcea1a8452da34e52bdd919744"} Jun 01 21:41:02.767647 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:41:02.767625 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-kfs2m" event={"ID":"bcc44458-cf91-4d2d-9b1b-e9da2db57098","Type":"ContainerStarted","Data":"168b958945759fa72ca086fcf6c2dd752a1557485d2ed7cb672efc9670af73f9"} Jun 01 21:41:02.769194 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:41:02.769173 2573 generic.go:358] "Generic (PLEG): container finished" podID="36aa25cf-725a-42f0-8f7b-938f5dbe52d4" containerID="1f2e3300fac5ea901f3efe3fdd1d315e6ca4aeae2021e006ce40bb33825f8450" exitCode=0 Jun 01 21:41:02.769293 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:41:02.769245 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mhq4q" event={"ID":"36aa25cf-725a-42f0-8f7b-938f5dbe52d4","Type":"ContainerDied","Data":"1f2e3300fac5ea901f3efe3fdd1d315e6ca4aeae2021e006ce40bb33825f8450"} Jun 01 21:41:02.770601 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:41:02.770574 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-qcs4d" event={"ID":"7ada00c1-dbeb-4df5-bfca-923ef619c2d1","Type":"ContainerStarted","Data":"874566b4ed0ae010f4ee71ab848c40b66a128c64ed8a3a70046db155882bf47e"} Jun 01 21:41:02.790634 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:41:02.790585 2573 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-241.ec2.internal" podStartSLOduration=20.790516046 podStartE2EDuration="20.790516046s" podCreationTimestamp="2026-06-01 21:40:42 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-06-01 21:40:45.742861149 +0000 UTC m=+5.462913093" watchObservedRunningTime="2026-06-01 21:41:02.790516046 +0000 UTC m=+22.510567990" Jun 01 21:41:02.790915 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:41:02.790890 2573 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-bfz2l" podStartSLOduration=4.61542853 podStartE2EDuration="22.790882557s" podCreationTimestamp="2026-06-01 21:40:40 +0000 UTC" firstStartedPulling="2026-06-01 21:40:43.500679339 +0000 UTC m=+3.220731263" lastFinishedPulling="2026-06-01 21:41:01.676133363 +0000 UTC m=+21.396185290" observedRunningTime="2026-06-01 21:41:02.790203748 +0000 UTC m=+22.510255692" watchObservedRunningTime="2026-06-01 21:41:02.790882557 +0000 UTC m=+22.510934496" Jun 01 21:41:02.805258 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:41:02.805199 2573 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-dns/node-resolver-kfs2m" podStartSLOduration=4.047325588 podStartE2EDuration="21.805187601s" podCreationTimestamp="2026-06-01 21:40:41 +0000 UTC" firstStartedPulling="2026-06-01 21:40:43.483237482 +0000 UTC m=+3.203289403" lastFinishedPulling="2026-06-01 21:41:01.241099487 +0000 UTC m=+20.961151416" observedRunningTime="2026-06-01 21:41:02.804511886 +0000 UTC m=+22.524563829" watchObservedRunningTime="2026-06-01 21:41:02.805187601 +0000 UTC m=+22.525239546" Jun 01 21:41:02.825090 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:41:02.825039 2573 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-multus/multus-4vd5l" podStartSLOduration=3.636094495 podStartE2EDuration="21.825025672s" podCreationTimestamp="2026-06-01 21:40:41 +0000 UTC" firstStartedPulling="2026-06-01 21:40:43.492722987 +0000 UTC m=+3.212774909" lastFinishedPulling="2026-06-01 21:41:01.681654161 +0000 UTC m=+21.401706086" observedRunningTime="2026-06-01 21:41:02.824928988 +0000 UTC m=+22.544980931" watchObservedRunningTime="2026-06-01 21:41:02.825025672 +0000 UTC m=+22.545077611" Jun 01 21:41:02.843526 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:41:02.843474 2573 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kube-system/konnectivity-agent-mwdhn" podStartSLOduration=4.669006733 podStartE2EDuration="22.843462022s" podCreationTimestamp="2026-06-01 21:40:40 +0000 UTC" firstStartedPulling="2026-06-01 21:40:43.499846016 +0000 UTC m=+3.219897938" lastFinishedPulling="2026-06-01 21:41:01.6743013 +0000 UTC m=+21.394353227" observedRunningTime="2026-06-01 21:41:02.843118539 +0000 UTC m=+22.563170483" watchObservedRunningTime="2026-06-01 21:41:02.843462022 +0000 UTC m=+22.563513964" Jun 01 21:41:02.884989 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:41:02.884940 2573 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-image-registry/node-ca-qcs4d" podStartSLOduration=9.467597242 podStartE2EDuration="22.88492355s" podCreationTimestamp="2026-06-01 21:40:40 +0000 UTC" firstStartedPulling="2026-06-01 21:40:43.493720049 +0000 UTC m=+3.213771970" lastFinishedPulling="2026-06-01 21:40:56.911046345 +0000 UTC m=+16.631098278" observedRunningTime="2026-06-01 21:41:02.884681079 +0000 UTC m=+22.604733021" watchObservedRunningTime="2026-06-01 21:41:02.88492355 +0000 UTC m=+22.604975508" Jun 01 21:41:03.506199 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:41:03.506174 2573 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jun 01 21:41:03.696384 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:41:03.696241 2573 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-06-01T21:41:03.506192932Z","UUID":"f2629619-83b7-464c-804f-4309960b7ef6","Handler":null,"Name":"","Endpoint":""} Jun 01 21:41:03.698032 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:41:03.698010 2573 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jun 01 21:41:03.698164 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:41:03.698039 2573 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jun 01 21:41:03.773990 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:41:03.773957 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c2qnd" event={"ID":"bd935121-781a-4420-80da-4b28e731a602","Type":"ContainerStarted","Data":"5cdc24aed70acd35c46b58e49561c71bd1879500fcb52ffa9671bd7ee3da6593"} Jun 01 21:41:03.775165 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:41:03.775126 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-c88gr" event={"ID":"fbd5149c-c44e-4b0d-959f-a8c158933a5c","Type":"ContainerStarted","Data":"cf4a05e8f742a9de194c4b9aef4c9b5f55068634d2527adc3e739da29aef3a76"} Jun 01 21:41:03.790723 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:41:03.790677 2573 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-c88gr" podStartSLOduration=9.37684144 podStartE2EDuration="22.790665742s" podCreationTimestamp="2026-06-01 21:40:41 +0000 UTC" firstStartedPulling="2026-06-01 21:40:43.497220341 +0000 UTC m=+3.217272266" lastFinishedPulling="2026-06-01 21:40:56.911044631 +0000 UTC m=+16.631096568" observedRunningTime="2026-06-01 21:41:03.790592806 +0000 UTC m=+23.510644748" watchObservedRunningTime="2026-06-01 21:41:03.790665742 +0000 UTC m=+23.510717684" Jun 01 21:41:04.673747 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:41:04.673719 2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z7k6x" Jun 01 21:41:04.673913 ip-10-0-138-241 kubenswrapper[2573]: E0601 21:41:04.673839 2573 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z7k6x" podUID="e2e547dd-415d-46fe-9608-4a3410d73834" Jun 01 21:41:04.673913 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:41:04.673877 2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w4d7s" Jun 01 21:41:04.674032 ip-10-0-138-241 kubenswrapper[2573]: E0601 21:41:04.673965 2573 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-w4d7s" podUID="f99cc923-cc5c-4895-8b2b-6ba878ba343b" Jun 01 21:41:04.780046 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:41:04.779969 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rcqkp_fcf0209a-2440-4267-834a-b176f4e337e4/ovn-acl-logging/0.log" Jun 01 21:41:04.780457 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:41:04.780371 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-rcqkp" event={"ID":"fcf0209a-2440-4267-834a-b176f4e337e4","Type":"ContainerStarted","Data":"ddc51ebb37f37a97ecc7b1726d965bbe7cea2a5976292d994e487d76b8f1c95a"} Jun 01 21:41:05.783975 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:41:05.783780 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c2qnd" event={"ID":"bd935121-781a-4420-80da-4b28e731a602","Type":"ContainerStarted","Data":"abe4ce73f8d355d07b29ccda4ce16116987824de4cd5dbe48f1b414191417248"} Jun 01 21:41:05.803172 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:41:05.803108 2573 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c2qnd" podStartSLOduration=3.446630567 podStartE2EDuration="24.803088818s" podCreationTimestamp="2026-06-01 21:40:41 +0000 UTC" firstStartedPulling="2026-06-01 21:40:43.501665947 +0000 UTC m=+3.221717867" lastFinishedPulling="2026-06-01 21:41:04.858124185 +0000 UTC m=+24.578176118" observedRunningTime="2026-06-01 21:41:05.803059106 +0000 UTC m=+25.523111071" watchObservedRunningTime="2026-06-01 21:41:05.803088818 +0000 UTC m=+25.523140766" Jun 01 21:41:06.674261 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:41:06.674222 2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w4d7s" Jun 01 21:41:06.674440 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:41:06.674266 2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z7k6x" Jun 01 21:41:06.674440 ip-10-0-138-241 kubenswrapper[2573]: E0601 21:41:06.674349 2573 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-w4d7s" podUID="f99cc923-cc5c-4895-8b2b-6ba878ba343b" Jun 01 21:41:06.674580 ip-10-0-138-241 kubenswrapper[2573]: E0601 21:41:06.674475 2573 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z7k6x" podUID="e2e547dd-415d-46fe-9608-4a3410d73834" Jun 01 21:41:06.789182 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:41:06.789109 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rcqkp_fcf0209a-2440-4267-834a-b176f4e337e4/ovn-acl-logging/0.log" Jun 01 21:41:06.790431 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:41:06.789583 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-rcqkp" event={"ID":"fcf0209a-2440-4267-834a-b176f4e337e4","Type":"ContainerStarted","Data":"b4c683940beee2b1abf5cb3ef6b2a4d5be333624f05d1be9ea50f2e2da0feb5c"} Jun 01 21:41:06.790431 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:41:06.790174 2573 scope.go:117] "RemoveContainer" containerID="be231aa51df163fd061be1066427213ead0e03360ae8c497ec218f19b815bb1b" Jun 01 21:41:06.813175 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:41:06.810681 2573 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-mwdhn" Jun 01 21:41:07.196339 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:41:07.196156 2573 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-mwdhn" Jun 01 21:41:07.197208 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:41:07.197190 2573 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-mwdhn" Jun 01 21:41:07.795057 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:41:07.795031 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rcqkp_fcf0209a-2440-4267-834a-b176f4e337e4/ovn-acl-logging/0.log" Jun 01 21:41:07.795481 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:41:07.795443 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-rcqkp" event={"ID":"fcf0209a-2440-4267-834a-b176f4e337e4","Type":"ContainerStarted","Data":"7ed35af844b289e13fc871a0bef31043bef6d4b02a4edffb6127500c66f7621c"} Jun 01 21:41:07.795964 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:41:07.795944 2573 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-rcqkp" Jun 01 21:41:07.796078 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:41:07.795993 2573 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-rcqkp" Jun 01 21:41:07.796078 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:41:07.796012 2573 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-rcqkp" Jun 01 21:41:07.796192 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:41:07.796172 2573 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-mwdhn" Jun 01 21:41:07.812836 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:41:07.812811 2573 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-rcqkp" Jun 01 21:41:07.812978 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:41:07.812879 2573 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-rcqkp" Jun 01 21:41:07.826159 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:41:07.826111 2573 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-rcqkp" podStartSLOduration=8.592846659 podStartE2EDuration="26.826094389s" podCreationTimestamp="2026-06-01 21:40:41 +0000 UTC" firstStartedPulling="2026-06-01 21:40:43.494186822 +0000 UTC m=+3.214238744" lastFinishedPulling="2026-06-01 21:41:01.727434542 +0000 UTC m=+21.447486474" observedRunningTime="2026-06-01 21:41:07.823989275 +0000 UTC m=+27.544041217" watchObservedRunningTime="2026-06-01 21:41:07.826094389 +0000 UTC m=+27.546146332" Jun 01 21:41:08.673615 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:41:08.673577 2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z7k6x" Jun 01 21:41:08.673783 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:41:08.673631 2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w4d7s" Jun 01 21:41:08.673783 ip-10-0-138-241 kubenswrapper[2573]: E0601 21:41:08.673717 2573 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z7k6x" podUID="e2e547dd-415d-46fe-9608-4a3410d73834" Jun 01 21:41:08.673905 ip-10-0-138-241 kubenswrapper[2573]: E0601 21:41:08.673825 2573 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-w4d7s" podUID="f99cc923-cc5c-4895-8b2b-6ba878ba343b" Jun 01 21:41:08.703647 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:41:08.703611 2573 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-z7k6x"] Jun 01 21:41:08.708154 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:41:08.708127 2573 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-w4d7s"] Jun 01 21:41:08.797142 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:41:08.797110 2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w4d7s" Jun 01 21:41:08.797142 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:41:08.797110 2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z7k6x" Jun 01 21:41:08.797652 ip-10-0-138-241 kubenswrapper[2573]: E0601 21:41:08.797319 2573 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-w4d7s" podUID="f99cc923-cc5c-4895-8b2b-6ba878ba343b" Jun 01 21:41:08.797652 ip-10-0-138-241 kubenswrapper[2573]: E0601 21:41:08.797345 2573 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z7k6x" podUID="e2e547dd-415d-46fe-9608-4a3410d73834" Jun 01 21:41:09.800587 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:41:09.800486 2573 generic.go:358] "Generic (PLEG): container finished" podID="36aa25cf-725a-42f0-8f7b-938f5dbe52d4" containerID="1456fcd5cb43621a7a513a5af49c8ee26fdd128e6a4915a60e219c8d9d1c3f7a" exitCode=0 Jun 01 21:41:09.800587 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:41:09.800566 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mhq4q" event={"ID":"36aa25cf-725a-42f0-8f7b-938f5dbe52d4","Type":"ContainerDied","Data":"1456fcd5cb43621a7a513a5af49c8ee26fdd128e6a4915a60e219c8d9d1c3f7a"} Jun 01 21:41:10.677506 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:41:10.676303 2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w4d7s" Jun 01 21:41:10.677506 ip-10-0-138-241 kubenswrapper[2573]: E0601 21:41:10.676467 2573 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-w4d7s" podUID="f99cc923-cc5c-4895-8b2b-6ba878ba343b" Jun 01 21:41:10.677506 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:41:10.676938 2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z7k6x" Jun 01 21:41:10.677506 ip-10-0-138-241 kubenswrapper[2573]: E0601 21:41:10.677060 2573 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z7k6x" podUID="e2e547dd-415d-46fe-9608-4a3410d73834" Jun 01 21:41:11.807003 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:41:11.806756 2573 generic.go:358] "Generic (PLEG): container finished" podID="36aa25cf-725a-42f0-8f7b-938f5dbe52d4" containerID="ee1549fb9af0da533e8857e4a1ce8abba6771008652948a677aba266baea96f2" exitCode=0 Jun 01 21:41:11.807003 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:41:11.806894 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mhq4q" event={"ID":"36aa25cf-725a-42f0-8f7b-938f5dbe52d4","Type":"ContainerDied","Data":"ee1549fb9af0da533e8857e4a1ce8abba6771008652948a677aba266baea96f2"} Jun 01 21:41:12.674014 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:41:12.673983 2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z7k6x" Jun 01 21:41:12.674137 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:41:12.673975 2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w4d7s" Jun 01 21:41:12.674137 ip-10-0-138-241 kubenswrapper[2573]: E0601 21:41:12.674112 2573 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-z7k6x" podUID="e2e547dd-415d-46fe-9608-4a3410d73834" Jun 01 21:41:12.674215 ip-10-0-138-241 kubenswrapper[2573]: E0601 21:41:12.674130 2573 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-w4d7s" podUID="f99cc923-cc5c-4895-8b2b-6ba878ba343b" Jun 01 21:41:12.811436 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:41:12.811350 2573 generic.go:358] "Generic (PLEG): container finished" podID="36aa25cf-725a-42f0-8f7b-938f5dbe52d4" containerID="5a2ea5a476168665997d0148d59d0bc90f099dff3373d22615375fc58f9540b9" exitCode=0 Jun 01 21:41:12.811436 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:41:12.811393 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mhq4q" event={"ID":"36aa25cf-725a-42f0-8f7b-938f5dbe52d4","Type":"ContainerDied","Data":"5a2ea5a476168665997d0148d59d0bc90f099dff3373d22615375fc58f9540b9"} Jun 01 21:41:14.354208 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:41:14.354185 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f99cc923-cc5c-4895-8b2b-6ba878ba343b-metrics-certs\") pod \"network-metrics-daemon-w4d7s\" (UID: \"f99cc923-cc5c-4895-8b2b-6ba878ba343b\") " pod="openshift-multus/network-metrics-daemon-w4d7s" Jun 01 21:41:14.354671 ip-10-0-138-241 kubenswrapper[2573]: E0601 21:41:14.354296 2573 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 01 21:41:14.354671 ip-10-0-138-241 kubenswrapper[2573]: E0601 21:41:14.354349 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f99cc923-cc5c-4895-8b2b-6ba878ba343b-metrics-certs podName:f99cc923-cc5c-4895-8b2b-6ba878ba343b nodeName:}" failed. No retries permitted until 2026-06-01 21:41:46.354331739 +0000 UTC m=+66.074383678 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f99cc923-cc5c-4895-8b2b-6ba878ba343b-metrics-certs") pod "network-metrics-daemon-w4d7s" (UID: "f99cc923-cc5c-4895-8b2b-6ba878ba343b") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 01 21:41:14.455456 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:41:14.455414 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-7dqw7\" (UniqueName: \"kubernetes.io/projected/e2e547dd-415d-46fe-9608-4a3410d73834-kube-api-access-7dqw7\") pod \"network-check-target-z7k6x\" (UID: \"e2e547dd-415d-46fe-9608-4a3410d73834\") " pod="openshift-network-diagnostics/network-check-target-z7k6x" Jun 01 21:41:14.455635 ip-10-0-138-241 kubenswrapper[2573]: E0601 21:41:14.455599 2573 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 01 21:41:14.455635 ip-10-0-138-241 kubenswrapper[2573]: E0601 21:41:14.455625 2573 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 01 21:41:14.455635 ip-10-0-138-241 kubenswrapper[2573]: E0601 21:41:14.455637 2573 projected.go:194] Error preparing data for projected volume kube-api-access-7dqw7 for pod openshift-network-diagnostics/network-check-target-z7k6x: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 01 21:41:14.455783 ip-10-0-138-241 kubenswrapper[2573]: E0601 21:41:14.455702 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e2e547dd-415d-46fe-9608-4a3410d73834-kube-api-access-7dqw7 podName:e2e547dd-415d-46fe-9608-4a3410d73834 nodeName:}" failed. No retries permitted until 2026-06-01 21:41:46.45568172 +0000 UTC m=+66.175733645 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-7dqw7" (UniqueName: "kubernetes.io/projected/e2e547dd-415d-46fe-9608-4a3410d73834-kube-api-access-7dqw7") pod "network-check-target-z7k6x" (UID: "e2e547dd-415d-46fe-9608-4a3410d73834") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 01 21:41:14.656288 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:41:14.656257 2573 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-138-241.ec2.internal" event="NodeReady" Jun 01 21:41:14.656471 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:41:14.656425 2573 kubelet_node_status.go:550] "Fast updating node status as it just became ready" Jun 01 21:41:14.674330 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:41:14.674300 2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w4d7s" Jun 01 21:41:14.674486 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:41:14.674472 2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z7k6x" Jun 01 21:41:14.677905 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:41:14.677303 2573 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jun 01 21:41:14.677905 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:41:14.677483 2573 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-dlgnh\"" Jun 01 21:41:14.677905 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:41:14.677565 2573 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jun 01 21:41:14.677905 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:41:14.677788 2573 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jun 01 21:41:14.677905 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:41:14.677799 2573 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-vgrbz\"" Jun 01 21:41:14.688903 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:41:14.688881 2573 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5f78687ff8-gl77x"] Jun 01 21:41:14.693397 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:41:14.693377 2573 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-77447d89bb-twkln"] Jun 01 21:41:14.693548 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:41:14.693513 2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5f78687ff8-gl77x" Jun 01 21:41:14.696285 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:41:14.696264 2573 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-bb6fb6964-vcwwk"] Jun 01 21:41:14.696477 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:41:14.696441 2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-77447d89bb-twkln" Jun 01 21:41:14.696589 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:41:14.696569 2573 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Jun 01 21:41:14.696666 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:41:14.696650 2573 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Jun 01 21:41:14.696726 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:41:14.696711 2573 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\"" Jun 01 21:41:14.696780 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:41:14.696576 2573 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Jun 01 21:41:14.699434 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:41:14.699417 2573 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-5f6f4f9b8b-7wqzv"] Jun 01 21:41:14.699559 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:41:14.699522 2573 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jun 01 21:41:14.699632 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:41:14.699594 2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-bb6fb6964-vcwwk" Jun 01 21:41:14.699689 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:41:14.699638 2573 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jun 01 21:41:14.700290 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:41:14.699849 2573 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jun 01 21:41:14.700290 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:41:14.699995 2573 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-vvsxk\"" Jun 01 21:41:14.705642 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:41:14.705618 2573 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\"" Jun 01 21:41:14.706171 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:41:14.706150 2573 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\"" Jun 01 21:41:14.732973 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:41:14.732924 2573 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5f78687ff8-gl77x"] Jun 01 21:41:14.733155 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:41:14.733086 2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5f6f4f9b8b-7wqzv" Jun 01 21:41:14.736323 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:41:14.734848 2573 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\"" Jun 01 21:41:14.737104 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:41:14.737078 2573 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jun 01 21:41:14.737336 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:41:14.737315 2573 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\"" Jun 01 21:41:14.737440 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:41:14.737397 2573 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\"" Jun 01 21:41:14.737646 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:41:14.737632 2573 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-5f6f4f9b8b-7wqzv"] Jun 01 21:41:14.738243 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:41:14.738223 2573 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-bb6fb6964-vcwwk"] Jun 01 21:41:14.738767 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:41:14.738747 2573 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-77447d89bb-twkln"] Jun 01 21:41:14.739348 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:41:14.739327 2573 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-pdmsk"] Jun 01 21:41:14.742222 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:41:14.742206 2573 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-pdmsk"] Jun 01 21:41:14.742311 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:41:14.742301 2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-pdmsk" Jun 01 21:41:14.745498 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:41:14.745477 2573 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jun 01 21:41:14.745613 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:41:14.745476 2573 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jun 01 21:41:14.745613 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:41:14.745478 2573 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-dqpsx\"" Jun 01 21:41:14.745948 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:41:14.745932 2573 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jun 01 21:41:14.756938 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:41:14.756917 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wrvgr\" (UniqueName: \"kubernetes.io/projected/3224f451-02bc-496d-9fa0-9045b1dbb798-kube-api-access-wrvgr\") pod \"image-registry-77447d89bb-twkln\" (UID: \"3224f451-02bc-496d-9fa0-9045b1dbb798\") " pod="openshift-image-registry/image-registry-77447d89bb-twkln" Jun 01 21:41:14.757058 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:41:14.756944 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/3224f451-02bc-496d-9fa0-9045b1dbb798-ca-trust-extracted\") pod \"image-registry-77447d89bb-twkln\" (UID: \"3224f451-02bc-496d-9fa0-9045b1dbb798\") " pod="openshift-image-registry/image-registry-77447d89bb-twkln" Jun 01 21:41:14.757058 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:41:14.756963 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/3224f451-02bc-496d-9fa0-9045b1dbb798-bound-sa-token\") pod \"image-registry-77447d89bb-twkln\" (UID: \"3224f451-02bc-496d-9fa0-9045b1dbb798\") " pod="openshift-image-registry/image-registry-77447d89bb-twkln" Jun 01 21:41:14.757140 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:41:14.757076 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ocpservice-ca\" (UniqueName: \"kubernetes.io/configmap/5e29e06d-c3f7-4b72-a9c7-99e78ad3946c-ocpservice-ca\") pod \"cluster-proxy-proxy-agent-bb6fb6964-vcwwk\" (UID: \"5e29e06d-c3f7-4b72-a9c7-99e78ad3946c\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-bb6fb6964-vcwwk" Jun 01 21:41:14.757140 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:41:14.757107 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t68qw\" (UniqueName: \"kubernetes.io/projected/5e29e06d-c3f7-4b72-a9c7-99e78ad3946c-kube-api-access-t68qw\") pod \"cluster-proxy-proxy-agent-bb6fb6964-vcwwk\" (UID: \"5e29e06d-c3f7-4b72-a9c7-99e78ad3946c\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-bb6fb6964-vcwwk" Jun 01 21:41:14.757140 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:41:14.757126 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/aaa23eec-441d-40fa-9d2a-54dbd73e994b-tmp\") pod \"klusterlet-addon-workmgr-5f6f4f9b8b-7wqzv\" (UID: \"aaa23eec-441d-40fa-9d2a-54dbd73e994b\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5f6f4f9b8b-7wqzv" Jun 01 21:41:14.757282 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:41:14.757144 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/3224f451-02bc-496d-9fa0-9045b1dbb798-registry-certificates\") pod \"image-registry-77447d89bb-twkln\" (UID: \"3224f451-02bc-496d-9fa0-9045b1dbb798\") " pod="openshift-image-registry/image-registry-77447d89bb-twkln" Jun 01 21:41:14.757282 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:41:14.757164 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/3224f451-02bc-496d-9fa0-9045b1dbb798-installation-pull-secrets\") pod \"image-registry-77447d89bb-twkln\" (UID: \"3224f451-02bc-496d-9fa0-9045b1dbb798\") " pod="openshift-image-registry/image-registry-77447d89bb-twkln" Jun 01 21:41:14.757282 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:41:14.757198 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mr6h4\" (UniqueName: \"kubernetes.io/projected/fd119bc7-9dc0-4e69-a312-61601c58c0af-kube-api-access-mr6h4\") pod \"managed-serviceaccount-addon-agent-5f78687ff8-gl77x\" (UID: \"fd119bc7-9dc0-4e69-a312-61601c58c0af\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5f78687ff8-gl77x" Jun 01 21:41:14.757282 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:41:14.757226 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/5e29e06d-c3f7-4b72-a9c7-99e78ad3946c-ca\") pod \"cluster-proxy-proxy-agent-bb6fb6964-vcwwk\" (UID: \"5e29e06d-c3f7-4b72-a9c7-99e78ad3946c\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-bb6fb6964-vcwwk" Jun 01 21:41:14.757428 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:41:14.757276 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/3224f451-02bc-496d-9fa0-9045b1dbb798-image-registry-private-configuration\") pod \"image-registry-77447d89bb-twkln\" (UID: \"3224f451-02bc-496d-9fa0-9045b1dbb798\") " pod="openshift-image-registry/image-registry-77447d89bb-twkln" Jun 01 21:41:14.757428 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:41:14.757309 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/3224f451-02bc-496d-9fa0-9045b1dbb798-registry-tls\") pod \"image-registry-77447d89bb-twkln\" (UID: \"3224f451-02bc-496d-9fa0-9045b1dbb798\") " pod="openshift-image-registry/image-registry-77447d89bb-twkln" Jun 01 21:41:14.757428 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:41:14.757342 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/fd119bc7-9dc0-4e69-a312-61601c58c0af-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-5f78687ff8-gl77x\" (UID: \"fd119bc7-9dc0-4e69-a312-61601c58c0af\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5f78687ff8-gl77x" Jun 01 21:41:14.757428 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:41:14.757373 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/5e29e06d-c3f7-4b72-a9c7-99e78ad3946c-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-bb6fb6964-vcwwk\" (UID: \"5e29e06d-c3f7-4b72-a9c7-99e78ad3946c\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-bb6fb6964-vcwwk" Jun 01 21:41:14.757428 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:41:14.757399 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/aaa23eec-441d-40fa-9d2a-54dbd73e994b-klusterlet-config\") pod \"klusterlet-addon-workmgr-5f6f4f9b8b-7wqzv\" (UID: \"aaa23eec-441d-40fa-9d2a-54dbd73e994b\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5f6f4f9b8b-7wqzv" Jun 01 21:41:14.757428 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:41:14.757422 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8njh6\" (UniqueName: \"kubernetes.io/projected/aaa23eec-441d-40fa-9d2a-54dbd73e994b-kube-api-access-8njh6\") pod \"klusterlet-addon-workmgr-5f6f4f9b8b-7wqzv\" (UID: \"aaa23eec-441d-40fa-9d2a-54dbd73e994b\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5f6f4f9b8b-7wqzv" Jun 01 21:41:14.757757 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:41:14.757445 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/3224f451-02bc-496d-9fa0-9045b1dbb798-trusted-ca\") pod \"image-registry-77447d89bb-twkln\" (UID: \"3224f451-02bc-496d-9fa0-9045b1dbb798\") " pod="openshift-image-registry/image-registry-77447d89bb-twkln" Jun 01 21:41:14.757757 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:41:14.757468 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/5e29e06d-c3f7-4b72-a9c7-99e78ad3946c-hub\") pod \"cluster-proxy-proxy-agent-bb6fb6964-vcwwk\" (UID: \"5e29e06d-c3f7-4b72-a9c7-99e78ad3946c\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-bb6fb6964-vcwwk" Jun 01 21:41:14.757757 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:41:14.757497 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/5e29e06d-c3f7-4b72-a9c7-99e78ad3946c-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-bb6fb6964-vcwwk\" (UID: \"5e29e06d-c3f7-4b72-a9c7-99e78ad3946c\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-bb6fb6964-vcwwk" Jun 01 21:41:14.805585 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:41:14.805553 2573 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-tnd8z"] Jun 01 21:41:14.808816 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:41:14.808786 2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-tnd8z" Jun 01 21:41:14.811642 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:41:14.811618 2573 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jun 01 21:41:14.811761 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:41:14.811733 2573 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-6g4lx\"" Jun 01 21:41:14.811811 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:41:14.811761 2573 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jun 01 21:41:14.816901 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:41:14.816876 2573 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-tnd8z"] Jun 01 21:41:14.858744 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:41:14.858713 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ocpservice-ca\" (UniqueName: \"kubernetes.io/configmap/5e29e06d-c3f7-4b72-a9c7-99e78ad3946c-ocpservice-ca\") pod \"cluster-proxy-proxy-agent-bb6fb6964-vcwwk\" (UID: \"5e29e06d-c3f7-4b72-a9c7-99e78ad3946c\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-bb6fb6964-vcwwk" Jun 01 21:41:14.858744 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:41:14.858748 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-t68qw\" (UniqueName: \"kubernetes.io/projected/5e29e06d-c3f7-4b72-a9c7-99e78ad3946c-kube-api-access-t68qw\") pod \"cluster-proxy-proxy-agent-bb6fb6964-vcwwk\" (UID: \"5e29e06d-c3f7-4b72-a9c7-99e78ad3946c\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-bb6fb6964-vcwwk" Jun 01 21:41:14.858987 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:41:14.858771 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/aaa23eec-441d-40fa-9d2a-54dbd73e994b-tmp\") pod \"klusterlet-addon-workmgr-5f6f4f9b8b-7wqzv\" (UID: \"aaa23eec-441d-40fa-9d2a-54dbd73e994b\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5f6f4f9b8b-7wqzv" Jun 01 21:41:14.858987 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:41:14.858812 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/3224f451-02bc-496d-9fa0-9045b1dbb798-registry-certificates\") pod \"image-registry-77447d89bb-twkln\" (UID: \"3224f451-02bc-496d-9fa0-9045b1dbb798\") " pod="openshift-image-registry/image-registry-77447d89bb-twkln" Jun 01 21:41:14.858987 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:41:14.858844 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/3224f451-02bc-496d-9fa0-9045b1dbb798-installation-pull-secrets\") pod \"image-registry-77447d89bb-twkln\" (UID: \"3224f451-02bc-496d-9fa0-9045b1dbb798\") " pod="openshift-image-registry/image-registry-77447d89bb-twkln" Jun 01 21:41:14.858987 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:41:14.858869 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-mr6h4\" (UniqueName: \"kubernetes.io/projected/fd119bc7-9dc0-4e69-a312-61601c58c0af-kube-api-access-mr6h4\") pod \"managed-serviceaccount-addon-agent-5f78687ff8-gl77x\" (UID: \"fd119bc7-9dc0-4e69-a312-61601c58c0af\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5f78687ff8-gl77x" Jun 01 21:41:14.858987 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:41:14.858886 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/5e29e06d-c3f7-4b72-a9c7-99e78ad3946c-ca\") pod \"cluster-proxy-proxy-agent-bb6fb6964-vcwwk\" (UID: \"5e29e06d-c3f7-4b72-a9c7-99e78ad3946c\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-bb6fb6964-vcwwk" Jun 01 21:41:14.859224 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:41:14.859202 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/aaa23eec-441d-40fa-9d2a-54dbd73e994b-tmp\") pod \"klusterlet-addon-workmgr-5f6f4f9b8b-7wqzv\" (UID: \"aaa23eec-441d-40fa-9d2a-54dbd73e994b\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5f6f4f9b8b-7wqzv" Jun 01 21:41:14.859289 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:41:14.858917 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/b57ae13c-b94a-4891-b4d2-0bd1a8873c4f-config-volume\") pod \"dns-default-tnd8z\" (UID: \"b57ae13c-b94a-4891-b4d2-0bd1a8873c4f\") " pod="openshift-dns/dns-default-tnd8z" Jun 01 21:41:14.859289 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:41:14.859274 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/3224f451-02bc-496d-9fa0-9045b1dbb798-image-registry-private-configuration\") pod \"image-registry-77447d89bb-twkln\" (UID: \"3224f451-02bc-496d-9fa0-9045b1dbb798\") " pod="openshift-image-registry/image-registry-77447d89bb-twkln" Jun 01 21:41:14.859395 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:41:14.859298 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/3224f451-02bc-496d-9fa0-9045b1dbb798-registry-tls\") pod \"image-registry-77447d89bb-twkln\" (UID: \"3224f451-02bc-496d-9fa0-9045b1dbb798\") " pod="openshift-image-registry/image-registry-77447d89bb-twkln" Jun 01 21:41:14.859395 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:41:14.859323 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/fd119bc7-9dc0-4e69-a312-61601c58c0af-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-5f78687ff8-gl77x\" (UID: \"fd119bc7-9dc0-4e69-a312-61601c58c0af\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5f78687ff8-gl77x" Jun 01 21:41:14.859395 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:41:14.859346 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/5e29e06d-c3f7-4b72-a9c7-99e78ad3946c-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-bb6fb6964-vcwwk\" (UID: \"5e29e06d-c3f7-4b72-a9c7-99e78ad3946c\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-bb6fb6964-vcwwk" Jun 01 21:41:14.859395 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:41:14.859367 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/aaa23eec-441d-40fa-9d2a-54dbd73e994b-klusterlet-config\") pod \"klusterlet-addon-workmgr-5f6f4f9b8b-7wqzv\" (UID: \"aaa23eec-441d-40fa-9d2a-54dbd73e994b\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5f6f4f9b8b-7wqzv" Jun 01 21:41:14.859395 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:41:14.859388 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-8njh6\" (UniqueName: \"kubernetes.io/projected/aaa23eec-441d-40fa-9d2a-54dbd73e994b-kube-api-access-8njh6\") pod \"klusterlet-addon-workmgr-5f6f4f9b8b-7wqzv\" (UID: \"aaa23eec-441d-40fa-9d2a-54dbd73e994b\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5f6f4f9b8b-7wqzv" Jun 01 21:41:14.859673 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:41:14.859416 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/3224f451-02bc-496d-9fa0-9045b1dbb798-trusted-ca\") pod \"image-registry-77447d89bb-twkln\" (UID: \"3224f451-02bc-496d-9fa0-9045b1dbb798\") " pod="openshift-image-registry/image-registry-77447d89bb-twkln" Jun 01 21:41:14.859673 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:41:14.859442 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nwhgp\" (UniqueName: \"kubernetes.io/projected/81b4edc5-d95f-4cae-9191-e27f75e48d8a-kube-api-access-nwhgp\") pod \"ingress-canary-pdmsk\" (UID: \"81b4edc5-d95f-4cae-9191-e27f75e48d8a\") " pod="openshift-ingress-canary/ingress-canary-pdmsk" Jun 01 21:41:14.859673 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:41:14.859468 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b57ae13c-b94a-4891-b4d2-0bd1a8873c4f-tmp-dir\") pod \"dns-default-tnd8z\" (UID: \"b57ae13c-b94a-4891-b4d2-0bd1a8873c4f\") " pod="openshift-dns/dns-default-tnd8z" Jun 01 21:41:14.859673 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:41:14.859502 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/5e29e06d-c3f7-4b72-a9c7-99e78ad3946c-hub\") pod \"cluster-proxy-proxy-agent-bb6fb6964-vcwwk\" (UID: \"5e29e06d-c3f7-4b72-a9c7-99e78ad3946c\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-bb6fb6964-vcwwk" Jun 01 21:41:14.859673 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:41:14.859552 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/5e29e06d-c3f7-4b72-a9c7-99e78ad3946c-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-bb6fb6964-vcwwk\" (UID: \"5e29e06d-c3f7-4b72-a9c7-99e78ad3946c\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-bb6fb6964-vcwwk" Jun 01 21:41:14.859673 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:41:14.859590 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-wrvgr\" (UniqueName: \"kubernetes.io/projected/3224f451-02bc-496d-9fa0-9045b1dbb798-kube-api-access-wrvgr\") pod \"image-registry-77447d89bb-twkln\" (UID: \"3224f451-02bc-496d-9fa0-9045b1dbb798\") " pod="openshift-image-registry/image-registry-77447d89bb-twkln" Jun 01 21:41:14.859673 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:41:14.859619 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/3224f451-02bc-496d-9fa0-9045b1dbb798-ca-trust-extracted\") pod \"image-registry-77447d89bb-twkln\" (UID: \"3224f451-02bc-496d-9fa0-9045b1dbb798\") " pod="openshift-image-registry/image-registry-77447d89bb-twkln" Jun 01 21:41:14.859673 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:41:14.859646 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/81b4edc5-d95f-4cae-9191-e27f75e48d8a-cert\") pod \"ingress-canary-pdmsk\" (UID: \"81b4edc5-d95f-4cae-9191-e27f75e48d8a\") " pod="openshift-ingress-canary/ingress-canary-pdmsk" Jun 01 21:41:14.859673 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:41:14.859672 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b57ae13c-b94a-4891-b4d2-0bd1a8873c4f-metrics-tls\") pod \"dns-default-tnd8z\" (UID: \"b57ae13c-b94a-4891-b4d2-0bd1a8873c4f\") " pod="openshift-dns/dns-default-tnd8z" Jun 01 21:41:14.860102 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:41:14.859695 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sq6nn\" (UniqueName: \"kubernetes.io/projected/b57ae13c-b94a-4891-b4d2-0bd1a8873c4f-kube-api-access-sq6nn\") pod \"dns-default-tnd8z\" (UID: \"b57ae13c-b94a-4891-b4d2-0bd1a8873c4f\") " pod="openshift-dns/dns-default-tnd8z" Jun 01 21:41:14.860102 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:41:14.859701 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ocpservice-ca\" (UniqueName: \"kubernetes.io/configmap/5e29e06d-c3f7-4b72-a9c7-99e78ad3946c-ocpservice-ca\") pod \"cluster-proxy-proxy-agent-bb6fb6964-vcwwk\" (UID: \"5e29e06d-c3f7-4b72-a9c7-99e78ad3946c\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-bb6fb6964-vcwwk" Jun 01 21:41:14.860102 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:41:14.859723 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/3224f451-02bc-496d-9fa0-9045b1dbb798-bound-sa-token\") pod \"image-registry-77447d89bb-twkln\" (UID: \"3224f451-02bc-496d-9fa0-9045b1dbb798\") " pod="openshift-image-registry/image-registry-77447d89bb-twkln" Jun 01 21:41:14.860102 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:41:14.859800 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/3224f451-02bc-496d-9fa0-9045b1dbb798-registry-certificates\") pod \"image-registry-77447d89bb-twkln\" (UID: \"3224f451-02bc-496d-9fa0-9045b1dbb798\") " pod="openshift-image-registry/image-registry-77447d89bb-twkln" Jun 01 21:41:14.860102 ip-10-0-138-241 kubenswrapper[2573]: E0601 21:41:14.859848 2573 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 01 21:41:14.860102 ip-10-0-138-241 kubenswrapper[2573]: E0601 21:41:14.859863 2573 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-77447d89bb-twkln: secret "image-registry-tls" not found Jun 01 21:41:14.860102 ip-10-0-138-241 kubenswrapper[2573]: E0601 21:41:14.859913 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3224f451-02bc-496d-9fa0-9045b1dbb798-registry-tls podName:3224f451-02bc-496d-9fa0-9045b1dbb798 nodeName:}" failed. No retries permitted until 2026-06-01 21:41:15.359893971 +0000 UTC m=+35.079945903 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/3224f451-02bc-496d-9fa0-9045b1dbb798-registry-tls") pod "image-registry-77447d89bb-twkln" (UID: "3224f451-02bc-496d-9fa0-9045b1dbb798") : secret "image-registry-tls" not found Jun 01 21:41:14.860417 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:41:14.860339 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/3224f451-02bc-496d-9fa0-9045b1dbb798-trusted-ca\") pod \"image-registry-77447d89bb-twkln\" (UID: \"3224f451-02bc-496d-9fa0-9045b1dbb798\") " pod="openshift-image-registry/image-registry-77447d89bb-twkln" Jun 01 21:41:14.861646 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:41:14.860659 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/3224f451-02bc-496d-9fa0-9045b1dbb798-ca-trust-extracted\") pod \"image-registry-77447d89bb-twkln\" (UID: \"3224f451-02bc-496d-9fa0-9045b1dbb798\") " pod="openshift-image-registry/image-registry-77447d89bb-twkln" Jun 01 21:41:14.864442 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:41:14.864419 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/3224f451-02bc-496d-9fa0-9045b1dbb798-image-registry-private-configuration\") pod \"image-registry-77447d89bb-twkln\" (UID: \"3224f451-02bc-496d-9fa0-9045b1dbb798\") " pod="openshift-image-registry/image-registry-77447d89bb-twkln" Jun 01 21:41:14.864630 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:41:14.864455 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/5e29e06d-c3f7-4b72-a9c7-99e78ad3946c-ca\") pod \"cluster-proxy-proxy-agent-bb6fb6964-vcwwk\" (UID: \"5e29e06d-c3f7-4b72-a9c7-99e78ad3946c\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-bb6fb6964-vcwwk" Jun 01 21:41:14.864736 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:41:14.864462 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/5e29e06d-c3f7-4b72-a9c7-99e78ad3946c-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-bb6fb6964-vcwwk\" (UID: \"5e29e06d-c3f7-4b72-a9c7-99e78ad3946c\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-bb6fb6964-vcwwk" Jun 01 21:41:14.864736 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:41:14.864648 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/5e29e06d-c3f7-4b72-a9c7-99e78ad3946c-hub\") pod \"cluster-proxy-proxy-agent-bb6fb6964-vcwwk\" (UID: \"5e29e06d-c3f7-4b72-a9c7-99e78ad3946c\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-bb6fb6964-vcwwk" Jun 01 21:41:14.865011 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:41:14.864988 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/aaa23eec-441d-40fa-9d2a-54dbd73e994b-klusterlet-config\") pod \"klusterlet-addon-workmgr-5f6f4f9b8b-7wqzv\" (UID: \"aaa23eec-441d-40fa-9d2a-54dbd73e994b\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5f6f4f9b8b-7wqzv" Jun 01 21:41:14.865084 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:41:14.865056 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/fd119bc7-9dc0-4e69-a312-61601c58c0af-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-5f78687ff8-gl77x\" (UID: \"fd119bc7-9dc0-4e69-a312-61601c58c0af\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5f78687ff8-gl77x" Jun 01 21:41:14.865506 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:41:14.865482 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/3224f451-02bc-496d-9fa0-9045b1dbb798-installation-pull-secrets\") pod \"image-registry-77447d89bb-twkln\" (UID: \"3224f451-02bc-496d-9fa0-9045b1dbb798\") " pod="openshift-image-registry/image-registry-77447d89bb-twkln" Jun 01 21:41:14.865625 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:41:14.865581 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/5e29e06d-c3f7-4b72-a9c7-99e78ad3946c-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-bb6fb6964-vcwwk\" (UID: \"5e29e06d-c3f7-4b72-a9c7-99e78ad3946c\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-bb6fb6964-vcwwk" Jun 01 21:41:14.868093 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:41:14.868048 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mr6h4\" (UniqueName: \"kubernetes.io/projected/fd119bc7-9dc0-4e69-a312-61601c58c0af-kube-api-access-mr6h4\") pod \"managed-serviceaccount-addon-agent-5f78687ff8-gl77x\" (UID: \"fd119bc7-9dc0-4e69-a312-61601c58c0af\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5f78687ff8-gl77x" Jun 01 21:41:14.869342 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:41:14.869322 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/3224f451-02bc-496d-9fa0-9045b1dbb798-bound-sa-token\") pod \"image-registry-77447d89bb-twkln\" (UID: \"3224f451-02bc-496d-9fa0-9045b1dbb798\") " pod="openshift-image-registry/image-registry-77447d89bb-twkln" Jun 01 21:41:14.870226 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:41:14.870202 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wrvgr\" (UniqueName: \"kubernetes.io/projected/3224f451-02bc-496d-9fa0-9045b1dbb798-kube-api-access-wrvgr\") pod \"image-registry-77447d89bb-twkln\" (UID: \"3224f451-02bc-496d-9fa0-9045b1dbb798\") " pod="openshift-image-registry/image-registry-77447d89bb-twkln" Jun 01 21:41:14.870408 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:41:14.870392 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8njh6\" (UniqueName: \"kubernetes.io/projected/aaa23eec-441d-40fa-9d2a-54dbd73e994b-kube-api-access-8njh6\") pod \"klusterlet-addon-workmgr-5f6f4f9b8b-7wqzv\" (UID: \"aaa23eec-441d-40fa-9d2a-54dbd73e994b\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5f6f4f9b8b-7wqzv" Jun 01 21:41:14.870499 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:41:14.870482 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t68qw\" (UniqueName: \"kubernetes.io/projected/5e29e06d-c3f7-4b72-a9c7-99e78ad3946c-kube-api-access-t68qw\") pod \"cluster-proxy-proxy-agent-bb6fb6964-vcwwk\" (UID: \"5e29e06d-c3f7-4b72-a9c7-99e78ad3946c\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-bb6fb6964-vcwwk" Jun 01 21:41:14.960677 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:41:14.960578 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/81b4edc5-d95f-4cae-9191-e27f75e48d8a-cert\") pod \"ingress-canary-pdmsk\" (UID: \"81b4edc5-d95f-4cae-9191-e27f75e48d8a\") " pod="openshift-ingress-canary/ingress-canary-pdmsk" Jun 01 21:41:14.960677 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:41:14.960627 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b57ae13c-b94a-4891-b4d2-0bd1a8873c4f-metrics-tls\") pod \"dns-default-tnd8z\" (UID: \"b57ae13c-b94a-4891-b4d2-0bd1a8873c4f\") " pod="openshift-dns/dns-default-tnd8z" Jun 01 21:41:14.960899 ip-10-0-138-241 kubenswrapper[2573]: E0601 21:41:14.960738 2573 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found Jun 01 21:41:14.960899 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:41:14.960750 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-sq6nn\" (UniqueName: \"kubernetes.io/projected/b57ae13c-b94a-4891-b4d2-0bd1a8873c4f-kube-api-access-sq6nn\") pod \"dns-default-tnd8z\" (UID: \"b57ae13c-b94a-4891-b4d2-0bd1a8873c4f\") " pod="openshift-dns/dns-default-tnd8z" Jun 01 21:41:14.960899 ip-10-0-138-241 kubenswrapper[2573]: E0601 21:41:14.960826 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/81b4edc5-d95f-4cae-9191-e27f75e48d8a-cert podName:81b4edc5-d95f-4cae-9191-e27f75e48d8a nodeName:}" failed. No retries permitted until 2026-06-01 21:41:15.460807111 +0000 UTC m=+35.180859051 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/81b4edc5-d95f-4cae-9191-e27f75e48d8a-cert") pod "ingress-canary-pdmsk" (UID: "81b4edc5-d95f-4cae-9191-e27f75e48d8a") : secret "canary-serving-cert" not found Jun 01 21:41:14.960899 ip-10-0-138-241 kubenswrapper[2573]: E0601 21:41:14.960883 2573 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 01 21:41:14.961100 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:41:14.960900 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/b57ae13c-b94a-4891-b4d2-0bd1a8873c4f-config-volume\") pod \"dns-default-tnd8z\" (UID: \"b57ae13c-b94a-4891-b4d2-0bd1a8873c4f\") " pod="openshift-dns/dns-default-tnd8z" Jun 01 21:41:14.961100 ip-10-0-138-241 kubenswrapper[2573]: E0601 21:41:14.960951 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b57ae13c-b94a-4891-b4d2-0bd1a8873c4f-metrics-tls podName:b57ae13c-b94a-4891-b4d2-0bd1a8873c4f nodeName:}" failed. No retries permitted until 2026-06-01 21:41:15.460932567 +0000 UTC m=+35.180984490 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b57ae13c-b94a-4891-b4d2-0bd1a8873c4f-metrics-tls") pod "dns-default-tnd8z" (UID: "b57ae13c-b94a-4891-b4d2-0bd1a8873c4f") : secret "dns-default-metrics-tls" not found Jun 01 21:41:14.961100 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:41:14.960986 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-nwhgp\" (UniqueName: \"kubernetes.io/projected/81b4edc5-d95f-4cae-9191-e27f75e48d8a-kube-api-access-nwhgp\") pod \"ingress-canary-pdmsk\" (UID: \"81b4edc5-d95f-4cae-9191-e27f75e48d8a\") " pod="openshift-ingress-canary/ingress-canary-pdmsk" Jun 01 21:41:14.961100 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:41:14.961013 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b57ae13c-b94a-4891-b4d2-0bd1a8873c4f-tmp-dir\") pod \"dns-default-tnd8z\" (UID: \"b57ae13c-b94a-4891-b4d2-0bd1a8873c4f\") " pod="openshift-dns/dns-default-tnd8z" Jun 01 21:41:14.961405 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:41:14.961388 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/b57ae13c-b94a-4891-b4d2-0bd1a8873c4f-config-volume\") pod \"dns-default-tnd8z\" (UID: \"b57ae13c-b94a-4891-b4d2-0bd1a8873c4f\") " pod="openshift-dns/dns-default-tnd8z" Jun 01 21:41:14.962046 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:41:14.962028 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b57ae13c-b94a-4891-b4d2-0bd1a8873c4f-tmp-dir\") pod \"dns-default-tnd8z\" (UID: \"b57ae13c-b94a-4891-b4d2-0bd1a8873c4f\") " pod="openshift-dns/dns-default-tnd8z" Jun 01 21:41:14.969558 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:41:14.969516 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sq6nn\" (UniqueName: \"kubernetes.io/projected/b57ae13c-b94a-4891-b4d2-0bd1a8873c4f-kube-api-access-sq6nn\") pod \"dns-default-tnd8z\" (UID: \"b57ae13c-b94a-4891-b4d2-0bd1a8873c4f\") " pod="openshift-dns/dns-default-tnd8z" Jun 01 21:41:14.969803 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:41:14.969783 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nwhgp\" (UniqueName: \"kubernetes.io/projected/81b4edc5-d95f-4cae-9191-e27f75e48d8a-kube-api-access-nwhgp\") pod \"ingress-canary-pdmsk\" (UID: \"81b4edc5-d95f-4cae-9191-e27f75e48d8a\") " pod="openshift-ingress-canary/ingress-canary-pdmsk" Jun 01 21:41:15.036998 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:41:15.036951 2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5f78687ff8-gl77x" Jun 01 21:41:15.049371 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:41:15.049335 2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-bb6fb6964-vcwwk" Jun 01 21:41:15.066496 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:41:15.066461 2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5f6f4f9b8b-7wqzv" Jun 01 21:41:15.206954 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:41:15.206666 2573 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5f78687ff8-gl77x"] Jun 01 21:41:15.210013 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:41:15.209984 2573 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-bb6fb6964-vcwwk"] Jun 01 21:41:15.211383 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:41:15.211357 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podfd119bc7_9dc0_4e69_a312_61601c58c0af.slice/crio-eb96a5deda0a9c7b5b041f3a6a7039a42028f3d2d00eb0877108b8d8539b2bbe WatchSource:0}: Error finding container eb96a5deda0a9c7b5b041f3a6a7039a42028f3d2d00eb0877108b8d8539b2bbe: Status 404 returned error can't find the container with id eb96a5deda0a9c7b5b041f3a6a7039a42028f3d2d00eb0877108b8d8539b2bbe Jun 01 21:41:15.214477 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:41:15.214432 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5e29e06d_c3f7_4b72_a9c7_99e78ad3946c.slice/crio-f8a0a8f98ae3b98cc4e16c67a34646d25e3386feb1ee68658b34478c8617b418 WatchSource:0}: Error finding container f8a0a8f98ae3b98cc4e16c67a34646d25e3386feb1ee68658b34478c8617b418: Status 404 returned error can't find the container with id f8a0a8f98ae3b98cc4e16c67a34646d25e3386feb1ee68658b34478c8617b418 Jun 01 21:41:15.237603 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:41:15.237552 2573 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-5f6f4f9b8b-7wqzv"] Jun 01 21:41:15.241809 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:41:15.241777 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaaa23eec_441d_40fa_9d2a_54dbd73e994b.slice/crio-3819aac002c029efa9a13db8d987ea4f1397941c6cbc5e781a5f06600993bcd3 WatchSource:0}: Error finding container 3819aac002c029efa9a13db8d987ea4f1397941c6cbc5e781a5f06600993bcd3: Status 404 returned error can't find the container with id 3819aac002c029efa9a13db8d987ea4f1397941c6cbc5e781a5f06600993bcd3 Jun 01 21:41:15.365058 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:41:15.365026 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/3224f451-02bc-496d-9fa0-9045b1dbb798-registry-tls\") pod \"image-registry-77447d89bb-twkln\" (UID: \"3224f451-02bc-496d-9fa0-9045b1dbb798\") " pod="openshift-image-registry/image-registry-77447d89bb-twkln" Jun 01 21:41:15.365460 ip-10-0-138-241 kubenswrapper[2573]: E0601 21:41:15.365160 2573 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 01 21:41:15.365460 ip-10-0-138-241 kubenswrapper[2573]: E0601 21:41:15.365176 2573 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-77447d89bb-twkln: secret "image-registry-tls" not found Jun 01 21:41:15.365460 ip-10-0-138-241 kubenswrapper[2573]: E0601 21:41:15.365234 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3224f451-02bc-496d-9fa0-9045b1dbb798-registry-tls podName:3224f451-02bc-496d-9fa0-9045b1dbb798 nodeName:}" failed. No retries permitted until 2026-06-01 21:41:16.3652186 +0000 UTC m=+36.085270521 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/3224f451-02bc-496d-9fa0-9045b1dbb798-registry-tls") pod "image-registry-77447d89bb-twkln" (UID: "3224f451-02bc-496d-9fa0-9045b1dbb798") : secret "image-registry-tls" not found Jun 01 21:41:15.465419 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:41:15.465329 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/81b4edc5-d95f-4cae-9191-e27f75e48d8a-cert\") pod \"ingress-canary-pdmsk\" (UID: \"81b4edc5-d95f-4cae-9191-e27f75e48d8a\") " pod="openshift-ingress-canary/ingress-canary-pdmsk" Jun 01 21:41:15.465419 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:41:15.465366 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b57ae13c-b94a-4891-b4d2-0bd1a8873c4f-metrics-tls\") pod \"dns-default-tnd8z\" (UID: \"b57ae13c-b94a-4891-b4d2-0bd1a8873c4f\") " pod="openshift-dns/dns-default-tnd8z" Jun 01 21:41:15.465675 ip-10-0-138-241 kubenswrapper[2573]: E0601 21:41:15.465480 2573 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found Jun 01 21:41:15.465675 ip-10-0-138-241 kubenswrapper[2573]: E0601 21:41:15.465558 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/81b4edc5-d95f-4cae-9191-e27f75e48d8a-cert podName:81b4edc5-d95f-4cae-9191-e27f75e48d8a nodeName:}" failed. No retries permitted until 2026-06-01 21:41:16.465525091 +0000 UTC m=+36.185577013 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/81b4edc5-d95f-4cae-9191-e27f75e48d8a-cert") pod "ingress-canary-pdmsk" (UID: "81b4edc5-d95f-4cae-9191-e27f75e48d8a") : secret "canary-serving-cert" not found Jun 01 21:41:15.465675 ip-10-0-138-241 kubenswrapper[2573]: E0601 21:41:15.465490 2573 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 01 21:41:15.465675 ip-10-0-138-241 kubenswrapper[2573]: E0601 21:41:15.465628 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b57ae13c-b94a-4891-b4d2-0bd1a8873c4f-metrics-tls podName:b57ae13c-b94a-4891-b4d2-0bd1a8873c4f nodeName:}" failed. No retries permitted until 2026-06-01 21:41:16.465611759 +0000 UTC m=+36.185663696 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b57ae13c-b94a-4891-b4d2-0bd1a8873c4f-metrics-tls") pod "dns-default-tnd8z" (UID: "b57ae13c-b94a-4891-b4d2-0bd1a8873c4f") : secret "dns-default-metrics-tls" not found Jun 01 21:41:15.818627 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:41:15.818590 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5f6f4f9b8b-7wqzv" event={"ID":"aaa23eec-441d-40fa-9d2a-54dbd73e994b","Type":"ContainerStarted","Data":"3819aac002c029efa9a13db8d987ea4f1397941c6cbc5e781a5f06600993bcd3"} Jun 01 21:41:15.819526 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:41:15.819489 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-bb6fb6964-vcwwk" event={"ID":"5e29e06d-c3f7-4b72-a9c7-99e78ad3946c","Type":"ContainerStarted","Data":"f8a0a8f98ae3b98cc4e16c67a34646d25e3386feb1ee68658b34478c8617b418"} Jun 01 21:41:15.820327 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:41:15.820303 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5f78687ff8-gl77x" event={"ID":"fd119bc7-9dc0-4e69-a312-61601c58c0af","Type":"ContainerStarted","Data":"eb96a5deda0a9c7b5b041f3a6a7039a42028f3d2d00eb0877108b8d8539b2bbe"} Jun 01 21:41:15.907971 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:41:15.907933 2573 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kube-system/global-pull-secret-syncer-hnntt"] Jun 01 21:41:15.911259 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:41:15.911233 2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hnntt" Jun 01 21:41:15.914689 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:41:15.914665 2573 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jun 01 21:41:15.924204 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:41:15.922424 2573 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-hnntt"] Jun 01 21:41:15.969901 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:41:15.969852 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2353b916-e500-4061-adcf-167688af93f0-original-pull-secret\") pod \"global-pull-secret-syncer-hnntt\" (UID: \"2353b916-e500-4061-adcf-167688af93f0\") " pod="kube-system/global-pull-secret-syncer-hnntt" Jun 01 21:41:15.969901 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:41:15.969897 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/2353b916-e500-4061-adcf-167688af93f0-kubelet-config\") pod \"global-pull-secret-syncer-hnntt\" (UID: \"2353b916-e500-4061-adcf-167688af93f0\") " pod="kube-system/global-pull-secret-syncer-hnntt" Jun 01 21:41:15.970148 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:41:15.969917 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/2353b916-e500-4061-adcf-167688af93f0-dbus\") pod \"global-pull-secret-syncer-hnntt\" (UID: \"2353b916-e500-4061-adcf-167688af93f0\") " pod="kube-system/global-pull-secret-syncer-hnntt" Jun 01 21:41:16.071017 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:41:16.070926 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2353b916-e500-4061-adcf-167688af93f0-original-pull-secret\") pod \"global-pull-secret-syncer-hnntt\" (UID: \"2353b916-e500-4061-adcf-167688af93f0\") " pod="kube-system/global-pull-secret-syncer-hnntt" Jun 01 21:41:16.071017 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:41:16.070973 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/2353b916-e500-4061-adcf-167688af93f0-kubelet-config\") pod \"global-pull-secret-syncer-hnntt\" (UID: \"2353b916-e500-4061-adcf-167688af93f0\") " pod="kube-system/global-pull-secret-syncer-hnntt" Jun 01 21:41:16.071017 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:41:16.070992 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/2353b916-e500-4061-adcf-167688af93f0-dbus\") pod \"global-pull-secret-syncer-hnntt\" (UID: \"2353b916-e500-4061-adcf-167688af93f0\") " pod="kube-system/global-pull-secret-syncer-hnntt" Jun 01 21:41:16.071300 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:41:16.071276 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/2353b916-e500-4061-adcf-167688af93f0-kubelet-config\") pod \"global-pull-secret-syncer-hnntt\" (UID: \"2353b916-e500-4061-adcf-167688af93f0\") " pod="kube-system/global-pull-secret-syncer-hnntt" Jun 01 21:41:16.071353 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:41:16.071310 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/2353b916-e500-4061-adcf-167688af93f0-dbus\") pod \"global-pull-secret-syncer-hnntt\" (UID: \"2353b916-e500-4061-adcf-167688af93f0\") " pod="kube-system/global-pull-secret-syncer-hnntt" Jun 01 21:41:16.081166 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:41:16.081102 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/2353b916-e500-4061-adcf-167688af93f0-original-pull-secret\") pod \"global-pull-secret-syncer-hnntt\" (UID: \"2353b916-e500-4061-adcf-167688af93f0\") " pod="kube-system/global-pull-secret-syncer-hnntt" Jun 01 21:41:16.224351 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:41:16.224309 2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hnntt" Jun 01 21:41:16.374791 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:41:16.374703 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/3224f451-02bc-496d-9fa0-9045b1dbb798-registry-tls\") pod \"image-registry-77447d89bb-twkln\" (UID: \"3224f451-02bc-496d-9fa0-9045b1dbb798\") " pod="openshift-image-registry/image-registry-77447d89bb-twkln" Jun 01 21:41:16.375235 ip-10-0-138-241 kubenswrapper[2573]: E0601 21:41:16.374881 2573 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 01 21:41:16.375235 ip-10-0-138-241 kubenswrapper[2573]: E0601 21:41:16.374904 2573 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-77447d89bb-twkln: secret "image-registry-tls" not found Jun 01 21:41:16.375235 ip-10-0-138-241 kubenswrapper[2573]: E0601 21:41:16.374972 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3224f451-02bc-496d-9fa0-9045b1dbb798-registry-tls podName:3224f451-02bc-496d-9fa0-9045b1dbb798 nodeName:}" failed. No retries permitted until 2026-06-01 21:41:18.374949546 +0000 UTC m=+38.095001487 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/3224f451-02bc-496d-9fa0-9045b1dbb798-registry-tls") pod "image-registry-77447d89bb-twkln" (UID: "3224f451-02bc-496d-9fa0-9045b1dbb798") : secret "image-registry-tls" not found Jun 01 21:41:16.406955 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:41:16.406790 2573 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-hnntt"] Jun 01 21:41:16.475908 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:41:16.475867 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/81b4edc5-d95f-4cae-9191-e27f75e48d8a-cert\") pod \"ingress-canary-pdmsk\" (UID: \"81b4edc5-d95f-4cae-9191-e27f75e48d8a\") " pod="openshift-ingress-canary/ingress-canary-pdmsk" Jun 01 21:41:16.476100 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:41:16.475915 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b57ae13c-b94a-4891-b4d2-0bd1a8873c4f-metrics-tls\") pod \"dns-default-tnd8z\" (UID: \"b57ae13c-b94a-4891-b4d2-0bd1a8873c4f\") " pod="openshift-dns/dns-default-tnd8z" Jun 01 21:41:16.476927 ip-10-0-138-241 kubenswrapper[2573]: E0601 21:41:16.476209 2573 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 01 21:41:16.476927 ip-10-0-138-241 kubenswrapper[2573]: E0601 21:41:16.476282 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b57ae13c-b94a-4891-b4d2-0bd1a8873c4f-metrics-tls podName:b57ae13c-b94a-4891-b4d2-0bd1a8873c4f nodeName:}" failed. No retries permitted until 2026-06-01 21:41:18.476262265 +0000 UTC m=+38.196314192 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b57ae13c-b94a-4891-b4d2-0bd1a8873c4f-metrics-tls") pod "dns-default-tnd8z" (UID: "b57ae13c-b94a-4891-b4d2-0bd1a8873c4f") : secret "dns-default-metrics-tls" not found Jun 01 21:41:16.476927 ip-10-0-138-241 kubenswrapper[2573]: E0601 21:41:16.476709 2573 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found Jun 01 21:41:16.476927 ip-10-0-138-241 kubenswrapper[2573]: E0601 21:41:16.476765 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/81b4edc5-d95f-4cae-9191-e27f75e48d8a-cert podName:81b4edc5-d95f-4cae-9191-e27f75e48d8a nodeName:}" failed. No retries permitted until 2026-06-01 21:41:18.4767487 +0000 UTC m=+38.196800627 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/81b4edc5-d95f-4cae-9191-e27f75e48d8a-cert") pod "ingress-canary-pdmsk" (UID: "81b4edc5-d95f-4cae-9191-e27f75e48d8a") : secret "canary-serving-cert" not found Jun 01 21:41:16.831841 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:41:16.831799 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-hnntt" event={"ID":"2353b916-e500-4061-adcf-167688af93f0","Type":"ContainerStarted","Data":"4cd7290f5d336166756add146e6e08b99dbefa98b4663f63ef1c40d5b1a02dc9"} Jun 01 21:41:18.397108 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:41:18.397015 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/3224f451-02bc-496d-9fa0-9045b1dbb798-registry-tls\") pod \"image-registry-77447d89bb-twkln\" (UID: \"3224f451-02bc-496d-9fa0-9045b1dbb798\") " pod="openshift-image-registry/image-registry-77447d89bb-twkln" Jun 01 21:41:18.397629 ip-10-0-138-241 kubenswrapper[2573]: E0601 21:41:18.397188 2573 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 01 21:41:18.397629 ip-10-0-138-241 kubenswrapper[2573]: E0601 21:41:18.397212 2573 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-77447d89bb-twkln: secret "image-registry-tls" not found Jun 01 21:41:18.397629 ip-10-0-138-241 kubenswrapper[2573]: E0601 21:41:18.397280 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3224f451-02bc-496d-9fa0-9045b1dbb798-registry-tls podName:3224f451-02bc-496d-9fa0-9045b1dbb798 nodeName:}" failed. No retries permitted until 2026-06-01 21:41:22.397258833 +0000 UTC m=+42.117310755 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/3224f451-02bc-496d-9fa0-9045b1dbb798-registry-tls") pod "image-registry-77447d89bb-twkln" (UID: "3224f451-02bc-496d-9fa0-9045b1dbb798") : secret "image-registry-tls" not found Jun 01 21:41:18.498356 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:41:18.498267 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/81b4edc5-d95f-4cae-9191-e27f75e48d8a-cert\") pod \"ingress-canary-pdmsk\" (UID: \"81b4edc5-d95f-4cae-9191-e27f75e48d8a\") " pod="openshift-ingress-canary/ingress-canary-pdmsk" Jun 01 21:41:18.498356 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:41:18.498312 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b57ae13c-b94a-4891-b4d2-0bd1a8873c4f-metrics-tls\") pod \"dns-default-tnd8z\" (UID: \"b57ae13c-b94a-4891-b4d2-0bd1a8873c4f\") " pod="openshift-dns/dns-default-tnd8z" Jun 01 21:41:18.499117 ip-10-0-138-241 kubenswrapper[2573]: E0601 21:41:18.498619 2573 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 01 21:41:18.499117 ip-10-0-138-241 kubenswrapper[2573]: E0601 21:41:18.498715 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b57ae13c-b94a-4891-b4d2-0bd1a8873c4f-metrics-tls podName:b57ae13c-b94a-4891-b4d2-0bd1a8873c4f nodeName:}" failed. No retries permitted until 2026-06-01 21:41:22.498672582 +0000 UTC m=+42.218724508 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b57ae13c-b94a-4891-b4d2-0bd1a8873c4f-metrics-tls") pod "dns-default-tnd8z" (UID: "b57ae13c-b94a-4891-b4d2-0bd1a8873c4f") : secret "dns-default-metrics-tls" not found Jun 01 21:41:18.499117 ip-10-0-138-241 kubenswrapper[2573]: E0601 21:41:18.498739 2573 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found Jun 01 21:41:18.499117 ip-10-0-138-241 kubenswrapper[2573]: E0601 21:41:18.498802 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/81b4edc5-d95f-4cae-9191-e27f75e48d8a-cert podName:81b4edc5-d95f-4cae-9191-e27f75e48d8a nodeName:}" failed. No retries permitted until 2026-06-01 21:41:22.498784193 +0000 UTC m=+42.218836123 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/81b4edc5-d95f-4cae-9191-e27f75e48d8a-cert") pod "ingress-canary-pdmsk" (UID: "81b4edc5-d95f-4cae-9191-e27f75e48d8a") : secret "canary-serving-cert" not found Jun 01 21:41:22.431094 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:41:22.431061 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/3224f451-02bc-496d-9fa0-9045b1dbb798-registry-tls\") pod \"image-registry-77447d89bb-twkln\" (UID: \"3224f451-02bc-496d-9fa0-9045b1dbb798\") " pod="openshift-image-registry/image-registry-77447d89bb-twkln" Jun 01 21:41:22.431581 ip-10-0-138-241 kubenswrapper[2573]: E0601 21:41:22.431229 2573 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 01 21:41:22.431581 ip-10-0-138-241 kubenswrapper[2573]: E0601 21:41:22.431254 2573 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-77447d89bb-twkln: secret "image-registry-tls" not found Jun 01 21:41:22.431581 ip-10-0-138-241 kubenswrapper[2573]: E0601 21:41:22.431342 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3224f451-02bc-496d-9fa0-9045b1dbb798-registry-tls podName:3224f451-02bc-496d-9fa0-9045b1dbb798 nodeName:}" failed. No retries permitted until 2026-06-01 21:41:30.431320684 +0000 UTC m=+50.151372618 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/3224f451-02bc-496d-9fa0-9045b1dbb798-registry-tls") pod "image-registry-77447d89bb-twkln" (UID: "3224f451-02bc-496d-9fa0-9045b1dbb798") : secret "image-registry-tls" not found Jun 01 21:41:22.532109 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:41:22.532067 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/81b4edc5-d95f-4cae-9191-e27f75e48d8a-cert\") pod \"ingress-canary-pdmsk\" (UID: \"81b4edc5-d95f-4cae-9191-e27f75e48d8a\") " pod="openshift-ingress-canary/ingress-canary-pdmsk" Jun 01 21:41:22.532109 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:41:22.532114 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b57ae13c-b94a-4891-b4d2-0bd1a8873c4f-metrics-tls\") pod \"dns-default-tnd8z\" (UID: \"b57ae13c-b94a-4891-b4d2-0bd1a8873c4f\") " pod="openshift-dns/dns-default-tnd8z" Jun 01 21:41:22.532339 ip-10-0-138-241 kubenswrapper[2573]: E0601 21:41:22.532227 2573 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 01 21:41:22.532339 ip-10-0-138-241 kubenswrapper[2573]: E0601 21:41:22.532292 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b57ae13c-b94a-4891-b4d2-0bd1a8873c4f-metrics-tls podName:b57ae13c-b94a-4891-b4d2-0bd1a8873c4f nodeName:}" failed. No retries permitted until 2026-06-01 21:41:30.532272518 +0000 UTC m=+50.252324465 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b57ae13c-b94a-4891-b4d2-0bd1a8873c4f-metrics-tls") pod "dns-default-tnd8z" (UID: "b57ae13c-b94a-4891-b4d2-0bd1a8873c4f") : secret "dns-default-metrics-tls" not found Jun 01 21:41:22.532339 ip-10-0-138-241 kubenswrapper[2573]: E0601 21:41:22.532229 2573 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found Jun 01 21:41:22.532483 ip-10-0-138-241 kubenswrapper[2573]: E0601 21:41:22.532391 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/81b4edc5-d95f-4cae-9191-e27f75e48d8a-cert podName:81b4edc5-d95f-4cae-9191-e27f75e48d8a nodeName:}" failed. No retries permitted until 2026-06-01 21:41:30.532370991 +0000 UTC m=+50.252422928 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/81b4edc5-d95f-4cae-9191-e27f75e48d8a-cert") pod "ingress-canary-pdmsk" (UID: "81b4edc5-d95f-4cae-9191-e27f75e48d8a") : secret "canary-serving-cert" not found Jun 01 21:41:27.855497 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:41:27.855412 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-bb6fb6964-vcwwk" event={"ID":"5e29e06d-c3f7-4b72-a9c7-99e78ad3946c","Type":"ContainerStarted","Data":"194812e5f2a689e76f805a790337541ced3700418cf938fec9749d04e7460a2b"} Jun 01 21:41:27.857263 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:41:27.857230 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-hnntt" event={"ID":"2353b916-e500-4061-adcf-167688af93f0","Type":"ContainerStarted","Data":"ed63046b1b7dedd1ddc24fba0ed84b7cadcdc1cc129c9e4a0578c9c37728c65c"} Jun 01 21:41:27.859011 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:41:27.858980 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5f6f4f9b8b-7wqzv" event={"ID":"aaa23eec-441d-40fa-9d2a-54dbd73e994b","Type":"ContainerStarted","Data":"30785a37077ce0d8117e02c460ab76d0bf7716c7a6cab9ca140e3a6662fee0ee"} Jun 01 21:41:27.859193 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:41:27.859172 2573 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5f6f4f9b8b-7wqzv" Jun 01 21:41:27.861488 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:41:27.861465 2573 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5f6f4f9b8b-7wqzv" Jun 01 21:41:27.862418 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:41:27.862393 2573 generic.go:358] "Generic (PLEG): container finished" podID="36aa25cf-725a-42f0-8f7b-938f5dbe52d4" containerID="c24214457d139746890cb05bfdd8272f6293bdf03dc4ad89139241b658b6c18b" exitCode=0 Jun 01 21:41:27.862515 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:41:27.862440 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mhq4q" event={"ID":"36aa25cf-725a-42f0-8f7b-938f5dbe52d4","Type":"ContainerDied","Data":"c24214457d139746890cb05bfdd8272f6293bdf03dc4ad89139241b658b6c18b"} Jun 01 21:41:27.873272 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:41:27.873134 2573 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-hnntt" podStartSLOduration=1.9493614049999999 podStartE2EDuration="12.873120614s" podCreationTimestamp="2026-06-01 21:41:15 +0000 UTC" firstStartedPulling="2026-06-01 21:41:16.415962502 +0000 UTC m=+36.136014439" lastFinishedPulling="2026-06-01 21:41:27.339721716 +0000 UTC m=+47.059773648" observedRunningTime="2026-06-01 21:41:27.873049137 +0000 UTC m=+47.593101081" watchObservedRunningTime="2026-06-01 21:41:27.873120614 +0000 UTC m=+47.593172558" Jun 01 21:41:27.911749 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:41:27.911687 2573 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5f6f4f9b8b-7wqzv" podStartSLOduration=23.801182064 podStartE2EDuration="35.911667142s" podCreationTimestamp="2026-06-01 21:40:52 +0000 UTC" firstStartedPulling="2026-06-01 21:41:15.243765813 +0000 UTC m=+34.963817738" lastFinishedPulling="2026-06-01 21:41:27.354250891 +0000 UTC m=+47.074302816" observedRunningTime="2026-06-01 21:41:27.910733351 +0000 UTC m=+47.630785295" watchObservedRunningTime="2026-06-01 21:41:27.911667142 +0000 UTC m=+47.631719077" Jun 01 21:41:28.867951 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:41:28.867758 2573 generic.go:358] "Generic (PLEG): container finished" podID="36aa25cf-725a-42f0-8f7b-938f5dbe52d4" containerID="127ba68d2826d27efaa297f313cd98ee78df7a1b1fd671a39041b52828fefb7c" exitCode=0 Jun 01 21:41:28.867951 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:41:28.867807 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mhq4q" event={"ID":"36aa25cf-725a-42f0-8f7b-938f5dbe52d4","Type":"ContainerDied","Data":"127ba68d2826d27efaa297f313cd98ee78df7a1b1fd671a39041b52828fefb7c"} Jun 01 21:41:28.872359 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:41:28.870842 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5f78687ff8-gl77x" event={"ID":"fd119bc7-9dc0-4e69-a312-61601c58c0af","Type":"ContainerStarted","Data":"94193a7f088d6f4ff5e4e6d087f9c5249369ecbbf02730a82b76d597b156e8e3"} Jun 01 21:41:28.907210 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:41:28.907129 2573 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5f78687ff8-gl77x" podStartSLOduration=24.34707719 podStartE2EDuration="36.907110257s" podCreationTimestamp="2026-06-01 21:40:52 +0000 UTC" firstStartedPulling="2026-06-01 21:41:15.213673172 +0000 UTC m=+34.933725097" lastFinishedPulling="2026-06-01 21:41:27.77370624 +0000 UTC m=+47.493758164" observedRunningTime="2026-06-01 21:41:28.906961779 +0000 UTC m=+48.627013727" watchObservedRunningTime="2026-06-01 21:41:28.907110257 +0000 UTC m=+48.627162201" Jun 01 21:41:29.875482 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:41:29.875435 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mhq4q" event={"ID":"36aa25cf-725a-42f0-8f7b-938f5dbe52d4","Type":"ContainerStarted","Data":"e5125c122d4f6569257852a34b34f1e4ee2c58946a00b3c2df7a7ce17890ae48"} Jun 01 21:41:29.900290 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:41:29.900240 2573 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-mhq4q" podStartSLOduration=5.057158895 podStartE2EDuration="48.900219471s" podCreationTimestamp="2026-06-01 21:40:41 +0000 UTC" firstStartedPulling="2026-06-01 21:40:43.49613918 +0000 UTC m=+3.216191116" lastFinishedPulling="2026-06-01 21:41:27.339199768 +0000 UTC m=+47.059251692" observedRunningTime="2026-06-01 21:41:29.897915275 +0000 UTC m=+49.617967219" watchObservedRunningTime="2026-06-01 21:41:29.900219471 +0000 UTC m=+49.620271616" Jun 01 21:41:30.502680 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:41:30.502640 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/3224f451-02bc-496d-9fa0-9045b1dbb798-registry-tls\") pod \"image-registry-77447d89bb-twkln\" (UID: \"3224f451-02bc-496d-9fa0-9045b1dbb798\") " pod="openshift-image-registry/image-registry-77447d89bb-twkln" Jun 01 21:41:30.502841 ip-10-0-138-241 kubenswrapper[2573]: E0601 21:41:30.502819 2573 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 01 21:41:30.502890 ip-10-0-138-241 kubenswrapper[2573]: E0601 21:41:30.502842 2573 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-77447d89bb-twkln: secret "image-registry-tls" not found Jun 01 21:41:30.502930 ip-10-0-138-241 kubenswrapper[2573]: E0601 21:41:30.502913 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3224f451-02bc-496d-9fa0-9045b1dbb798-registry-tls podName:3224f451-02bc-496d-9fa0-9045b1dbb798 nodeName:}" failed. No retries permitted until 2026-06-01 21:41:46.502897023 +0000 UTC m=+66.222948943 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/3224f451-02bc-496d-9fa0-9045b1dbb798-registry-tls") pod "image-registry-77447d89bb-twkln" (UID: "3224f451-02bc-496d-9fa0-9045b1dbb798") : secret "image-registry-tls" not found Jun 01 21:41:30.603283 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:41:30.603252 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/81b4edc5-d95f-4cae-9191-e27f75e48d8a-cert\") pod \"ingress-canary-pdmsk\" (UID: \"81b4edc5-d95f-4cae-9191-e27f75e48d8a\") " pod="openshift-ingress-canary/ingress-canary-pdmsk" Jun 01 21:41:30.603445 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:41:30.603294 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b57ae13c-b94a-4891-b4d2-0bd1a8873c4f-metrics-tls\") pod \"dns-default-tnd8z\" (UID: \"b57ae13c-b94a-4891-b4d2-0bd1a8873c4f\") " pod="openshift-dns/dns-default-tnd8z" Jun 01 21:41:30.603521 ip-10-0-138-241 kubenswrapper[2573]: E0601 21:41:30.603482 2573 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 01 21:41:30.603600 ip-10-0-138-241 kubenswrapper[2573]: E0601 21:41:30.603549 2573 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found Jun 01 21:41:30.603710 ip-10-0-138-241 kubenswrapper[2573]: E0601 21:41:30.603560 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b57ae13c-b94a-4891-b4d2-0bd1a8873c4f-metrics-tls podName:b57ae13c-b94a-4891-b4d2-0bd1a8873c4f nodeName:}" failed. No retries permitted until 2026-06-01 21:41:46.603522361 +0000 UTC m=+66.323574282 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b57ae13c-b94a-4891-b4d2-0bd1a8873c4f-metrics-tls") pod "dns-default-tnd8z" (UID: "b57ae13c-b94a-4891-b4d2-0bd1a8873c4f") : secret "dns-default-metrics-tls" not found Jun 01 21:41:30.603710 ip-10-0-138-241 kubenswrapper[2573]: E0601 21:41:30.603634 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/81b4edc5-d95f-4cae-9191-e27f75e48d8a-cert podName:81b4edc5-d95f-4cae-9191-e27f75e48d8a nodeName:}" failed. No retries permitted until 2026-06-01 21:41:46.603613891 +0000 UTC m=+66.323665816 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/81b4edc5-d95f-4cae-9191-e27f75e48d8a-cert") pod "ingress-canary-pdmsk" (UID: "81b4edc5-d95f-4cae-9191-e27f75e48d8a") : secret "canary-serving-cert" not found Jun 01 21:41:30.880352 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:41:30.880312 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-bb6fb6964-vcwwk" event={"ID":"5e29e06d-c3f7-4b72-a9c7-99e78ad3946c","Type":"ContainerStarted","Data":"8db9e0efeb656decc8939aa6a44749d94f9de8f520b76bc8de52bb48f5c8ef4b"} Jun 01 21:41:30.880352 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:41:30.880354 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-bb6fb6964-vcwwk" event={"ID":"5e29e06d-c3f7-4b72-a9c7-99e78ad3946c","Type":"ContainerStarted","Data":"74303e56dc65aedc30398ab1b10ff31c4af1c0a03d4ebb426f34cf1726d9c5d1"} Jun 01 21:41:30.901127 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:41:30.901086 2573 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-bb6fb6964-vcwwk" podStartSLOduration=23.648243881 podStartE2EDuration="38.901072722s" podCreationTimestamp="2026-06-01 21:40:52 +0000 UTC" firstStartedPulling="2026-06-01 21:41:15.216298735 +0000 UTC m=+34.936350663" lastFinishedPulling="2026-06-01 21:41:30.469127569 +0000 UTC m=+50.189179504" observedRunningTime="2026-06-01 21:41:30.900751239 +0000 UTC m=+50.620803184" watchObservedRunningTime="2026-06-01 21:41:30.901072722 +0000 UTC m=+50.621124642" Jun 01 21:41:39.811383 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:41:39.811352 2573 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-rcqkp" Jun 01 21:41:46.429438 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:41:46.429392 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f99cc923-cc5c-4895-8b2b-6ba878ba343b-metrics-certs\") pod \"network-metrics-daemon-w4d7s\" (UID: \"f99cc923-cc5c-4895-8b2b-6ba878ba343b\") " pod="openshift-multus/network-metrics-daemon-w4d7s" Jun 01 21:41:46.432706 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:41:46.432676 2573 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jun 01 21:41:46.440375 ip-10-0-138-241 kubenswrapper[2573]: E0601 21:41:46.440356 2573 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jun 01 21:41:46.440483 ip-10-0-138-241 kubenswrapper[2573]: E0601 21:41:46.440451 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f99cc923-cc5c-4895-8b2b-6ba878ba343b-metrics-certs podName:f99cc923-cc5c-4895-8b2b-6ba878ba343b nodeName:}" failed. No retries permitted until 2026-06-01 21:42:50.440433424 +0000 UTC m=+130.160485345 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f99cc923-cc5c-4895-8b2b-6ba878ba343b-metrics-certs") pod "network-metrics-daemon-w4d7s" (UID: "f99cc923-cc5c-4895-8b2b-6ba878ba343b") : secret "metrics-daemon-secret" not found Jun 01 21:41:46.530402 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:41:46.530361 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/3224f451-02bc-496d-9fa0-9045b1dbb798-registry-tls\") pod \"image-registry-77447d89bb-twkln\" (UID: \"3224f451-02bc-496d-9fa0-9045b1dbb798\") " pod="openshift-image-registry/image-registry-77447d89bb-twkln" Jun 01 21:41:46.530578 ip-10-0-138-241 kubenswrapper[2573]: E0601 21:41:46.530505 2573 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 01 21:41:46.530578 ip-10-0-138-241 kubenswrapper[2573]: E0601 21:41:46.530528 2573 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-77447d89bb-twkln: secret "image-registry-tls" not found Jun 01 21:41:46.530694 ip-10-0-138-241 kubenswrapper[2573]: E0601 21:41:46.530603 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3224f451-02bc-496d-9fa0-9045b1dbb798-registry-tls podName:3224f451-02bc-496d-9fa0-9045b1dbb798 nodeName:}" failed. No retries permitted until 2026-06-01 21:42:18.530586757 +0000 UTC m=+98.250638678 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/3224f451-02bc-496d-9fa0-9045b1dbb798-registry-tls") pod "image-registry-77447d89bb-twkln" (UID: "3224f451-02bc-496d-9fa0-9045b1dbb798") : secret "image-registry-tls" not found Jun 01 21:41:46.530694 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:41:46.530507 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-7dqw7\" (UniqueName: \"kubernetes.io/projected/e2e547dd-415d-46fe-9608-4a3410d73834-kube-api-access-7dqw7\") pod \"network-check-target-z7k6x\" (UID: \"e2e547dd-415d-46fe-9608-4a3410d73834\") " pod="openshift-network-diagnostics/network-check-target-z7k6x" Jun 01 21:41:46.533569 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:41:46.533548 2573 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jun 01 21:41:46.544233 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:41:46.544211 2573 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jun 01 21:41:46.554580 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:41:46.554553 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7dqw7\" (UniqueName: \"kubernetes.io/projected/e2e547dd-415d-46fe-9608-4a3410d73834-kube-api-access-7dqw7\") pod \"network-check-target-z7k6x\" (UID: \"e2e547dd-415d-46fe-9608-4a3410d73834\") " pod="openshift-network-diagnostics/network-check-target-z7k6x" Jun 01 21:41:46.631836 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:41:46.631800 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/81b4edc5-d95f-4cae-9191-e27f75e48d8a-cert\") pod \"ingress-canary-pdmsk\" (UID: \"81b4edc5-d95f-4cae-9191-e27f75e48d8a\") " pod="openshift-ingress-canary/ingress-canary-pdmsk" Jun 01 21:41:46.631836 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:41:46.631837 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b57ae13c-b94a-4891-b4d2-0bd1a8873c4f-metrics-tls\") pod \"dns-default-tnd8z\" (UID: \"b57ae13c-b94a-4891-b4d2-0bd1a8873c4f\") " pod="openshift-dns/dns-default-tnd8z" Jun 01 21:41:46.632089 ip-10-0-138-241 kubenswrapper[2573]: E0601 21:41:46.631940 2573 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 01 21:41:46.632089 ip-10-0-138-241 kubenswrapper[2573]: E0601 21:41:46.631974 2573 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found Jun 01 21:41:46.632089 ip-10-0-138-241 kubenswrapper[2573]: E0601 21:41:46.631992 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b57ae13c-b94a-4891-b4d2-0bd1a8873c4f-metrics-tls podName:b57ae13c-b94a-4891-b4d2-0bd1a8873c4f nodeName:}" failed. No retries permitted until 2026-06-01 21:42:18.631978977 +0000 UTC m=+98.352030898 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b57ae13c-b94a-4891-b4d2-0bd1a8873c4f-metrics-tls") pod "dns-default-tnd8z" (UID: "b57ae13c-b94a-4891-b4d2-0bd1a8873c4f") : secret "dns-default-metrics-tls" not found Jun 01 21:41:46.632089 ip-10-0-138-241 kubenswrapper[2573]: E0601 21:41:46.632052 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/81b4edc5-d95f-4cae-9191-e27f75e48d8a-cert podName:81b4edc5-d95f-4cae-9191-e27f75e48d8a nodeName:}" failed. No retries permitted until 2026-06-01 21:42:18.632033304 +0000 UTC m=+98.352085225 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/81b4edc5-d95f-4cae-9191-e27f75e48d8a-cert") pod "ingress-canary-pdmsk" (UID: "81b4edc5-d95f-4cae-9191-e27f75e48d8a") : secret "canary-serving-cert" not found Jun 01 21:41:46.796717 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:41:46.796639 2573 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-dlgnh\"" Jun 01 21:41:46.804298 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:41:46.804275 2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-z7k6x" Jun 01 21:41:46.928880 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:41:46.928849 2573 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-z7k6x"] Jun 01 21:41:46.931736 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:41:46.931709 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode2e547dd_415d_46fe_9608_4a3410d73834.slice/crio-3fa94a1afc8d441debe4f0a7db92db241a1d259a4ae5900bf8e6599078fec7b2 WatchSource:0}: Error finding container 3fa94a1afc8d441debe4f0a7db92db241a1d259a4ae5900bf8e6599078fec7b2: Status 404 returned error can't find the container with id 3fa94a1afc8d441debe4f0a7db92db241a1d259a4ae5900bf8e6599078fec7b2 Jun 01 21:41:47.922450 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:41:47.922410 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-z7k6x" event={"ID":"e2e547dd-415d-46fe-9608-4a3410d73834","Type":"ContainerStarted","Data":"3fa94a1afc8d441debe4f0a7db92db241a1d259a4ae5900bf8e6599078fec7b2"} Jun 01 21:41:51.932528 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:41:51.932491 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-z7k6x" event={"ID":"e2e547dd-415d-46fe-9608-4a3410d73834","Type":"ContainerStarted","Data":"acf9a884a4c5e3c96cb72674228b1ab7969465388e3bdcb4b74816050aeef116"} Jun 01 21:41:51.932946 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:41:51.932646 2573 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-z7k6x" Jun 01 21:41:51.948940 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:41:51.948895 2573 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-z7k6x" podStartSLOduration=67.927987326 podStartE2EDuration="1m11.948882033s" podCreationTimestamp="2026-06-01 21:40:40 +0000 UTC" firstStartedPulling="2026-06-01 21:41:46.933603285 +0000 UTC m=+66.653655207" lastFinishedPulling="2026-06-01 21:41:50.954497982 +0000 UTC m=+70.674549914" observedRunningTime="2026-06-01 21:41:51.948280033 +0000 UTC m=+71.668331977" watchObservedRunningTime="2026-06-01 21:41:51.948882033 +0000 UTC m=+71.668934016" Jun 01 21:42:18.577402 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:42:18.577359 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/3224f451-02bc-496d-9fa0-9045b1dbb798-registry-tls\") pod \"image-registry-77447d89bb-twkln\" (UID: \"3224f451-02bc-496d-9fa0-9045b1dbb798\") " pod="openshift-image-registry/image-registry-77447d89bb-twkln" Jun 01 21:42:18.577820 ip-10-0-138-241 kubenswrapper[2573]: E0601 21:42:18.577488 2573 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 01 21:42:18.577820 ip-10-0-138-241 kubenswrapper[2573]: E0601 21:42:18.577502 2573 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-77447d89bb-twkln: secret "image-registry-tls" not found Jun 01 21:42:18.577820 ip-10-0-138-241 kubenswrapper[2573]: E0601 21:42:18.577589 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3224f451-02bc-496d-9fa0-9045b1dbb798-registry-tls podName:3224f451-02bc-496d-9fa0-9045b1dbb798 nodeName:}" failed. No retries permitted until 2026-06-01 21:43:22.57757109 +0000 UTC m=+162.297623013 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/3224f451-02bc-496d-9fa0-9045b1dbb798-registry-tls") pod "image-registry-77447d89bb-twkln" (UID: "3224f451-02bc-496d-9fa0-9045b1dbb798") : secret "image-registry-tls" not found Jun 01 21:42:18.677977 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:42:18.677939 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/81b4edc5-d95f-4cae-9191-e27f75e48d8a-cert\") pod \"ingress-canary-pdmsk\" (UID: \"81b4edc5-d95f-4cae-9191-e27f75e48d8a\") " pod="openshift-ingress-canary/ingress-canary-pdmsk" Jun 01 21:42:18.677977 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:42:18.677972 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b57ae13c-b94a-4891-b4d2-0bd1a8873c4f-metrics-tls\") pod \"dns-default-tnd8z\" (UID: \"b57ae13c-b94a-4891-b4d2-0bd1a8873c4f\") " pod="openshift-dns/dns-default-tnd8z" Jun 01 21:42:18.678187 ip-10-0-138-241 kubenswrapper[2573]: E0601 21:42:18.678082 2573 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found Jun 01 21:42:18.678187 ip-10-0-138-241 kubenswrapper[2573]: E0601 21:42:18.678108 2573 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 01 21:42:18.678187 ip-10-0-138-241 kubenswrapper[2573]: E0601 21:42:18.678155 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/81b4edc5-d95f-4cae-9191-e27f75e48d8a-cert podName:81b4edc5-d95f-4cae-9191-e27f75e48d8a nodeName:}" failed. No retries permitted until 2026-06-01 21:43:22.678140323 +0000 UTC m=+162.398192243 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/81b4edc5-d95f-4cae-9191-e27f75e48d8a-cert") pod "ingress-canary-pdmsk" (UID: "81b4edc5-d95f-4cae-9191-e27f75e48d8a") : secret "canary-serving-cert" not found Jun 01 21:42:18.678187 ip-10-0-138-241 kubenswrapper[2573]: E0601 21:42:18.678173 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b57ae13c-b94a-4891-b4d2-0bd1a8873c4f-metrics-tls podName:b57ae13c-b94a-4891-b4d2-0bd1a8873c4f nodeName:}" failed. No retries permitted until 2026-06-01 21:43:22.678161732 +0000 UTC m=+162.398213652 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b57ae13c-b94a-4891-b4d2-0bd1a8873c4f-metrics-tls") pod "dns-default-tnd8z" (UID: "b57ae13c-b94a-4891-b4d2-0bd1a8873c4f") : secret "dns-default-metrics-tls" not found Jun 01 21:42:22.937066 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:42:22.937038 2573 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-z7k6x" Jun 01 21:42:50.528477 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:42:50.528431 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f99cc923-cc5c-4895-8b2b-6ba878ba343b-metrics-certs\") pod \"network-metrics-daemon-w4d7s\" (UID: \"f99cc923-cc5c-4895-8b2b-6ba878ba343b\") " pod="openshift-multus/network-metrics-daemon-w4d7s" Jun 01 21:42:50.528930 ip-10-0-138-241 kubenswrapper[2573]: E0601 21:42:50.528596 2573 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jun 01 21:42:50.528930 ip-10-0-138-241 kubenswrapper[2573]: E0601 21:42:50.528676 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f99cc923-cc5c-4895-8b2b-6ba878ba343b-metrics-certs podName:f99cc923-cc5c-4895-8b2b-6ba878ba343b nodeName:}" failed. No retries permitted until 2026-06-01 21:44:52.528657254 +0000 UTC m=+252.248709175 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f99cc923-cc5c-4895-8b2b-6ba878ba343b-metrics-certs") pod "network-metrics-daemon-w4d7s" (UID: "f99cc923-cc5c-4895-8b2b-6ba878ba343b") : secret "metrics-daemon-secret" not found Jun 01 21:43:06.392594 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:43:06.392564 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-kfs2m_bcc44458-cf91-4d2d-9b1b-e9da2db57098/dns-node-resolver/0.log" Jun 01 21:43:07.397268 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:43:07.397240 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-qcs4d_7ada00c1-dbeb-4df5-bfca-923ef619c2d1/node-ca/0.log" Jun 01 21:43:17.687357 ip-10-0-138-241 kubenswrapper[2573]: E0601 21:43:17.687310 2573 pod_workers.go:1301] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-w4d7s" podUID="f99cc923-cc5c-4895-8b2b-6ba878ba343b" Jun 01 21:43:17.743780 ip-10-0-138-241 kubenswrapper[2573]: E0601 21:43:17.743741 2573 pod_workers.go:1301] "Error syncing pod, skipping" err="unmounted volumes=[registry-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-image-registry/image-registry-77447d89bb-twkln" podUID="3224f451-02bc-496d-9fa0-9045b1dbb798" Jun 01 21:43:17.771091 ip-10-0-138-241 kubenswrapper[2573]: E0601 21:43:17.771045 2573 pod_workers.go:1301] "Error syncing pod, skipping" err="unmounted volumes=[cert], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-ingress-canary/ingress-canary-pdmsk" podUID="81b4edc5-d95f-4cae-9191-e27f75e48d8a" Jun 01 21:43:17.818477 ip-10-0-138-241 kubenswrapper[2573]: E0601 21:43:17.818432 2573 pod_workers.go:1301] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-tnd8z" podUID="b57ae13c-b94a-4891-b4d2-0bd1a8873c4f" Jun 01 21:43:18.128789 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:43:18.128757 2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-77447d89bb-twkln" Jun 01 21:43:18.128996 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:43:18.128757 2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-tnd8z" Jun 01 21:43:22.663011 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:43:22.662968 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/3224f451-02bc-496d-9fa0-9045b1dbb798-registry-tls\") pod \"image-registry-77447d89bb-twkln\" (UID: \"3224f451-02bc-496d-9fa0-9045b1dbb798\") " pod="openshift-image-registry/image-registry-77447d89bb-twkln" Jun 01 21:43:22.665385 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:43:22.665359 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/3224f451-02bc-496d-9fa0-9045b1dbb798-registry-tls\") pod \"image-registry-77447d89bb-twkln\" (UID: \"3224f451-02bc-496d-9fa0-9045b1dbb798\") " pod="openshift-image-registry/image-registry-77447d89bb-twkln" Jun 01 21:43:22.763329 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:43:22.763285 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/81b4edc5-d95f-4cae-9191-e27f75e48d8a-cert\") pod \"ingress-canary-pdmsk\" (UID: \"81b4edc5-d95f-4cae-9191-e27f75e48d8a\") " pod="openshift-ingress-canary/ingress-canary-pdmsk" Jun 01 21:43:22.763329 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:43:22.763331 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b57ae13c-b94a-4891-b4d2-0bd1a8873c4f-metrics-tls\") pod \"dns-default-tnd8z\" (UID: \"b57ae13c-b94a-4891-b4d2-0bd1a8873c4f\") " pod="openshift-dns/dns-default-tnd8z" Jun 01 21:43:22.765747 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:43:22.765712 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/81b4edc5-d95f-4cae-9191-e27f75e48d8a-cert\") pod \"ingress-canary-pdmsk\" (UID: \"81b4edc5-d95f-4cae-9191-e27f75e48d8a\") " pod="openshift-ingress-canary/ingress-canary-pdmsk" Jun 01 21:43:22.765747 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:43:22.765739 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b57ae13c-b94a-4891-b4d2-0bd1a8873c4f-metrics-tls\") pod \"dns-default-tnd8z\" (UID: \"b57ae13c-b94a-4891-b4d2-0bd1a8873c4f\") " pod="openshift-dns/dns-default-tnd8z" Jun 01 21:43:22.934196 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:43:22.934113 2573 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-6g4lx\"" Jun 01 21:43:22.934196 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:43:22.934172 2573 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-vvsxk\"" Jun 01 21:43:22.939823 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:43:22.939798 2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-77447d89bb-twkln" Jun 01 21:43:22.939932 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:43:22.939837 2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-tnd8z" Jun 01 21:43:23.283438 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:43:23.283365 2573 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-tnd8z"] Jun 01 21:43:23.290007 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:43:23.288959 2573 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-77447d89bb-twkln"] Jun 01 21:43:24.143881 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:43:24.143837 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-77447d89bb-twkln" event={"ID":"3224f451-02bc-496d-9fa0-9045b1dbb798","Type":"ContainerStarted","Data":"230fe3948974b975b11382f3538aa6d49791523015797e3a40bbe5f233d21136"} Jun 01 21:43:24.143881 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:43:24.143884 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-77447d89bb-twkln" event={"ID":"3224f451-02bc-496d-9fa0-9045b1dbb798","Type":"ContainerStarted","Data":"b80c570f8423ea3bdc75fbc9ba7f404faf753c066e9b18fbd703afa6f134e600"} Jun 01 21:43:24.144399 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:43:24.143909 2573 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-77447d89bb-twkln" Jun 01 21:43:24.144848 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:43:24.144828 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-tnd8z" event={"ID":"b57ae13c-b94a-4891-b4d2-0bd1a8873c4f","Type":"ContainerStarted","Data":"f7fc16cca518b15cb791ad11800c3eccd316ddcd0401e1c6be4ace0da62a0712"} Jun 01 21:43:24.167496 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:43:24.167443 2573 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-image-registry/image-registry-77447d89bb-twkln" podStartSLOduration=152.167428484 podStartE2EDuration="2m32.167428484s" podCreationTimestamp="2026-06-01 21:40:52 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-06-01 21:43:24.166640654 +0000 UTC m=+163.886692600" watchObservedRunningTime="2026-06-01 21:43:24.167428484 +0000 UTC m=+163.887480427" Jun 01 21:43:25.148710 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:43:25.148678 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-tnd8z" event={"ID":"b57ae13c-b94a-4891-b4d2-0bd1a8873c4f","Type":"ContainerStarted","Data":"3ea69fad7e9ac1f59cdb896f47cbd08119063a538aaf706a38a71d2abeb3be1a"} Jun 01 21:43:25.148710 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:43:25.148716 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-tnd8z" event={"ID":"b57ae13c-b94a-4891-b4d2-0bd1a8873c4f","Type":"ContainerStarted","Data":"9b0a5e1ad0154f551571a06acee5e25e706d6cee9d5371c0d27a4aadead452d9"} Jun 01 21:43:25.167260 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:43:25.167209 2573 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-dns/dns-default-tnd8z" podStartSLOduration=129.6450319 podStartE2EDuration="2m11.167194888s" podCreationTimestamp="2026-06-01 21:41:14 +0000 UTC" firstStartedPulling="2026-06-01 21:43:23.291363353 +0000 UTC m=+163.011415277" lastFinishedPulling="2026-06-01 21:43:24.813526332 +0000 UTC m=+164.533578265" observedRunningTime="2026-06-01 21:43:25.165449442 +0000 UTC m=+164.885501384" watchObservedRunningTime="2026-06-01 21:43:25.167194888 +0000 UTC m=+164.887246830" Jun 01 21:43:26.151512 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:43:26.151482 2573 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-tnd8z" Jun 01 21:43:27.572688 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:43:27.572648 2573 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-7lrz6"] Jun 01 21:43:27.575501 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:43:27.575478 2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-7lrz6" Jun 01 21:43:27.578735 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:43:27.578712 2573 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jun 01 21:43:27.578893 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:43:27.578867 2573 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jun 01 21:43:27.580243 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:43:27.580223 2573 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jun 01 21:43:27.580583 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:43:27.580525 2573 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jun 01 21:43:27.580681 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:43:27.580624 2573 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-n68nj\"" Jun 01 21:43:27.587252 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:43:27.587230 2573 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-7lrz6"] Jun 01 21:43:27.702431 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:43:27.702396 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9257de6e-8c40-4471-82e8-f6fd9d898198-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-7lrz6\" (UID: \"9257de6e-8c40-4471-82e8-f6fd9d898198\") " pod="openshift-insights/insights-runtime-extractor-7lrz6" Jun 01 21:43:27.702431 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:43:27.702431 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7cc8h\" (UniqueName: \"kubernetes.io/projected/9257de6e-8c40-4471-82e8-f6fd9d898198-kube-api-access-7cc8h\") pod \"insights-runtime-extractor-7lrz6\" (UID: \"9257de6e-8c40-4471-82e8-f6fd9d898198\") " pod="openshift-insights/insights-runtime-extractor-7lrz6" Jun 01 21:43:27.702680 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:43:27.702458 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9257de6e-8c40-4471-82e8-f6fd9d898198-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-7lrz6\" (UID: \"9257de6e-8c40-4471-82e8-f6fd9d898198\") " pod="openshift-insights/insights-runtime-extractor-7lrz6" Jun 01 21:43:27.702680 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:43:27.702590 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/9257de6e-8c40-4471-82e8-f6fd9d898198-data-volume\") pod \"insights-runtime-extractor-7lrz6\" (UID: \"9257de6e-8c40-4471-82e8-f6fd9d898198\") " pod="openshift-insights/insights-runtime-extractor-7lrz6" Jun 01 21:43:27.702680 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:43:27.702623 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/9257de6e-8c40-4471-82e8-f6fd9d898198-crio-socket\") pod \"insights-runtime-extractor-7lrz6\" (UID: \"9257de6e-8c40-4471-82e8-f6fd9d898198\") " pod="openshift-insights/insights-runtime-extractor-7lrz6" Jun 01 21:43:27.803710 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:43:27.803681 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/9257de6e-8c40-4471-82e8-f6fd9d898198-crio-socket\") pod \"insights-runtime-extractor-7lrz6\" (UID: \"9257de6e-8c40-4471-82e8-f6fd9d898198\") " pod="openshift-insights/insights-runtime-extractor-7lrz6" Jun 01 21:43:27.803886 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:43:27.803721 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9257de6e-8c40-4471-82e8-f6fd9d898198-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-7lrz6\" (UID: \"9257de6e-8c40-4471-82e8-f6fd9d898198\") " pod="openshift-insights/insights-runtime-extractor-7lrz6" Jun 01 21:43:27.803886 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:43:27.803744 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-7cc8h\" (UniqueName: \"kubernetes.io/projected/9257de6e-8c40-4471-82e8-f6fd9d898198-kube-api-access-7cc8h\") pod \"insights-runtime-extractor-7lrz6\" (UID: \"9257de6e-8c40-4471-82e8-f6fd9d898198\") " pod="openshift-insights/insights-runtime-extractor-7lrz6" Jun 01 21:43:27.803886 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:43:27.803782 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9257de6e-8c40-4471-82e8-f6fd9d898198-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-7lrz6\" (UID: \"9257de6e-8c40-4471-82e8-f6fd9d898198\") " pod="openshift-insights/insights-runtime-extractor-7lrz6" Jun 01 21:43:27.803886 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:43:27.803827 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/9257de6e-8c40-4471-82e8-f6fd9d898198-crio-socket\") pod \"insights-runtime-extractor-7lrz6\" (UID: \"9257de6e-8c40-4471-82e8-f6fd9d898198\") " pod="openshift-insights/insights-runtime-extractor-7lrz6" Jun 01 21:43:27.803886 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:43:27.803881 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/9257de6e-8c40-4471-82e8-f6fd9d898198-data-volume\") pod \"insights-runtime-extractor-7lrz6\" (UID: \"9257de6e-8c40-4471-82e8-f6fd9d898198\") " pod="openshift-insights/insights-runtime-extractor-7lrz6" Jun 01 21:43:27.804223 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:43:27.804199 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/9257de6e-8c40-4471-82e8-f6fd9d898198-data-volume\") pod \"insights-runtime-extractor-7lrz6\" (UID: \"9257de6e-8c40-4471-82e8-f6fd9d898198\") " pod="openshift-insights/insights-runtime-extractor-7lrz6" Jun 01 21:43:27.804467 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:43:27.804444 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9257de6e-8c40-4471-82e8-f6fd9d898198-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-7lrz6\" (UID: \"9257de6e-8c40-4471-82e8-f6fd9d898198\") " pod="openshift-insights/insights-runtime-extractor-7lrz6" Jun 01 21:43:27.806117 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:43:27.806091 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9257de6e-8c40-4471-82e8-f6fd9d898198-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-7lrz6\" (UID: \"9257de6e-8c40-4471-82e8-f6fd9d898198\") " pod="openshift-insights/insights-runtime-extractor-7lrz6" Jun 01 21:43:27.816801 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:43:27.816776 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7cc8h\" (UniqueName: \"kubernetes.io/projected/9257de6e-8c40-4471-82e8-f6fd9d898198-kube-api-access-7cc8h\") pod \"insights-runtime-extractor-7lrz6\" (UID: \"9257de6e-8c40-4471-82e8-f6fd9d898198\") " pod="openshift-insights/insights-runtime-extractor-7lrz6" Jun 01 21:43:27.860199 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:43:27.860164 2573 prober.go:120] "Probe failed" probeType="Readiness" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5f6f4f9b8b-7wqzv" podUID="aaa23eec-441d-40fa-9d2a-54dbd73e994b" containerName="acm-agent" probeResult="failure" output="Get \"http://10.132.0.8:8000/readyz\": dial tcp 10.132.0.8:8000: connect: connection refused" Jun 01 21:43:27.883994 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:43:27.883963 2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-7lrz6" Jun 01 21:43:28.009269 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:43:28.009235 2573 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-7lrz6"] Jun 01 21:43:28.012318 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:43:28.012289 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9257de6e_8c40_4471_82e8_f6fd9d898198.slice/crio-d09884a3380636d3a9987922261a17291756a172a8e4b3f45f0b59412e3424ae WatchSource:0}: Error finding container d09884a3380636d3a9987922261a17291756a172a8e4b3f45f0b59412e3424ae: Status 404 returned error can't find the container with id d09884a3380636d3a9987922261a17291756a172a8e4b3f45f0b59412e3424ae Jun 01 21:43:28.161245 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:43:28.161152 2573 generic.go:358] "Generic (PLEG): container finished" podID="aaa23eec-441d-40fa-9d2a-54dbd73e994b" containerID="30785a37077ce0d8117e02c460ab76d0bf7716c7a6cab9ca140e3a6662fee0ee" exitCode=1 Jun 01 21:43:28.161245 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:43:28.161225 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5f6f4f9b8b-7wqzv" event={"ID":"aaa23eec-441d-40fa-9d2a-54dbd73e994b","Type":"ContainerDied","Data":"30785a37077ce0d8117e02c460ab76d0bf7716c7a6cab9ca140e3a6662fee0ee"} Jun 01 21:43:28.161614 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:43:28.161597 2573 scope.go:117] "RemoveContainer" containerID="30785a37077ce0d8117e02c460ab76d0bf7716c7a6cab9ca140e3a6662fee0ee" Jun 01 21:43:28.162495 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:43:28.162473 2573 generic.go:358] "Generic (PLEG): container finished" podID="fd119bc7-9dc0-4e69-a312-61601c58c0af" containerID="94193a7f088d6f4ff5e4e6d087f9c5249369ecbbf02730a82b76d597b156e8e3" exitCode=255 Jun 01 21:43:28.162633 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:43:28.162556 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5f78687ff8-gl77x" event={"ID":"fd119bc7-9dc0-4e69-a312-61601c58c0af","Type":"ContainerDied","Data":"94193a7f088d6f4ff5e4e6d087f9c5249369ecbbf02730a82b76d597b156e8e3"} Jun 01 21:43:28.162862 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:43:28.162849 2573 scope.go:117] "RemoveContainer" containerID="94193a7f088d6f4ff5e4e6d087f9c5249369ecbbf02730a82b76d597b156e8e3" Jun 01 21:43:28.164037 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:43:28.163939 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-7lrz6" event={"ID":"9257de6e-8c40-4471-82e8-f6fd9d898198","Type":"ContainerStarted","Data":"298cda250b972781c26d2195e23a6fc145caa2b9b2967b60c863c47c4b7ded8f"} Jun 01 21:43:28.164037 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:43:28.163968 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-7lrz6" event={"ID":"9257de6e-8c40-4471-82e8-f6fd9d898198","Type":"ContainerStarted","Data":"d09884a3380636d3a9987922261a17291756a172a8e4b3f45f0b59412e3424ae"} Jun 01 21:43:28.673637 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:43:28.673597 2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-pdmsk" Jun 01 21:43:28.677031 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:43:28.677008 2573 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-dqpsx\"" Jun 01 21:43:28.684608 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:43:28.684590 2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-pdmsk" Jun 01 21:43:28.817440 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:43:28.817412 2573 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-pdmsk"] Jun 01 21:43:28.857396 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:43:28.857365 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod81b4edc5_d95f_4cae_9191_e27f75e48d8a.slice/crio-7017fc202591e653ca6315a8e4c7300bf1b24b2a7f3aade8a50da0b8afe9bd17 WatchSource:0}: Error finding container 7017fc202591e653ca6315a8e4c7300bf1b24b2a7f3aade8a50da0b8afe9bd17: Status 404 returned error can't find the container with id 7017fc202591e653ca6315a8e4c7300bf1b24b2a7f3aade8a50da0b8afe9bd17 Jun 01 21:43:29.168234 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:43:29.168197 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5f78687ff8-gl77x" event={"ID":"fd119bc7-9dc0-4e69-a312-61601c58c0af","Type":"ContainerStarted","Data":"c41a06d6ce498b3dc4beb45b2e81da7eff0ddcd063d799bbd9ba9a07fa577327"} Jun 01 21:43:29.169281 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:43:29.169256 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-pdmsk" event={"ID":"81b4edc5-d95f-4cae-9191-e27f75e48d8a","Type":"ContainerStarted","Data":"7017fc202591e653ca6315a8e4c7300bf1b24b2a7f3aade8a50da0b8afe9bd17"} Jun 01 21:43:29.170691 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:43:29.170666 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-7lrz6" event={"ID":"9257de6e-8c40-4471-82e8-f6fd9d898198","Type":"ContainerStarted","Data":"a4222656c9edb06273a3902d1871eb65fe3197b1c728de092acfbaade3596e4e"} Jun 01 21:43:29.172014 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:43:29.171995 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5f6f4f9b8b-7wqzv" event={"ID":"aaa23eec-441d-40fa-9d2a-54dbd73e994b","Type":"ContainerStarted","Data":"0079657bbf0282390a34884990443b08e1fc512b44319562f4cd26ed08f81ecf"} Jun 01 21:43:29.172256 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:43:29.172243 2573 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5f6f4f9b8b-7wqzv" Jun 01 21:43:29.172880 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:43:29.172863 2573 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-5f6f4f9b8b-7wqzv" Jun 01 21:43:30.677954 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:43:30.677752 2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w4d7s" Jun 01 21:43:31.182359 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:43:31.182260 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-pdmsk" event={"ID":"81b4edc5-d95f-4cae-9191-e27f75e48d8a","Type":"ContainerStarted","Data":"074195e3de3e1f1bf0979918bd8ddb7e171a872e349b1d4bea90c3dc489454db"} Jun 01 21:43:31.183963 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:43:31.183937 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-7lrz6" event={"ID":"9257de6e-8c40-4471-82e8-f6fd9d898198","Type":"ContainerStarted","Data":"fecedfad2894545e40e90a658e7e16aacbd756e79752f655d0c12c56ebcfee75"} Jun 01 21:43:31.199042 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:43:31.198997 2573 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-pdmsk" podStartSLOduration=135.141052243 podStartE2EDuration="2m17.198983975s" podCreationTimestamp="2026-06-01 21:41:14 +0000 UTC" firstStartedPulling="2026-06-01 21:43:28.859382345 +0000 UTC m=+168.579434266" lastFinishedPulling="2026-06-01 21:43:30.917314078 +0000 UTC m=+170.637365998" observedRunningTime="2026-06-01 21:43:31.198216537 +0000 UTC m=+170.918268481" watchObservedRunningTime="2026-06-01 21:43:31.198983975 +0000 UTC m=+170.919035918" Jun 01 21:43:31.215658 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:43:31.215610 2573 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-7lrz6" podStartSLOduration=1.374001653 podStartE2EDuration="4.215594298s" podCreationTimestamp="2026-06-01 21:43:27 +0000 UTC" firstStartedPulling="2026-06-01 21:43:28.076982041 +0000 UTC m=+167.797033962" lastFinishedPulling="2026-06-01 21:43:30.918574557 +0000 UTC m=+170.638626607" observedRunningTime="2026-06-01 21:43:31.215261282 +0000 UTC m=+170.935313225" watchObservedRunningTime="2026-06-01 21:43:31.215594298 +0000 UTC m=+170.935646260" Jun 01 21:43:36.156133 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:43:36.156101 2573 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-tnd8z" Jun 01 21:43:42.377667 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:43:42.377628 2573 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-jbmfp"] Jun 01 21:43:42.381114 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:43:42.381096 2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-jbmfp" Jun 01 21:43:42.384167 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:43:42.384141 2573 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jun 01 21:43:42.384297 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:43:42.384166 2573 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jun 01 21:43:42.384297 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:43:42.384201 2573 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jun 01 21:43:42.386380 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:43:42.386169 2573 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jun 01 21:43:42.386380 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:43:42.386231 2573 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jun 01 21:43:42.386380 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:43:42.386241 2573 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-hq6rf\"" Jun 01 21:43:42.386380 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:43:42.386258 2573 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jun 01 21:43:42.521443 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:43:42.521401 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/1b742974-2d33-4c56-9778-341b869d773e-node-exporter-accelerators-collector-config\") pod \"node-exporter-jbmfp\" (UID: \"1b742974-2d33-4c56-9778-341b869d773e\") " pod="openshift-monitoring/node-exporter-jbmfp" Jun 01 21:43:42.521443 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:43:42.521447 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/1b742974-2d33-4c56-9778-341b869d773e-root\") pod \"node-exporter-jbmfp\" (UID: \"1b742974-2d33-4c56-9778-341b869d773e\") " pod="openshift-monitoring/node-exporter-jbmfp" Jun 01 21:43:42.521683 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:43:42.521473 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1b742974-2d33-4c56-9778-341b869d773e-sys\") pod \"node-exporter-jbmfp\" (UID: \"1b742974-2d33-4c56-9778-341b869d773e\") " pod="openshift-monitoring/node-exporter-jbmfp" Jun 01 21:43:42.521683 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:43:42.521529 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/1b742974-2d33-4c56-9778-341b869d773e-node-exporter-textfile\") pod \"node-exporter-jbmfp\" (UID: \"1b742974-2d33-4c56-9778-341b869d773e\") " pod="openshift-monitoring/node-exporter-jbmfp" Jun 01 21:43:42.521683 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:43:42.521577 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1b742974-2d33-4c56-9778-341b869d773e-metrics-client-ca\") pod \"node-exporter-jbmfp\" (UID: \"1b742974-2d33-4c56-9778-341b869d773e\") " pod="openshift-monitoring/node-exporter-jbmfp" Jun 01 21:43:42.521683 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:43:42.521596 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/1b742974-2d33-4c56-9778-341b869d773e-node-exporter-wtmp\") pod \"node-exporter-jbmfp\" (UID: \"1b742974-2d33-4c56-9778-341b869d773e\") " pod="openshift-monitoring/node-exporter-jbmfp" Jun 01 21:43:42.521683 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:43:42.521621 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1b742974-2d33-4c56-9778-341b869d773e-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-jbmfp\" (UID: \"1b742974-2d33-4c56-9778-341b869d773e\") " pod="openshift-monitoring/node-exporter-jbmfp" Jun 01 21:43:42.521683 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:43:42.521680 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5bw9w\" (UniqueName: \"kubernetes.io/projected/1b742974-2d33-4c56-9778-341b869d773e-kube-api-access-5bw9w\") pod \"node-exporter-jbmfp\" (UID: \"1b742974-2d33-4c56-9778-341b869d773e\") " pod="openshift-monitoring/node-exporter-jbmfp" Jun 01 21:43:42.521861 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:43:42.521718 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/1b742974-2d33-4c56-9778-341b869d773e-node-exporter-tls\") pod \"node-exporter-jbmfp\" (UID: \"1b742974-2d33-4c56-9778-341b869d773e\") " pod="openshift-monitoring/node-exporter-jbmfp" Jun 01 21:43:42.622873 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:43:42.622837 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/1b742974-2d33-4c56-9778-341b869d773e-node-exporter-accelerators-collector-config\") pod \"node-exporter-jbmfp\" (UID: \"1b742974-2d33-4c56-9778-341b869d773e\") " pod="openshift-monitoring/node-exporter-jbmfp" Jun 01 21:43:42.622873 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:43:42.622873 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/1b742974-2d33-4c56-9778-341b869d773e-root\") pod \"node-exporter-jbmfp\" (UID: \"1b742974-2d33-4c56-9778-341b869d773e\") " pod="openshift-monitoring/node-exporter-jbmfp" Jun 01 21:43:42.623104 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:43:42.622892 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1b742974-2d33-4c56-9778-341b869d773e-sys\") pod \"node-exporter-jbmfp\" (UID: \"1b742974-2d33-4c56-9778-341b869d773e\") " pod="openshift-monitoring/node-exporter-jbmfp" Jun 01 21:43:42.623104 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:43:42.622931 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/1b742974-2d33-4c56-9778-341b869d773e-node-exporter-textfile\") pod \"node-exporter-jbmfp\" (UID: \"1b742974-2d33-4c56-9778-341b869d773e\") " pod="openshift-monitoring/node-exporter-jbmfp" Jun 01 21:43:42.623104 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:43:42.622955 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1b742974-2d33-4c56-9778-341b869d773e-metrics-client-ca\") pod \"node-exporter-jbmfp\" (UID: \"1b742974-2d33-4c56-9778-341b869d773e\") " pod="openshift-monitoring/node-exporter-jbmfp" Jun 01 21:43:42.623104 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:43:42.622963 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/1b742974-2d33-4c56-9778-341b869d773e-root\") pod \"node-exporter-jbmfp\" (UID: \"1b742974-2d33-4c56-9778-341b869d773e\") " pod="openshift-monitoring/node-exporter-jbmfp" Jun 01 21:43:42.623104 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:43:42.622973 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/1b742974-2d33-4c56-9778-341b869d773e-node-exporter-wtmp\") pod \"node-exporter-jbmfp\" (UID: \"1b742974-2d33-4c56-9778-341b869d773e\") " pod="openshift-monitoring/node-exporter-jbmfp" Jun 01 21:43:42.623104 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:43:42.623022 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1b742974-2d33-4c56-9778-341b869d773e-sys\") pod \"node-exporter-jbmfp\" (UID: \"1b742974-2d33-4c56-9778-341b869d773e\") " pod="openshift-monitoring/node-exporter-jbmfp" Jun 01 21:43:42.623104 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:43:42.623061 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1b742974-2d33-4c56-9778-341b869d773e-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-jbmfp\" (UID: \"1b742974-2d33-4c56-9778-341b869d773e\") " pod="openshift-monitoring/node-exporter-jbmfp" Jun 01 21:43:42.623390 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:43:42.623115 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-5bw9w\" (UniqueName: \"kubernetes.io/projected/1b742974-2d33-4c56-9778-341b869d773e-kube-api-access-5bw9w\") pod \"node-exporter-jbmfp\" (UID: \"1b742974-2d33-4c56-9778-341b869d773e\") " pod="openshift-monitoring/node-exporter-jbmfp" Jun 01 21:43:42.623390 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:43:42.623146 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/1b742974-2d33-4c56-9778-341b869d773e-node-exporter-tls\") pod \"node-exporter-jbmfp\" (UID: \"1b742974-2d33-4c56-9778-341b869d773e\") " pod="openshift-monitoring/node-exporter-jbmfp" Jun 01 21:43:42.623390 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:43:42.623188 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/1b742974-2d33-4c56-9778-341b869d773e-node-exporter-wtmp\") pod \"node-exporter-jbmfp\" (UID: \"1b742974-2d33-4c56-9778-341b869d773e\") " pod="openshift-monitoring/node-exporter-jbmfp" Jun 01 21:43:42.623390 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:43:42.623362 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/1b742974-2d33-4c56-9778-341b869d773e-node-exporter-textfile\") pod \"node-exporter-jbmfp\" (UID: \"1b742974-2d33-4c56-9778-341b869d773e\") " pod="openshift-monitoring/node-exporter-jbmfp" Jun 01 21:43:42.623638 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:43:42.623608 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1b742974-2d33-4c56-9778-341b869d773e-metrics-client-ca\") pod \"node-exporter-jbmfp\" (UID: \"1b742974-2d33-4c56-9778-341b869d773e\") " pod="openshift-monitoring/node-exporter-jbmfp" Jun 01 21:43:42.623838 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:43:42.623821 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/1b742974-2d33-4c56-9778-341b869d773e-node-exporter-accelerators-collector-config\") pod \"node-exporter-jbmfp\" (UID: \"1b742974-2d33-4c56-9778-341b869d773e\") " pod="openshift-monitoring/node-exporter-jbmfp" Jun 01 21:43:42.625491 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:43:42.625471 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1b742974-2d33-4c56-9778-341b869d773e-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-jbmfp\" (UID: \"1b742974-2d33-4c56-9778-341b869d773e\") " pod="openshift-monitoring/node-exporter-jbmfp" Jun 01 21:43:42.625614 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:43:42.625598 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/1b742974-2d33-4c56-9778-341b869d773e-node-exporter-tls\") pod \"node-exporter-jbmfp\" (UID: \"1b742974-2d33-4c56-9778-341b869d773e\") " pod="openshift-monitoring/node-exporter-jbmfp" Jun 01 21:43:42.632115 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:43:42.632048 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5bw9w\" (UniqueName: \"kubernetes.io/projected/1b742974-2d33-4c56-9778-341b869d773e-kube-api-access-5bw9w\") pod \"node-exporter-jbmfp\" (UID: \"1b742974-2d33-4c56-9778-341b869d773e\") " pod="openshift-monitoring/node-exporter-jbmfp" Jun 01 21:43:42.690989 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:43:42.690957 2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-jbmfp" Jun 01 21:43:42.698698 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:43:42.698668 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1b742974_2d33_4c56_9778_341b869d773e.slice/crio-8b592688382f9deec9b88a2b80192b77677755fc91bb4dc8c1fc5ea93b5e2b8e WatchSource:0}: Error finding container 8b592688382f9deec9b88a2b80192b77677755fc91bb4dc8c1fc5ea93b5e2b8e: Status 404 returned error can't find the container with id 8b592688382f9deec9b88a2b80192b77677755fc91bb4dc8c1fc5ea93b5e2b8e Jun 01 21:43:42.944271 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:43:42.944189 2573 patch_prober.go:28] interesting pod/image-registry-77447d89bb-twkln container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jun 01 21:43:42.944271 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:43:42.944256 2573 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-77447d89bb-twkln" podUID="3224f451-02bc-496d-9fa0-9045b1dbb798" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jun 01 21:43:43.213576 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:43:43.213482 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-jbmfp" event={"ID":"1b742974-2d33-4c56-9778-341b869d773e","Type":"ContainerStarted","Data":"8b592688382f9deec9b88a2b80192b77677755fc91bb4dc8c1fc5ea93b5e2b8e"} Jun 01 21:43:44.217912 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:43:44.217881 2573 generic.go:358] "Generic (PLEG): container finished" podID="1b742974-2d33-4c56-9778-341b869d773e" containerID="a75979d44fb42151fc7bf2b1e6fcfe35ba1db496bc4485cf3231e756cab24b8e" exitCode=0 Jun 01 21:43:44.218307 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:43:44.217975 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-jbmfp" event={"ID":"1b742974-2d33-4c56-9778-341b869d773e","Type":"ContainerDied","Data":"a75979d44fb42151fc7bf2b1e6fcfe35ba1db496bc4485cf3231e756cab24b8e"} Jun 01 21:43:45.153406 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:43:45.153376 2573 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-77447d89bb-twkln" Jun 01 21:43:45.222400 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:43:45.222369 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-jbmfp" event={"ID":"1b742974-2d33-4c56-9778-341b869d773e","Type":"ContainerStarted","Data":"585df5f0796fb27be4f51099ba40482e6b6ad593366077252e5db1347016f256"} Jun 01 21:43:45.222400 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:43:45.222405 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-jbmfp" event={"ID":"1b742974-2d33-4c56-9778-341b869d773e","Type":"ContainerStarted","Data":"80ba3f10d839e5cb07b10e0eeff7b796cd6d6e2f903a26c4ce78370f97c4ec33"} Jun 01 21:43:45.253376 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:43:45.253321 2573 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-jbmfp" podStartSLOduration=2.1553828409999998 podStartE2EDuration="3.253305412s" podCreationTimestamp="2026-06-01 21:43:42 +0000 UTC" firstStartedPulling="2026-06-01 21:43:42.700487332 +0000 UTC m=+182.420539252" lastFinishedPulling="2026-06-01 21:43:43.798409883 +0000 UTC m=+183.518461823" observedRunningTime="2026-06-01 21:43:45.25214446 +0000 UTC m=+184.972196415" watchObservedRunningTime="2026-06-01 21:43:45.253305412 +0000 UTC m=+184.973357355" Jun 01 21:43:50.021704 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:43:50.021669 2573 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-77447d89bb-twkln"] Jun 01 21:44:15.041128 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:44:15.041060 2573 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-image-registry/image-registry-77447d89bb-twkln" podUID="3224f451-02bc-496d-9fa0-9045b1dbb798" containerName="registry" containerID="cri-o://230fe3948974b975b11382f3538aa6d49791523015797e3a40bbe5f233d21136" gracePeriod=30 Jun 01 21:44:15.149218 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:44:15.149189 2573 patch_prober.go:28] interesting pod/image-registry-77447d89bb-twkln container/registry namespace/openshift-image-registry: Readiness probe status=failure output="Get \"https://10.132.0.6:5000/healthz\": dial tcp 10.132.0.6:5000: connect: connection refused" start-of-body= Jun 01 21:44:15.149370 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:44:15.149252 2573 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-77447d89bb-twkln" podUID="3224f451-02bc-496d-9fa0-9045b1dbb798" containerName="registry" probeResult="failure" output="Get \"https://10.132.0.6:5000/healthz\": dial tcp 10.132.0.6:5000: connect: connection refused" Jun 01 21:44:15.292909 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:44:15.292845 2573 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-77447d89bb-twkln" Jun 01 21:44:15.302289 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:44:15.302264 2573 generic.go:358] "Generic (PLEG): container finished" podID="3224f451-02bc-496d-9fa0-9045b1dbb798" containerID="230fe3948974b975b11382f3538aa6d49791523015797e3a40bbe5f233d21136" exitCode=0 Jun 01 21:44:15.302415 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:44:15.302310 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-77447d89bb-twkln" event={"ID":"3224f451-02bc-496d-9fa0-9045b1dbb798","Type":"ContainerDied","Data":"230fe3948974b975b11382f3538aa6d49791523015797e3a40bbe5f233d21136"} Jun 01 21:44:15.302415 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:44:15.302315 2573 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-77447d89bb-twkln" Jun 01 21:44:15.302415 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:44:15.302336 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-77447d89bb-twkln" event={"ID":"3224f451-02bc-496d-9fa0-9045b1dbb798","Type":"ContainerDied","Data":"b80c570f8423ea3bdc75fbc9ba7f404faf753c066e9b18fbd703afa6f134e600"} Jun 01 21:44:15.302415 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:44:15.302351 2573 scope.go:117] "RemoveContainer" containerID="230fe3948974b975b11382f3538aa6d49791523015797e3a40bbe5f233d21136" Jun 01 21:44:15.309602 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:44:15.309585 2573 scope.go:117] "RemoveContainer" containerID="230fe3948974b975b11382f3538aa6d49791523015797e3a40bbe5f233d21136" Jun 01 21:44:15.309869 ip-10-0-138-241 kubenswrapper[2573]: E0601 21:44:15.309848 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"230fe3948974b975b11382f3538aa6d49791523015797e3a40bbe5f233d21136\": container with ID starting with 230fe3948974b975b11382f3538aa6d49791523015797e3a40bbe5f233d21136 not found: ID does not exist" containerID="230fe3948974b975b11382f3538aa6d49791523015797e3a40bbe5f233d21136" Jun 01 21:44:15.309936 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:44:15.309879 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"230fe3948974b975b11382f3538aa6d49791523015797e3a40bbe5f233d21136"} err="failed to get container status \"230fe3948974b975b11382f3538aa6d49791523015797e3a40bbe5f233d21136\": rpc error: code = NotFound desc = could not find container \"230fe3948974b975b11382f3538aa6d49791523015797e3a40bbe5f233d21136\": container with ID starting with 230fe3948974b975b11382f3538aa6d49791523015797e3a40bbe5f233d21136 not found: ID does not exist" Jun 01 21:44:15.367942 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:44:15.367914 2573 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/3224f451-02bc-496d-9fa0-9045b1dbb798-image-registry-private-configuration\") pod \"3224f451-02bc-496d-9fa0-9045b1dbb798\" (UID: \"3224f451-02bc-496d-9fa0-9045b1dbb798\") " Jun 01 21:44:15.367942 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:44:15.367946 2573 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/3224f451-02bc-496d-9fa0-9045b1dbb798-bound-sa-token\") pod \"3224f451-02bc-496d-9fa0-9045b1dbb798\" (UID: \"3224f451-02bc-496d-9fa0-9045b1dbb798\") " Jun 01 21:44:15.368164 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:44:15.367977 2573 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/3224f451-02bc-496d-9fa0-9045b1dbb798-ca-trust-extracted\") pod \"3224f451-02bc-496d-9fa0-9045b1dbb798\" (UID: \"3224f451-02bc-496d-9fa0-9045b1dbb798\") " Jun 01 21:44:15.368164 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:44:15.367997 2573 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/3224f451-02bc-496d-9fa0-9045b1dbb798-trusted-ca\") pod \"3224f451-02bc-496d-9fa0-9045b1dbb798\" (UID: \"3224f451-02bc-496d-9fa0-9045b1dbb798\") " Jun 01 21:44:15.368476 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:44:15.368452 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3224f451-02bc-496d-9fa0-9045b1dbb798-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "3224f451-02bc-496d-9fa0-9045b1dbb798" (UID: "3224f451-02bc-496d-9fa0-9045b1dbb798"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 01 21:44:15.370328 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:44:15.370303 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3224f451-02bc-496d-9fa0-9045b1dbb798-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "3224f451-02bc-496d-9fa0-9045b1dbb798" (UID: "3224f451-02bc-496d-9fa0-9045b1dbb798"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 01 21:44:15.370396 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:44:15.370326 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3224f451-02bc-496d-9fa0-9045b1dbb798-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "3224f451-02bc-496d-9fa0-9045b1dbb798" (UID: "3224f451-02bc-496d-9fa0-9045b1dbb798"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 01 21:44:15.378288 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:44:15.378259 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3224f451-02bc-496d-9fa0-9045b1dbb798-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "3224f451-02bc-496d-9fa0-9045b1dbb798" (UID: "3224f451-02bc-496d-9fa0-9045b1dbb798"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 01 21:44:15.469060 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:44:15.469021 2573 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/3224f451-02bc-496d-9fa0-9045b1dbb798-registry-certificates\") pod \"3224f451-02bc-496d-9fa0-9045b1dbb798\" (UID: \"3224f451-02bc-496d-9fa0-9045b1dbb798\") " Jun 01 21:44:15.469060 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:44:15.469066 2573 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wrvgr\" (UniqueName: \"kubernetes.io/projected/3224f451-02bc-496d-9fa0-9045b1dbb798-kube-api-access-wrvgr\") pod \"3224f451-02bc-496d-9fa0-9045b1dbb798\" (UID: \"3224f451-02bc-496d-9fa0-9045b1dbb798\") " Jun 01 21:44:15.469308 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:44:15.469088 2573 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/3224f451-02bc-496d-9fa0-9045b1dbb798-installation-pull-secrets\") pod \"3224f451-02bc-496d-9fa0-9045b1dbb798\" (UID: \"3224f451-02bc-496d-9fa0-9045b1dbb798\") " Jun 01 21:44:15.469308 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:44:15.469120 2573 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/3224f451-02bc-496d-9fa0-9045b1dbb798-registry-tls\") pod \"3224f451-02bc-496d-9fa0-9045b1dbb798\" (UID: \"3224f451-02bc-496d-9fa0-9045b1dbb798\") " Jun 01 21:44:15.469422 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:44:15.469337 2573 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/3224f451-02bc-496d-9fa0-9045b1dbb798-image-registry-private-configuration\") on node \"ip-10-0-138-241.ec2.internal\" DevicePath \"\"" Jun 01 21:44:15.469422 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:44:15.469353 2573 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/3224f451-02bc-496d-9fa0-9045b1dbb798-bound-sa-token\") on node \"ip-10-0-138-241.ec2.internal\" DevicePath \"\"" Jun 01 21:44:15.469422 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:44:15.469369 2573 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/3224f451-02bc-496d-9fa0-9045b1dbb798-ca-trust-extracted\") on node \"ip-10-0-138-241.ec2.internal\" DevicePath \"\"" Jun 01 21:44:15.469422 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:44:15.469383 2573 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/3224f451-02bc-496d-9fa0-9045b1dbb798-trusted-ca\") on node \"ip-10-0-138-241.ec2.internal\" DevicePath \"\"" Jun 01 21:44:15.469616 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:44:15.469497 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3224f451-02bc-496d-9fa0-9045b1dbb798-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "3224f451-02bc-496d-9fa0-9045b1dbb798" (UID: "3224f451-02bc-496d-9fa0-9045b1dbb798"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 01 21:44:15.471337 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:44:15.471302 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3224f451-02bc-496d-9fa0-9045b1dbb798-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "3224f451-02bc-496d-9fa0-9045b1dbb798" (UID: "3224f451-02bc-496d-9fa0-9045b1dbb798"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 01 21:44:15.471337 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:44:15.471305 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3224f451-02bc-496d-9fa0-9045b1dbb798-kube-api-access-wrvgr" (OuterVolumeSpecName: "kube-api-access-wrvgr") pod "3224f451-02bc-496d-9fa0-9045b1dbb798" (UID: "3224f451-02bc-496d-9fa0-9045b1dbb798"). InnerVolumeSpecName "kube-api-access-wrvgr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 01 21:44:15.471477 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:44:15.471308 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3224f451-02bc-496d-9fa0-9045b1dbb798-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "3224f451-02bc-496d-9fa0-9045b1dbb798" (UID: "3224f451-02bc-496d-9fa0-9045b1dbb798"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 01 21:44:15.570013 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:44:15.569980 2573 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/3224f451-02bc-496d-9fa0-9045b1dbb798-registry-certificates\") on node \"ip-10-0-138-241.ec2.internal\" DevicePath \"\"" Jun 01 21:44:15.570013 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:44:15.570012 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wrvgr\" (UniqueName: \"kubernetes.io/projected/3224f451-02bc-496d-9fa0-9045b1dbb798-kube-api-access-wrvgr\") on node \"ip-10-0-138-241.ec2.internal\" DevicePath \"\"" Jun 01 21:44:15.570200 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:44:15.570022 2573 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/3224f451-02bc-496d-9fa0-9045b1dbb798-installation-pull-secrets\") on node \"ip-10-0-138-241.ec2.internal\" DevicePath \"\"" Jun 01 21:44:15.570200 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:44:15.570032 2573 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/3224f451-02bc-496d-9fa0-9045b1dbb798-registry-tls\") on node \"ip-10-0-138-241.ec2.internal\" DevicePath \"\"" Jun 01 21:44:15.624841 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:44:15.624810 2573 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-77447d89bb-twkln"] Jun 01 21:44:15.628841 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:44:15.628820 2573 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-77447d89bb-twkln"] Jun 01 21:44:16.677039 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:44:16.677006 2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3224f451-02bc-496d-9fa0-9045b1dbb798" path="/var/lib/kubelet/pods/3224f451-02bc-496d-9fa0-9045b1dbb798/volumes" Jun 01 21:44:25.051147 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:44:25.051103 2573 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-bb6fb6964-vcwwk" podUID="5e29e06d-c3f7-4b72-a9c7-99e78ad3946c" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500" Jun 01 21:44:35.051019 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:44:35.050979 2573 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-bb6fb6964-vcwwk" podUID="5e29e06d-c3f7-4b72-a9c7-99e78ad3946c" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500" Jun 01 21:44:45.051275 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:44:45.051231 2573 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-bb6fb6964-vcwwk" podUID="5e29e06d-c3f7-4b72-a9c7-99e78ad3946c" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500" Jun 01 21:44:45.051795 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:44:45.051312 2573 kubelet.go:2658] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-bb6fb6964-vcwwk" Jun 01 21:44:45.051843 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:44:45.051824 2573 kuberuntime_manager.go:1107] "Message for Container of pod" containerName="service-proxy" containerStatusID={"Type":"cri-o","ID":"8db9e0efeb656decc8939aa6a44749d94f9de8f520b76bc8de52bb48f5c8ef4b"} pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-bb6fb6964-vcwwk" containerMessage="Container service-proxy failed liveness probe, will be restarted" Jun 01 21:44:45.051885 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:44:45.051860 2573 kuberuntime_container.go:864] "Killing container with a grace period" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-bb6fb6964-vcwwk" podUID="5e29e06d-c3f7-4b72-a9c7-99e78ad3946c" containerName="service-proxy" containerID="cri-o://8db9e0efeb656decc8939aa6a44749d94f9de8f520b76bc8de52bb48f5c8ef4b" gracePeriod=30 Jun 01 21:44:45.381058 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:44:45.381015 2573 generic.go:358] "Generic (PLEG): container finished" podID="5e29e06d-c3f7-4b72-a9c7-99e78ad3946c" containerID="8db9e0efeb656decc8939aa6a44749d94f9de8f520b76bc8de52bb48f5c8ef4b" exitCode=2 Jun 01 21:44:45.381229 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:44:45.381065 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-bb6fb6964-vcwwk" event={"ID":"5e29e06d-c3f7-4b72-a9c7-99e78ad3946c","Type":"ContainerDied","Data":"8db9e0efeb656decc8939aa6a44749d94f9de8f520b76bc8de52bb48f5c8ef4b"} Jun 01 21:44:45.381229 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:44:45.381091 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-bb6fb6964-vcwwk" event={"ID":"5e29e06d-c3f7-4b72-a9c7-99e78ad3946c","Type":"ContainerStarted","Data":"28f3e4036e7a3394d42b10aecf9327bf13305ac4ce0ffe45fad11733c2f4237f"} Jun 01 21:44:52.531197 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:44:52.531157 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f99cc923-cc5c-4895-8b2b-6ba878ba343b-metrics-certs\") pod \"network-metrics-daemon-w4d7s\" (UID: \"f99cc923-cc5c-4895-8b2b-6ba878ba343b\") " pod="openshift-multus/network-metrics-daemon-w4d7s" Jun 01 21:44:52.533505 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:44:52.533480 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f99cc923-cc5c-4895-8b2b-6ba878ba343b-metrics-certs\") pod \"network-metrics-daemon-w4d7s\" (UID: \"f99cc923-cc5c-4895-8b2b-6ba878ba343b\") " pod="openshift-multus/network-metrics-daemon-w4d7s" Jun 01 21:44:52.582216 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:44:52.582186 2573 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-vgrbz\"" Jun 01 21:44:52.589675 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:44:52.589653 2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w4d7s" Jun 01 21:44:52.712809 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:44:52.712778 2573 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-w4d7s"] Jun 01 21:44:52.717822 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:44:52.717794 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf99cc923_cc5c_4895_8b2b_6ba878ba343b.slice/crio-848b79119635cdccafb37c191b0cd7a0dc04e9882e6e52bd57838c65f2c51dde WatchSource:0}: Error finding container 848b79119635cdccafb37c191b0cd7a0dc04e9882e6e52bd57838c65f2c51dde: Status 404 returned error can't find the container with id 848b79119635cdccafb37c191b0cd7a0dc04e9882e6e52bd57838c65f2c51dde Jun 01 21:44:53.402742 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:44:53.402707 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-w4d7s" event={"ID":"f99cc923-cc5c-4895-8b2b-6ba878ba343b","Type":"ContainerStarted","Data":"848b79119635cdccafb37c191b0cd7a0dc04e9882e6e52bd57838c65f2c51dde"} Jun 01 21:44:54.406523 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:44:54.406483 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-w4d7s" event={"ID":"f99cc923-cc5c-4895-8b2b-6ba878ba343b","Type":"ContainerStarted","Data":"f607de9a3f23fff894a130deed4f5d68d6a347ee4fa602b3cd8e2b1fe3cf1f34"} Jun 01 21:44:54.406523 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:44:54.406525 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-w4d7s" event={"ID":"f99cc923-cc5c-4895-8b2b-6ba878ba343b","Type":"ContainerStarted","Data":"f199274fb8f2d4b157c3eaf0632698c9706c5062337c86fe6ebcc7842225b135"} Jun 01 21:44:54.424406 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:44:54.424354 2573 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-w4d7s" podStartSLOduration=253.239126742 podStartE2EDuration="4m14.424340772s" podCreationTimestamp="2026-06-01 21:40:40 +0000 UTC" firstStartedPulling="2026-06-01 21:44:52.719582285 +0000 UTC m=+252.439634219" lastFinishedPulling="2026-06-01 21:44:53.90479632 +0000 UTC m=+253.624848249" observedRunningTime="2026-06-01 21:44:54.422859764 +0000 UTC m=+254.142911707" watchObservedRunningTime="2026-06-01 21:44:54.424340772 +0000 UTC m=+254.144392714" Jun 01 21:45:06.781635 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:45:06.781601 2573 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-7d9f684f78-cjls7"] Jun 01 21:45:06.781998 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:45:06.781814 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3224f451-02bc-496d-9fa0-9045b1dbb798" containerName="registry" Jun 01 21:45:06.781998 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:45:06.781824 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="3224f451-02bc-496d-9fa0-9045b1dbb798" containerName="registry" Jun 01 21:45:06.781998 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:45:06.781869 2573 memory_manager.go:356] "RemoveStaleState removing state" podUID="3224f451-02bc-496d-9fa0-9045b1dbb798" containerName="registry" Jun 01 21:45:06.784779 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:45:06.784759 2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-7d9f684f78-cjls7" Jun 01 21:45:06.787834 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:45:06.787816 2573 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\"" Jun 01 21:45:06.787957 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:45:06.787882 2573 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\"" Jun 01 21:45:06.787957 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:45:06.787916 2573 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\"" Jun 01 21:45:06.787957 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:45:06.787928 2573 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\"" Jun 01 21:45:06.788110 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:45:06.787988 2573 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\"" Jun 01 21:45:06.788110 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:45:06.787992 2573 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-2nqpr\"" Jun 01 21:45:06.800836 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:45:06.795658 2573 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\"" Jun 01 21:45:06.802373 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:45:06.802348 2573 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-7d9f684f78-cjls7"] Jun 01 21:45:06.830049 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:45:06.830012 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4883866e-975d-4eb8-8d98-062ea8cd2a5a-serving-certs-ca-bundle\") pod \"telemeter-client-7d9f684f78-cjls7\" (UID: \"4883866e-975d-4eb8-8d98-062ea8cd2a5a\") " pod="openshift-monitoring/telemeter-client-7d9f684f78-cjls7" Jun 01 21:45:06.830049 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:45:06.830046 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4883866e-975d-4eb8-8d98-062ea8cd2a5a-telemeter-trusted-ca-bundle\") pod \"telemeter-client-7d9f684f78-cjls7\" (UID: \"4883866e-975d-4eb8-8d98-062ea8cd2a5a\") " pod="openshift-monitoring/telemeter-client-7d9f684f78-cjls7" Jun 01 21:45:06.830251 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:45:06.830071 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fwpdc\" (UniqueName: \"kubernetes.io/projected/4883866e-975d-4eb8-8d98-062ea8cd2a5a-kube-api-access-fwpdc\") pod \"telemeter-client-7d9f684f78-cjls7\" (UID: \"4883866e-975d-4eb8-8d98-062ea8cd2a5a\") " pod="openshift-monitoring/telemeter-client-7d9f684f78-cjls7" Jun 01 21:45:06.830251 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:45:06.830157 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/4883866e-975d-4eb8-8d98-062ea8cd2a5a-federate-client-tls\") pod \"telemeter-client-7d9f684f78-cjls7\" (UID: \"4883866e-975d-4eb8-8d98-062ea8cd2a5a\") " pod="openshift-monitoring/telemeter-client-7d9f684f78-cjls7" Jun 01 21:45:06.830251 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:45:06.830199 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/4883866e-975d-4eb8-8d98-062ea8cd2a5a-telemeter-client-tls\") pod \"telemeter-client-7d9f684f78-cjls7\" (UID: \"4883866e-975d-4eb8-8d98-062ea8cd2a5a\") " pod="openshift-monitoring/telemeter-client-7d9f684f78-cjls7" Jun 01 21:45:06.830251 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:45:06.830229 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/4883866e-975d-4eb8-8d98-062ea8cd2a5a-secret-telemeter-client\") pod \"telemeter-client-7d9f684f78-cjls7\" (UID: \"4883866e-975d-4eb8-8d98-062ea8cd2a5a\") " pod="openshift-monitoring/telemeter-client-7d9f684f78-cjls7" Jun 01 21:45:06.830376 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:45:06.830282 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/4883866e-975d-4eb8-8d98-062ea8cd2a5a-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-7d9f684f78-cjls7\" (UID: \"4883866e-975d-4eb8-8d98-062ea8cd2a5a\") " pod="openshift-monitoring/telemeter-client-7d9f684f78-cjls7" Jun 01 21:45:06.830376 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:45:06.830319 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4883866e-975d-4eb8-8d98-062ea8cd2a5a-metrics-client-ca\") pod \"telemeter-client-7d9f684f78-cjls7\" (UID: \"4883866e-975d-4eb8-8d98-062ea8cd2a5a\") " pod="openshift-monitoring/telemeter-client-7d9f684f78-cjls7" Jun 01 21:45:06.931638 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:45:06.931593 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/4883866e-975d-4eb8-8d98-062ea8cd2a5a-secret-telemeter-client\") pod \"telemeter-client-7d9f684f78-cjls7\" (UID: \"4883866e-975d-4eb8-8d98-062ea8cd2a5a\") " pod="openshift-monitoring/telemeter-client-7d9f684f78-cjls7" Jun 01 21:45:06.931822 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:45:06.931649 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/4883866e-975d-4eb8-8d98-062ea8cd2a5a-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-7d9f684f78-cjls7\" (UID: \"4883866e-975d-4eb8-8d98-062ea8cd2a5a\") " pod="openshift-monitoring/telemeter-client-7d9f684f78-cjls7" Jun 01 21:45:06.931822 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:45:06.931680 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4883866e-975d-4eb8-8d98-062ea8cd2a5a-metrics-client-ca\") pod \"telemeter-client-7d9f684f78-cjls7\" (UID: \"4883866e-975d-4eb8-8d98-062ea8cd2a5a\") " pod="openshift-monitoring/telemeter-client-7d9f684f78-cjls7" Jun 01 21:45:06.931822 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:45:06.931711 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4883866e-975d-4eb8-8d98-062ea8cd2a5a-serving-certs-ca-bundle\") pod \"telemeter-client-7d9f684f78-cjls7\" (UID: \"4883866e-975d-4eb8-8d98-062ea8cd2a5a\") " pod="openshift-monitoring/telemeter-client-7d9f684f78-cjls7" Jun 01 21:45:06.931822 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:45:06.931734 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4883866e-975d-4eb8-8d98-062ea8cd2a5a-telemeter-trusted-ca-bundle\") pod \"telemeter-client-7d9f684f78-cjls7\" (UID: \"4883866e-975d-4eb8-8d98-062ea8cd2a5a\") " pod="openshift-monitoring/telemeter-client-7d9f684f78-cjls7" Jun 01 21:45:06.931822 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:45:06.931769 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-fwpdc\" (UniqueName: \"kubernetes.io/projected/4883866e-975d-4eb8-8d98-062ea8cd2a5a-kube-api-access-fwpdc\") pod \"telemeter-client-7d9f684f78-cjls7\" (UID: \"4883866e-975d-4eb8-8d98-062ea8cd2a5a\") " pod="openshift-monitoring/telemeter-client-7d9f684f78-cjls7" Jun 01 21:45:06.932068 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:45:06.931813 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/4883866e-975d-4eb8-8d98-062ea8cd2a5a-federate-client-tls\") pod \"telemeter-client-7d9f684f78-cjls7\" (UID: \"4883866e-975d-4eb8-8d98-062ea8cd2a5a\") " pod="openshift-monitoring/telemeter-client-7d9f684f78-cjls7" Jun 01 21:45:06.932068 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:45:06.931893 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/4883866e-975d-4eb8-8d98-062ea8cd2a5a-telemeter-client-tls\") pod \"telemeter-client-7d9f684f78-cjls7\" (UID: \"4883866e-975d-4eb8-8d98-062ea8cd2a5a\") " pod="openshift-monitoring/telemeter-client-7d9f684f78-cjls7" Jun 01 21:45:06.932618 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:45:06.932585 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4883866e-975d-4eb8-8d98-062ea8cd2a5a-serving-certs-ca-bundle\") pod \"telemeter-client-7d9f684f78-cjls7\" (UID: \"4883866e-975d-4eb8-8d98-062ea8cd2a5a\") " pod="openshift-monitoring/telemeter-client-7d9f684f78-cjls7" Jun 01 21:45:06.932758 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:45:06.932715 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4883866e-975d-4eb8-8d98-062ea8cd2a5a-metrics-client-ca\") pod \"telemeter-client-7d9f684f78-cjls7\" (UID: \"4883866e-975d-4eb8-8d98-062ea8cd2a5a\") " pod="openshift-monitoring/telemeter-client-7d9f684f78-cjls7" Jun 01 21:45:06.933007 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:45:06.932982 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4883866e-975d-4eb8-8d98-062ea8cd2a5a-telemeter-trusted-ca-bundle\") pod \"telemeter-client-7d9f684f78-cjls7\" (UID: \"4883866e-975d-4eb8-8d98-062ea8cd2a5a\") " pod="openshift-monitoring/telemeter-client-7d9f684f78-cjls7" Jun 01 21:45:06.934274 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:45:06.934242 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/4883866e-975d-4eb8-8d98-062ea8cd2a5a-secret-telemeter-client\") pod \"telemeter-client-7d9f684f78-cjls7\" (UID: \"4883866e-975d-4eb8-8d98-062ea8cd2a5a\") " pod="openshift-monitoring/telemeter-client-7d9f684f78-cjls7" Jun 01 21:45:06.934378 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:45:06.934336 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/4883866e-975d-4eb8-8d98-062ea8cd2a5a-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-7d9f684f78-cjls7\" (UID: \"4883866e-975d-4eb8-8d98-062ea8cd2a5a\") " pod="openshift-monitoring/telemeter-client-7d9f684f78-cjls7" Jun 01 21:45:06.934591 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:45:06.934574 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/4883866e-975d-4eb8-8d98-062ea8cd2a5a-telemeter-client-tls\") pod \"telemeter-client-7d9f684f78-cjls7\" (UID: \"4883866e-975d-4eb8-8d98-062ea8cd2a5a\") " pod="openshift-monitoring/telemeter-client-7d9f684f78-cjls7" Jun 01 21:45:06.934653 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:45:06.934577 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/4883866e-975d-4eb8-8d98-062ea8cd2a5a-federate-client-tls\") pod \"telemeter-client-7d9f684f78-cjls7\" (UID: \"4883866e-975d-4eb8-8d98-062ea8cd2a5a\") " pod="openshift-monitoring/telemeter-client-7d9f684f78-cjls7" Jun 01 21:45:06.940096 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:45:06.940066 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fwpdc\" (UniqueName: \"kubernetes.io/projected/4883866e-975d-4eb8-8d98-062ea8cd2a5a-kube-api-access-fwpdc\") pod \"telemeter-client-7d9f684f78-cjls7\" (UID: \"4883866e-975d-4eb8-8d98-062ea8cd2a5a\") " pod="openshift-monitoring/telemeter-client-7d9f684f78-cjls7" Jun 01 21:45:07.093654 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:45:07.093622 2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-7d9f684f78-cjls7" Jun 01 21:45:07.223665 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:45:07.223620 2573 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-7d9f684f78-cjls7"] Jun 01 21:45:07.225791 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:45:07.225762 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4883866e_975d_4eb8_8d98_062ea8cd2a5a.slice/crio-6ff3b248c873542fe013e639e68ffe1c8a63a505c5424877a7e941987b169436 WatchSource:0}: Error finding container 6ff3b248c873542fe013e639e68ffe1c8a63a505c5424877a7e941987b169436: Status 404 returned error can't find the container with id 6ff3b248c873542fe013e639e68ffe1c8a63a505c5424877a7e941987b169436 Jun 01 21:45:07.440390 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:45:07.440309 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-7d9f684f78-cjls7" event={"ID":"4883866e-975d-4eb8-8d98-062ea8cd2a5a","Type":"ContainerStarted","Data":"6ff3b248c873542fe013e639e68ffe1c8a63a505c5424877a7e941987b169436"} Jun 01 21:45:10.454132 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:45:10.454101 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-7d9f684f78-cjls7" event={"ID":"4883866e-975d-4eb8-8d98-062ea8cd2a5a","Type":"ContainerStarted","Data":"96de039313cb0395dfe456e2af647419d304afda638693239bb83ade5ac3f585"} Jun 01 21:45:11.458090 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:45:11.458051 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-7d9f684f78-cjls7" event={"ID":"4883866e-975d-4eb8-8d98-062ea8cd2a5a","Type":"ContainerStarted","Data":"a8413998e7b3127f5da77064dab270fed0c24c3b2438e0ef47c5f4e7a2bb3c66"} Jun 01 21:45:11.458090 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:45:11.458090 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-7d9f684f78-cjls7" event={"ID":"4883866e-975d-4eb8-8d98-062ea8cd2a5a","Type":"ContainerStarted","Data":"bdd19bd4dee26ff16c989810ee33cf7017ab4d0fecb218c9a3a4246ba30df7d9"} Jun 01 21:45:11.481988 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:45:11.481940 2573 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-7d9f684f78-cjls7" podStartSLOduration=2.206534003 podStartE2EDuration="5.481927086s" podCreationTimestamp="2026-06-01 21:45:06 +0000 UTC" firstStartedPulling="2026-06-01 21:45:07.227754164 +0000 UTC m=+266.947806086" lastFinishedPulling="2026-06-01 21:45:10.503147246 +0000 UTC m=+270.223199169" observedRunningTime="2026-06-01 21:45:11.480031087 +0000 UTC m=+271.200083030" watchObservedRunningTime="2026-06-01 21:45:11.481927086 +0000 UTC m=+271.201979028" Jun 01 21:45:40.634764 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:45:40.634721 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rcqkp_fcf0209a-2440-4267-834a-b176f4e337e4/ovn-acl-logging/0.log" Jun 01 21:45:40.636616 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:45:40.636588 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rcqkp_fcf0209a-2440-4267-834a-b176f4e337e4/ovn-acl-logging/0.log" Jun 01 21:45:40.638652 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:45:40.638632 2573 kubelet.go:1628] "Image garbage collection succeeded" Jun 01 21:48:24.312523 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:48:24.312484 2573 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-64fd4455d7-nqvzv"] Jun 01 21:48:24.315461 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:48:24.315440 2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-64fd4455d7-nqvzv" Jun 01 21:48:24.320288 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:48:24.320254 2573 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-r5796\"" Jun 01 21:48:24.320288 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:48:24.320267 2573 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jun 01 21:48:24.320288 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:48:24.320270 2573 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Jun 01 21:48:24.320519 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:48:24.320342 2573 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Jun 01 21:48:24.320519 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:48:24.320254 2573 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Jun 01 21:48:24.320519 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:48:24.320422 2573 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jun 01 21:48:24.323864 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:48:24.323843 2573 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-64fd4455d7-nqvzv"] Jun 01 21:48:24.335953 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:48:24.335927 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4fftz\" (UniqueName: \"kubernetes.io/projected/f6942086-3c8d-4f2e-8d27-1329e670b4ad-kube-api-access-4fftz\") pod \"lws-controller-manager-64fd4455d7-nqvzv\" (UID: \"f6942086-3c8d-4f2e-8d27-1329e670b4ad\") " pod="openshift-lws-operator/lws-controller-manager-64fd4455d7-nqvzv" Jun 01 21:48:24.336111 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:48:24.335963 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f6942086-3c8d-4f2e-8d27-1329e670b4ad-cert\") pod \"lws-controller-manager-64fd4455d7-nqvzv\" (UID: \"f6942086-3c8d-4f2e-8d27-1329e670b4ad\") " pod="openshift-lws-operator/lws-controller-manager-64fd4455d7-nqvzv" Jun 01 21:48:24.336111 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:48:24.335999 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/f6942086-3c8d-4f2e-8d27-1329e670b4ad-manager-config\") pod \"lws-controller-manager-64fd4455d7-nqvzv\" (UID: \"f6942086-3c8d-4f2e-8d27-1329e670b4ad\") " pod="openshift-lws-operator/lws-controller-manager-64fd4455d7-nqvzv" Jun 01 21:48:24.336207 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:48:24.336145 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/f6942086-3c8d-4f2e-8d27-1329e670b4ad-metrics-cert\") pod \"lws-controller-manager-64fd4455d7-nqvzv\" (UID: \"f6942086-3c8d-4f2e-8d27-1329e670b4ad\") " pod="openshift-lws-operator/lws-controller-manager-64fd4455d7-nqvzv" Jun 01 21:48:24.437469 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:48:24.437427 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/f6942086-3c8d-4f2e-8d27-1329e670b4ad-metrics-cert\") pod \"lws-controller-manager-64fd4455d7-nqvzv\" (UID: \"f6942086-3c8d-4f2e-8d27-1329e670b4ad\") " pod="openshift-lws-operator/lws-controller-manager-64fd4455d7-nqvzv" Jun 01 21:48:24.437469 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:48:24.437475 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-4fftz\" (UniqueName: \"kubernetes.io/projected/f6942086-3c8d-4f2e-8d27-1329e670b4ad-kube-api-access-4fftz\") pod \"lws-controller-manager-64fd4455d7-nqvzv\" (UID: \"f6942086-3c8d-4f2e-8d27-1329e670b4ad\") " pod="openshift-lws-operator/lws-controller-manager-64fd4455d7-nqvzv" Jun 01 21:48:24.437748 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:48:24.437552 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f6942086-3c8d-4f2e-8d27-1329e670b4ad-cert\") pod \"lws-controller-manager-64fd4455d7-nqvzv\" (UID: \"f6942086-3c8d-4f2e-8d27-1329e670b4ad\") " pod="openshift-lws-operator/lws-controller-manager-64fd4455d7-nqvzv" Jun 01 21:48:24.437748 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:48:24.437598 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/f6942086-3c8d-4f2e-8d27-1329e670b4ad-manager-config\") pod \"lws-controller-manager-64fd4455d7-nqvzv\" (UID: \"f6942086-3c8d-4f2e-8d27-1329e670b4ad\") " pod="openshift-lws-operator/lws-controller-manager-64fd4455d7-nqvzv" Jun 01 21:48:24.438226 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:48:24.438205 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/f6942086-3c8d-4f2e-8d27-1329e670b4ad-manager-config\") pod \"lws-controller-manager-64fd4455d7-nqvzv\" (UID: \"f6942086-3c8d-4f2e-8d27-1329e670b4ad\") " pod="openshift-lws-operator/lws-controller-manager-64fd4455d7-nqvzv" Jun 01 21:48:24.440037 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:48:24.440013 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f6942086-3c8d-4f2e-8d27-1329e670b4ad-cert\") pod \"lws-controller-manager-64fd4455d7-nqvzv\" (UID: \"f6942086-3c8d-4f2e-8d27-1329e670b4ad\") " pod="openshift-lws-operator/lws-controller-manager-64fd4455d7-nqvzv" Jun 01 21:48:24.440037 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:48:24.440030 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/f6942086-3c8d-4f2e-8d27-1329e670b4ad-metrics-cert\") pod \"lws-controller-manager-64fd4455d7-nqvzv\" (UID: \"f6942086-3c8d-4f2e-8d27-1329e670b4ad\") " pod="openshift-lws-operator/lws-controller-manager-64fd4455d7-nqvzv" Jun 01 21:48:24.450837 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:48:24.450815 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4fftz\" (UniqueName: \"kubernetes.io/projected/f6942086-3c8d-4f2e-8d27-1329e670b4ad-kube-api-access-4fftz\") pod \"lws-controller-manager-64fd4455d7-nqvzv\" (UID: \"f6942086-3c8d-4f2e-8d27-1329e670b4ad\") " pod="openshift-lws-operator/lws-controller-manager-64fd4455d7-nqvzv" Jun 01 21:48:24.629509 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:48:24.629412 2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-64fd4455d7-nqvzv" Jun 01 21:48:24.755805 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:48:24.755770 2573 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-64fd4455d7-nqvzv"] Jun 01 21:48:24.758647 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:48:24.758621 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf6942086_3c8d_4f2e_8d27_1329e670b4ad.slice/crio-00924eec9fa4df23ff3790d2df3403f002b98b6399792dcc792608888247e001 WatchSource:0}: Error finding container 00924eec9fa4df23ff3790d2df3403f002b98b6399792dcc792608888247e001: Status 404 returned error can't find the container with id 00924eec9fa4df23ff3790d2df3403f002b98b6399792dcc792608888247e001 Jun 01 21:48:24.760357 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:48:24.760342 2573 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jun 01 21:48:24.949457 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:48:24.949370 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-64fd4455d7-nqvzv" event={"ID":"f6942086-3c8d-4f2e-8d27-1329e670b4ad","Type":"ContainerStarted","Data":"00924eec9fa4df23ff3790d2df3403f002b98b6399792dcc792608888247e001"} Jun 01 21:48:29.963674 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:48:29.963634 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-64fd4455d7-nqvzv" event={"ID":"f6942086-3c8d-4f2e-8d27-1329e670b4ad","Type":"ContainerStarted","Data":"78b32592a9b41ce9f993d1151730438e95173abdd0a725bbd0cd496ca23c20e6"} Jun 01 21:48:29.964035 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:48:29.963747 2573 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-64fd4455d7-nqvzv" Jun 01 21:48:29.984516 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:48:29.984464 2573 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-64fd4455d7-nqvzv" podStartSLOduration=1.319602092 podStartE2EDuration="5.984449634s" podCreationTimestamp="2026-06-01 21:48:24 +0000 UTC" firstStartedPulling="2026-06-01 21:48:24.760472955 +0000 UTC m=+464.480524888" lastFinishedPulling="2026-06-01 21:48:29.425320504 +0000 UTC m=+469.145372430" observedRunningTime="2026-06-01 21:48:29.983676307 +0000 UTC m=+469.703728249" watchObservedRunningTime="2026-06-01 21:48:29.984449634 +0000 UTC m=+469.704501608" Jun 01 21:48:30.836749 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:48:30.836714 2573 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5568c4bcd5-29s6k"] Jun 01 21:48:30.839894 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:48:30.839874 2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-5568c4bcd5-29s6k" Jun 01 21:48:30.844682 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:48:30.844659 2573 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jun 01 21:48:30.844804 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:48:30.844736 2573 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jun 01 21:48:30.845348 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:48:30.845325 2573 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jun 01 21:48:30.845653 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:48:30.845635 2573 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-d54dr\"" Jun 01 21:48:30.846317 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:48:30.846298 2573 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jun 01 21:48:30.884504 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:48:30.884465 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/d226ebd5-dfa6-4f73-a032-3f59142dc353-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5568c4bcd5-29s6k\" (UID: \"d226ebd5-dfa6-4f73-a032-3f59142dc353\") " pod="opendatahub/opendatahub-operator-controller-manager-5568c4bcd5-29s6k" Jun 01 21:48:30.884659 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:48:30.884509 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f6h78\" (UniqueName: \"kubernetes.io/projected/d226ebd5-dfa6-4f73-a032-3f59142dc353-kube-api-access-f6h78\") pod \"opendatahub-operator-controller-manager-5568c4bcd5-29s6k\" (UID: \"d226ebd5-dfa6-4f73-a032-3f59142dc353\") " pod="opendatahub/opendatahub-operator-controller-manager-5568c4bcd5-29s6k" Jun 01 21:48:30.884659 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:48:30.884559 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/d226ebd5-dfa6-4f73-a032-3f59142dc353-manifests\") pod \"opendatahub-operator-controller-manager-5568c4bcd5-29s6k\" (UID: \"d226ebd5-dfa6-4f73-a032-3f59142dc353\") " pod="opendatahub/opendatahub-operator-controller-manager-5568c4bcd5-29s6k" Jun 01 21:48:30.884659 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:48:30.884587 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d226ebd5-dfa6-4f73-a032-3f59142dc353-webhook-cert\") pod \"opendatahub-operator-controller-manager-5568c4bcd5-29s6k\" (UID: \"d226ebd5-dfa6-4f73-a032-3f59142dc353\") " pod="opendatahub/opendatahub-operator-controller-manager-5568c4bcd5-29s6k" Jun 01 21:48:30.884659 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:48:30.884614 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d226ebd5-dfa6-4f73-a032-3f59142dc353-tmp\") pod \"opendatahub-operator-controller-manager-5568c4bcd5-29s6k\" (UID: \"d226ebd5-dfa6-4f73-a032-3f59142dc353\") " pod="opendatahub/opendatahub-operator-controller-manager-5568c4bcd5-29s6k" Jun 01 21:48:30.899277 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:48:30.899249 2573 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5568c4bcd5-29s6k"] Jun 01 21:48:30.985195 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:48:30.985161 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d226ebd5-dfa6-4f73-a032-3f59142dc353-tmp\") pod \"opendatahub-operator-controller-manager-5568c4bcd5-29s6k\" (UID: \"d226ebd5-dfa6-4f73-a032-3f59142dc353\") " pod="opendatahub/opendatahub-operator-controller-manager-5568c4bcd5-29s6k" Jun 01 21:48:30.985690 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:48:30.985204 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/d226ebd5-dfa6-4f73-a032-3f59142dc353-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5568c4bcd5-29s6k\" (UID: \"d226ebd5-dfa6-4f73-a032-3f59142dc353\") " pod="opendatahub/opendatahub-operator-controller-manager-5568c4bcd5-29s6k" Jun 01 21:48:30.985690 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:48:30.985338 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-f6h78\" (UniqueName: \"kubernetes.io/projected/d226ebd5-dfa6-4f73-a032-3f59142dc353-kube-api-access-f6h78\") pod \"opendatahub-operator-controller-manager-5568c4bcd5-29s6k\" (UID: \"d226ebd5-dfa6-4f73-a032-3f59142dc353\") " pod="opendatahub/opendatahub-operator-controller-manager-5568c4bcd5-29s6k" Jun 01 21:48:30.985690 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:48:30.985398 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/d226ebd5-dfa6-4f73-a032-3f59142dc353-manifests\") pod \"opendatahub-operator-controller-manager-5568c4bcd5-29s6k\" (UID: \"d226ebd5-dfa6-4f73-a032-3f59142dc353\") " pod="opendatahub/opendatahub-operator-controller-manager-5568c4bcd5-29s6k" Jun 01 21:48:30.985690 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:48:30.985449 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d226ebd5-dfa6-4f73-a032-3f59142dc353-webhook-cert\") pod \"opendatahub-operator-controller-manager-5568c4bcd5-29s6k\" (UID: \"d226ebd5-dfa6-4f73-a032-3f59142dc353\") " pod="opendatahub/opendatahub-operator-controller-manager-5568c4bcd5-29s6k" Jun 01 21:48:30.985872 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:48:30.985783 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/d226ebd5-dfa6-4f73-a032-3f59142dc353-manifests\") pod \"opendatahub-operator-controller-manager-5568c4bcd5-29s6k\" (UID: \"d226ebd5-dfa6-4f73-a032-3f59142dc353\") " pod="opendatahub/opendatahub-operator-controller-manager-5568c4bcd5-29s6k" Jun 01 21:48:30.987522 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:48:30.987496 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d226ebd5-dfa6-4f73-a032-3f59142dc353-tmp\") pod \"opendatahub-operator-controller-manager-5568c4bcd5-29s6k\" (UID: \"d226ebd5-dfa6-4f73-a032-3f59142dc353\") " pod="opendatahub/opendatahub-operator-controller-manager-5568c4bcd5-29s6k" Jun 01 21:48:30.987743 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:48:30.987725 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d226ebd5-dfa6-4f73-a032-3f59142dc353-webhook-cert\") pod \"opendatahub-operator-controller-manager-5568c4bcd5-29s6k\" (UID: \"d226ebd5-dfa6-4f73-a032-3f59142dc353\") " pod="opendatahub/opendatahub-operator-controller-manager-5568c4bcd5-29s6k" Jun 01 21:48:30.987906 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:48:30.987888 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/d226ebd5-dfa6-4f73-a032-3f59142dc353-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5568c4bcd5-29s6k\" (UID: \"d226ebd5-dfa6-4f73-a032-3f59142dc353\") " pod="opendatahub/opendatahub-operator-controller-manager-5568c4bcd5-29s6k" Jun 01 21:48:30.999978 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:48:30.999951 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-f6h78\" (UniqueName: \"kubernetes.io/projected/d226ebd5-dfa6-4f73-a032-3f59142dc353-kube-api-access-f6h78\") pod \"opendatahub-operator-controller-manager-5568c4bcd5-29s6k\" (UID: \"d226ebd5-dfa6-4f73-a032-3f59142dc353\") " pod="opendatahub/opendatahub-operator-controller-manager-5568c4bcd5-29s6k" Jun 01 21:48:31.149254 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:48:31.149163 2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-5568c4bcd5-29s6k" Jun 01 21:48:31.279096 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:48:31.279060 2573 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5568c4bcd5-29s6k"] Jun 01 21:48:31.282047 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:48:31.282020 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd226ebd5_dfa6_4f73_a032_3f59142dc353.slice/crio-c5cd34c1100ed773c81e5256fd38e2281fcd192f7c19c8e6caac28d4570799ec WatchSource:0}: Error finding container c5cd34c1100ed773c81e5256fd38e2281fcd192f7c19c8e6caac28d4570799ec: Status 404 returned error can't find the container with id c5cd34c1100ed773c81e5256fd38e2281fcd192f7c19c8e6caac28d4570799ec Jun 01 21:48:31.970864 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:48:31.970818 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5568c4bcd5-29s6k" event={"ID":"d226ebd5-dfa6-4f73-a032-3f59142dc353","Type":"ContainerStarted","Data":"c5cd34c1100ed773c81e5256fd38e2281fcd192f7c19c8e6caac28d4570799ec"} Jun 01 21:48:34.980344 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:48:34.980305 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5568c4bcd5-29s6k" event={"ID":"d226ebd5-dfa6-4f73-a032-3f59142dc353","Type":"ContainerStarted","Data":"7ef69cf7bde9e4511b28d8170e85cf83d9567bad533a20be9380f1171c657a3c"} Jun 01 21:48:35.984839 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:48:35.984802 2573 generic.go:358] "Generic (PLEG): container finished" podID="d226ebd5-dfa6-4f73-a032-3f59142dc353" containerID="7ef69cf7bde9e4511b28d8170e85cf83d9567bad533a20be9380f1171c657a3c" exitCode=0 Jun 01 21:48:35.985309 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:48:35.984844 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5568c4bcd5-29s6k" event={"ID":"d226ebd5-dfa6-4f73-a032-3f59142dc353","Type":"ContainerDied","Data":"7ef69cf7bde9e4511b28d8170e85cf83d9567bad533a20be9380f1171c657a3c"} Jun 01 21:48:36.988719 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:48:36.988686 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5568c4bcd5-29s6k" event={"ID":"d226ebd5-dfa6-4f73-a032-3f59142dc353","Type":"ContainerStarted","Data":"f26c52b4fe2416482b8de9cfc83c897b3354f83b927cb69ee52dcb0fe139daac"} Jun 01 21:48:36.989121 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:48:36.988819 2573 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-5568c4bcd5-29s6k" Jun 01 21:48:37.012075 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:48:37.012029 2573 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-5568c4bcd5-29s6k" podStartSLOduration=1.932356218 podStartE2EDuration="7.012013842s" podCreationTimestamp="2026-06-01 21:48:30 +0000 UTC" firstStartedPulling="2026-06-01 21:48:31.283787947 +0000 UTC m=+471.003839868" lastFinishedPulling="2026-06-01 21:48:36.363445572 +0000 UTC m=+476.083497492" observedRunningTime="2026-06-01 21:48:37.010430477 +0000 UTC m=+476.730482435" watchObservedRunningTime="2026-06-01 21:48:37.012013842 +0000 UTC m=+476.732065785" Jun 01 21:48:40.969156 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:48:40.969128 2573 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-64fd4455d7-nqvzv" Jun 01 21:48:47.994077 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:48:47.994038 2573 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-5568c4bcd5-29s6k" Jun 01 21:48:53.459183 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:48:53.459147 2573 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-67dd84b75b-ppsmf"] Jun 01 21:48:53.475601 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:48:53.475572 2573 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-67dd84b75b-ppsmf"] Jun 01 21:48:53.475784 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:48:53.475692 2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-67dd84b75b-ppsmf" Jun 01 21:48:53.479276 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:48:53.479248 2573 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jun 01 21:48:53.480941 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:48:53.480917 2573 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jun 01 21:48:53.481035 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:48:53.480956 2573 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-6j6zf\"" Jun 01 21:48:53.481035 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:48:53.481026 2573 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Jun 01 21:48:53.481127 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:48:53.481116 2573 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Jun 01 21:48:53.561343 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:48:53.561307 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/613a1517-3530-4b64-abc3-ea3c2e095200-tmp\") pod \"kube-auth-proxy-67dd84b75b-ppsmf\" (UID: \"613a1517-3530-4b64-abc3-ea3c2e095200\") " pod="openshift-ingress/kube-auth-proxy-67dd84b75b-ppsmf" Jun 01 21:48:53.561343 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:48:53.561354 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c8w68\" (UniqueName: \"kubernetes.io/projected/613a1517-3530-4b64-abc3-ea3c2e095200-kube-api-access-c8w68\") pod \"kube-auth-proxy-67dd84b75b-ppsmf\" (UID: \"613a1517-3530-4b64-abc3-ea3c2e095200\") " pod="openshift-ingress/kube-auth-proxy-67dd84b75b-ppsmf" Jun 01 21:48:53.561609 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:48:53.561460 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/613a1517-3530-4b64-abc3-ea3c2e095200-tls-certs\") pod \"kube-auth-proxy-67dd84b75b-ppsmf\" (UID: \"613a1517-3530-4b64-abc3-ea3c2e095200\") " pod="openshift-ingress/kube-auth-proxy-67dd84b75b-ppsmf" Jun 01 21:48:53.662623 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:48:53.662574 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/613a1517-3530-4b64-abc3-ea3c2e095200-tmp\") pod \"kube-auth-proxy-67dd84b75b-ppsmf\" (UID: \"613a1517-3530-4b64-abc3-ea3c2e095200\") " pod="openshift-ingress/kube-auth-proxy-67dd84b75b-ppsmf" Jun 01 21:48:53.662623 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:48:53.662617 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-c8w68\" (UniqueName: \"kubernetes.io/projected/613a1517-3530-4b64-abc3-ea3c2e095200-kube-api-access-c8w68\") pod \"kube-auth-proxy-67dd84b75b-ppsmf\" (UID: \"613a1517-3530-4b64-abc3-ea3c2e095200\") " pod="openshift-ingress/kube-auth-proxy-67dd84b75b-ppsmf" Jun 01 21:48:53.662841 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:48:53.662737 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/613a1517-3530-4b64-abc3-ea3c2e095200-tls-certs\") pod \"kube-auth-proxy-67dd84b75b-ppsmf\" (UID: \"613a1517-3530-4b64-abc3-ea3c2e095200\") " pod="openshift-ingress/kube-auth-proxy-67dd84b75b-ppsmf" Jun 01 21:48:53.664839 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:48:53.664807 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/613a1517-3530-4b64-abc3-ea3c2e095200-tmp\") pod \"kube-auth-proxy-67dd84b75b-ppsmf\" (UID: \"613a1517-3530-4b64-abc3-ea3c2e095200\") " pod="openshift-ingress/kube-auth-proxy-67dd84b75b-ppsmf" Jun 01 21:48:53.665044 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:48:53.665028 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/613a1517-3530-4b64-abc3-ea3c2e095200-tls-certs\") pod \"kube-auth-proxy-67dd84b75b-ppsmf\" (UID: \"613a1517-3530-4b64-abc3-ea3c2e095200\") " pod="openshift-ingress/kube-auth-proxy-67dd84b75b-ppsmf" Jun 01 21:48:53.671075 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:48:53.671055 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c8w68\" (UniqueName: \"kubernetes.io/projected/613a1517-3530-4b64-abc3-ea3c2e095200-kube-api-access-c8w68\") pod \"kube-auth-proxy-67dd84b75b-ppsmf\" (UID: \"613a1517-3530-4b64-abc3-ea3c2e095200\") " pod="openshift-ingress/kube-auth-proxy-67dd84b75b-ppsmf" Jun 01 21:48:53.785671 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:48:53.785578 2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-67dd84b75b-ppsmf" Jun 01 21:48:53.922405 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:48:53.922267 2573 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-67dd84b75b-ppsmf"] Jun 01 21:48:53.925157 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:48:53.925123 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod613a1517_3530_4b64_abc3_ea3c2e095200.slice/crio-c65ebffea178fd7828eb01088fce11b1eb95d9accecd8a8875c6d618683a0bb8 WatchSource:0}: Error finding container c65ebffea178fd7828eb01088fce11b1eb95d9accecd8a8875c6d618683a0bb8: Status 404 returned error can't find the container with id c65ebffea178fd7828eb01088fce11b1eb95d9accecd8a8875c6d618683a0bb8 Jun 01 21:48:54.035513 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:48:54.035473 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-67dd84b75b-ppsmf" event={"ID":"613a1517-3530-4b64-abc3-ea3c2e095200","Type":"ContainerStarted","Data":"c65ebffea178fd7828eb01088fce11b1eb95d9accecd8a8875c6d618683a0bb8"} Jun 01 21:48:58.048117 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:48:58.048083 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-67dd84b75b-ppsmf" event={"ID":"613a1517-3530-4b64-abc3-ea3c2e095200","Type":"ContainerStarted","Data":"934f12975872ba75c1e673e4ad81a8a726b6a7321941153eb138d22e08ef756b"} Jun 01 21:48:58.066836 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:48:58.066784 2573 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-67dd84b75b-ppsmf" podStartSLOduration=1.99035753 podStartE2EDuration="5.066767626s" podCreationTimestamp="2026-06-01 21:48:53 +0000 UTC" firstStartedPulling="2026-06-01 21:48:53.927016944 +0000 UTC m=+493.647068864" lastFinishedPulling="2026-06-01 21:48:57.003427039 +0000 UTC m=+496.723478960" observedRunningTime="2026-06-01 21:48:58.066053722 +0000 UTC m=+497.786105664" watchObservedRunningTime="2026-06-01 21:48:58.066767626 +0000 UTC m=+497.786819569" Jun 01 21:50:40.654478 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:50:40.654446 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rcqkp_fcf0209a-2440-4267-834a-b176f4e337e4/ovn-acl-logging/0.log" Jun 01 21:50:40.655004 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:50:40.654724 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rcqkp_fcf0209a-2440-4267-834a-b176f4e337e4/ovn-acl-logging/0.log" Jun 01 21:50:42.554129 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:50:42.554033 2573 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-6cb54b5c86-4kdq7"] Jun 01 21:50:42.557143 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:50:42.557125 2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-6cb54b5c86-4kdq7" Jun 01 21:50:42.560311 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:50:42.560289 2573 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\"" Jun 01 21:50:42.560444 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:50:42.560316 2573 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jun 01 21:50:42.560444 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:50:42.560371 2573 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-vr4b7\"" Jun 01 21:50:42.561867 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:50:42.561850 2573 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\"" Jun 01 21:50:42.561974 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:50:42.561887 2573 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jun 01 21:50:42.570960 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:50:42.570939 2573 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-6cb54b5c86-4kdq7"] Jun 01 21:50:42.572043 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:50:42.572027 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/afbcd9bd-2c42-406e-ae2a-68ba6ed4f478-plugin-serving-cert\") pod \"kuadrant-console-plugin-6cb54b5c86-4kdq7\" (UID: \"afbcd9bd-2c42-406e-ae2a-68ba6ed4f478\") " pod="kuadrant-system/kuadrant-console-plugin-6cb54b5c86-4kdq7" Jun 01 21:50:42.572102 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:50:42.572065 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jq95m\" (UniqueName: \"kubernetes.io/projected/afbcd9bd-2c42-406e-ae2a-68ba6ed4f478-kube-api-access-jq95m\") pod \"kuadrant-console-plugin-6cb54b5c86-4kdq7\" (UID: \"afbcd9bd-2c42-406e-ae2a-68ba6ed4f478\") " pod="kuadrant-system/kuadrant-console-plugin-6cb54b5c86-4kdq7" Jun 01 21:50:42.572102 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:50:42.572089 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/afbcd9bd-2c42-406e-ae2a-68ba6ed4f478-nginx-conf\") pod \"kuadrant-console-plugin-6cb54b5c86-4kdq7\" (UID: \"afbcd9bd-2c42-406e-ae2a-68ba6ed4f478\") " pod="kuadrant-system/kuadrant-console-plugin-6cb54b5c86-4kdq7" Jun 01 21:50:42.672555 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:50:42.672498 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-jq95m\" (UniqueName: \"kubernetes.io/projected/afbcd9bd-2c42-406e-ae2a-68ba6ed4f478-kube-api-access-jq95m\") pod \"kuadrant-console-plugin-6cb54b5c86-4kdq7\" (UID: \"afbcd9bd-2c42-406e-ae2a-68ba6ed4f478\") " pod="kuadrant-system/kuadrant-console-plugin-6cb54b5c86-4kdq7" Jun 01 21:50:42.672721 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:50:42.672563 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/afbcd9bd-2c42-406e-ae2a-68ba6ed4f478-nginx-conf\") pod \"kuadrant-console-plugin-6cb54b5c86-4kdq7\" (UID: \"afbcd9bd-2c42-406e-ae2a-68ba6ed4f478\") " pod="kuadrant-system/kuadrant-console-plugin-6cb54b5c86-4kdq7" Jun 01 21:50:42.672721 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:50:42.672622 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/afbcd9bd-2c42-406e-ae2a-68ba6ed4f478-plugin-serving-cert\") pod \"kuadrant-console-plugin-6cb54b5c86-4kdq7\" (UID: \"afbcd9bd-2c42-406e-ae2a-68ba6ed4f478\") " pod="kuadrant-system/kuadrant-console-plugin-6cb54b5c86-4kdq7" Jun 01 21:50:42.672793 ip-10-0-138-241 kubenswrapper[2573]: E0601 21:50:42.672746 2573 secret.go:189] Couldn't get secret kuadrant-system/plugin-serving-cert: secret "plugin-serving-cert" not found Jun 01 21:50:42.672840 ip-10-0-138-241 kubenswrapper[2573]: E0601 21:50:42.672824 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/afbcd9bd-2c42-406e-ae2a-68ba6ed4f478-plugin-serving-cert podName:afbcd9bd-2c42-406e-ae2a-68ba6ed4f478 nodeName:}" failed. No retries permitted until 2026-06-01 21:50:43.172793734 +0000 UTC m=+602.892845658 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "plugin-serving-cert" (UniqueName: "kubernetes.io/secret/afbcd9bd-2c42-406e-ae2a-68ba6ed4f478-plugin-serving-cert") pod "kuadrant-console-plugin-6cb54b5c86-4kdq7" (UID: "afbcd9bd-2c42-406e-ae2a-68ba6ed4f478") : secret "plugin-serving-cert" not found Jun 01 21:50:42.673301 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:50:42.673279 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/afbcd9bd-2c42-406e-ae2a-68ba6ed4f478-nginx-conf\") pod \"kuadrant-console-plugin-6cb54b5c86-4kdq7\" (UID: \"afbcd9bd-2c42-406e-ae2a-68ba6ed4f478\") " pod="kuadrant-system/kuadrant-console-plugin-6cb54b5c86-4kdq7" Jun 01 21:50:42.682507 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:50:42.682480 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jq95m\" (UniqueName: \"kubernetes.io/projected/afbcd9bd-2c42-406e-ae2a-68ba6ed4f478-kube-api-access-jq95m\") pod \"kuadrant-console-plugin-6cb54b5c86-4kdq7\" (UID: \"afbcd9bd-2c42-406e-ae2a-68ba6ed4f478\") " pod="kuadrant-system/kuadrant-console-plugin-6cb54b5c86-4kdq7" Jun 01 21:50:43.175772 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:50:43.175736 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/afbcd9bd-2c42-406e-ae2a-68ba6ed4f478-plugin-serving-cert\") pod \"kuadrant-console-plugin-6cb54b5c86-4kdq7\" (UID: \"afbcd9bd-2c42-406e-ae2a-68ba6ed4f478\") " pod="kuadrant-system/kuadrant-console-plugin-6cb54b5c86-4kdq7" Jun 01 21:50:43.178213 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:50:43.178187 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/afbcd9bd-2c42-406e-ae2a-68ba6ed4f478-plugin-serving-cert\") pod \"kuadrant-console-plugin-6cb54b5c86-4kdq7\" (UID: \"afbcd9bd-2c42-406e-ae2a-68ba6ed4f478\") " pod="kuadrant-system/kuadrant-console-plugin-6cb54b5c86-4kdq7" Jun 01 21:50:43.465729 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:50:43.465622 2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-6cb54b5c86-4kdq7" Jun 01 21:50:43.588228 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:50:43.588202 2573 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-6cb54b5c86-4kdq7"] Jun 01 21:50:43.590975 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:50:43.590948 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podafbcd9bd_2c42_406e_ae2a_68ba6ed4f478.slice/crio-c583d0ac49580731a4c773b93995d855885d8e26bee4f6001090a048134d2b57 WatchSource:0}: Error finding container c583d0ac49580731a4c773b93995d855885d8e26bee4f6001090a048134d2b57: Status 404 returned error can't find the container with id c583d0ac49580731a4c773b93995d855885d8e26bee4f6001090a048134d2b57 Jun 01 21:50:44.335902 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:50:44.335864 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-6cb54b5c86-4kdq7" event={"ID":"afbcd9bd-2c42-406e-ae2a-68ba6ed4f478","Type":"ContainerStarted","Data":"c583d0ac49580731a4c773b93995d855885d8e26bee4f6001090a048134d2b57"} Jun 01 21:50:52.778348 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:50:52.778313 2573 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-84b657d985-99jbh"] Jun 01 21:50:52.788232 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:50:52.788204 2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-84b657d985-99jbh" Jun 01 21:50:52.792029 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:50:52.792004 2573 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-j8zj4\"" Jun 01 21:50:52.798507 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:50:52.798463 2573 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-84b657d985-99jbh"] Jun 01 21:50:52.847094 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:50:52.847056 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ac3cdbf3-6b97-4914-895d-e49c147c2700-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-84b657d985-99jbh\" (UID: \"ac3cdbf3-6b97-4914-895d-e49c147c2700\") " pod="kuadrant-system/kuadrant-operator-controller-manager-84b657d985-99jbh" Jun 01 21:50:52.847277 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:50:52.847134 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sk8bl\" (UniqueName: \"kubernetes.io/projected/ac3cdbf3-6b97-4914-895d-e49c147c2700-kube-api-access-sk8bl\") pod \"kuadrant-operator-controller-manager-84b657d985-99jbh\" (UID: \"ac3cdbf3-6b97-4914-895d-e49c147c2700\") " pod="kuadrant-system/kuadrant-operator-controller-manager-84b657d985-99jbh" Jun 01 21:50:52.948473 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:50:52.948418 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ac3cdbf3-6b97-4914-895d-e49c147c2700-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-84b657d985-99jbh\" (UID: \"ac3cdbf3-6b97-4914-895d-e49c147c2700\") " pod="kuadrant-system/kuadrant-operator-controller-manager-84b657d985-99jbh" Jun 01 21:50:52.948663 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:50:52.948494 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-sk8bl\" (UniqueName: \"kubernetes.io/projected/ac3cdbf3-6b97-4914-895d-e49c147c2700-kube-api-access-sk8bl\") pod \"kuadrant-operator-controller-manager-84b657d985-99jbh\" (UID: \"ac3cdbf3-6b97-4914-895d-e49c147c2700\") " pod="kuadrant-system/kuadrant-operator-controller-manager-84b657d985-99jbh" Jun 01 21:50:52.948839 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:50:52.948811 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ac3cdbf3-6b97-4914-895d-e49c147c2700-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-84b657d985-99jbh\" (UID: \"ac3cdbf3-6b97-4914-895d-e49c147c2700\") " pod="kuadrant-system/kuadrant-operator-controller-manager-84b657d985-99jbh" Jun 01 21:50:52.958209 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:50:52.958180 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sk8bl\" (UniqueName: \"kubernetes.io/projected/ac3cdbf3-6b97-4914-895d-e49c147c2700-kube-api-access-sk8bl\") pod \"kuadrant-operator-controller-manager-84b657d985-99jbh\" (UID: \"ac3cdbf3-6b97-4914-895d-e49c147c2700\") " pod="kuadrant-system/kuadrant-operator-controller-manager-84b657d985-99jbh" Jun 01 21:50:53.102192 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:50:53.102142 2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-84b657d985-99jbh" Jun 01 21:50:53.236070 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:50:53.236031 2573 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-84b657d985-99jbh"] Jun 01 21:50:53.245294 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:50:53.245261 2573 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-84b657d985-99jbh"] Jun 01 21:50:53.261922 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:50:53.261886 2573 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-84b657d985-vgh9n"] Jun 01 21:50:53.276810 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:50:53.276778 2573 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-84b657d985-vgh9n"] Jun 01 21:50:53.276810 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:50:53.276781 2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-84b657d985-vgh9n" Jun 01 21:50:53.350945 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:50:53.350911 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/14b47f69-03be-4d43-8898-eb710a9c4629-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-84b657d985-vgh9n\" (UID: \"14b47f69-03be-4d43-8898-eb710a9c4629\") " pod="kuadrant-system/kuadrant-operator-controller-manager-84b657d985-vgh9n" Jun 01 21:50:53.351140 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:50:53.350959 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rppvs\" (UniqueName: \"kubernetes.io/projected/14b47f69-03be-4d43-8898-eb710a9c4629-kube-api-access-rppvs\") pod \"kuadrant-operator-controller-manager-84b657d985-vgh9n\" (UID: \"14b47f69-03be-4d43-8898-eb710a9c4629\") " pod="kuadrant-system/kuadrant-operator-controller-manager-84b657d985-vgh9n" Jun 01 21:50:53.451817 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:50:53.451713 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-rppvs\" (UniqueName: \"kubernetes.io/projected/14b47f69-03be-4d43-8898-eb710a9c4629-kube-api-access-rppvs\") pod \"kuadrant-operator-controller-manager-84b657d985-vgh9n\" (UID: \"14b47f69-03be-4d43-8898-eb710a9c4629\") " pod="kuadrant-system/kuadrant-operator-controller-manager-84b657d985-vgh9n" Jun 01 21:50:53.451991 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:50:53.451818 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/14b47f69-03be-4d43-8898-eb710a9c4629-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-84b657d985-vgh9n\" (UID: \"14b47f69-03be-4d43-8898-eb710a9c4629\") " pod="kuadrant-system/kuadrant-operator-controller-manager-84b657d985-vgh9n" Jun 01 21:50:53.452225 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:50:53.452206 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/14b47f69-03be-4d43-8898-eb710a9c4629-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-84b657d985-vgh9n\" (UID: \"14b47f69-03be-4d43-8898-eb710a9c4629\") " pod="kuadrant-system/kuadrant-operator-controller-manager-84b657d985-vgh9n" Jun 01 21:50:53.460902 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:50:53.460872 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rppvs\" (UniqueName: \"kubernetes.io/projected/14b47f69-03be-4d43-8898-eb710a9c4629-kube-api-access-rppvs\") pod \"kuadrant-operator-controller-manager-84b657d985-vgh9n\" (UID: \"14b47f69-03be-4d43-8898-eb710a9c4629\") " pod="kuadrant-system/kuadrant-operator-controller-manager-84b657d985-vgh9n" Jun 01 21:50:53.591050 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:50:53.591012 2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-84b657d985-vgh9n" Jun 01 21:51:05.649733 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:51:05.649708 2573 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-84b657d985-vgh9n"] Jun 01 21:51:05.652479 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:51:05.652441 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod14b47f69_03be_4d43_8898_eb710a9c4629.slice/crio-5aa71c27fa8437f70b2969767ef7920cc35addfd6544fdaff5b7c1b05355a4dd WatchSource:0}: Error finding container 5aa71c27fa8437f70b2969767ef7920cc35addfd6544fdaff5b7c1b05355a4dd: Status 404 returned error can't find the container with id 5aa71c27fa8437f70b2969767ef7920cc35addfd6544fdaff5b7c1b05355a4dd Jun 01 21:51:06.402748 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:51:06.402714 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-84b657d985-vgh9n" event={"ID":"14b47f69-03be-4d43-8898-eb710a9c4629","Type":"ContainerStarted","Data":"5aa71c27fa8437f70b2969767ef7920cc35addfd6544fdaff5b7c1b05355a4dd"} Jun 01 21:51:06.403960 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:51:06.403932 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-6cb54b5c86-4kdq7" event={"ID":"afbcd9bd-2c42-406e-ae2a-68ba6ed4f478","Type":"ContainerStarted","Data":"b4c4564b87da72310e07e2989c73613079f7d426e339bd2312c45fa8e6a05565"} Jun 01 21:51:06.420992 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:51:06.420939 2573 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-6cb54b5c86-4kdq7" podStartSLOduration=2.427200078 podStartE2EDuration="24.420923815s" podCreationTimestamp="2026-06-01 21:50:42 +0000 UTC" firstStartedPulling="2026-06-01 21:50:43.592194994 +0000 UTC m=+603.312246914" lastFinishedPulling="2026-06-01 21:51:05.585918718 +0000 UTC m=+625.305970651" observedRunningTime="2026-06-01 21:51:06.419335606 +0000 UTC m=+626.139387550" watchObservedRunningTime="2026-06-01 21:51:06.420923815 +0000 UTC m=+626.140975757" Jun 01 21:51:08.034237 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:51:08.034201 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podac3cdbf3_6b97_4914_895d_e49c147c2700.slice/crio-6e72b97e65f61a98bf4ec8370f51cceeab6218ec2809cd0e6e5ba510e5e49cf0 WatchSource:0}: Error finding container 6e72b97e65f61a98bf4ec8370f51cceeab6218ec2809cd0e6e5ba510e5e49cf0: Status 404 returned error can't find the container with id 6e72b97e65f61a98bf4ec8370f51cceeab6218ec2809cd0e6e5ba510e5e49cf0 Jun 01 21:51:11.424918 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:51:11.424872 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-84b657d985-vgh9n" event={"ID":"14b47f69-03be-4d43-8898-eb710a9c4629","Type":"ContainerStarted","Data":"8875a4658b04b0c36cf6569b90c7de51d9a048eda72927491edcd4335f6a719d"} Jun 01 21:51:11.425407 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:51:11.425059 2573 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-84b657d985-vgh9n" Jun 01 21:51:11.426527 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:51:11.426502 2573 generic.go:358] "Generic (PLEG): container finished" podID="ac3cdbf3-6b97-4914-895d-e49c147c2700" containerID="adf5dddb0491d8c49d359d44ec32ef4055c409ac3f8a93c1acd3874c17788cf5" exitCode=1 Jun 01 21:51:11.447279 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:51:11.447232 2573 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-84b657d985-vgh9n" podStartSLOduration=13.450062911 podStartE2EDuration="18.447217631s" podCreationTimestamp="2026-06-01 21:50:53 +0000 UTC" firstStartedPulling="2026-06-01 21:51:05.655078923 +0000 UTC m=+625.375130848" lastFinishedPulling="2026-06-01 21:51:10.652233633 +0000 UTC m=+630.372285568" observedRunningTime="2026-06-01 21:51:11.44491002 +0000 UTC m=+631.164961963" watchObservedRunningTime="2026-06-01 21:51:11.447217631 +0000 UTC m=+631.167269574" Jun 01 21:51:11.447409 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:51:11.447307 2573 status_manager.go:895] "Failed to get status for pod" podUID="ac3cdbf3-6b97-4914-895d-e49c147c2700" pod="kuadrant-system/kuadrant-operator-controller-manager-84b657d985-99jbh" err="pods \"kuadrant-operator-controller-manager-84b657d985-99jbh\" is forbidden: User \"system:node:ip-10-0-138-241.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-241.ec2.internal' and this object" Jun 01 21:51:11.454933 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:51:11.454916 2573 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-84b657d985-99jbh" Jun 01 21:51:11.457362 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:51:11.457339 2573 status_manager.go:895] "Failed to get status for pod" podUID="ac3cdbf3-6b97-4914-895d-e49c147c2700" pod="kuadrant-system/kuadrant-operator-controller-manager-84b657d985-99jbh" err="pods \"kuadrant-operator-controller-manager-84b657d985-99jbh\" is forbidden: User \"system:node:ip-10-0-138-241.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-241.ec2.internal' and this object" Jun 01 21:51:11.508453 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:51:11.508420 2573 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ac3cdbf3-6b97-4914-895d-e49c147c2700-extensions-socket-volume\") pod \"ac3cdbf3-6b97-4914-895d-e49c147c2700\" (UID: \"ac3cdbf3-6b97-4914-895d-e49c147c2700\") " Jun 01 21:51:11.508629 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:51:11.508508 2573 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sk8bl\" (UniqueName: \"kubernetes.io/projected/ac3cdbf3-6b97-4914-895d-e49c147c2700-kube-api-access-sk8bl\") pod \"ac3cdbf3-6b97-4914-895d-e49c147c2700\" (UID: \"ac3cdbf3-6b97-4914-895d-e49c147c2700\") " Jun 01 21:51:11.508741 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:51:11.508718 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ac3cdbf3-6b97-4914-895d-e49c147c2700-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "ac3cdbf3-6b97-4914-895d-e49c147c2700" (UID: "ac3cdbf3-6b97-4914-895d-e49c147c2700"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 01 21:51:11.510611 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:51:11.510591 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ac3cdbf3-6b97-4914-895d-e49c147c2700-kube-api-access-sk8bl" (OuterVolumeSpecName: "kube-api-access-sk8bl") pod "ac3cdbf3-6b97-4914-895d-e49c147c2700" (UID: "ac3cdbf3-6b97-4914-895d-e49c147c2700"). InnerVolumeSpecName "kube-api-access-sk8bl". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 01 21:51:11.609063 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:51:11.609023 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sk8bl\" (UniqueName: \"kubernetes.io/projected/ac3cdbf3-6b97-4914-895d-e49c147c2700-kube-api-access-sk8bl\") on node \"ip-10-0-138-241.ec2.internal\" DevicePath \"\"" Jun 01 21:51:11.609063 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:51:11.609057 2573 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ac3cdbf3-6b97-4914-895d-e49c147c2700-extensions-socket-volume\") on node \"ip-10-0-138-241.ec2.internal\" DevicePath \"\"" Jun 01 21:51:12.430363 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:51:12.430332 2573 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-84b657d985-99jbh" Jun 01 21:51:12.430363 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:51:12.430366 2573 scope.go:117] "RemoveContainer" containerID="adf5dddb0491d8c49d359d44ec32ef4055c409ac3f8a93c1acd3874c17788cf5" Jun 01 21:51:12.433260 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:51:12.433223 2573 status_manager.go:895] "Failed to get status for pod" podUID="ac3cdbf3-6b97-4914-895d-e49c147c2700" pod="kuadrant-system/kuadrant-operator-controller-manager-84b657d985-99jbh" err="pods \"kuadrant-operator-controller-manager-84b657d985-99jbh\" is forbidden: User \"system:node:ip-10-0-138-241.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-241.ec2.internal' and this object" Jun 01 21:51:12.441917 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:51:12.441891 2573 status_manager.go:895] "Failed to get status for pod" podUID="ac3cdbf3-6b97-4914-895d-e49c147c2700" pod="kuadrant-system/kuadrant-operator-controller-manager-84b657d985-99jbh" err="pods \"kuadrant-operator-controller-manager-84b657d985-99jbh\" is forbidden: User \"system:node:ip-10-0-138-241.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-241.ec2.internal' and this object" Jun 01 21:51:12.678299 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:51:12.678255 2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ac3cdbf3-6b97-4914-895d-e49c147c2700" path="/var/lib/kubelet/pods/ac3cdbf3-6b97-4914-895d-e49c147c2700/volumes" Jun 01 21:51:22.432957 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:51:22.432928 2573 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-84b657d985-vgh9n" Jun 01 21:51:34.748587 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:51:34.748524 2573 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-84b657d985-vgh9n"] Jun 01 21:51:34.749119 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:51:34.748786 2573 kuberuntime_container.go:864] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-84b657d985-vgh9n" podUID="14b47f69-03be-4d43-8898-eb710a9c4629" containerName="manager" containerID="cri-o://8875a4658b04b0c36cf6569b90c7de51d9a048eda72927491edcd4335f6a719d" gracePeriod=10 Jun 01 21:51:34.992587 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:51:34.992564 2573 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-84b657d985-vgh9n" Jun 01 21:51:35.109851 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:51:35.109809 2573 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rppvs\" (UniqueName: \"kubernetes.io/projected/14b47f69-03be-4d43-8898-eb710a9c4629-kube-api-access-rppvs\") pod \"14b47f69-03be-4d43-8898-eb710a9c4629\" (UID: \"14b47f69-03be-4d43-8898-eb710a9c4629\") " Jun 01 21:51:35.110020 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:51:35.109904 2573 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/14b47f69-03be-4d43-8898-eb710a9c4629-extensions-socket-volume\") pod \"14b47f69-03be-4d43-8898-eb710a9c4629\" (UID: \"14b47f69-03be-4d43-8898-eb710a9c4629\") " Jun 01 21:51:35.110395 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:51:35.110366 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/14b47f69-03be-4d43-8898-eb710a9c4629-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "14b47f69-03be-4d43-8898-eb710a9c4629" (UID: "14b47f69-03be-4d43-8898-eb710a9c4629"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 01 21:51:35.112005 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:51:35.111982 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/14b47f69-03be-4d43-8898-eb710a9c4629-kube-api-access-rppvs" (OuterVolumeSpecName: "kube-api-access-rppvs") pod "14b47f69-03be-4d43-8898-eb710a9c4629" (UID: "14b47f69-03be-4d43-8898-eb710a9c4629"). InnerVolumeSpecName "kube-api-access-rppvs". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 01 21:51:35.210506 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:51:35.210470 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rppvs\" (UniqueName: \"kubernetes.io/projected/14b47f69-03be-4d43-8898-eb710a9c4629-kube-api-access-rppvs\") on node \"ip-10-0-138-241.ec2.internal\" DevicePath \"\"" Jun 01 21:51:35.210506 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:51:35.210502 2573 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/14b47f69-03be-4d43-8898-eb710a9c4629-extensions-socket-volume\") on node \"ip-10-0-138-241.ec2.internal\" DevicePath \"\"" Jun 01 21:51:35.501186 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:51:35.501093 2573 generic.go:358] "Generic (PLEG): container finished" podID="14b47f69-03be-4d43-8898-eb710a9c4629" containerID="8875a4658b04b0c36cf6569b90c7de51d9a048eda72927491edcd4335f6a719d" exitCode=0 Jun 01 21:51:35.501186 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:51:35.501134 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-84b657d985-vgh9n" event={"ID":"14b47f69-03be-4d43-8898-eb710a9c4629","Type":"ContainerDied","Data":"8875a4658b04b0c36cf6569b90c7de51d9a048eda72927491edcd4335f6a719d"} Jun 01 21:51:35.501186 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:51:35.501173 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-84b657d985-vgh9n" event={"ID":"14b47f69-03be-4d43-8898-eb710a9c4629","Type":"ContainerDied","Data":"5aa71c27fa8437f70b2969767ef7920cc35addfd6544fdaff5b7c1b05355a4dd"} Jun 01 21:51:35.501186 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:51:35.501188 2573 scope.go:117] "RemoveContainer" containerID="8875a4658b04b0c36cf6569b90c7de51d9a048eda72927491edcd4335f6a719d" Jun 01 21:51:35.501459 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:51:35.501187 2573 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-84b657d985-vgh9n" Jun 01 21:51:35.509914 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:51:35.509884 2573 scope.go:117] "RemoveContainer" containerID="8875a4658b04b0c36cf6569b90c7de51d9a048eda72927491edcd4335f6a719d" Jun 01 21:51:35.510221 ip-10-0-138-241 kubenswrapper[2573]: E0601 21:51:35.510193 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8875a4658b04b0c36cf6569b90c7de51d9a048eda72927491edcd4335f6a719d\": container with ID starting with 8875a4658b04b0c36cf6569b90c7de51d9a048eda72927491edcd4335f6a719d not found: ID does not exist" containerID="8875a4658b04b0c36cf6569b90c7de51d9a048eda72927491edcd4335f6a719d" Jun 01 21:51:35.510337 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:51:35.510233 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8875a4658b04b0c36cf6569b90c7de51d9a048eda72927491edcd4335f6a719d"} err="failed to get container status \"8875a4658b04b0c36cf6569b90c7de51d9a048eda72927491edcd4335f6a719d\": rpc error: code = NotFound desc = could not find container \"8875a4658b04b0c36cf6569b90c7de51d9a048eda72927491edcd4335f6a719d\": container with ID starting with 8875a4658b04b0c36cf6569b90c7de51d9a048eda72927491edcd4335f6a719d not found: ID does not exist" Jun 01 21:51:35.525409 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:51:35.525374 2573 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-84b657d985-vgh9n"] Jun 01 21:51:35.529625 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:51:35.529600 2573 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-84b657d985-vgh9n"] Jun 01 21:51:36.677484 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:51:36.677453 2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="14b47f69-03be-4d43-8898-eb710a9c4629" path="/var/lib/kubelet/pods/14b47f69-03be-4d43-8898-eb710a9c4629/volumes" Jun 01 21:51:43.263029 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:51:43.262991 2573 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-ddtwb"] Jun 01 21:51:43.263439 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:51:43.263249 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="14b47f69-03be-4d43-8898-eb710a9c4629" containerName="manager" Jun 01 21:51:43.263439 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:51:43.263259 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="14b47f69-03be-4d43-8898-eb710a9c4629" containerName="manager" Jun 01 21:51:43.263439 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:51:43.263265 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ac3cdbf3-6b97-4914-895d-e49c147c2700" containerName="manager" Jun 01 21:51:43.263439 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:51:43.263271 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="ac3cdbf3-6b97-4914-895d-e49c147c2700" containerName="manager" Jun 01 21:51:43.263439 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:51:43.263322 2573 memory_manager.go:356] "RemoveStaleState removing state" podUID="14b47f69-03be-4d43-8898-eb710a9c4629" containerName="manager" Jun 01 21:51:43.263439 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:51:43.263331 2573 memory_manager.go:356] "RemoveStaleState removing state" podUID="ac3cdbf3-6b97-4914-895d-e49c147c2700" containerName="manager" Jun 01 21:51:43.300802 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:51:43.300769 2573 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-ddtwb"] Jun 01 21:51:43.300802 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:51:43.300801 2573 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-ddtwb"] Jun 01 21:51:43.301006 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:51:43.300906 2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-78c99df468-ddtwb" Jun 01 21:51:43.303811 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:51:43.303786 2573 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" Jun 01 21:51:43.476463 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:51:43.476422 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/716c4733-0b19-48f3-ba2c-4b6e6d09cdd6-config-file\") pod \"limitador-limitador-78c99df468-ddtwb\" (UID: \"716c4733-0b19-48f3-ba2c-4b6e6d09cdd6\") " pod="kuadrant-system/limitador-limitador-78c99df468-ddtwb" Jun 01 21:51:43.476670 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:51:43.476499 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ppvxb\" (UniqueName: \"kubernetes.io/projected/716c4733-0b19-48f3-ba2c-4b6e6d09cdd6-kube-api-access-ppvxb\") pod \"limitador-limitador-78c99df468-ddtwb\" (UID: \"716c4733-0b19-48f3-ba2c-4b6e6d09cdd6\") " pod="kuadrant-system/limitador-limitador-78c99df468-ddtwb" Jun 01 21:51:43.577741 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:51:43.577708 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-ppvxb\" (UniqueName: \"kubernetes.io/projected/716c4733-0b19-48f3-ba2c-4b6e6d09cdd6-kube-api-access-ppvxb\") pod \"limitador-limitador-78c99df468-ddtwb\" (UID: \"716c4733-0b19-48f3-ba2c-4b6e6d09cdd6\") " pod="kuadrant-system/limitador-limitador-78c99df468-ddtwb" Jun 01 21:51:43.577897 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:51:43.577770 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/716c4733-0b19-48f3-ba2c-4b6e6d09cdd6-config-file\") pod \"limitador-limitador-78c99df468-ddtwb\" (UID: \"716c4733-0b19-48f3-ba2c-4b6e6d09cdd6\") " pod="kuadrant-system/limitador-limitador-78c99df468-ddtwb" Jun 01 21:51:43.578305 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:51:43.578288 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/716c4733-0b19-48f3-ba2c-4b6e6d09cdd6-config-file\") pod \"limitador-limitador-78c99df468-ddtwb\" (UID: \"716c4733-0b19-48f3-ba2c-4b6e6d09cdd6\") " pod="kuadrant-system/limitador-limitador-78c99df468-ddtwb" Jun 01 21:51:43.586973 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:51:43.586941 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ppvxb\" (UniqueName: \"kubernetes.io/projected/716c4733-0b19-48f3-ba2c-4b6e6d09cdd6-kube-api-access-ppvxb\") pod \"limitador-limitador-78c99df468-ddtwb\" (UID: \"716c4733-0b19-48f3-ba2c-4b6e6d09cdd6\") " pod="kuadrant-system/limitador-limitador-78c99df468-ddtwb" Jun 01 21:51:43.610978 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:51:43.610946 2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-78c99df468-ddtwb" Jun 01 21:51:43.734027 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:51:43.734002 2573 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-ddtwb"] Jun 01 21:51:43.736447 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:51:43.736417 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod716c4733_0b19_48f3_ba2c_4b6e6d09cdd6.slice/crio-5f09024ca609c4591d4461c2161b46c74632560ff3c6d7ab60f99f9ef01a7590 WatchSource:0}: Error finding container 5f09024ca609c4591d4461c2161b46c74632560ff3c6d7ab60f99f9ef01a7590: Status 404 returned error can't find the container with id 5f09024ca609c4591d4461c2161b46c74632560ff3c6d7ab60f99f9ef01a7590 Jun 01 21:51:43.749789 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:51:43.749764 2573 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-f99f4b5cd-2v7rr"] Jun 01 21:51:43.754456 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:51:43.754434 2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-f99f4b5cd-2v7rr" Jun 01 21:51:43.757206 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:51:43.757188 2573 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-m9ds2\"" Jun 01 21:51:43.760918 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:51:43.760889 2573 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-f99f4b5cd-2v7rr"] Jun 01 21:51:43.778336 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:51:43.778305 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bl6vj\" (UniqueName: \"kubernetes.io/projected/69e298b1-7f6f-4890-9d2b-842396a3f3fd-kube-api-access-bl6vj\") pod \"authorino-f99f4b5cd-2v7rr\" (UID: \"69e298b1-7f6f-4890-9d2b-842396a3f3fd\") " pod="kuadrant-system/authorino-f99f4b5cd-2v7rr" Jun 01 21:51:43.879128 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:51:43.879036 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-bl6vj\" (UniqueName: \"kubernetes.io/projected/69e298b1-7f6f-4890-9d2b-842396a3f3fd-kube-api-access-bl6vj\") pod \"authorino-f99f4b5cd-2v7rr\" (UID: \"69e298b1-7f6f-4890-9d2b-842396a3f3fd\") " pod="kuadrant-system/authorino-f99f4b5cd-2v7rr" Jun 01 21:51:43.888276 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:51:43.888248 2573 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-7498df8756-6kw9w"] Jun 01 21:51:43.891139 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:51:43.891118 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bl6vj\" (UniqueName: \"kubernetes.io/projected/69e298b1-7f6f-4890-9d2b-842396a3f3fd-kube-api-access-bl6vj\") pod \"authorino-f99f4b5cd-2v7rr\" (UID: \"69e298b1-7f6f-4890-9d2b-842396a3f3fd\") " pod="kuadrant-system/authorino-f99f4b5cd-2v7rr" Jun 01 21:51:43.891596 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:51:43.891581 2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7498df8756-6kw9w" Jun 01 21:51:43.896910 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:51:43.896889 2573 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7498df8756-6kw9w"] Jun 01 21:51:43.979894 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:51:43.979861 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9hsd9\" (UniqueName: \"kubernetes.io/projected/93654fdd-203d-4b0f-842d-90323507305d-kube-api-access-9hsd9\") pod \"authorino-7498df8756-6kw9w\" (UID: \"93654fdd-203d-4b0f-842d-90323507305d\") " pod="kuadrant-system/authorino-7498df8756-6kw9w" Jun 01 21:51:44.064998 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:51:44.064959 2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-f99f4b5cd-2v7rr" Jun 01 21:51:44.081011 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:51:44.080980 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-9hsd9\" (UniqueName: \"kubernetes.io/projected/93654fdd-203d-4b0f-842d-90323507305d-kube-api-access-9hsd9\") pod \"authorino-7498df8756-6kw9w\" (UID: \"93654fdd-203d-4b0f-842d-90323507305d\") " pod="kuadrant-system/authorino-7498df8756-6kw9w" Jun 01 21:51:44.090034 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:51:44.090007 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9hsd9\" (UniqueName: \"kubernetes.io/projected/93654fdd-203d-4b0f-842d-90323507305d-kube-api-access-9hsd9\") pod \"authorino-7498df8756-6kw9w\" (UID: \"93654fdd-203d-4b0f-842d-90323507305d\") " pod="kuadrant-system/authorino-7498df8756-6kw9w" Jun 01 21:51:44.205951 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:51:44.205913 2573 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-f99f4b5cd-2v7rr"] Jun 01 21:51:44.206121 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:51:44.206087 2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7498df8756-6kw9w" Jun 01 21:51:44.210072 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:51:44.210014 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod69e298b1_7f6f_4890_9d2b_842396a3f3fd.slice/crio-16d35d50d33b2a9a5e203ba9219bed7e5b52859733299748a6b1836b3606b175 WatchSource:0}: Error finding container 16d35d50d33b2a9a5e203ba9219bed7e5b52859733299748a6b1836b3606b175: Status 404 returned error can't find the container with id 16d35d50d33b2a9a5e203ba9219bed7e5b52859733299748a6b1836b3606b175 Jun 01 21:51:44.394429 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:51:44.394380 2573 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7498df8756-6kw9w"] Jun 01 21:51:44.399470 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:51:44.399395 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod93654fdd_203d_4b0f_842d_90323507305d.slice/crio-cbeae8cf72debb32216d8869a3bcba765cb67d7b74cb398318910dc0b0f00d2a WatchSource:0}: Error finding container cbeae8cf72debb32216d8869a3bcba765cb67d7b74cb398318910dc0b0f00d2a: Status 404 returned error can't find the container with id cbeae8cf72debb32216d8869a3bcba765cb67d7b74cb398318910dc0b0f00d2a Jun 01 21:51:44.531203 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:51:44.531149 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7498df8756-6kw9w" event={"ID":"93654fdd-203d-4b0f-842d-90323507305d","Type":"ContainerStarted","Data":"cbeae8cf72debb32216d8869a3bcba765cb67d7b74cb398318910dc0b0f00d2a"} Jun 01 21:51:44.532475 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:51:44.532430 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-f99f4b5cd-2v7rr" event={"ID":"69e298b1-7f6f-4890-9d2b-842396a3f3fd","Type":"ContainerStarted","Data":"16d35d50d33b2a9a5e203ba9219bed7e5b52859733299748a6b1836b3606b175"} Jun 01 21:51:44.534641 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:51:44.534607 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-78c99df468-ddtwb" event={"ID":"716c4733-0b19-48f3-ba2c-4b6e6d09cdd6","Type":"ContainerStarted","Data":"5f09024ca609c4591d4461c2161b46c74632560ff3c6d7ab60f99f9ef01a7590"} Jun 01 21:51:48.553129 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:51:48.553028 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7498df8756-6kw9w" event={"ID":"93654fdd-203d-4b0f-842d-90323507305d","Type":"ContainerStarted","Data":"662a4edc0fe533c2ccc141c24bb1f582199268a2efb305ffac0eaacf83132d36"} Jun 01 21:51:48.554389 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:51:48.554360 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-f99f4b5cd-2v7rr" event={"ID":"69e298b1-7f6f-4890-9d2b-842396a3f3fd","Type":"ContainerStarted","Data":"4dd580d9db3a4ee47f794cef8eae25d91eb3ba3c635c5a7af1ef76acf5717256"} Jun 01 21:51:48.555689 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:51:48.555664 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-78c99df468-ddtwb" event={"ID":"716c4733-0b19-48f3-ba2c-4b6e6d09cdd6","Type":"ContainerStarted","Data":"4fd7910540fd4e288b8aa4268b929a58a6900d9cc271199c69b06aec334a8070"} Jun 01 21:51:48.555781 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:51:48.555766 2573 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-78c99df468-ddtwb" Jun 01 21:51:48.568441 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:51:48.568391 2573 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/authorino-7498df8756-6kw9w" podStartSLOduration=1.625201134 podStartE2EDuration="5.568377762s" podCreationTimestamp="2026-06-01 21:51:43 +0000 UTC" firstStartedPulling="2026-06-01 21:51:44.401072048 +0000 UTC m=+664.121123991" lastFinishedPulling="2026-06-01 21:51:48.344248698 +0000 UTC m=+668.064300619" observedRunningTime="2026-06-01 21:51:48.568167225 +0000 UTC m=+668.288219169" watchObservedRunningTime="2026-06-01 21:51:48.568377762 +0000 UTC m=+668.288429706" Jun 01 21:51:48.586274 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:51:48.586224 2573 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-78c99df468-ddtwb" podStartSLOduration=1.039879301 podStartE2EDuration="5.586208877s" podCreationTimestamp="2026-06-01 21:51:43 +0000 UTC" firstStartedPulling="2026-06-01 21:51:43.738360891 +0000 UTC m=+663.458412812" lastFinishedPulling="2026-06-01 21:51:48.284690463 +0000 UTC m=+668.004742388" observedRunningTime="2026-06-01 21:51:48.584345267 +0000 UTC m=+668.304397245" watchObservedRunningTime="2026-06-01 21:51:48.586208877 +0000 UTC m=+668.306260827" Jun 01 21:51:48.599572 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:51:48.599510 2573 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-f99f4b5cd-2v7rr"] Jun 01 21:51:48.611342 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:51:48.611288 2573 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/authorino-f99f4b5cd-2v7rr" podStartSLOduration=1.486700178 podStartE2EDuration="5.611273652s" podCreationTimestamp="2026-06-01 21:51:43 +0000 UTC" firstStartedPulling="2026-06-01 21:51:44.211848398 +0000 UTC m=+663.931900333" lastFinishedPulling="2026-06-01 21:51:48.336421872 +0000 UTC m=+668.056473807" observedRunningTime="2026-06-01 21:51:48.610960885 +0000 UTC m=+668.331012830" watchObservedRunningTime="2026-06-01 21:51:48.611273652 +0000 UTC m=+668.331325595" Jun 01 21:51:50.562608 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:51:50.562516 2573 kuberuntime_container.go:864] "Killing container with a grace period" pod="kuadrant-system/authorino-f99f4b5cd-2v7rr" podUID="69e298b1-7f6f-4890-9d2b-842396a3f3fd" containerName="authorino" containerID="cri-o://4dd580d9db3a4ee47f794cef8eae25d91eb3ba3c635c5a7af1ef76acf5717256" gracePeriod=30 Jun 01 21:51:50.803402 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:51:50.803372 2573 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-f99f4b5cd-2v7rr" Jun 01 21:51:50.833808 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:51:50.833714 2573 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bl6vj\" (UniqueName: \"kubernetes.io/projected/69e298b1-7f6f-4890-9d2b-842396a3f3fd-kube-api-access-bl6vj\") pod \"69e298b1-7f6f-4890-9d2b-842396a3f3fd\" (UID: \"69e298b1-7f6f-4890-9d2b-842396a3f3fd\") " Jun 01 21:51:50.835702 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:51:50.835675 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/69e298b1-7f6f-4890-9d2b-842396a3f3fd-kube-api-access-bl6vj" (OuterVolumeSpecName: "kube-api-access-bl6vj") pod "69e298b1-7f6f-4890-9d2b-842396a3f3fd" (UID: "69e298b1-7f6f-4890-9d2b-842396a3f3fd"). InnerVolumeSpecName "kube-api-access-bl6vj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 01 21:51:50.935274 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:51:50.935233 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bl6vj\" (UniqueName: \"kubernetes.io/projected/69e298b1-7f6f-4890-9d2b-842396a3f3fd-kube-api-access-bl6vj\") on node \"ip-10-0-138-241.ec2.internal\" DevicePath \"\"" Jun 01 21:51:51.567469 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:51:51.567435 2573 generic.go:358] "Generic (PLEG): container finished" podID="69e298b1-7f6f-4890-9d2b-842396a3f3fd" containerID="4dd580d9db3a4ee47f794cef8eae25d91eb3ba3c635c5a7af1ef76acf5717256" exitCode=0 Jun 01 21:51:51.567891 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:51:51.567480 2573 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-f99f4b5cd-2v7rr" Jun 01 21:51:51.567891 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:51:51.567521 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-f99f4b5cd-2v7rr" event={"ID":"69e298b1-7f6f-4890-9d2b-842396a3f3fd","Type":"ContainerDied","Data":"4dd580d9db3a4ee47f794cef8eae25d91eb3ba3c635c5a7af1ef76acf5717256"} Jun 01 21:51:51.567891 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:51:51.567576 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-f99f4b5cd-2v7rr" event={"ID":"69e298b1-7f6f-4890-9d2b-842396a3f3fd","Type":"ContainerDied","Data":"16d35d50d33b2a9a5e203ba9219bed7e5b52859733299748a6b1836b3606b175"} Jun 01 21:51:51.567891 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:51:51.567594 2573 scope.go:117] "RemoveContainer" containerID="4dd580d9db3a4ee47f794cef8eae25d91eb3ba3c635c5a7af1ef76acf5717256" Jun 01 21:51:51.575500 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:51:51.575484 2573 scope.go:117] "RemoveContainer" containerID="4dd580d9db3a4ee47f794cef8eae25d91eb3ba3c635c5a7af1ef76acf5717256" Jun 01 21:51:51.575783 ip-10-0-138-241 kubenswrapper[2573]: E0601 21:51:51.575762 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4dd580d9db3a4ee47f794cef8eae25d91eb3ba3c635c5a7af1ef76acf5717256\": container with ID starting with 4dd580d9db3a4ee47f794cef8eae25d91eb3ba3c635c5a7af1ef76acf5717256 not found: ID does not exist" containerID="4dd580d9db3a4ee47f794cef8eae25d91eb3ba3c635c5a7af1ef76acf5717256" Jun 01 21:51:51.575878 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:51:51.575790 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4dd580d9db3a4ee47f794cef8eae25d91eb3ba3c635c5a7af1ef76acf5717256"} err="failed to get container status \"4dd580d9db3a4ee47f794cef8eae25d91eb3ba3c635c5a7af1ef76acf5717256\": rpc error: code = NotFound desc = could not find container \"4dd580d9db3a4ee47f794cef8eae25d91eb3ba3c635c5a7af1ef76acf5717256\": container with ID starting with 4dd580d9db3a4ee47f794cef8eae25d91eb3ba3c635c5a7af1ef76acf5717256 not found: ID does not exist" Jun 01 21:51:51.588017 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:51:51.587993 2573 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-f99f4b5cd-2v7rr"] Jun 01 21:51:51.591260 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:51:51.591235 2573 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-f99f4b5cd-2v7rr"] Jun 01 21:51:52.677347 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:51:52.677314 2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="69e298b1-7f6f-4890-9d2b-842396a3f3fd" path="/var/lib/kubelet/pods/69e298b1-7f6f-4890-9d2b-842396a3f3fd/volumes" Jun 01 21:51:59.560203 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:51:59.560171 2573 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-78c99df468-ddtwb" Jun 01 21:53:13.724586 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:53:13.724528 2573 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-8b475cf9f-kz99d"] Jun 01 21:53:13.725072 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:53:13.724820 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="69e298b1-7f6f-4890-9d2b-842396a3f3fd" containerName="authorino" Jun 01 21:53:13.725072 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:53:13.724831 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="69e298b1-7f6f-4890-9d2b-842396a3f3fd" containerName="authorino" Jun 01 21:53:13.725072 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:53:13.724873 2573 memory_manager.go:356] "RemoveStaleState removing state" podUID="69e298b1-7f6f-4890-9d2b-842396a3f3fd" containerName="authorino" Jun 01 21:53:13.727721 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:53:13.727704 2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-8b475cf9f-kz99d" Jun 01 21:53:13.732796 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:53:13.732769 2573 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-8b475cf9f-kz99d"] Jun 01 21:53:13.746059 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:53:13.746030 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wl2d5\" (UniqueName: \"kubernetes.io/projected/4bd7e704-b342-4013-a8eb-73c2a12de584-kube-api-access-wl2d5\") pod \"authorino-8b475cf9f-kz99d\" (UID: \"4bd7e704-b342-4013-a8eb-73c2a12de584\") " pod="kuadrant-system/authorino-8b475cf9f-kz99d" Jun 01 21:53:13.846974 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:53:13.846932 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-wl2d5\" (UniqueName: \"kubernetes.io/projected/4bd7e704-b342-4013-a8eb-73c2a12de584-kube-api-access-wl2d5\") pod \"authorino-8b475cf9f-kz99d\" (UID: \"4bd7e704-b342-4013-a8eb-73c2a12de584\") " pod="kuadrant-system/authorino-8b475cf9f-kz99d" Jun 01 21:53:13.855243 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:53:13.855213 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wl2d5\" (UniqueName: \"kubernetes.io/projected/4bd7e704-b342-4013-a8eb-73c2a12de584-kube-api-access-wl2d5\") pod \"authorino-8b475cf9f-kz99d\" (UID: \"4bd7e704-b342-4013-a8eb-73c2a12de584\") " pod="kuadrant-system/authorino-8b475cf9f-kz99d" Jun 01 21:53:13.967122 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:53:13.967078 2573 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-8b475cf9f-kz99d"] Jun 01 21:53:13.967297 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:53:13.967286 2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-8b475cf9f-kz99d" Jun 01 21:53:13.992172 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:53:13.992134 2573 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-8688c48586-7kr59"] Jun 01 21:53:13.996453 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:53:13.996434 2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-8688c48586-7kr59" Jun 01 21:53:14.003245 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:53:14.003197 2573 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-8688c48586-7kr59"] Jun 01 21:53:14.048177 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:53:14.048150 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gf6tg\" (UniqueName: \"kubernetes.io/projected/14c13811-9e9c-4a6a-864f-bc03bc21e419-kube-api-access-gf6tg\") pod \"authorino-8688c48586-7kr59\" (UID: \"14c13811-9e9c-4a6a-864f-bc03bc21e419\") " pod="kuadrant-system/authorino-8688c48586-7kr59" Jun 01 21:53:14.098068 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:53:14.098036 2573 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-8b475cf9f-kz99d"] Jun 01 21:53:14.101759 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:53:14.101729 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4bd7e704_b342_4013_a8eb_73c2a12de584.slice/crio-dcf912f9042e0e606ed400cb880e2ea86603d5455615556b42cb473ddee9afff WatchSource:0}: Error finding container dcf912f9042e0e606ed400cb880e2ea86603d5455615556b42cb473ddee9afff: Status 404 returned error can't find the container with id dcf912f9042e0e606ed400cb880e2ea86603d5455615556b42cb473ddee9afff Jun 01 21:53:14.149523 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:53:14.149482 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-gf6tg\" (UniqueName: \"kubernetes.io/projected/14c13811-9e9c-4a6a-864f-bc03bc21e419-kube-api-access-gf6tg\") pod \"authorino-8688c48586-7kr59\" (UID: \"14c13811-9e9c-4a6a-864f-bc03bc21e419\") " pod="kuadrant-system/authorino-8688c48586-7kr59" Jun 01 21:53:14.158041 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:53:14.158013 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gf6tg\" (UniqueName: \"kubernetes.io/projected/14c13811-9e9c-4a6a-864f-bc03bc21e419-kube-api-access-gf6tg\") pod \"authorino-8688c48586-7kr59\" (UID: \"14c13811-9e9c-4a6a-864f-bc03bc21e419\") " pod="kuadrant-system/authorino-8688c48586-7kr59" Jun 01 21:53:14.162581 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:53:14.162556 2573 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-8688c48586-7kr59"] Jun 01 21:53:14.162769 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:53:14.162758 2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-8688c48586-7kr59" Jun 01 21:53:14.188578 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:53:14.188528 2573 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-7c846b5758-fk27t"] Jun 01 21:53:14.192934 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:53:14.192897 2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7c846b5758-fk27t" Jun 01 21:53:14.195871 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:53:14.195848 2573 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\"" Jun 01 21:53:14.200927 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:53:14.200749 2573 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7c846b5758-fk27t"] Jun 01 21:53:14.250126 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:53:14.250097 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vrq99\" (UniqueName: \"kubernetes.io/projected/790b3718-a27a-4ecf-b7cd-46a81acbcd18-kube-api-access-vrq99\") pod \"authorino-7c846b5758-fk27t\" (UID: \"790b3718-a27a-4ecf-b7cd-46a81acbcd18\") " pod="kuadrant-system/authorino-7c846b5758-fk27t" Jun 01 21:53:14.250240 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:53:14.250141 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/790b3718-a27a-4ecf-b7cd-46a81acbcd18-tls-cert\") pod \"authorino-7c846b5758-fk27t\" (UID: \"790b3718-a27a-4ecf-b7cd-46a81acbcd18\") " pod="kuadrant-system/authorino-7c846b5758-fk27t" Jun 01 21:53:14.290346 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:53:14.290313 2573 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-8688c48586-7kr59"] Jun 01 21:53:14.293354 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:53:14.293317 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod14c13811_9e9c_4a6a_864f_bc03bc21e419.slice/crio-3daa8102e7b0b084cede544e1835d04448102648105cf77b9fa6c65c3de6c6b9 WatchSource:0}: Error finding container 3daa8102e7b0b084cede544e1835d04448102648105cf77b9fa6c65c3de6c6b9: Status 404 returned error can't find the container with id 3daa8102e7b0b084cede544e1835d04448102648105cf77b9fa6c65c3de6c6b9 Jun 01 21:53:14.350871 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:53:14.350839 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-vrq99\" (UniqueName: \"kubernetes.io/projected/790b3718-a27a-4ecf-b7cd-46a81acbcd18-kube-api-access-vrq99\") pod \"authorino-7c846b5758-fk27t\" (UID: \"790b3718-a27a-4ecf-b7cd-46a81acbcd18\") " pod="kuadrant-system/authorino-7c846b5758-fk27t" Jun 01 21:53:14.351033 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:53:14.350878 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/790b3718-a27a-4ecf-b7cd-46a81acbcd18-tls-cert\") pod \"authorino-7c846b5758-fk27t\" (UID: \"790b3718-a27a-4ecf-b7cd-46a81acbcd18\") " pod="kuadrant-system/authorino-7c846b5758-fk27t" Jun 01 21:53:14.353243 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:53:14.353218 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/790b3718-a27a-4ecf-b7cd-46a81acbcd18-tls-cert\") pod \"authorino-7c846b5758-fk27t\" (UID: \"790b3718-a27a-4ecf-b7cd-46a81acbcd18\") " pod="kuadrant-system/authorino-7c846b5758-fk27t" Jun 01 21:53:14.359085 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:53:14.359065 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vrq99\" (UniqueName: \"kubernetes.io/projected/790b3718-a27a-4ecf-b7cd-46a81acbcd18-kube-api-access-vrq99\") pod \"authorino-7c846b5758-fk27t\" (UID: \"790b3718-a27a-4ecf-b7cd-46a81acbcd18\") " pod="kuadrant-system/authorino-7c846b5758-fk27t" Jun 01 21:53:14.503774 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:53:14.503750 2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7c846b5758-fk27t" Jun 01 21:53:14.638282 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:53:14.638257 2573 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7c846b5758-fk27t"] Jun 01 21:53:14.644648 ip-10-0-138-241 kubenswrapper[2573]: W0601 21:53:14.644622 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod790b3718_a27a_4ecf_b7cd_46a81acbcd18.slice/crio-afe0d839a55df86ab2b831bab61e9e13b7fb4f51dfb8b58c1e5eabb373528e01 WatchSource:0}: Error finding container afe0d839a55df86ab2b831bab61e9e13b7fb4f51dfb8b58c1e5eabb373528e01: Status 404 returned error can't find the container with id afe0d839a55df86ab2b831bab61e9e13b7fb4f51dfb8b58c1e5eabb373528e01 Jun 01 21:53:14.813547 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:53:14.813509 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-8688c48586-7kr59" event={"ID":"14c13811-9e9c-4a6a-864f-bc03bc21e419","Type":"ContainerStarted","Data":"5d1fe0898a7112209963ce388482ecdf6f07a6bcd7aa895a8a3784f0beaf0e44"} Jun 01 21:53:14.813995 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:53:14.813576 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-8688c48586-7kr59" event={"ID":"14c13811-9e9c-4a6a-864f-bc03bc21e419","Type":"ContainerStarted","Data":"3daa8102e7b0b084cede544e1835d04448102648105cf77b9fa6c65c3de6c6b9"} Jun 01 21:53:14.813995 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:53:14.813585 2573 kuberuntime_container.go:864] "Killing container with a grace period" pod="kuadrant-system/authorino-8688c48586-7kr59" podUID="14c13811-9e9c-4a6a-864f-bc03bc21e419" containerName="authorino" containerID="cri-o://5d1fe0898a7112209963ce388482ecdf6f07a6bcd7aa895a8a3784f0beaf0e44" gracePeriod=30 Jun 01 21:53:14.814679 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:53:14.814654 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7c846b5758-fk27t" event={"ID":"790b3718-a27a-4ecf-b7cd-46a81acbcd18","Type":"ContainerStarted","Data":"afe0d839a55df86ab2b831bab61e9e13b7fb4f51dfb8b58c1e5eabb373528e01"} Jun 01 21:53:14.815970 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:53:14.815948 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-8b475cf9f-kz99d" event={"ID":"4bd7e704-b342-4013-a8eb-73c2a12de584","Type":"ContainerStarted","Data":"5655ed74aacf1cbd2588f2662d86ce2772280775e3ee820546598c117f695493"} Jun 01 21:53:14.816058 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:53:14.815976 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-8b475cf9f-kz99d" event={"ID":"4bd7e704-b342-4013-a8eb-73c2a12de584","Type":"ContainerStarted","Data":"dcf912f9042e0e606ed400cb880e2ea86603d5455615556b42cb473ddee9afff"} Jun 01 21:53:14.816058 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:53:14.816027 2573 kuberuntime_container.go:864] "Killing container with a grace period" pod="kuadrant-system/authorino-8b475cf9f-kz99d" podUID="4bd7e704-b342-4013-a8eb-73c2a12de584" containerName="authorino" containerID="cri-o://5655ed74aacf1cbd2588f2662d86ce2772280775e3ee820546598c117f695493" gracePeriod=30 Jun 01 21:53:14.829430 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:53:14.829388 2573 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/authorino-8688c48586-7kr59" podStartSLOduration=1.511081562 podStartE2EDuration="1.829375583s" podCreationTimestamp="2026-06-01 21:53:13 +0000 UTC" firstStartedPulling="2026-06-01 21:53:14.294658431 +0000 UTC m=+754.014710356" lastFinishedPulling="2026-06-01 21:53:14.612952451 +0000 UTC m=+754.333004377" observedRunningTime="2026-06-01 21:53:14.828618188 +0000 UTC m=+754.548670131" watchObservedRunningTime="2026-06-01 21:53:14.829375583 +0000 UTC m=+754.549427526" Jun 01 21:53:14.843579 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:53:14.843512 2573 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/authorino-8b475cf9f-kz99d" podStartSLOduration=1.47920489 podStartE2EDuration="1.843497692s" podCreationTimestamp="2026-06-01 21:53:13 +0000 UTC" firstStartedPulling="2026-06-01 21:53:14.103068451 +0000 UTC m=+753.823120371" lastFinishedPulling="2026-06-01 21:53:14.467361034 +0000 UTC m=+754.187413173" observedRunningTime="2026-06-01 21:53:14.84329483 +0000 UTC m=+754.563346773" watchObservedRunningTime="2026-06-01 21:53:14.843497692 +0000 UTC m=+754.563549635" Jun 01 21:53:15.072115 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:53:15.072094 2573 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-8b475cf9f-kz99d" Jun 01 21:53:15.080838 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:53:15.080809 2573 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-8688c48586-7kr59" Jun 01 21:53:15.159334 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:53:15.159299 2573 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gf6tg\" (UniqueName: \"kubernetes.io/projected/14c13811-9e9c-4a6a-864f-bc03bc21e419-kube-api-access-gf6tg\") pod \"14c13811-9e9c-4a6a-864f-bc03bc21e419\" (UID: \"14c13811-9e9c-4a6a-864f-bc03bc21e419\") " Jun 01 21:53:15.159557 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:53:15.159379 2573 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wl2d5\" (UniqueName: \"kubernetes.io/projected/4bd7e704-b342-4013-a8eb-73c2a12de584-kube-api-access-wl2d5\") pod \"4bd7e704-b342-4013-a8eb-73c2a12de584\" (UID: \"4bd7e704-b342-4013-a8eb-73c2a12de584\") " Jun 01 21:53:15.161416 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:53:15.161386 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/14c13811-9e9c-4a6a-864f-bc03bc21e419-kube-api-access-gf6tg" (OuterVolumeSpecName: "kube-api-access-gf6tg") pod "14c13811-9e9c-4a6a-864f-bc03bc21e419" (UID: "14c13811-9e9c-4a6a-864f-bc03bc21e419"). InnerVolumeSpecName "kube-api-access-gf6tg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 01 21:53:15.161568 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:53:15.161434 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4bd7e704-b342-4013-a8eb-73c2a12de584-kube-api-access-wl2d5" (OuterVolumeSpecName: "kube-api-access-wl2d5") pod "4bd7e704-b342-4013-a8eb-73c2a12de584" (UID: "4bd7e704-b342-4013-a8eb-73c2a12de584"). InnerVolumeSpecName "kube-api-access-wl2d5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 01 21:53:15.260503 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:53:15.260462 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gf6tg\" (UniqueName: \"kubernetes.io/projected/14c13811-9e9c-4a6a-864f-bc03bc21e419-kube-api-access-gf6tg\") on node \"ip-10-0-138-241.ec2.internal\" DevicePath \"\"" Jun 01 21:53:15.260503 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:53:15.260498 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wl2d5\" (UniqueName: \"kubernetes.io/projected/4bd7e704-b342-4013-a8eb-73c2a12de584-kube-api-access-wl2d5\") on node \"ip-10-0-138-241.ec2.internal\" DevicePath \"\"" Jun 01 21:53:15.820574 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:53:15.820524 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7c846b5758-fk27t" event={"ID":"790b3718-a27a-4ecf-b7cd-46a81acbcd18","Type":"ContainerStarted","Data":"69b6e849ae6662f4516c92b00631824532827e0eb59cad48f1ac63b5861c696c"} Jun 01 21:53:15.821624 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:53:15.821594 2573 generic.go:358] "Generic (PLEG): container finished" podID="4bd7e704-b342-4013-a8eb-73c2a12de584" containerID="5655ed74aacf1cbd2588f2662d86ce2772280775e3ee820546598c117f695493" exitCode=0 Jun 01 21:53:15.821770 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:53:15.821635 2573 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-8b475cf9f-kz99d" Jun 01 21:53:15.821770 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:53:15.821674 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-8b475cf9f-kz99d" event={"ID":"4bd7e704-b342-4013-a8eb-73c2a12de584","Type":"ContainerDied","Data":"5655ed74aacf1cbd2588f2662d86ce2772280775e3ee820546598c117f695493"} Jun 01 21:53:15.821770 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:53:15.821711 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-8b475cf9f-kz99d" event={"ID":"4bd7e704-b342-4013-a8eb-73c2a12de584","Type":"ContainerDied","Data":"dcf912f9042e0e606ed400cb880e2ea86603d5455615556b42cb473ddee9afff"} Jun 01 21:53:15.821770 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:53:15.821734 2573 scope.go:117] "RemoveContainer" containerID="5655ed74aacf1cbd2588f2662d86ce2772280775e3ee820546598c117f695493" Jun 01 21:53:15.822824 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:53:15.822796 2573 generic.go:358] "Generic (PLEG): container finished" podID="14c13811-9e9c-4a6a-864f-bc03bc21e419" containerID="5d1fe0898a7112209963ce388482ecdf6f07a6bcd7aa895a8a3784f0beaf0e44" exitCode=0 Jun 01 21:53:15.822929 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:53:15.822833 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-8688c48586-7kr59" event={"ID":"14c13811-9e9c-4a6a-864f-bc03bc21e419","Type":"ContainerDied","Data":"5d1fe0898a7112209963ce388482ecdf6f07a6bcd7aa895a8a3784f0beaf0e44"} Jun 01 21:53:15.822929 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:53:15.822841 2573 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-8688c48586-7kr59" Jun 01 21:53:15.822929 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:53:15.822855 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-8688c48586-7kr59" event={"ID":"14c13811-9e9c-4a6a-864f-bc03bc21e419","Type":"ContainerDied","Data":"3daa8102e7b0b084cede544e1835d04448102648105cf77b9fa6c65c3de6c6b9"} Jun 01 21:53:15.829772 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:53:15.829750 2573 scope.go:117] "RemoveContainer" containerID="5655ed74aacf1cbd2588f2662d86ce2772280775e3ee820546598c117f695493" Jun 01 21:53:15.830051 ip-10-0-138-241 kubenswrapper[2573]: E0601 21:53:15.830018 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5655ed74aacf1cbd2588f2662d86ce2772280775e3ee820546598c117f695493\": container with ID starting with 5655ed74aacf1cbd2588f2662d86ce2772280775e3ee820546598c117f695493 not found: ID does not exist" containerID="5655ed74aacf1cbd2588f2662d86ce2772280775e3ee820546598c117f695493" Jun 01 21:53:15.830147 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:53:15.830049 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5655ed74aacf1cbd2588f2662d86ce2772280775e3ee820546598c117f695493"} err="failed to get container status \"5655ed74aacf1cbd2588f2662d86ce2772280775e3ee820546598c117f695493\": rpc error: code = NotFound desc = could not find container \"5655ed74aacf1cbd2588f2662d86ce2772280775e3ee820546598c117f695493\": container with ID starting with 5655ed74aacf1cbd2588f2662d86ce2772280775e3ee820546598c117f695493 not found: ID does not exist" Jun 01 21:53:15.830147 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:53:15.830072 2573 scope.go:117] "RemoveContainer" containerID="5d1fe0898a7112209963ce388482ecdf6f07a6bcd7aa895a8a3784f0beaf0e44" Jun 01 21:53:15.837311 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:53:15.837293 2573 scope.go:117] "RemoveContainer" containerID="5d1fe0898a7112209963ce388482ecdf6f07a6bcd7aa895a8a3784f0beaf0e44" Jun 01 21:53:15.837627 ip-10-0-138-241 kubenswrapper[2573]: E0601 21:53:15.837608 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5d1fe0898a7112209963ce388482ecdf6f07a6bcd7aa895a8a3784f0beaf0e44\": container with ID starting with 5d1fe0898a7112209963ce388482ecdf6f07a6bcd7aa895a8a3784f0beaf0e44 not found: ID does not exist" containerID="5d1fe0898a7112209963ce388482ecdf6f07a6bcd7aa895a8a3784f0beaf0e44" Jun 01 21:53:15.837687 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:53:15.837634 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5d1fe0898a7112209963ce388482ecdf6f07a6bcd7aa895a8a3784f0beaf0e44"} err="failed to get container status \"5d1fe0898a7112209963ce388482ecdf6f07a6bcd7aa895a8a3784f0beaf0e44\": rpc error: code = NotFound desc = could not find container \"5d1fe0898a7112209963ce388482ecdf6f07a6bcd7aa895a8a3784f0beaf0e44\": container with ID starting with 5d1fe0898a7112209963ce388482ecdf6f07a6bcd7aa895a8a3784f0beaf0e44 not found: ID does not exist" Jun 01 21:53:15.838257 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:53:15.838221 2573 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/authorino-7c846b5758-fk27t" podStartSLOduration=1.512292778 podStartE2EDuration="1.838209043s" podCreationTimestamp="2026-06-01 21:53:14 +0000 UTC" firstStartedPulling="2026-06-01 21:53:14.645981316 +0000 UTC m=+754.366033241" lastFinishedPulling="2026-06-01 21:53:14.971897581 +0000 UTC m=+754.691949506" observedRunningTime="2026-06-01 21:53:15.83659367 +0000 UTC m=+755.556645612" watchObservedRunningTime="2026-06-01 21:53:15.838209043 +0000 UTC m=+755.558260986" Jun 01 21:53:15.852680 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:53:15.852653 2573 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-8b475cf9f-kz99d"] Jun 01 21:53:15.858670 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:53:15.858646 2573 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-8b475cf9f-kz99d"] Jun 01 21:53:15.863475 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:53:15.863438 2573 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7498df8756-6kw9w"] Jun 01 21:53:15.863711 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:53:15.863686 2573 kuberuntime_container.go:864] "Killing container with a grace period" pod="kuadrant-system/authorino-7498df8756-6kw9w" podUID="93654fdd-203d-4b0f-842d-90323507305d" containerName="authorino" containerID="cri-o://662a4edc0fe533c2ccc141c24bb1f582199268a2efb305ffac0eaacf83132d36" gracePeriod=30 Jun 01 21:53:15.878688 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:53:15.878655 2573 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-8688c48586-7kr59"] Jun 01 21:53:15.888084 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:53:15.883840 2573 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-8688c48586-7kr59"] Jun 01 21:53:16.115862 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:53:16.115838 2573 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7498df8756-6kw9w" Jun 01 21:53:16.167818 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:53:16.167776 2573 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9hsd9\" (UniqueName: \"kubernetes.io/projected/93654fdd-203d-4b0f-842d-90323507305d-kube-api-access-9hsd9\") pod \"93654fdd-203d-4b0f-842d-90323507305d\" (UID: \"93654fdd-203d-4b0f-842d-90323507305d\") " Jun 01 21:53:16.169809 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:53:16.169783 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/93654fdd-203d-4b0f-842d-90323507305d-kube-api-access-9hsd9" (OuterVolumeSpecName: "kube-api-access-9hsd9") pod "93654fdd-203d-4b0f-842d-90323507305d" (UID: "93654fdd-203d-4b0f-842d-90323507305d"). InnerVolumeSpecName "kube-api-access-9hsd9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 01 21:53:16.268722 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:53:16.268689 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9hsd9\" (UniqueName: \"kubernetes.io/projected/93654fdd-203d-4b0f-842d-90323507305d-kube-api-access-9hsd9\") on node \"ip-10-0-138-241.ec2.internal\" DevicePath \"\"" Jun 01 21:53:16.677980 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:53:16.677937 2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="14c13811-9e9c-4a6a-864f-bc03bc21e419" path="/var/lib/kubelet/pods/14c13811-9e9c-4a6a-864f-bc03bc21e419/volumes" Jun 01 21:53:16.678321 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:53:16.678301 2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4bd7e704-b342-4013-a8eb-73c2a12de584" path="/var/lib/kubelet/pods/4bd7e704-b342-4013-a8eb-73c2a12de584/volumes" Jun 01 21:53:16.826868 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:53:16.826828 2573 generic.go:358] "Generic (PLEG): container finished" podID="93654fdd-203d-4b0f-842d-90323507305d" containerID="662a4edc0fe533c2ccc141c24bb1f582199268a2efb305ffac0eaacf83132d36" exitCode=0 Jun 01 21:53:16.827308 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:53:16.826886 2573 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7498df8756-6kw9w" Jun 01 21:53:16.827308 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:53:16.826903 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7498df8756-6kw9w" event={"ID":"93654fdd-203d-4b0f-842d-90323507305d","Type":"ContainerDied","Data":"662a4edc0fe533c2ccc141c24bb1f582199268a2efb305ffac0eaacf83132d36"} Jun 01 21:53:16.827308 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:53:16.826934 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7498df8756-6kw9w" event={"ID":"93654fdd-203d-4b0f-842d-90323507305d","Type":"ContainerDied","Data":"cbeae8cf72debb32216d8869a3bcba765cb67d7b74cb398318910dc0b0f00d2a"} Jun 01 21:53:16.827308 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:53:16.826951 2573 scope.go:117] "RemoveContainer" containerID="662a4edc0fe533c2ccc141c24bb1f582199268a2efb305ffac0eaacf83132d36" Jun 01 21:53:16.834727 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:53:16.834708 2573 scope.go:117] "RemoveContainer" containerID="662a4edc0fe533c2ccc141c24bb1f582199268a2efb305ffac0eaacf83132d36" Jun 01 21:53:16.835003 ip-10-0-138-241 kubenswrapper[2573]: E0601 21:53:16.834982 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"662a4edc0fe533c2ccc141c24bb1f582199268a2efb305ffac0eaacf83132d36\": container with ID starting with 662a4edc0fe533c2ccc141c24bb1f582199268a2efb305ffac0eaacf83132d36 not found: ID does not exist" containerID="662a4edc0fe533c2ccc141c24bb1f582199268a2efb305ffac0eaacf83132d36" Jun 01 21:53:16.835065 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:53:16.835016 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"662a4edc0fe533c2ccc141c24bb1f582199268a2efb305ffac0eaacf83132d36"} err="failed to get container status \"662a4edc0fe533c2ccc141c24bb1f582199268a2efb305ffac0eaacf83132d36\": rpc error: code = NotFound desc = could not find container \"662a4edc0fe533c2ccc141c24bb1f582199268a2efb305ffac0eaacf83132d36\": container with ID starting with 662a4edc0fe533c2ccc141c24bb1f582199268a2efb305ffac0eaacf83132d36 not found: ID does not exist" Jun 01 21:53:16.844069 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:53:16.844040 2573 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7498df8756-6kw9w"] Jun 01 21:53:16.847453 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:53:16.847431 2573 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-7498df8756-6kw9w"] Jun 01 21:53:18.678061 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:53:18.678029 2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="93654fdd-203d-4b0f-842d-90323507305d" path="/var/lib/kubelet/pods/93654fdd-203d-4b0f-842d-90323507305d/volumes" Jun 01 21:53:56.086229 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:53:56.086190 2573 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-ddtwb"] Jun 01 21:55:40.676676 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:55:40.676583 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rcqkp_fcf0209a-2440-4267-834a-b176f4e337e4/ovn-acl-logging/0.log" Jun 01 21:55:40.677803 ip-10-0-138-241 kubenswrapper[2573]: I0601 21:55:40.677777 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rcqkp_fcf0209a-2440-4267-834a-b176f4e337e4/ovn-acl-logging/0.log" Jun 01 22:00:00.144841 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:00:00.144804 2573 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-key-cleanup-29672520-sqjm8"] Jun 01 22:00:00.145308 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:00:00.145065 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4bd7e704-b342-4013-a8eb-73c2a12de584" containerName="authorino" Jun 01 22:00:00.145308 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:00:00.145076 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="4bd7e704-b342-4013-a8eb-73c2a12de584" containerName="authorino" Jun 01 22:00:00.145308 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:00:00.145084 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="14c13811-9e9c-4a6a-864f-bc03bc21e419" containerName="authorino" Jun 01 22:00:00.145308 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:00:00.145091 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="14c13811-9e9c-4a6a-864f-bc03bc21e419" containerName="authorino" Jun 01 22:00:00.145308 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:00:00.145104 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="93654fdd-203d-4b0f-842d-90323507305d" containerName="authorino" Jun 01 22:00:00.145308 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:00:00.145109 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="93654fdd-203d-4b0f-842d-90323507305d" containerName="authorino" Jun 01 22:00:00.145308 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:00:00.145160 2573 memory_manager.go:356] "RemoveStaleState removing state" podUID="4bd7e704-b342-4013-a8eb-73c2a12de584" containerName="authorino" Jun 01 22:00:00.145308 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:00:00.145168 2573 memory_manager.go:356] "RemoveStaleState removing state" podUID="14c13811-9e9c-4a6a-864f-bc03bc21e419" containerName="authorino" Jun 01 22:00:00.145308 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:00:00.145177 2573 memory_manager.go:356] "RemoveStaleState removing state" podUID="93654fdd-203d-4b0f-842d-90323507305d" containerName="authorino" Jun 01 22:00:00.147983 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:00:00.147966 2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29672520-sqjm8" Jun 01 22:00:00.150916 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:00:00.150894 2573 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-96mhz\"" Jun 01 22:00:00.154990 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:00:00.154487 2573 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-29672520-sqjm8"] Jun 01 22:00:00.268966 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:00:00.268924 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8dhzc\" (UniqueName: \"kubernetes.io/projected/04a7e4da-2f2b-4ad9-8d4c-7dc2879ab9d7-kube-api-access-8dhzc\") pod \"maas-api-key-cleanup-29672520-sqjm8\" (UID: \"04a7e4da-2f2b-4ad9-8d4c-7dc2879ab9d7\") " pod="opendatahub/maas-api-key-cleanup-29672520-sqjm8" Jun 01 22:00:00.369977 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:00:00.369934 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-8dhzc\" (UniqueName: \"kubernetes.io/projected/04a7e4da-2f2b-4ad9-8d4c-7dc2879ab9d7-kube-api-access-8dhzc\") pod \"maas-api-key-cleanup-29672520-sqjm8\" (UID: \"04a7e4da-2f2b-4ad9-8d4c-7dc2879ab9d7\") " pod="opendatahub/maas-api-key-cleanup-29672520-sqjm8" Jun 01 22:00:00.379073 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:00:00.379034 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8dhzc\" (UniqueName: \"kubernetes.io/projected/04a7e4da-2f2b-4ad9-8d4c-7dc2879ab9d7-kube-api-access-8dhzc\") pod \"maas-api-key-cleanup-29672520-sqjm8\" (UID: \"04a7e4da-2f2b-4ad9-8d4c-7dc2879ab9d7\") " pod="opendatahub/maas-api-key-cleanup-29672520-sqjm8" Jun 01 22:00:00.459030 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:00:00.458942 2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29672520-sqjm8" Jun 01 22:00:00.586328 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:00:00.586291 2573 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-29672520-sqjm8"] Jun 01 22:00:00.589444 ip-10-0-138-241 kubenswrapper[2573]: W0601 22:00:00.589406 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod04a7e4da_2f2b_4ad9_8d4c_7dc2879ab9d7.slice/crio-d933d5ee0f92c6aced6bcae0377311ba079987474d240292b054f470bb49d62c WatchSource:0}: Error finding container d933d5ee0f92c6aced6bcae0377311ba079987474d240292b054f470bb49d62c: Status 404 returned error can't find the container with id d933d5ee0f92c6aced6bcae0377311ba079987474d240292b054f470bb49d62c Jun 01 22:00:00.591258 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:00:00.591239 2573 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jun 01 22:00:00.963523 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:00:00.963483 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29672520-sqjm8" event={"ID":"04a7e4da-2f2b-4ad9-8d4c-7dc2879ab9d7","Type":"ContainerStarted","Data":"d933d5ee0f92c6aced6bcae0377311ba079987474d240292b054f470bb49d62c"} Jun 01 22:00:18.014955 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:00:18.014869 2573 generic.go:358] "Generic (PLEG): container finished" podID="04a7e4da-2f2b-4ad9-8d4c-7dc2879ab9d7" containerID="2371c769bca9bfa65748329065fbc446bd207f044ad4d3b54e251b04b9a0d84c" exitCode=7 Jun 01 22:00:18.014955 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:00:18.014938 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29672520-sqjm8" event={"ID":"04a7e4da-2f2b-4ad9-8d4c-7dc2879ab9d7","Type":"ContainerDied","Data":"2371c769bca9bfa65748329065fbc446bd207f044ad4d3b54e251b04b9a0d84c"} Jun 01 22:00:18.015329 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:00:18.015151 2573 scope.go:117] "RemoveContainer" containerID="2371c769bca9bfa65748329065fbc446bd207f044ad4d3b54e251b04b9a0d84c" Jun 01 22:00:19.020151 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:00:19.020059 2573 generic.go:358] "Generic (PLEG): container finished" podID="04a7e4da-2f2b-4ad9-8d4c-7dc2879ab9d7" containerID="83bda4e107ff6928460f868c792b885f88b006b7115ceb97575f4d798eebcda8" exitCode=7 Jun 01 22:00:19.020151 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:00:19.020131 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29672520-sqjm8" event={"ID":"04a7e4da-2f2b-4ad9-8d4c-7dc2879ab9d7","Type":"ContainerDied","Data":"83bda4e107ff6928460f868c792b885f88b006b7115ceb97575f4d798eebcda8"} Jun 01 22:00:19.020660 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:00:19.020167 2573 scope.go:117] "RemoveContainer" containerID="2371c769bca9bfa65748329065fbc446bd207f044ad4d3b54e251b04b9a0d84c" Jun 01 22:00:19.020660 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:00:19.020351 2573 scope.go:117] "RemoveContainer" containerID="83bda4e107ff6928460f868c792b885f88b006b7115ceb97575f4d798eebcda8" Jun 01 22:00:19.020660 ip-10-0-138-241 kubenswrapper[2573]: E0601 22:00:19.020573 2573 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-29672520-sqjm8_opendatahub(04a7e4da-2f2b-4ad9-8d4c-7dc2879ab9d7)\"" pod="opendatahub/maas-api-key-cleanup-29672520-sqjm8" podUID="04a7e4da-2f2b-4ad9-8d4c-7dc2879ab9d7" Jun 01 22:00:20.024189 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:00:20.024160 2573 scope.go:117] "RemoveContainer" containerID="83bda4e107ff6928460f868c792b885f88b006b7115ceb97575f4d798eebcda8" Jun 01 22:00:20.024679 ip-10-0-138-241 kubenswrapper[2573]: E0601 22:00:20.024422 2573 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-29672520-sqjm8_opendatahub(04a7e4da-2f2b-4ad9-8d4c-7dc2879ab9d7)\"" pod="opendatahub/maas-api-key-cleanup-29672520-sqjm8" podUID="04a7e4da-2f2b-4ad9-8d4c-7dc2879ab9d7" Jun 01 22:00:33.674070 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:00:33.674037 2573 scope.go:117] "RemoveContainer" containerID="83bda4e107ff6928460f868c792b885f88b006b7115ceb97575f4d798eebcda8" Jun 01 22:00:34.068981 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:00:34.068944 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29672520-sqjm8" event={"ID":"04a7e4da-2f2b-4ad9-8d4c-7dc2879ab9d7","Type":"ContainerStarted","Data":"b2b250cc2be0588066bcd36cec65bf9abc128be4a4dc2eeec27da28a81be79c9"} Jun 01 22:00:34.088074 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:00:34.088022 2573 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="opendatahub/maas-api-key-cleanup-29672520-sqjm8" podStartSLOduration=17.497471639 podStartE2EDuration="34.088005101s" podCreationTimestamp="2026-06-01 22:00:00 +0000 UTC" firstStartedPulling="2026-06-01 22:00:00.591423752 +0000 UTC m=+1160.311475691" lastFinishedPulling="2026-06-01 22:00:17.18195723 +0000 UTC m=+1176.902009153" observedRunningTime="2026-06-01 22:00:34.086609197 +0000 UTC m=+1193.806661137" watchObservedRunningTime="2026-06-01 22:00:34.088005101 +0000 UTC m=+1193.808057044" Jun 01 22:00:35.074631 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:00:35.074598 2573 generic.go:358] "Generic (PLEG): container finished" podID="04a7e4da-2f2b-4ad9-8d4c-7dc2879ab9d7" containerID="b2b250cc2be0588066bcd36cec65bf9abc128be4a4dc2eeec27da28a81be79c9" exitCode=7 Jun 01 22:00:35.075036 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:00:35.074660 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29672520-sqjm8" event={"ID":"04a7e4da-2f2b-4ad9-8d4c-7dc2879ab9d7","Type":"ContainerDied","Data":"b2b250cc2be0588066bcd36cec65bf9abc128be4a4dc2eeec27da28a81be79c9"} Jun 01 22:00:35.075036 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:00:35.074696 2573 scope.go:117] "RemoveContainer" containerID="83bda4e107ff6928460f868c792b885f88b006b7115ceb97575f4d798eebcda8" Jun 01 22:00:35.075036 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:00:35.074977 2573 scope.go:117] "RemoveContainer" containerID="b2b250cc2be0588066bcd36cec65bf9abc128be4a4dc2eeec27da28a81be79c9" Jun 01 22:00:35.075207 ip-10-0-138-241 kubenswrapper[2573]: E0601 22:00:35.075189 2573 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-29672520-sqjm8_opendatahub(04a7e4da-2f2b-4ad9-8d4c-7dc2879ab9d7)\"" pod="opendatahub/maas-api-key-cleanup-29672520-sqjm8" podUID="04a7e4da-2f2b-4ad9-8d4c-7dc2879ab9d7" Jun 01 22:00:35.097058 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:00:35.097033 2573 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-key-cleanup-29672520-sqjm8"] Jun 01 22:00:36.195059 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:00:36.195032 2573 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29672520-sqjm8" Jun 01 22:00:36.361437 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:00:36.361334 2573 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8dhzc\" (UniqueName: \"kubernetes.io/projected/04a7e4da-2f2b-4ad9-8d4c-7dc2879ab9d7-kube-api-access-8dhzc\") pod \"04a7e4da-2f2b-4ad9-8d4c-7dc2879ab9d7\" (UID: \"04a7e4da-2f2b-4ad9-8d4c-7dc2879ab9d7\") " Jun 01 22:00:36.363502 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:00:36.363477 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/04a7e4da-2f2b-4ad9-8d4c-7dc2879ab9d7-kube-api-access-8dhzc" (OuterVolumeSpecName: "kube-api-access-8dhzc") pod "04a7e4da-2f2b-4ad9-8d4c-7dc2879ab9d7" (UID: "04a7e4da-2f2b-4ad9-8d4c-7dc2879ab9d7"). InnerVolumeSpecName "kube-api-access-8dhzc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 01 22:00:36.462355 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:00:36.462303 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8dhzc\" (UniqueName: \"kubernetes.io/projected/04a7e4da-2f2b-4ad9-8d4c-7dc2879ab9d7-kube-api-access-8dhzc\") on node \"ip-10-0-138-241.ec2.internal\" DevicePath \"\"" Jun 01 22:00:37.085857 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:00:37.085819 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29672520-sqjm8" event={"ID":"04a7e4da-2f2b-4ad9-8d4c-7dc2879ab9d7","Type":"ContainerDied","Data":"d933d5ee0f92c6aced6bcae0377311ba079987474d240292b054f470bb49d62c"} Jun 01 22:00:37.085857 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:00:37.085842 2573 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29672520-sqjm8" Jun 01 22:00:37.086105 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:00:37.085870 2573 scope.go:117] "RemoveContainer" containerID="b2b250cc2be0588066bcd36cec65bf9abc128be4a4dc2eeec27da28a81be79c9" Jun 01 22:00:37.103312 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:00:37.103276 2573 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-key-cleanup-29672520-sqjm8"] Jun 01 22:00:37.106033 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:00:37.106008 2573 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-key-cleanup-29672520-sqjm8"] Jun 01 22:00:38.677353 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:00:38.677319 2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="04a7e4da-2f2b-4ad9-8d4c-7dc2879ab9d7" path="/var/lib/kubelet/pods/04a7e4da-2f2b-4ad9-8d4c-7dc2879ab9d7/volumes" Jun 01 22:00:40.695988 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:00:40.695963 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rcqkp_fcf0209a-2440-4267-834a-b176f4e337e4/ovn-acl-logging/0.log" Jun 01 22:00:40.697252 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:00:40.697233 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rcqkp_fcf0209a-2440-4267-834a-b176f4e337e4/ovn-acl-logging/0.log" Jun 01 22:05:40.714213 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:05:40.714175 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rcqkp_fcf0209a-2440-4267-834a-b176f4e337e4/ovn-acl-logging/0.log" Jun 01 22:05:40.716293 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:05:40.716273 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rcqkp_fcf0209a-2440-4267-834a-b176f4e337e4/ovn-acl-logging/0.log" Jun 01 22:10:40.733330 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:10:40.733305 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rcqkp_fcf0209a-2440-4267-834a-b176f4e337e4/ovn-acl-logging/0.log" Jun 01 22:10:40.734583 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:10:40.734561 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rcqkp_fcf0209a-2440-4267-834a-b176f4e337e4/ovn-acl-logging/0.log" Jun 01 22:14:08.435530 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:14:08.435493 2573 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-85664567db-tvhd4"] Jun 01 22:14:08.436031 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:14:08.435782 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="04a7e4da-2f2b-4ad9-8d4c-7dc2879ab9d7" containerName="cleanup" Jun 01 22:14:08.436031 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:14:08.435794 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="04a7e4da-2f2b-4ad9-8d4c-7dc2879ab9d7" containerName="cleanup" Jun 01 22:14:08.436031 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:14:08.435808 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="04a7e4da-2f2b-4ad9-8d4c-7dc2879ab9d7" containerName="cleanup" Jun 01 22:14:08.436031 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:14:08.435813 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="04a7e4da-2f2b-4ad9-8d4c-7dc2879ab9d7" containerName="cleanup" Jun 01 22:14:08.436031 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:14:08.435820 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="04a7e4da-2f2b-4ad9-8d4c-7dc2879ab9d7" containerName="cleanup" Jun 01 22:14:08.436031 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:14:08.435825 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="04a7e4da-2f2b-4ad9-8d4c-7dc2879ab9d7" containerName="cleanup" Jun 01 22:14:08.436031 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:14:08.435875 2573 memory_manager.go:356] "RemoveStaleState removing state" podUID="04a7e4da-2f2b-4ad9-8d4c-7dc2879ab9d7" containerName="cleanup" Jun 01 22:14:08.436031 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:14:08.435882 2573 memory_manager.go:356] "RemoveStaleState removing state" podUID="04a7e4da-2f2b-4ad9-8d4c-7dc2879ab9d7" containerName="cleanup" Jun 01 22:14:08.438645 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:14:08.438627 2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-85664567db-tvhd4" Jun 01 22:14:08.441705 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:14:08.441681 2573 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"authorino-oidc-ca\"" Jun 01 22:14:08.446267 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:14:08.446241 2573 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-85664567db-tvhd4"] Jun 01 22:14:08.487133 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:14:08.487082 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/0d6e9812-bf0f-42ea-9db4-cd07b1c3d0f5-tls-cert\") pod \"authorino-85664567db-tvhd4\" (UID: \"0d6e9812-bf0f-42ea-9db4-cd07b1c3d0f5\") " pod="kuadrant-system/authorino-85664567db-tvhd4" Jun 01 22:14:08.487319 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:14:08.487167 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oidc-ca\" (UniqueName: \"kubernetes.io/configmap/0d6e9812-bf0f-42ea-9db4-cd07b1c3d0f5-oidc-ca\") pod \"authorino-85664567db-tvhd4\" (UID: \"0d6e9812-bf0f-42ea-9db4-cd07b1c3d0f5\") " pod="kuadrant-system/authorino-85664567db-tvhd4" Jun 01 22:14:08.487319 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:14:08.487191 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hxn6p\" (UniqueName: \"kubernetes.io/projected/0d6e9812-bf0f-42ea-9db4-cd07b1c3d0f5-kube-api-access-hxn6p\") pod \"authorino-85664567db-tvhd4\" (UID: \"0d6e9812-bf0f-42ea-9db4-cd07b1c3d0f5\") " pod="kuadrant-system/authorino-85664567db-tvhd4" Jun 01 22:14:08.588138 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:14:08.588106 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/0d6e9812-bf0f-42ea-9db4-cd07b1c3d0f5-tls-cert\") pod \"authorino-85664567db-tvhd4\" (UID: \"0d6e9812-bf0f-42ea-9db4-cd07b1c3d0f5\") " pod="kuadrant-system/authorino-85664567db-tvhd4" Jun 01 22:14:08.588283 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:14:08.588166 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"oidc-ca\" (UniqueName: \"kubernetes.io/configmap/0d6e9812-bf0f-42ea-9db4-cd07b1c3d0f5-oidc-ca\") pod \"authorino-85664567db-tvhd4\" (UID: \"0d6e9812-bf0f-42ea-9db4-cd07b1c3d0f5\") " pod="kuadrant-system/authorino-85664567db-tvhd4" Jun 01 22:14:08.588283 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:14:08.588272 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-hxn6p\" (UniqueName: \"kubernetes.io/projected/0d6e9812-bf0f-42ea-9db4-cd07b1c3d0f5-kube-api-access-hxn6p\") pod \"authorino-85664567db-tvhd4\" (UID: \"0d6e9812-bf0f-42ea-9db4-cd07b1c3d0f5\") " pod="kuadrant-system/authorino-85664567db-tvhd4" Jun 01 22:14:08.588912 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:14:08.588891 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oidc-ca\" (UniqueName: \"kubernetes.io/configmap/0d6e9812-bf0f-42ea-9db4-cd07b1c3d0f5-oidc-ca\") pod \"authorino-85664567db-tvhd4\" (UID: \"0d6e9812-bf0f-42ea-9db4-cd07b1c3d0f5\") " pod="kuadrant-system/authorino-85664567db-tvhd4" Jun 01 22:14:08.590592 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:14:08.590568 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/0d6e9812-bf0f-42ea-9db4-cd07b1c3d0f5-tls-cert\") pod \"authorino-85664567db-tvhd4\" (UID: \"0d6e9812-bf0f-42ea-9db4-cd07b1c3d0f5\") " pod="kuadrant-system/authorino-85664567db-tvhd4" Jun 01 22:14:08.600417 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:14:08.600391 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hxn6p\" (UniqueName: \"kubernetes.io/projected/0d6e9812-bf0f-42ea-9db4-cd07b1c3d0f5-kube-api-access-hxn6p\") pod \"authorino-85664567db-tvhd4\" (UID: \"0d6e9812-bf0f-42ea-9db4-cd07b1c3d0f5\") " pod="kuadrant-system/authorino-85664567db-tvhd4" Jun 01 22:14:08.748116 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:14:08.748017 2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-85664567db-tvhd4" Jun 01 22:14:08.873140 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:14:08.873100 2573 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-85664567db-tvhd4"] Jun 01 22:14:08.875911 ip-10-0-138-241 kubenswrapper[2573]: W0601 22:14:08.875883 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0d6e9812_bf0f_42ea_9db4_cd07b1c3d0f5.slice/crio-762d0392f6160d3ae34f2370c56d2004dfb2df84f8f600bbeef871e588ec0741 WatchSource:0}: Error finding container 762d0392f6160d3ae34f2370c56d2004dfb2df84f8f600bbeef871e588ec0741: Status 404 returned error can't find the container with id 762d0392f6160d3ae34f2370c56d2004dfb2df84f8f600bbeef871e588ec0741 Jun 01 22:14:08.877294 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:14:08.877273 2573 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jun 01 22:14:09.363623 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:14:09.363586 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-85664567db-tvhd4" event={"ID":"0d6e9812-bf0f-42ea-9db4-cd07b1c3d0f5","Type":"ContainerStarted","Data":"762d0392f6160d3ae34f2370c56d2004dfb2df84f8f600bbeef871e588ec0741"} Jun 01 22:14:10.371606 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:14:10.371564 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-85664567db-tvhd4" event={"ID":"0d6e9812-bf0f-42ea-9db4-cd07b1c3d0f5","Type":"ContainerStarted","Data":"543ae62afd47aa495a59c9bcda114537e502f5be78d010af28e42ac8f4ea0fae"} Jun 01 22:14:10.388803 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:14:10.388752 2573 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/authorino-85664567db-tvhd4" podStartSLOduration=1.9007310849999999 podStartE2EDuration="2.388737907s" podCreationTimestamp="2026-06-01 22:14:08 +0000 UTC" firstStartedPulling="2026-06-01 22:14:08.8774628 +0000 UTC m=+2008.597514728" lastFinishedPulling="2026-06-01 22:14:09.36546963 +0000 UTC m=+2009.085521550" observedRunningTime="2026-06-01 22:14:10.386972462 +0000 UTC m=+2010.107024406" watchObservedRunningTime="2026-06-01 22:14:10.388737907 +0000 UTC m=+2010.108789850" Jun 01 22:14:10.412357 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:14:10.412320 2573 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7c846b5758-fk27t"] Jun 01 22:14:10.412571 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:14:10.412498 2573 kuberuntime_container.go:864] "Killing container with a grace period" pod="kuadrant-system/authorino-7c846b5758-fk27t" podUID="790b3718-a27a-4ecf-b7cd-46a81acbcd18" containerName="authorino" containerID="cri-o://69b6e849ae6662f4516c92b00631824532827e0eb59cad48f1ac63b5861c696c" gracePeriod=30 Jun 01 22:14:10.664863 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:14:10.664839 2573 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7c846b5758-fk27t" Jun 01 22:14:10.807221 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:14:10.807184 2573 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/790b3718-a27a-4ecf-b7cd-46a81acbcd18-tls-cert\") pod \"790b3718-a27a-4ecf-b7cd-46a81acbcd18\" (UID: \"790b3718-a27a-4ecf-b7cd-46a81acbcd18\") " Jun 01 22:14:10.807380 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:14:10.807239 2573 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vrq99\" (UniqueName: \"kubernetes.io/projected/790b3718-a27a-4ecf-b7cd-46a81acbcd18-kube-api-access-vrq99\") pod \"790b3718-a27a-4ecf-b7cd-46a81acbcd18\" (UID: \"790b3718-a27a-4ecf-b7cd-46a81acbcd18\") " Jun 01 22:14:10.809240 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:14:10.809210 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/790b3718-a27a-4ecf-b7cd-46a81acbcd18-kube-api-access-vrq99" (OuterVolumeSpecName: "kube-api-access-vrq99") pod "790b3718-a27a-4ecf-b7cd-46a81acbcd18" (UID: "790b3718-a27a-4ecf-b7cd-46a81acbcd18"). InnerVolumeSpecName "kube-api-access-vrq99". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 01 22:14:10.816657 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:14:10.816631 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/790b3718-a27a-4ecf-b7cd-46a81acbcd18-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "790b3718-a27a-4ecf-b7cd-46a81acbcd18" (UID: "790b3718-a27a-4ecf-b7cd-46a81acbcd18"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 01 22:14:10.908219 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:14:10.908110 2573 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/790b3718-a27a-4ecf-b7cd-46a81acbcd18-tls-cert\") on node \"ip-10-0-138-241.ec2.internal\" DevicePath \"\"" Jun 01 22:14:10.908219 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:14:10.908158 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vrq99\" (UniqueName: \"kubernetes.io/projected/790b3718-a27a-4ecf-b7cd-46a81acbcd18-kube-api-access-vrq99\") on node \"ip-10-0-138-241.ec2.internal\" DevicePath \"\"" Jun 01 22:14:11.375402 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:14:11.375365 2573 generic.go:358] "Generic (PLEG): container finished" podID="790b3718-a27a-4ecf-b7cd-46a81acbcd18" containerID="69b6e849ae6662f4516c92b00631824532827e0eb59cad48f1ac63b5861c696c" exitCode=0 Jun 01 22:14:11.375853 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:14:11.375420 2573 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7c846b5758-fk27t" Jun 01 22:14:11.375853 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:14:11.375445 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7c846b5758-fk27t" event={"ID":"790b3718-a27a-4ecf-b7cd-46a81acbcd18","Type":"ContainerDied","Data":"69b6e849ae6662f4516c92b00631824532827e0eb59cad48f1ac63b5861c696c"} Jun 01 22:14:11.375853 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:14:11.375477 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7c846b5758-fk27t" event={"ID":"790b3718-a27a-4ecf-b7cd-46a81acbcd18","Type":"ContainerDied","Data":"afe0d839a55df86ab2b831bab61e9e13b7fb4f51dfb8b58c1e5eabb373528e01"} Jun 01 22:14:11.375853 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:14:11.375491 2573 scope.go:117] "RemoveContainer" containerID="69b6e849ae6662f4516c92b00631824532827e0eb59cad48f1ac63b5861c696c" Jun 01 22:14:11.383528 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:14:11.383502 2573 scope.go:117] "RemoveContainer" containerID="69b6e849ae6662f4516c92b00631824532827e0eb59cad48f1ac63b5861c696c" Jun 01 22:14:11.383840 ip-10-0-138-241 kubenswrapper[2573]: E0601 22:14:11.383795 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"69b6e849ae6662f4516c92b00631824532827e0eb59cad48f1ac63b5861c696c\": container with ID starting with 69b6e849ae6662f4516c92b00631824532827e0eb59cad48f1ac63b5861c696c not found: ID does not exist" containerID="69b6e849ae6662f4516c92b00631824532827e0eb59cad48f1ac63b5861c696c" Jun 01 22:14:11.383886 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:14:11.383853 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"69b6e849ae6662f4516c92b00631824532827e0eb59cad48f1ac63b5861c696c"} err="failed to get container status \"69b6e849ae6662f4516c92b00631824532827e0eb59cad48f1ac63b5861c696c\": rpc error: code = NotFound desc = could not find container \"69b6e849ae6662f4516c92b00631824532827e0eb59cad48f1ac63b5861c696c\": container with ID starting with 69b6e849ae6662f4516c92b00631824532827e0eb59cad48f1ac63b5861c696c not found: ID does not exist" Jun 01 22:14:11.395798 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:14:11.395748 2573 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7c846b5758-fk27t"] Jun 01 22:14:11.398946 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:14:11.398921 2573 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-7c846b5758-fk27t"] Jun 01 22:14:12.678129 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:14:12.678092 2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="790b3718-a27a-4ecf-b7cd-46a81acbcd18" path="/var/lib/kubelet/pods/790b3718-a27a-4ecf-b7cd-46a81acbcd18/volumes" Jun 01 22:14:20.227808 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:14:20.227769 2573 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-ddtwb"] Jun 01 22:14:27.538766 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:14:27.538735 2573 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-5489f4f4bc-m6dlt"] Jun 01 22:14:27.539112 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:14:27.538992 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="790b3718-a27a-4ecf-b7cd-46a81acbcd18" containerName="authorino" Jun 01 22:14:27.539112 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:14:27.539003 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="790b3718-a27a-4ecf-b7cd-46a81acbcd18" containerName="authorino" Jun 01 22:14:27.539112 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:14:27.539053 2573 memory_manager.go:356] "RemoveStaleState removing state" podUID="790b3718-a27a-4ecf-b7cd-46a81acbcd18" containerName="authorino" Jun 01 22:14:27.539112 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:14:27.539065 2573 memory_manager.go:356] "RemoveStaleState removing state" podUID="04a7e4da-2f2b-4ad9-8d4c-7dc2879ab9d7" containerName="cleanup" Jun 01 22:14:27.543405 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:14:27.543383 2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-5489f4f4bc-m6dlt" Jun 01 22:14:27.546266 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:14:27.546246 2573 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jun 01 22:14:27.547847 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:14:27.547828 2573 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jun 01 22:14:27.547915 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:14:27.547848 2573 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\"" Jun 01 22:14:27.547915 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:14:27.547887 2573 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-lnk8n\"" Jun 01 22:14:27.552322 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:14:27.552299 2573 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-5489f4f4bc-m6dlt"] Jun 01 22:14:27.643969 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:14:27.643923 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/5eef219b-e796-4aa9-bf30-50a9f4f56e7b-kserve-provision-location\") pod \"e2e-distinct-simulated-kserve-5489f4f4bc-m6dlt\" (UID: \"5eef219b-e796-4aa9-bf30-50a9f4f56e7b\") " pod="llm/e2e-distinct-simulated-kserve-5489f4f4bc-m6dlt" Jun 01 22:14:27.643969 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:14:27.643970 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5eef219b-e796-4aa9-bf30-50a9f4f56e7b-model-cache\") pod \"e2e-distinct-simulated-kserve-5489f4f4bc-m6dlt\" (UID: \"5eef219b-e796-4aa9-bf30-50a9f4f56e7b\") " pod="llm/e2e-distinct-simulated-kserve-5489f4f4bc-m6dlt" Jun 01 22:14:27.644174 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:14:27.644040 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2z4lr\" (UniqueName: \"kubernetes.io/projected/5eef219b-e796-4aa9-bf30-50a9f4f56e7b-kube-api-access-2z4lr\") pod \"e2e-distinct-simulated-kserve-5489f4f4bc-m6dlt\" (UID: \"5eef219b-e796-4aa9-bf30-50a9f4f56e7b\") " pod="llm/e2e-distinct-simulated-kserve-5489f4f4bc-m6dlt" Jun 01 22:14:27.644174 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:14:27.644088 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5eef219b-e796-4aa9-bf30-50a9f4f56e7b-tls-certs\") pod \"e2e-distinct-simulated-kserve-5489f4f4bc-m6dlt\" (UID: \"5eef219b-e796-4aa9-bf30-50a9f4f56e7b\") " pod="llm/e2e-distinct-simulated-kserve-5489f4f4bc-m6dlt" Jun 01 22:14:27.644174 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:14:27.644120 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5eef219b-e796-4aa9-bf30-50a9f4f56e7b-dshm\") pod \"e2e-distinct-simulated-kserve-5489f4f4bc-m6dlt\" (UID: \"5eef219b-e796-4aa9-bf30-50a9f4f56e7b\") " pod="llm/e2e-distinct-simulated-kserve-5489f4f4bc-m6dlt" Jun 01 22:14:27.644272 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:14:27.644177 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5eef219b-e796-4aa9-bf30-50a9f4f56e7b-home\") pod \"e2e-distinct-simulated-kserve-5489f4f4bc-m6dlt\" (UID: \"5eef219b-e796-4aa9-bf30-50a9f4f56e7b\") " pod="llm/e2e-distinct-simulated-kserve-5489f4f4bc-m6dlt" Jun 01 22:14:27.745355 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:14:27.745317 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5eef219b-e796-4aa9-bf30-50a9f4f56e7b-model-cache\") pod \"e2e-distinct-simulated-kserve-5489f4f4bc-m6dlt\" (UID: \"5eef219b-e796-4aa9-bf30-50a9f4f56e7b\") " pod="llm/e2e-distinct-simulated-kserve-5489f4f4bc-m6dlt" Jun 01 22:14:27.745557 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:14:27.745364 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-2z4lr\" (UniqueName: \"kubernetes.io/projected/5eef219b-e796-4aa9-bf30-50a9f4f56e7b-kube-api-access-2z4lr\") pod \"e2e-distinct-simulated-kserve-5489f4f4bc-m6dlt\" (UID: \"5eef219b-e796-4aa9-bf30-50a9f4f56e7b\") " pod="llm/e2e-distinct-simulated-kserve-5489f4f4bc-m6dlt" Jun 01 22:14:27.745557 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:14:27.745389 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5eef219b-e796-4aa9-bf30-50a9f4f56e7b-tls-certs\") pod \"e2e-distinct-simulated-kserve-5489f4f4bc-m6dlt\" (UID: \"5eef219b-e796-4aa9-bf30-50a9f4f56e7b\") " pod="llm/e2e-distinct-simulated-kserve-5489f4f4bc-m6dlt" Jun 01 22:14:27.745557 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:14:27.745415 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5eef219b-e796-4aa9-bf30-50a9f4f56e7b-dshm\") pod \"e2e-distinct-simulated-kserve-5489f4f4bc-m6dlt\" (UID: \"5eef219b-e796-4aa9-bf30-50a9f4f56e7b\") " pod="llm/e2e-distinct-simulated-kserve-5489f4f4bc-m6dlt" Jun 01 22:14:27.745557 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:14:27.745440 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5eef219b-e796-4aa9-bf30-50a9f4f56e7b-home\") pod \"e2e-distinct-simulated-kserve-5489f4f4bc-m6dlt\" (UID: \"5eef219b-e796-4aa9-bf30-50a9f4f56e7b\") " pod="llm/e2e-distinct-simulated-kserve-5489f4f4bc-m6dlt" Jun 01 22:14:27.745557 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:14:27.745466 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/5eef219b-e796-4aa9-bf30-50a9f4f56e7b-kserve-provision-location\") pod \"e2e-distinct-simulated-kserve-5489f4f4bc-m6dlt\" (UID: \"5eef219b-e796-4aa9-bf30-50a9f4f56e7b\") " pod="llm/e2e-distinct-simulated-kserve-5489f4f4bc-m6dlt" Jun 01 22:14:27.745826 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:14:27.745799 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5eef219b-e796-4aa9-bf30-50a9f4f56e7b-model-cache\") pod \"e2e-distinct-simulated-kserve-5489f4f4bc-m6dlt\" (UID: \"5eef219b-e796-4aa9-bf30-50a9f4f56e7b\") " pod="llm/e2e-distinct-simulated-kserve-5489f4f4bc-m6dlt" Jun 01 22:14:27.745886 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:14:27.745857 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/5eef219b-e796-4aa9-bf30-50a9f4f56e7b-kserve-provision-location\") pod \"e2e-distinct-simulated-kserve-5489f4f4bc-m6dlt\" (UID: \"5eef219b-e796-4aa9-bf30-50a9f4f56e7b\") " pod="llm/e2e-distinct-simulated-kserve-5489f4f4bc-m6dlt" Jun 01 22:14:27.745944 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:14:27.745913 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5eef219b-e796-4aa9-bf30-50a9f4f56e7b-home\") pod \"e2e-distinct-simulated-kserve-5489f4f4bc-m6dlt\" (UID: \"5eef219b-e796-4aa9-bf30-50a9f4f56e7b\") " pod="llm/e2e-distinct-simulated-kserve-5489f4f4bc-m6dlt" Jun 01 22:14:27.747863 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:14:27.747841 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5eef219b-e796-4aa9-bf30-50a9f4f56e7b-dshm\") pod \"e2e-distinct-simulated-kserve-5489f4f4bc-m6dlt\" (UID: \"5eef219b-e796-4aa9-bf30-50a9f4f56e7b\") " pod="llm/e2e-distinct-simulated-kserve-5489f4f4bc-m6dlt" Jun 01 22:14:27.747973 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:14:27.747957 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5eef219b-e796-4aa9-bf30-50a9f4f56e7b-tls-certs\") pod \"e2e-distinct-simulated-kserve-5489f4f4bc-m6dlt\" (UID: \"5eef219b-e796-4aa9-bf30-50a9f4f56e7b\") " pod="llm/e2e-distinct-simulated-kserve-5489f4f4bc-m6dlt" Jun 01 22:14:27.753141 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:14:27.753113 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2z4lr\" (UniqueName: \"kubernetes.io/projected/5eef219b-e796-4aa9-bf30-50a9f4f56e7b-kube-api-access-2z4lr\") pod \"e2e-distinct-simulated-kserve-5489f4f4bc-m6dlt\" (UID: \"5eef219b-e796-4aa9-bf30-50a9f4f56e7b\") " pod="llm/e2e-distinct-simulated-kserve-5489f4f4bc-m6dlt" Jun 01 22:14:27.854156 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:14:27.854100 2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-5489f4f4bc-m6dlt" Jun 01 22:14:27.987554 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:14:27.987468 2573 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-5489f4f4bc-m6dlt"] Jun 01 22:14:27.990217 ip-10-0-138-241 kubenswrapper[2573]: W0601 22:14:27.990191 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5eef219b_e796_4aa9_bf30_50a9f4f56e7b.slice/crio-356a9b151942268a74697969bbe2faeed30408d1c010a282f6906633b8e9fdb1 WatchSource:0}: Error finding container 356a9b151942268a74697969bbe2faeed30408d1c010a282f6906633b8e9fdb1: Status 404 returned error can't find the container with id 356a9b151942268a74697969bbe2faeed30408d1c010a282f6906633b8e9fdb1 Jun 01 22:14:28.116682 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:14:28.116599 2573 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-ddtwb"] Jun 01 22:14:28.430217 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:14:28.430123 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-5489f4f4bc-m6dlt" event={"ID":"5eef219b-e796-4aa9-bf30-50a9f4f56e7b","Type":"ContainerStarted","Data":"356a9b151942268a74697969bbe2faeed30408d1c010a282f6906633b8e9fdb1"} Jun 01 22:14:31.441315 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:14:31.441276 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-5489f4f4bc-m6dlt" event={"ID":"5eef219b-e796-4aa9-bf30-50a9f4f56e7b","Type":"ContainerStarted","Data":"33c8d2f7a0e087ce405ac67aaf15eabe77165ebf399b2af5540d9b953a498e38"} Jun 01 22:14:33.448551 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:14:33.448497 2573 generic.go:358] "Generic (PLEG): container finished" podID="5eef219b-e796-4aa9-bf30-50a9f4f56e7b" containerID="33c8d2f7a0e087ce405ac67aaf15eabe77165ebf399b2af5540d9b953a498e38" exitCode=0 Jun 01 22:14:33.448907 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:14:33.448577 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-5489f4f4bc-m6dlt" event={"ID":"5eef219b-e796-4aa9-bf30-50a9f4f56e7b","Type":"ContainerDied","Data":"33c8d2f7a0e087ce405ac67aaf15eabe77165ebf399b2af5540d9b953a498e38"} Jun 01 22:14:35.457042 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:14:35.457001 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-5489f4f4bc-m6dlt" event={"ID":"5eef219b-e796-4aa9-bf30-50a9f4f56e7b","Type":"ContainerStarted","Data":"7810b59ebd5ed4496bcd66091f5aedfa5ba4ad3c8e9bf27c5eb8119105d7a760"} Jun 01 22:14:35.476979 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:14:35.476904 2573 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-5489f4f4bc-m6dlt" podStartSLOduration=2.016017109 podStartE2EDuration="8.476887691s" podCreationTimestamp="2026-06-01 22:14:27 +0000 UTC" firstStartedPulling="2026-06-01 22:14:27.993159755 +0000 UTC m=+2027.713211694" lastFinishedPulling="2026-06-01 22:14:34.454030355 +0000 UTC m=+2034.174082276" observedRunningTime="2026-06-01 22:14:35.474291272 +0000 UTC m=+2035.194343215" watchObservedRunningTime="2026-06-01 22:14:35.476887691 +0000 UTC m=+2035.196939993" Jun 01 22:14:37.854790 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:14:37.854746 2573 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-5489f4f4bc-m6dlt" Jun 01 22:14:37.855149 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:14:37.854806 2573 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-5489f4f4bc-m6dlt" Jun 01 22:14:37.867373 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:14:37.867341 2573 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-5489f4f4bc-m6dlt" Jun 01 22:14:38.480424 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:14:38.480387 2573 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-5489f4f4bc-m6dlt" Jun 01 22:14:40.422581 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:14:40.422522 2573 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-ddtwb"] Jun 01 22:14:51.737498 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:14:51.737467 2573 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9bthpk"] Jun 01 22:14:51.742577 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:14:51.742556 2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9bthpk" Jun 01 22:14:51.745382 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:14:51.745359 2573 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\"" Jun 01 22:14:51.748692 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:14:51.748665 2573 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9bthpk"] Jun 01 22:14:51.845620 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:14:51.845575 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7rhkn\" (UniqueName: \"kubernetes.io/projected/89bf1068-3455-4c94-b28a-b0ac05a796eb-kube-api-access-7rhkn\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9bthpk\" (UID: \"89bf1068-3455-4c94-b28a-b0ac05a796eb\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9bthpk" Jun 01 22:14:51.845821 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:14:51.845629 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/89bf1068-3455-4c94-b28a-b0ac05a796eb-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9bthpk\" (UID: \"89bf1068-3455-4c94-b28a-b0ac05a796eb\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9bthpk" Jun 01 22:14:51.845821 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:14:51.845721 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/89bf1068-3455-4c94-b28a-b0ac05a796eb-kserve-provision-location\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9bthpk\" (UID: \"89bf1068-3455-4c94-b28a-b0ac05a796eb\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9bthpk" Jun 01 22:14:51.845821 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:14:51.845765 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/89bf1068-3455-4c94-b28a-b0ac05a796eb-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9bthpk\" (UID: \"89bf1068-3455-4c94-b28a-b0ac05a796eb\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9bthpk" Jun 01 22:14:51.845821 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:14:51.845810 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/89bf1068-3455-4c94-b28a-b0ac05a796eb-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9bthpk\" (UID: \"89bf1068-3455-4c94-b28a-b0ac05a796eb\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9bthpk" Jun 01 22:14:51.845991 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:14:51.845844 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/89bf1068-3455-4c94-b28a-b0ac05a796eb-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9bthpk\" (UID: \"89bf1068-3455-4c94-b28a-b0ac05a796eb\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9bthpk" Jun 01 22:14:51.946815 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:14:51.946777 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/89bf1068-3455-4c94-b28a-b0ac05a796eb-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9bthpk\" (UID: \"89bf1068-3455-4c94-b28a-b0ac05a796eb\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9bthpk" Jun 01 22:14:51.946990 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:14:51.946824 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-7rhkn\" (UniqueName: \"kubernetes.io/projected/89bf1068-3455-4c94-b28a-b0ac05a796eb-kube-api-access-7rhkn\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9bthpk\" (UID: \"89bf1068-3455-4c94-b28a-b0ac05a796eb\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9bthpk" Jun 01 22:14:51.946990 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:14:51.946854 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/89bf1068-3455-4c94-b28a-b0ac05a796eb-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9bthpk\" (UID: \"89bf1068-3455-4c94-b28a-b0ac05a796eb\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9bthpk" Jun 01 22:14:51.946990 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:14:51.946882 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/89bf1068-3455-4c94-b28a-b0ac05a796eb-kserve-provision-location\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9bthpk\" (UID: \"89bf1068-3455-4c94-b28a-b0ac05a796eb\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9bthpk" Jun 01 22:14:51.946990 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:14:51.946899 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/89bf1068-3455-4c94-b28a-b0ac05a796eb-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9bthpk\" (UID: \"89bf1068-3455-4c94-b28a-b0ac05a796eb\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9bthpk" Jun 01 22:14:51.946990 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:14:51.946920 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/89bf1068-3455-4c94-b28a-b0ac05a796eb-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9bthpk\" (UID: \"89bf1068-3455-4c94-b28a-b0ac05a796eb\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9bthpk" Jun 01 22:14:51.947335 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:14:51.947313 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/89bf1068-3455-4c94-b28a-b0ac05a796eb-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9bthpk\" (UID: \"89bf1068-3455-4c94-b28a-b0ac05a796eb\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9bthpk" Jun 01 22:14:51.947396 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:14:51.947357 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/89bf1068-3455-4c94-b28a-b0ac05a796eb-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9bthpk\" (UID: \"89bf1068-3455-4c94-b28a-b0ac05a796eb\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9bthpk" Jun 01 22:14:51.947396 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:14:51.947378 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/89bf1068-3455-4c94-b28a-b0ac05a796eb-kserve-provision-location\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9bthpk\" (UID: \"89bf1068-3455-4c94-b28a-b0ac05a796eb\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9bthpk" Jun 01 22:14:51.949243 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:14:51.949221 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/89bf1068-3455-4c94-b28a-b0ac05a796eb-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9bthpk\" (UID: \"89bf1068-3455-4c94-b28a-b0ac05a796eb\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9bthpk" Jun 01 22:14:51.949385 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:14:51.949369 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/89bf1068-3455-4c94-b28a-b0ac05a796eb-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9bthpk\" (UID: \"89bf1068-3455-4c94-b28a-b0ac05a796eb\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9bthpk" Jun 01 22:14:51.954409 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:14:51.954383 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7rhkn\" (UniqueName: \"kubernetes.io/projected/89bf1068-3455-4c94-b28a-b0ac05a796eb-kube-api-access-7rhkn\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9bthpk\" (UID: \"89bf1068-3455-4c94-b28a-b0ac05a796eb\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9bthpk" Jun 01 22:14:52.053327 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:14:52.053237 2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9bthpk" Jun 01 22:14:52.182905 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:14:52.182879 2573 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9bthpk"] Jun 01 22:14:52.185358 ip-10-0-138-241 kubenswrapper[2573]: W0601 22:14:52.185328 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod89bf1068_3455_4c94_b28a_b0ac05a796eb.slice/crio-f9c3c6171b1a21f1357e1a5aca5932f2b57d638ff208fe8d45449e7484d40b91 WatchSource:0}: Error finding container f9c3c6171b1a21f1357e1a5aca5932f2b57d638ff208fe8d45449e7484d40b91: Status 404 returned error can't find the container with id f9c3c6171b1a21f1357e1a5aca5932f2b57d638ff208fe8d45449e7484d40b91 Jun 01 22:14:52.512508 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:14:52.512467 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9bthpk" event={"ID":"89bf1068-3455-4c94-b28a-b0ac05a796eb","Type":"ContainerStarted","Data":"e808632aa4468fea8a4e394395b9fbc15de273e66fe7e808a437d82c32db62fd"} Jun 01 22:14:52.512508 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:14:52.512512 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9bthpk" event={"ID":"89bf1068-3455-4c94-b28a-b0ac05a796eb","Type":"ContainerStarted","Data":"f9c3c6171b1a21f1357e1a5aca5932f2b57d638ff208fe8d45449e7484d40b91"} Jun 01 22:14:52.519229 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:14:52.519195 2573 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-ddtwb"] Jun 01 22:15:00.132337 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:15:00.132299 2573 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-key-cleanup-29672535-jrz2z"] Jun 01 22:15:00.150311 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:15:00.150280 2573 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-29672535-jrz2z"] Jun 01 22:15:00.150502 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:15:00.150393 2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29672535-jrz2z" Jun 01 22:15:00.153392 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:15:00.153370 2573 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-96mhz\"" Jun 01 22:15:00.215092 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:15:00.215051 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hwm4j\" (UniqueName: \"kubernetes.io/projected/5211e3c4-f063-4d34-94aa-58dc3ae0c8b4-kube-api-access-hwm4j\") pod \"maas-api-key-cleanup-29672535-jrz2z\" (UID: \"5211e3c4-f063-4d34-94aa-58dc3ae0c8b4\") " pod="opendatahub/maas-api-key-cleanup-29672535-jrz2z" Jun 01 22:15:00.316509 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:15:00.316465 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-hwm4j\" (UniqueName: \"kubernetes.io/projected/5211e3c4-f063-4d34-94aa-58dc3ae0c8b4-kube-api-access-hwm4j\") pod \"maas-api-key-cleanup-29672535-jrz2z\" (UID: \"5211e3c4-f063-4d34-94aa-58dc3ae0c8b4\") " pod="opendatahub/maas-api-key-cleanup-29672535-jrz2z" Jun 01 22:15:00.324670 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:15:00.324638 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hwm4j\" (UniqueName: \"kubernetes.io/projected/5211e3c4-f063-4d34-94aa-58dc3ae0c8b4-kube-api-access-hwm4j\") pod \"maas-api-key-cleanup-29672535-jrz2z\" (UID: \"5211e3c4-f063-4d34-94aa-58dc3ae0c8b4\") " pod="opendatahub/maas-api-key-cleanup-29672535-jrz2z" Jun 01 22:15:00.461462 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:15:00.461367 2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29672535-jrz2z" Jun 01 22:15:00.587047 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:15:00.587020 2573 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-29672535-jrz2z"] Jun 01 22:15:00.589484 ip-10-0-138-241 kubenswrapper[2573]: W0601 22:15:00.589451 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5211e3c4_f063_4d34_94aa_58dc3ae0c8b4.slice/crio-052b14b050c194d2cdd99c09e01f0dc774b4adfb252d8fbb2734cfb3b9294bc5 WatchSource:0}: Error finding container 052b14b050c194d2cdd99c09e01f0dc774b4adfb252d8fbb2734cfb3b9294bc5: Status 404 returned error can't find the container with id 052b14b050c194d2cdd99c09e01f0dc774b4adfb252d8fbb2734cfb3b9294bc5 Jun 01 22:15:01.540807 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:15:01.540708 2573 generic.go:358] "Generic (PLEG): container finished" podID="5211e3c4-f063-4d34-94aa-58dc3ae0c8b4" containerID="f9da5b6cda0f807f70663f41c400115b2bedbaf3f023920733bd6cbe98308758" exitCode=7 Jun 01 22:15:01.541182 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:15:01.540795 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29672535-jrz2z" event={"ID":"5211e3c4-f063-4d34-94aa-58dc3ae0c8b4","Type":"ContainerDied","Data":"f9da5b6cda0f807f70663f41c400115b2bedbaf3f023920733bd6cbe98308758"} Jun 01 22:15:01.541182 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:15:01.540847 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29672535-jrz2z" event={"ID":"5211e3c4-f063-4d34-94aa-58dc3ae0c8b4","Type":"ContainerStarted","Data":"052b14b050c194d2cdd99c09e01f0dc774b4adfb252d8fbb2734cfb3b9294bc5"} Jun 01 22:15:01.541182 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:15:01.540994 2573 scope.go:117] "RemoveContainer" containerID="f9da5b6cda0f807f70663f41c400115b2bedbaf3f023920733bd6cbe98308758" Jun 01 22:15:02.546693 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:15:02.546655 2573 generic.go:358] "Generic (PLEG): container finished" podID="5211e3c4-f063-4d34-94aa-58dc3ae0c8b4" containerID="30fc393a1f080ce4caf7caa270d2c3064e7ca9f46eb0f94c82e2acdd482f987f" exitCode=7 Jun 01 22:15:02.547163 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:15:02.546778 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29672535-jrz2z" event={"ID":"5211e3c4-f063-4d34-94aa-58dc3ae0c8b4","Type":"ContainerDied","Data":"30fc393a1f080ce4caf7caa270d2c3064e7ca9f46eb0f94c82e2acdd482f987f"} Jun 01 22:15:02.547163 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:15:02.546818 2573 scope.go:117] "RemoveContainer" containerID="f9da5b6cda0f807f70663f41c400115b2bedbaf3f023920733bd6cbe98308758" Jun 01 22:15:02.547163 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:15:02.546986 2573 scope.go:117] "RemoveContainer" containerID="30fc393a1f080ce4caf7caa270d2c3064e7ca9f46eb0f94c82e2acdd482f987f" Jun 01 22:15:02.547344 ip-10-0-138-241 kubenswrapper[2573]: E0601 22:15:02.547207 2573 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-29672535-jrz2z_opendatahub(5211e3c4-f063-4d34-94aa-58dc3ae0c8b4)\"" pod="opendatahub/maas-api-key-cleanup-29672535-jrz2z" podUID="5211e3c4-f063-4d34-94aa-58dc3ae0c8b4" Jun 01 22:15:03.551111 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:15:03.551086 2573 scope.go:117] "RemoveContainer" containerID="30fc393a1f080ce4caf7caa270d2c3064e7ca9f46eb0f94c82e2acdd482f987f" Jun 01 22:15:03.551482 ip-10-0-138-241 kubenswrapper[2573]: E0601 22:15:03.551273 2573 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-29672535-jrz2z_opendatahub(5211e3c4-f063-4d34-94aa-58dc3ae0c8b4)\"" pod="opendatahub/maas-api-key-cleanup-29672535-jrz2z" podUID="5211e3c4-f063-4d34-94aa-58dc3ae0c8b4" Jun 01 22:15:04.614699 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:15:04.614663 2573 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-p2l47"] Jun 01 22:15:04.619456 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:15:04.619437 2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-p2l47" Jun 01 22:15:04.622998 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:15:04.622965 2573 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\"" Jun 01 22:15:04.637765 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:15:04.637723 2573 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-p2l47"] Jun 01 22:15:04.756557 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:15:04.756502 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/88603a8e-0273-4168-820f-5793085778d7-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-59dcdf78f5-p2l47\" (UID: \"88603a8e-0273-4168-820f-5793085778d7\") " pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-p2l47" Jun 01 22:15:04.756733 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:15:04.756599 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/88603a8e-0273-4168-820f-5793085778d7-kserve-provision-location\") pod \"facebook-opt-125m-simulated-kserve-59dcdf78f5-p2l47\" (UID: \"88603a8e-0273-4168-820f-5793085778d7\") " pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-p2l47" Jun 01 22:15:04.756733 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:15:04.756629 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/88603a8e-0273-4168-820f-5793085778d7-home\") pod \"facebook-opt-125m-simulated-kserve-59dcdf78f5-p2l47\" (UID: \"88603a8e-0273-4168-820f-5793085778d7\") " pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-p2l47" Jun 01 22:15:04.756733 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:15:04.756705 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/88603a8e-0273-4168-820f-5793085778d7-model-cache\") pod \"facebook-opt-125m-simulated-kserve-59dcdf78f5-p2l47\" (UID: \"88603a8e-0273-4168-820f-5793085778d7\") " pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-p2l47" Jun 01 22:15:04.756840 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:15:04.756736 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/88603a8e-0273-4168-820f-5793085778d7-dshm\") pod \"facebook-opt-125m-simulated-kserve-59dcdf78f5-p2l47\" (UID: \"88603a8e-0273-4168-820f-5793085778d7\") " pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-p2l47" Jun 01 22:15:04.756840 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:15:04.756782 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2sl4r\" (UniqueName: \"kubernetes.io/projected/88603a8e-0273-4168-820f-5793085778d7-kube-api-access-2sl4r\") pod \"facebook-opt-125m-simulated-kserve-59dcdf78f5-p2l47\" (UID: \"88603a8e-0273-4168-820f-5793085778d7\") " pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-p2l47" Jun 01 22:15:04.857938 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:15:04.857892 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/88603a8e-0273-4168-820f-5793085778d7-kserve-provision-location\") pod \"facebook-opt-125m-simulated-kserve-59dcdf78f5-p2l47\" (UID: \"88603a8e-0273-4168-820f-5793085778d7\") " pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-p2l47" Jun 01 22:15:04.857938 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:15:04.857936 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/88603a8e-0273-4168-820f-5793085778d7-home\") pod \"facebook-opt-125m-simulated-kserve-59dcdf78f5-p2l47\" (UID: \"88603a8e-0273-4168-820f-5793085778d7\") " pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-p2l47" Jun 01 22:15:04.858171 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:15:04.858066 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/88603a8e-0273-4168-820f-5793085778d7-model-cache\") pod \"facebook-opt-125m-simulated-kserve-59dcdf78f5-p2l47\" (UID: \"88603a8e-0273-4168-820f-5793085778d7\") " pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-p2l47" Jun 01 22:15:04.858171 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:15:04.858110 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/88603a8e-0273-4168-820f-5793085778d7-dshm\") pod \"facebook-opt-125m-simulated-kserve-59dcdf78f5-p2l47\" (UID: \"88603a8e-0273-4168-820f-5793085778d7\") " pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-p2l47" Jun 01 22:15:04.858171 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:15:04.858164 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-2sl4r\" (UniqueName: \"kubernetes.io/projected/88603a8e-0273-4168-820f-5793085778d7-kube-api-access-2sl4r\") pod \"facebook-opt-125m-simulated-kserve-59dcdf78f5-p2l47\" (UID: \"88603a8e-0273-4168-820f-5793085778d7\") " pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-p2l47" Jun 01 22:15:04.858333 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:15:04.858211 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/88603a8e-0273-4168-820f-5793085778d7-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-59dcdf78f5-p2l47\" (UID: \"88603a8e-0273-4168-820f-5793085778d7\") " pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-p2l47" Jun 01 22:15:04.858388 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:15:04.858362 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/88603a8e-0273-4168-820f-5793085778d7-home\") pod \"facebook-opt-125m-simulated-kserve-59dcdf78f5-p2l47\" (UID: \"88603a8e-0273-4168-820f-5793085778d7\") " pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-p2l47" Jun 01 22:15:04.858444 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:15:04.858387 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/88603a8e-0273-4168-820f-5793085778d7-kserve-provision-location\") pod \"facebook-opt-125m-simulated-kserve-59dcdf78f5-p2l47\" (UID: \"88603a8e-0273-4168-820f-5793085778d7\") " pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-p2l47" Jun 01 22:15:04.858444 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:15:04.858427 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/88603a8e-0273-4168-820f-5793085778d7-model-cache\") pod \"facebook-opt-125m-simulated-kserve-59dcdf78f5-p2l47\" (UID: \"88603a8e-0273-4168-820f-5793085778d7\") " pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-p2l47" Jun 01 22:15:04.860410 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:15:04.860384 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/88603a8e-0273-4168-820f-5793085778d7-dshm\") pod \"facebook-opt-125m-simulated-kserve-59dcdf78f5-p2l47\" (UID: \"88603a8e-0273-4168-820f-5793085778d7\") " pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-p2l47" Jun 01 22:15:04.860788 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:15:04.860765 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/88603a8e-0273-4168-820f-5793085778d7-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-59dcdf78f5-p2l47\" (UID: \"88603a8e-0273-4168-820f-5793085778d7\") " pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-p2l47" Jun 01 22:15:04.866944 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:15:04.866886 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2sl4r\" (UniqueName: \"kubernetes.io/projected/88603a8e-0273-4168-820f-5793085778d7-kube-api-access-2sl4r\") pod \"facebook-opt-125m-simulated-kserve-59dcdf78f5-p2l47\" (UID: \"88603a8e-0273-4168-820f-5793085778d7\") " pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-p2l47" Jun 01 22:15:04.934391 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:15:04.934362 2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-p2l47" Jun 01 22:15:05.064641 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:15:05.064359 2573 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-p2l47"] Jun 01 22:15:05.067101 ip-10-0-138-241 kubenswrapper[2573]: W0601 22:15:05.067073 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod88603a8e_0273_4168_820f_5793085778d7.slice/crio-6260f6b0b0484dcf72b59a157cbafa4d7dd0824d7a6d9438e1ed96e28fa34049 WatchSource:0}: Error finding container 6260f6b0b0484dcf72b59a157cbafa4d7dd0824d7a6d9438e1ed96e28fa34049: Status 404 returned error can't find the container with id 6260f6b0b0484dcf72b59a157cbafa4d7dd0824d7a6d9438e1ed96e28fa34049 Jun 01 22:15:05.557384 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:15:05.557272 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-p2l47" event={"ID":"88603a8e-0273-4168-820f-5793085778d7","Type":"ContainerStarted","Data":"868ab0254d5f4b6c22a241aa6cb242c2a1335b3f5ed7553b08d6310659cd2edf"} Jun 01 22:15:05.557384 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:15:05.557324 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-p2l47" event={"ID":"88603a8e-0273-4168-820f-5793085778d7","Type":"ContainerStarted","Data":"6260f6b0b0484dcf72b59a157cbafa4d7dd0824d7a6d9438e1ed96e28fa34049"} Jun 01 22:15:05.558569 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:15:05.558519 2573 generic.go:358] "Generic (PLEG): container finished" podID="89bf1068-3455-4c94-b28a-b0ac05a796eb" containerID="e808632aa4468fea8a4e394395b9fbc15de273e66fe7e808a437d82c32db62fd" exitCode=0 Jun 01 22:15:05.558688 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:15:05.558569 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9bthpk" event={"ID":"89bf1068-3455-4c94-b28a-b0ac05a796eb","Type":"ContainerDied","Data":"e808632aa4468fea8a4e394395b9fbc15de273e66fe7e808a437d82c32db62fd"} Jun 01 22:15:05.922676 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:15:05.921166 2573 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-ddtwb"] Jun 01 22:15:06.566171 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:15:06.566138 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9bthpk" event={"ID":"89bf1068-3455-4c94-b28a-b0ac05a796eb","Type":"ContainerStarted","Data":"4884c151e0877c436ff67525de1a57b3960cf5339d682cb0de6fda045e517a6d"} Jun 01 22:15:06.567403 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:15:06.567376 2573 generic.go:358] "Generic (PLEG): container finished" podID="88603a8e-0273-4168-820f-5793085778d7" containerID="868ab0254d5f4b6c22a241aa6cb242c2a1335b3f5ed7553b08d6310659cd2edf" exitCode=0 Jun 01 22:15:06.567574 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:15:06.567455 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-p2l47" event={"ID":"88603a8e-0273-4168-820f-5793085778d7","Type":"ContainerDied","Data":"868ab0254d5f4b6c22a241aa6cb242c2a1335b3f5ed7553b08d6310659cd2edf"} Jun 01 22:15:06.586307 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:15:06.586250 2573 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9bthpk" podStartSLOduration=15.375151663 podStartE2EDuration="15.586233479s" podCreationTimestamp="2026-06-01 22:14:51 +0000 UTC" firstStartedPulling="2026-06-01 22:15:05.559126834 +0000 UTC m=+2065.279178755" lastFinishedPulling="2026-06-01 22:15:05.770208633 +0000 UTC m=+2065.490260571" observedRunningTime="2026-06-01 22:15:06.583393615 +0000 UTC m=+2066.303445559" watchObservedRunningTime="2026-06-01 22:15:06.586233479 +0000 UTC m=+2066.306285421" Jun 01 22:15:07.572390 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:15:07.572353 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-p2l47" event={"ID":"88603a8e-0273-4168-820f-5793085778d7","Type":"ContainerStarted","Data":"c9792a187d262e763e3875b6580858a442903b95751256ecac44c0dcb416afe4"} Jun 01 22:15:07.591106 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:15:07.591042 2573 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-p2l47" podStartSLOduration=3.379080251 podStartE2EDuration="3.591024949s" podCreationTimestamp="2026-06-01 22:15:04 +0000 UTC" firstStartedPulling="2026-06-01 22:15:06.56794503 +0000 UTC m=+2066.287996951" lastFinishedPulling="2026-06-01 22:15:06.779889724 +0000 UTC m=+2066.499941649" observedRunningTime="2026-06-01 22:15:07.589062243 +0000 UTC m=+2067.309114197" watchObservedRunningTime="2026-06-01 22:15:07.591024949 +0000 UTC m=+2067.311076892" Jun 01 22:15:10.820075 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:15:10.820034 2573 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-78c99df468-ddtwb"] Jun 01 22:15:12.054350 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:15:12.054303 2573 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9bthpk" Jun 01 22:15:12.054784 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:15:12.054363 2573 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9bthpk" Jun 01 22:15:12.067120 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:15:12.067094 2573 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9bthpk" Jun 01 22:15:12.599419 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:15:12.599385 2573 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9bthpk" Jun 01 22:15:14.934803 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:15:14.934760 2573 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-p2l47" Jun 01 22:15:14.935265 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:15:14.934815 2573 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-p2l47" Jun 01 22:15:14.947192 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:15:14.947162 2573 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-p2l47" Jun 01 22:15:15.609285 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:15:15.609255 2573 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-59dcdf78f5-p2l47" Jun 01 22:15:15.673849 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:15:15.673813 2573 scope.go:117] "RemoveContainer" containerID="30fc393a1f080ce4caf7caa270d2c3064e7ca9f46eb0f94c82e2acdd482f987f" Jun 01 22:15:16.602698 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:15:16.602661 2573 generic.go:358] "Generic (PLEG): container finished" podID="5211e3c4-f063-4d34-94aa-58dc3ae0c8b4" containerID="23f153157020acc675a6617f5646dcb0d417ad5d70a2454a1e9aceec560d3a45" exitCode=7 Jun 01 22:15:16.603112 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:15:16.602735 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29672535-jrz2z" event={"ID":"5211e3c4-f063-4d34-94aa-58dc3ae0c8b4","Type":"ContainerDied","Data":"23f153157020acc675a6617f5646dcb0d417ad5d70a2454a1e9aceec560d3a45"} Jun 01 22:15:16.603112 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:15:16.602782 2573 scope.go:117] "RemoveContainer" containerID="30fc393a1f080ce4caf7caa270d2c3064e7ca9f46eb0f94c82e2acdd482f987f" Jun 01 22:15:16.603196 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:15:16.603181 2573 scope.go:117] "RemoveContainer" containerID="23f153157020acc675a6617f5646dcb0d417ad5d70a2454a1e9aceec560d3a45" Jun 01 22:15:16.603427 ip-10-0-138-241 kubenswrapper[2573]: E0601 22:15:16.603409 2573 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-29672535-jrz2z_opendatahub(5211e3c4-f063-4d34-94aa-58dc3ae0c8b4)\"" pod="opendatahub/maas-api-key-cleanup-29672535-jrz2z" podUID="5211e3c4-f063-4d34-94aa-58dc3ae0c8b4" Jun 01 22:15:17.626684 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:15:17.626649 2573 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-key-cleanup-29672535-jrz2z"] Jun 01 22:15:17.755442 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:15:17.755414 2573 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29672535-jrz2z" Jun 01 22:15:17.872699 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:15:17.872660 2573 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hwm4j\" (UniqueName: \"kubernetes.io/projected/5211e3c4-f063-4d34-94aa-58dc3ae0c8b4-kube-api-access-hwm4j\") pod \"5211e3c4-f063-4d34-94aa-58dc3ae0c8b4\" (UID: \"5211e3c4-f063-4d34-94aa-58dc3ae0c8b4\") " Jun 01 22:15:17.874762 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:15:17.874728 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5211e3c4-f063-4d34-94aa-58dc3ae0c8b4-kube-api-access-hwm4j" (OuterVolumeSpecName: "kube-api-access-hwm4j") pod "5211e3c4-f063-4d34-94aa-58dc3ae0c8b4" (UID: "5211e3c4-f063-4d34-94aa-58dc3ae0c8b4"). InnerVolumeSpecName "kube-api-access-hwm4j". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 01 22:15:17.973965 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:15:17.973929 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hwm4j\" (UniqueName: \"kubernetes.io/projected/5211e3c4-f063-4d34-94aa-58dc3ae0c8b4-kube-api-access-hwm4j\") on node \"ip-10-0-138-241.ec2.internal\" DevicePath \"\"" Jun 01 22:15:18.612419 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:15:18.612390 2573 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29672535-jrz2z" Jun 01 22:15:18.612629 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:15:18.612388 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29672535-jrz2z" event={"ID":"5211e3c4-f063-4d34-94aa-58dc3ae0c8b4","Type":"ContainerDied","Data":"052b14b050c194d2cdd99c09e01f0dc774b4adfb252d8fbb2734cfb3b9294bc5"} Jun 01 22:15:18.612629 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:15:18.612517 2573 scope.go:117] "RemoveContainer" containerID="23f153157020acc675a6617f5646dcb0d417ad5d70a2454a1e9aceec560d3a45" Jun 01 22:15:18.632900 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:15:18.632870 2573 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-key-cleanup-29672535-jrz2z"] Jun 01 22:15:18.638215 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:15:18.638190 2573 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-key-cleanup-29672535-jrz2z"] Jun 01 22:15:18.685510 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:15:18.685465 2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5211e3c4-f063-4d34-94aa-58dc3ae0c8b4" path="/var/lib/kubelet/pods/5211e3c4-f063-4d34-94aa-58dc3ae0c8b4/volumes" Jun 01 22:15:40.753749 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:15:40.753720 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rcqkp_fcf0209a-2440-4267-834a-b176f4e337e4/ovn-acl-logging/0.log" Jun 01 22:15:40.756865 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:15:40.755675 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rcqkp_fcf0209a-2440-4267-834a-b176f4e337e4/ovn-acl-logging/0.log" Jun 01 22:20:40.776981 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:20:40.776862 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rcqkp_fcf0209a-2440-4267-834a-b176f4e337e4/ovn-acl-logging/0.log" Jun 01 22:20:40.780329 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:20:40.780307 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rcqkp_fcf0209a-2440-4267-834a-b176f4e337e4/ovn-acl-logging/0.log" Jun 01 22:20:59.901552 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:20:59.901509 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-85664567db-tvhd4_0d6e9812-bf0f-42ea-9db4-cd07b1c3d0f5/authorino/0.log" Jun 01 22:21:05.491195 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:21:05.491168 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5568c4bcd5-29s6k_d226ebd5-dfa6-4f73-a032-3f59142dc353/copy-manifests/0.log" Jun 01 22:21:05.505213 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:21:05.505189 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5568c4bcd5-29s6k_d226ebd5-dfa6-4f73-a032-3f59142dc353/manager/0.log" Jun 01 22:21:07.294301 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:21:07.294270 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-85664567db-tvhd4_0d6e9812-bf0f-42ea-9db4-cd07b1c3d0f5/authorino/0.log" Jun 01 22:21:07.662236 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:21:07.662212 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-6cb54b5c86-4kdq7_afbcd9bd-2c42-406e-ae2a-68ba6ed4f478/kuadrant-console-plugin/0.log" Jun 01 22:21:08.041640 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:21:08.041522 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-78c99df468-ddtwb_716c4733-0b19-48f3-ba2c-4b6e6d09cdd6/limitador/0.log" Jun 01 22:21:08.763828 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:21:08.763799 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-67dd84b75b-ppsmf_613a1517-3530-4b64-abc3-ea3c2e095200/kube-auth-proxy/0.log" Jun 01 22:21:09.712872 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:21:09.712846 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-5489f4f4bc-m6dlt_5eef219b-e796-4aa9-bf30-50a9f4f56e7b/storage-initializer/0.log" Jun 01 22:21:09.720524 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:21:09.720501 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-5489f4f4bc-m6dlt_5eef219b-e796-4aa9-bf30-50a9f4f56e7b/main/0.log" Jun 01 22:21:09.974912 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:21:09.974837 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9bthpk_89bf1068-3455-4c94-b28a-b0ac05a796eb/storage-initializer/0.log" Jun 01 22:21:09.988555 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:21:09.988508 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9bthpk_89bf1068-3455-4c94-b28a-b0ac05a796eb/main/0.log" Jun 01 22:21:10.110617 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:21:10.110585 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-59dcdf78f5-p2l47_88603a8e-0273-4168-820f-5793085778d7/main/0.log" Jun 01 22:21:10.124096 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:21:10.124066 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-59dcdf78f5-p2l47_88603a8e-0273-4168-820f-5793085778d7/storage-initializer/0.log" Jun 01 22:21:14.565514 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:21:14.565478 2573 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-must-gather-q9gp2/must-gather-j9ckd"] Jun 01 22:21:14.565991 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:21:14.565778 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5211e3c4-f063-4d34-94aa-58dc3ae0c8b4" containerName="cleanup" Jun 01 22:21:14.565991 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:21:14.565788 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="5211e3c4-f063-4d34-94aa-58dc3ae0c8b4" containerName="cleanup" Jun 01 22:21:14.565991 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:21:14.565804 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5211e3c4-f063-4d34-94aa-58dc3ae0c8b4" containerName="cleanup" Jun 01 22:21:14.565991 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:21:14.565809 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="5211e3c4-f063-4d34-94aa-58dc3ae0c8b4" containerName="cleanup" Jun 01 22:21:14.565991 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:21:14.565858 2573 memory_manager.go:356] "RemoveStaleState removing state" podUID="5211e3c4-f063-4d34-94aa-58dc3ae0c8b4" containerName="cleanup" Jun 01 22:21:14.565991 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:21:14.565866 2573 memory_manager.go:356] "RemoveStaleState removing state" podUID="5211e3c4-f063-4d34-94aa-58dc3ae0c8b4" containerName="cleanup" Jun 01 22:21:14.565991 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:21:14.565921 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5211e3c4-f063-4d34-94aa-58dc3ae0c8b4" containerName="cleanup" Jun 01 22:21:14.565991 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:21:14.565927 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="5211e3c4-f063-4d34-94aa-58dc3ae0c8b4" containerName="cleanup" Jun 01 22:21:14.565991 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:21:14.565969 2573 memory_manager.go:356] "RemoveStaleState removing state" podUID="5211e3c4-f063-4d34-94aa-58dc3ae0c8b4" containerName="cleanup" Jun 01 22:21:14.568718 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:21:14.568700 2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-q9gp2/must-gather-j9ckd" Jun 01 22:21:14.575130 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:21:14.574812 2573 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-q9gp2\"/\"kube-root-ca.crt\"" Jun 01 22:21:14.575130 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:21:14.574818 2573 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-q9gp2\"/\"default-dockercfg-5t7cj\"" Jun 01 22:21:14.576560 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:21:14.576528 2573 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-q9gp2\"/\"openshift-service-ca.crt\"" Jun 01 22:21:14.592968 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:21:14.592939 2573 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-q9gp2/must-gather-j9ckd"] Jun 01 22:21:14.600854 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:21:14.600828 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gdb6k\" (UniqueName: \"kubernetes.io/projected/a56b6ed1-72c2-4ca4-a97f-72c3c90c1134-kube-api-access-gdb6k\") pod \"must-gather-j9ckd\" (UID: \"a56b6ed1-72c2-4ca4-a97f-72c3c90c1134\") " pod="openshift-must-gather-q9gp2/must-gather-j9ckd" Jun 01 22:21:14.601025 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:21:14.600873 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/a56b6ed1-72c2-4ca4-a97f-72c3c90c1134-must-gather-output\") pod \"must-gather-j9ckd\" (UID: \"a56b6ed1-72c2-4ca4-a97f-72c3c90c1134\") " pod="openshift-must-gather-q9gp2/must-gather-j9ckd" Jun 01 22:21:14.701439 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:21:14.701405 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/a56b6ed1-72c2-4ca4-a97f-72c3c90c1134-must-gather-output\") pod \"must-gather-j9ckd\" (UID: \"a56b6ed1-72c2-4ca4-a97f-72c3c90c1134\") " pod="openshift-must-gather-q9gp2/must-gather-j9ckd" Jun 01 22:21:14.701664 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:21:14.701486 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-gdb6k\" (UniqueName: \"kubernetes.io/projected/a56b6ed1-72c2-4ca4-a97f-72c3c90c1134-kube-api-access-gdb6k\") pod \"must-gather-j9ckd\" (UID: \"a56b6ed1-72c2-4ca4-a97f-72c3c90c1134\") " pod="openshift-must-gather-q9gp2/must-gather-j9ckd" Jun 01 22:21:14.701878 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:21:14.701856 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/a56b6ed1-72c2-4ca4-a97f-72c3c90c1134-must-gather-output\") pod \"must-gather-j9ckd\" (UID: \"a56b6ed1-72c2-4ca4-a97f-72c3c90c1134\") " pod="openshift-must-gather-q9gp2/must-gather-j9ckd" Jun 01 22:21:14.712718 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:21:14.712694 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gdb6k\" (UniqueName: \"kubernetes.io/projected/a56b6ed1-72c2-4ca4-a97f-72c3c90c1134-kube-api-access-gdb6k\") pod \"must-gather-j9ckd\" (UID: \"a56b6ed1-72c2-4ca4-a97f-72c3c90c1134\") " pod="openshift-must-gather-q9gp2/must-gather-j9ckd" Jun 01 22:21:14.878356 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:21:14.878260 2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-q9gp2/must-gather-j9ckd" Jun 01 22:21:15.018987 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:21:15.018958 2573 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-q9gp2/must-gather-j9ckd"] Jun 01 22:21:15.021004 ip-10-0-138-241 kubenswrapper[2573]: W0601 22:21:15.020978 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda56b6ed1_72c2_4ca4_a97f_72c3c90c1134.slice/crio-e9d593dc4501362452822710dcaa53e2a721221388315bef4832bc2e5708d442 WatchSource:0}: Error finding container e9d593dc4501362452822710dcaa53e2a721221388315bef4832bc2e5708d442: Status 404 returned error can't find the container with id e9d593dc4501362452822710dcaa53e2a721221388315bef4832bc2e5708d442 Jun 01 22:21:15.022680 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:21:15.022663 2573 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jun 01 22:21:15.747935 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:21:15.747896 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-q9gp2/must-gather-j9ckd" event={"ID":"a56b6ed1-72c2-4ca4-a97f-72c3c90c1134","Type":"ContainerStarted","Data":"e9d593dc4501362452822710dcaa53e2a721221388315bef4832bc2e5708d442"} Jun 01 22:21:16.754064 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:21:16.754026 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-q9gp2/must-gather-j9ckd" event={"ID":"a56b6ed1-72c2-4ca4-a97f-72c3c90c1134","Type":"ContainerStarted","Data":"ca5e9c6e69a603e5ba1fc67b38a1b96980a544c665b73e5ea35c857a253190d1"} Jun 01 22:21:16.754649 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:21:16.754623 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-q9gp2/must-gather-j9ckd" event={"ID":"a56b6ed1-72c2-4ca4-a97f-72c3c90c1134","Type":"ContainerStarted","Data":"47df08613e849b5740482d4f437dc1346ac5b02f17445b48f0aec8036ded68b5"} Jun 01 22:21:16.785725 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:21:16.785661 2573 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-must-gather-q9gp2/must-gather-j9ckd" podStartSLOduration=2.037256849 podStartE2EDuration="2.78564047s" podCreationTimestamp="2026-06-01 22:21:14 +0000 UTC" firstStartedPulling="2026-06-01 22:21:15.022791459 +0000 UTC m=+2434.742843380" lastFinishedPulling="2026-06-01 22:21:15.77117508 +0000 UTC m=+2435.491227001" observedRunningTime="2026-06-01 22:21:16.783926084 +0000 UTC m=+2436.503978032" watchObservedRunningTime="2026-06-01 22:21:16.78564047 +0000 UTC m=+2436.505692414" Jun 01 22:21:18.033835 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:21:18.033797 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-hnntt_2353b916-e500-4061-adcf-167688af93f0/global-pull-secret-syncer/0.log" Jun 01 22:21:18.330176 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:21:18.330146 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-mwdhn_9a3c0ed4-56c7-4f73-a903-7b8fe4652a41/konnectivity-agent/0.log" Jun 01 22:21:18.549483 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:21:18.549449 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-138-241.ec2.internal_bbf2997371dfbc83d81e28332ffdac56/haproxy/0.log" Jun 01 22:21:23.068297 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:21:23.068259 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-85664567db-tvhd4_0d6e9812-bf0f-42ea-9db4-cd07b1c3d0f5/authorino/0.log" Jun 01 22:21:23.527209 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:21:23.527129 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-6cb54b5c86-4kdq7_afbcd9bd-2c42-406e-ae2a-68ba6ed4f478/kuadrant-console-plugin/0.log" Jun 01 22:21:23.691482 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:21:23.691452 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-78c99df468-ddtwb_716c4733-0b19-48f3-ba2c-4b6e6d09cdd6/limitador/0.log" Jun 01 22:21:26.228649 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:21:26.228609 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-jbmfp_1b742974-2d33-4c56-9778-341b869d773e/node-exporter/0.log" Jun 01 22:21:26.277175 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:21:26.277143 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-jbmfp_1b742974-2d33-4c56-9778-341b869d773e/kube-rbac-proxy/0.log" Jun 01 22:21:26.349547 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:21:26.349510 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-jbmfp_1b742974-2d33-4c56-9778-341b869d773e/init-textfile/0.log" Jun 01 22:21:27.092642 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:21:27.092600 2573 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-must-gather-q9gp2/perf-node-gather-daemonset-g89tg"] Jun 01 22:21:27.098008 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:21:27.097976 2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-q9gp2/perf-node-gather-daemonset-g89tg" Jun 01 22:21:27.113981 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:21:27.113930 2573 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-q9gp2/perf-node-gather-daemonset-g89tg"] Jun 01 22:21:27.226070 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:21:27.226033 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/745890e0-eee4-4321-96d7-dbef62d156f9-proc\") pod \"perf-node-gather-daemonset-g89tg\" (UID: \"745890e0-eee4-4321-96d7-dbef62d156f9\") " pod="openshift-must-gather-q9gp2/perf-node-gather-daemonset-g89tg" Jun 01 22:21:27.226239 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:21:27.226082 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/745890e0-eee4-4321-96d7-dbef62d156f9-podres\") pod \"perf-node-gather-daemonset-g89tg\" (UID: \"745890e0-eee4-4321-96d7-dbef62d156f9\") " pod="openshift-must-gather-q9gp2/perf-node-gather-daemonset-g89tg" Jun 01 22:21:27.226239 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:21:27.226158 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/745890e0-eee4-4321-96d7-dbef62d156f9-sys\") pod \"perf-node-gather-daemonset-g89tg\" (UID: \"745890e0-eee4-4321-96d7-dbef62d156f9\") " pod="openshift-must-gather-q9gp2/perf-node-gather-daemonset-g89tg" Jun 01 22:21:27.226239 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:21:27.226230 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/745890e0-eee4-4321-96d7-dbef62d156f9-lib-modules\") pod \"perf-node-gather-daemonset-g89tg\" (UID: \"745890e0-eee4-4321-96d7-dbef62d156f9\") " pod="openshift-must-gather-q9gp2/perf-node-gather-daemonset-g89tg" Jun 01 22:21:27.226349 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:21:27.226260 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kvnvb\" (UniqueName: \"kubernetes.io/projected/745890e0-eee4-4321-96d7-dbef62d156f9-kube-api-access-kvnvb\") pod \"perf-node-gather-daemonset-g89tg\" (UID: \"745890e0-eee4-4321-96d7-dbef62d156f9\") " pod="openshift-must-gather-q9gp2/perf-node-gather-daemonset-g89tg" Jun 01 22:21:27.327551 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:21:27.327511 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/745890e0-eee4-4321-96d7-dbef62d156f9-sys\") pod \"perf-node-gather-daemonset-g89tg\" (UID: \"745890e0-eee4-4321-96d7-dbef62d156f9\") " pod="openshift-must-gather-q9gp2/perf-node-gather-daemonset-g89tg" Jun 01 22:21:27.327942 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:21:27.327597 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/745890e0-eee4-4321-96d7-dbef62d156f9-lib-modules\") pod \"perf-node-gather-daemonset-g89tg\" (UID: \"745890e0-eee4-4321-96d7-dbef62d156f9\") " pod="openshift-must-gather-q9gp2/perf-node-gather-daemonset-g89tg" Jun 01 22:21:27.327942 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:21:27.327620 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-kvnvb\" (UniqueName: \"kubernetes.io/projected/745890e0-eee4-4321-96d7-dbef62d156f9-kube-api-access-kvnvb\") pod \"perf-node-gather-daemonset-g89tg\" (UID: \"745890e0-eee4-4321-96d7-dbef62d156f9\") " pod="openshift-must-gather-q9gp2/perf-node-gather-daemonset-g89tg" Jun 01 22:21:27.327942 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:21:27.327632 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/745890e0-eee4-4321-96d7-dbef62d156f9-sys\") pod \"perf-node-gather-daemonset-g89tg\" (UID: \"745890e0-eee4-4321-96d7-dbef62d156f9\") " pod="openshift-must-gather-q9gp2/perf-node-gather-daemonset-g89tg" Jun 01 22:21:27.327942 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:21:27.327660 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/745890e0-eee4-4321-96d7-dbef62d156f9-proc\") pod \"perf-node-gather-daemonset-g89tg\" (UID: \"745890e0-eee4-4321-96d7-dbef62d156f9\") " pod="openshift-must-gather-q9gp2/perf-node-gather-daemonset-g89tg" Jun 01 22:21:27.327942 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:21:27.327695 2573 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/745890e0-eee4-4321-96d7-dbef62d156f9-podres\") pod \"perf-node-gather-daemonset-g89tg\" (UID: \"745890e0-eee4-4321-96d7-dbef62d156f9\") " pod="openshift-must-gather-q9gp2/perf-node-gather-daemonset-g89tg" Jun 01 22:21:27.327942 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:21:27.327804 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/745890e0-eee4-4321-96d7-dbef62d156f9-proc\") pod \"perf-node-gather-daemonset-g89tg\" (UID: \"745890e0-eee4-4321-96d7-dbef62d156f9\") " pod="openshift-must-gather-q9gp2/perf-node-gather-daemonset-g89tg" Jun 01 22:21:27.327942 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:21:27.327808 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/745890e0-eee4-4321-96d7-dbef62d156f9-lib-modules\") pod \"perf-node-gather-daemonset-g89tg\" (UID: \"745890e0-eee4-4321-96d7-dbef62d156f9\") " pod="openshift-must-gather-q9gp2/perf-node-gather-daemonset-g89tg" Jun 01 22:21:27.327942 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:21:27.327806 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/745890e0-eee4-4321-96d7-dbef62d156f9-podres\") pod \"perf-node-gather-daemonset-g89tg\" (UID: \"745890e0-eee4-4321-96d7-dbef62d156f9\") " pod="openshift-must-gather-q9gp2/perf-node-gather-daemonset-g89tg" Jun 01 22:21:27.347006 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:21:27.346931 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kvnvb\" (UniqueName: \"kubernetes.io/projected/745890e0-eee4-4321-96d7-dbef62d156f9-kube-api-access-kvnvb\") pod \"perf-node-gather-daemonset-g89tg\" (UID: \"745890e0-eee4-4321-96d7-dbef62d156f9\") " pod="openshift-must-gather-q9gp2/perf-node-gather-daemonset-g89tg" Jun 01 22:21:27.411605 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:21:27.411566 2573 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-q9gp2/perf-node-gather-daemonset-g89tg" Jun 01 22:21:27.442481 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:21:27.442450 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-7d9f684f78-cjls7_4883866e-975d-4eb8-8d98-062ea8cd2a5a/telemeter-client/0.log" Jun 01 22:21:27.488745 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:21:27.488718 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-7d9f684f78-cjls7_4883866e-975d-4eb8-8d98-062ea8cd2a5a/reload/0.log" Jun 01 22:21:27.556591 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:21:27.556556 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-7d9f684f78-cjls7_4883866e-975d-4eb8-8d98-062ea8cd2a5a/kube-rbac-proxy/0.log" Jun 01 22:21:27.576142 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:21:27.576040 2573 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-q9gp2/perf-node-gather-daemonset-g89tg"] Jun 01 22:21:27.578456 ip-10-0-138-241 kubenswrapper[2573]: W0601 22:21:27.578425 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod745890e0_eee4_4321_96d7_dbef62d156f9.slice/crio-fa892bd7b91cccd32218451749daf4e7dd6c91e912621f8b5af73a671ded26da WatchSource:0}: Error finding container fa892bd7b91cccd32218451749daf4e7dd6c91e912621f8b5af73a671ded26da: Status 404 returned error can't find the container with id fa892bd7b91cccd32218451749daf4e7dd6c91e912621f8b5af73a671ded26da Jun 01 22:21:27.799294 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:21:27.799262 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-q9gp2/perf-node-gather-daemonset-g89tg" event={"ID":"745890e0-eee4-4321-96d7-dbef62d156f9","Type":"ContainerStarted","Data":"1102bbbeeda12ad24777d09ef744a9c4f053fe8608e8c7aaf7b7d06bcb30a2c6"} Jun 01 22:21:27.799463 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:21:27.799300 2573 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-q9gp2/perf-node-gather-daemonset-g89tg" event={"ID":"745890e0-eee4-4321-96d7-dbef62d156f9","Type":"ContainerStarted","Data":"fa892bd7b91cccd32218451749daf4e7dd6c91e912621f8b5af73a671ded26da"} Jun 01 22:21:27.799463 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:21:27.799384 2573 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-q9gp2/perf-node-gather-daemonset-g89tg" Jun 01 22:21:27.836841 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:21:27.836767 2573 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-must-gather-q9gp2/perf-node-gather-daemonset-g89tg" podStartSLOduration=0.836747768 podStartE2EDuration="836.747768ms" podCreationTimestamp="2026-06-01 22:21:27 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-06-01 22:21:27.828125084 +0000 UTC m=+2447.548177026" watchObservedRunningTime="2026-06-01 22:21:27.836747768 +0000 UTC m=+2447.556799710" Jun 01 22:21:31.568070 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:21:31.568044 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-tnd8z_b57ae13c-b94a-4891-b4d2-0bd1a8873c4f/dns/0.log" Jun 01 22:21:31.591734 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:21:31.591706 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-tnd8z_b57ae13c-b94a-4891-b4d2-0bd1a8873c4f/kube-rbac-proxy/0.log" Jun 01 22:21:31.680630 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:21:31.680586 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-kfs2m_bcc44458-cf91-4d2d-9b1b-e9da2db57098/dns-node-resolver/0.log" Jun 01 22:21:32.226012 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:21:32.225986 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-qcs4d_7ada00c1-dbeb-4df5-bfca-923ef619c2d1/node-ca/0.log" Jun 01 22:21:33.167066 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:21:33.167037 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-67dd84b75b-ppsmf_613a1517-3530-4b64-abc3-ea3c2e095200/kube-auth-proxy/0.log" Jun 01 22:21:33.808934 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:21:33.808897 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-pdmsk_81b4edc5-d95f-4cae-9191-e27f75e48d8a/serve-healthcheck-canary/0.log" Jun 01 22:21:33.813690 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:21:33.813664 2573 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-q9gp2/perf-node-gather-daemonset-g89tg" Jun 01 22:21:34.459616 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:21:34.459581 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-7lrz6_9257de6e-8c40-4471-82e8-f6fd9d898198/kube-rbac-proxy/0.log" Jun 01 22:21:34.493421 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:21:34.493396 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-7lrz6_9257de6e-8c40-4471-82e8-f6fd9d898198/exporter/0.log" Jun 01 22:21:34.532198 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:21:34.532165 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-7lrz6_9257de6e-8c40-4471-82e8-f6fd9d898198/extractor/0.log" Jun 01 22:21:37.195881 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:21:37.195851 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5568c4bcd5-29s6k_d226ebd5-dfa6-4f73-a032-3f59142dc353/manager/0.log" Jun 01 22:21:37.215516 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:21:37.215486 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5568c4bcd5-29s6k_d226ebd5-dfa6-4f73-a032-3f59142dc353/copy-manifests/0.log" Jun 01 22:21:38.847435 ip-10-0-138-241 kubenswrapper[2573]: I0601 22:21:38.847409 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-64fd4455d7-nqvzv_f6942086-3c8d-4f2e-8d27-1329e670b4ad/manager/0.log"