Jun 03 16:17:08.788826 ip-10-0-128-88 systemd[1]: Starting Kubernetes Kubelet... Jun 03 16:17:09.268616 ip-10-0-128-88 kubenswrapper[2568]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jun 03 16:17:09.268616 ip-10-0-128-88 kubenswrapper[2568]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jun 03 16:17:09.268616 ip-10-0-128-88 kubenswrapper[2568]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jun 03 16:17:09.268616 ip-10-0-128-88 kubenswrapper[2568]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jun 03 16:17:09.268616 ip-10-0-128-88 kubenswrapper[2568]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jun 03 16:17:09.273274 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:09.273184 2568 server.go:212] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jun 03 16:17:09.277905 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.277891 2568 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 03 16:17:09.277905 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.277905 2568 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController Jun 03 16:17:09.277974 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.277909 2568 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 03 16:17:09.277974 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.277913 2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 03 16:17:09.277974 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.277916 2568 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 03 16:17:09.277974 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.277919 2568 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 03 16:17:09.277974 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.277924 2568 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 03 16:17:09.277974 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.277929 2568 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 03 16:17:09.277974 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.277932 2568 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations Jun 03 16:17:09.277974 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.277935 2568 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 03 16:17:09.277974 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.277938 2568 feature_gate.go:328] unrecognized feature gate: Example Jun 03 16:17:09.277974 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.277948 2568 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 03 16:17:09.277974 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.277951 2568 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 03 16:17:09.277974 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.277954 2568 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 03 16:17:09.277974 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.277956 2568 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 03 16:17:09.277974 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.277959 2568 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 03 16:17:09.277974 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.277961 2568 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 03 16:17:09.277974 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.277964 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 03 16:17:09.277974 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.277967 2568 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 03 16:17:09.277974 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.277969 2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 03 16:17:09.277974 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.277972 2568 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 03 16:17:09.277974 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.277975 2568 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 03 16:17:09.278447 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.277977 2568 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 03 16:17:09.278447 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.277980 2568 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 03 16:17:09.278447 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.277983 2568 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 03 16:17:09.278447 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.277986 2568 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 03 16:17:09.278447 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.277988 2568 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 03 16:17:09.278447 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.277991 2568 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 03 16:17:09.278447 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.277993 2568 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS Jun 03 16:17:09.278447 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.277996 2568 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 03 16:17:09.278447 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.277998 2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 03 16:17:09.278447 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.278001 2568 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 03 16:17:09.278447 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.278003 2568 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 03 16:17:09.278447 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.278005 2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 03 16:17:09.278447 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.278008 2568 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 03 16:17:09.278447 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.278011 2568 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 03 16:17:09.278447 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.278013 2568 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 03 16:17:09.278447 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.278015 2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 03 16:17:09.278447 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.278018 2568 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI Jun 03 16:17:09.278447 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.278021 2568 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 03 16:17:09.278447 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.278023 2568 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 03 16:17:09.278447 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.278025 2568 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 03 16:17:09.278949 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.278028 2568 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 03 16:17:09.278949 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.278030 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 03 16:17:09.278949 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.278033 2568 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 03 16:17:09.278949 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.278035 2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 03 16:17:09.278949 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.278038 2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 03 16:17:09.278949 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.278040 2568 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 03 16:17:09.278949 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.278044 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 03 16:17:09.278949 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.278047 2568 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 03 16:17:09.278949 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.278049 2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 03 16:17:09.278949 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.278051 2568 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 03 16:17:09.278949 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.278054 2568 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 03 16:17:09.278949 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.278056 2568 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 03 16:17:09.278949 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.278059 2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 03 16:17:09.278949 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.278062 2568 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 03 16:17:09.278949 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.278065 2568 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 03 16:17:09.278949 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.278068 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 03 16:17:09.278949 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.278070 2568 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 03 16:17:09.278949 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.278073 2568 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 03 16:17:09.278949 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.278076 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 03 16:17:09.278949 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.278078 2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 03 16:17:09.279427 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.278081 2568 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 03 16:17:09.279427 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.278083 2568 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 03 16:17:09.279427 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.278086 2568 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 03 16:17:09.279427 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.278089 2568 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 03 16:17:09.279427 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.278091 2568 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 03 16:17:09.279427 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.278094 2568 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 03 16:17:09.279427 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.278096 2568 feature_gate.go:328] unrecognized feature gate: Example2 Jun 03 16:17:09.279427 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.278099 2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 03 16:17:09.279427 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.278103 2568 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 03 16:17:09.279427 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.278106 2568 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 03 16:17:09.279427 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.278108 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 03 16:17:09.279427 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.278110 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 03 16:17:09.279427 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.278113 2568 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 03 16:17:09.279427 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.278116 2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 03 16:17:09.279427 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.278118 2568 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 03 16:17:09.279427 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.278122 2568 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 03 16:17:09.279427 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.278126 2568 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 03 16:17:09.279427 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.278128 2568 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 03 16:17:09.279427 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.278131 2568 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 03 16:17:09.279896 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.278134 2568 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 03 16:17:09.279896 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.278137 2568 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 03 16:17:09.279896 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.278139 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 03 16:17:09.279896 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.278142 2568 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 03 16:17:09.279896 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.278144 2568 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 03 16:17:09.281122 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.281110 2568 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 03 16:17:09.281122 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.281122 2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 03 16:17:09.281184 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.281125 2568 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 03 16:17:09.281184 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.281129 2568 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController Jun 03 16:17:09.281184 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.281132 2568 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 03 16:17:09.281184 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.281134 2568 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 03 16:17:09.281184 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.281137 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 03 16:17:09.281184 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.281140 2568 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 03 16:17:09.281184 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.281142 2568 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 03 16:17:09.281184 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.281145 2568 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 03 16:17:09.281184 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.281148 2568 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 03 16:17:09.281184 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.281151 2568 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 03 16:17:09.281184 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.281154 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 03 16:17:09.281184 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.281156 2568 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 03 16:17:09.281184 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.281159 2568 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 03 16:17:09.281184 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.281161 2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 03 16:17:09.281184 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.281164 2568 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 03 16:17:09.281184 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.281166 2568 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 03 16:17:09.281184 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.281169 2568 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 03 16:17:09.281184 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.281172 2568 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 03 16:17:09.281184 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.281175 2568 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 03 16:17:09.281184 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.281179 2568 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 03 16:17:09.281675 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.281182 2568 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 03 16:17:09.281675 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.281185 2568 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 03 16:17:09.281675 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.281188 2568 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 03 16:17:09.281675 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.281190 2568 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 03 16:17:09.281675 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.281194 2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 03 16:17:09.281675 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.281197 2568 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 03 16:17:09.281675 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.281200 2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 03 16:17:09.281675 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.281202 2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 03 16:17:09.281675 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.281204 2568 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 03 16:17:09.281675 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.281207 2568 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 03 16:17:09.281675 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.281209 2568 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 03 16:17:09.281675 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.281212 2568 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 03 16:17:09.281675 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.281214 2568 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 03 16:17:09.281675 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.281217 2568 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 03 16:17:09.281675 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.281219 2568 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 03 16:17:09.281675 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.281222 2568 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 03 16:17:09.281675 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.281224 2568 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 03 16:17:09.281675 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.281226 2568 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 03 16:17:09.281675 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.281229 2568 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 03 16:17:09.282142 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.281231 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 03 16:17:09.282142 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.281234 2568 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 03 16:17:09.282142 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.281237 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 03 16:17:09.282142 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.281239 2568 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 03 16:17:09.282142 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.281242 2568 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 03 16:17:09.282142 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.281244 2568 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 03 16:17:09.282142 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.281247 2568 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 03 16:17:09.282142 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.281249 2568 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 03 16:17:09.282142 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.281252 2568 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 03 16:17:09.282142 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.281254 2568 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 03 16:17:09.282142 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.281257 2568 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 03 16:17:09.282142 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.281259 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 03 16:17:09.282142 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.281262 2568 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS Jun 03 16:17:09.282142 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.281264 2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 03 16:17:09.282142 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.281268 2568 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 03 16:17:09.282142 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.281270 2568 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 03 16:17:09.282142 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.281273 2568 feature_gate.go:328] unrecognized feature gate: Example2 Jun 03 16:17:09.282142 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.281275 2568 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 03 16:17:09.282142 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.281278 2568 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 03 16:17:09.282142 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.281280 2568 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 03 16:17:09.282646 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.281283 2568 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 03 16:17:09.282646 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.281285 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 03 16:17:09.282646 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.281288 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 03 16:17:09.282646 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.281290 2568 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations Jun 03 16:17:09.282646 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.281292 2568 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 03 16:17:09.282646 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.281295 2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 03 16:17:09.282646 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.281298 2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 03 16:17:09.282646 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.281300 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 03 16:17:09.282646 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.281302 2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 03 16:17:09.282646 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.281305 2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 03 16:17:09.282646 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.281307 2568 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 03 16:17:09.282646 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.281310 2568 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 03 16:17:09.282646 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.281312 2568 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI Jun 03 16:17:09.282646 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.281315 2568 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 03 16:17:09.282646 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.281317 2568 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 03 16:17:09.282646 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.281321 2568 feature_gate.go:328] unrecognized feature gate: Example Jun 03 16:17:09.282646 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.281323 2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 03 16:17:09.282646 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.281326 2568 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 03 16:17:09.282646 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.281328 2568 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 03 16:17:09.282646 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.281331 2568 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 03 16:17:09.283160 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.281333 2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 03 16:17:09.283160 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.281336 2568 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 03 16:17:09.283160 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.281338 2568 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 03 16:17:09.283160 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.281341 2568 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 03 16:17:09.283160 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.281343 2568 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 03 16:17:09.283160 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:09.281424 2568 flags.go:64] FLAG: --address="0.0.0.0" Jun 03 16:17:09.283160 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:09.281432 2568 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jun 03 16:17:09.283160 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:09.281442 2568 flags.go:64] FLAG: --anonymous-auth="true" Jun 03 16:17:09.283160 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:09.281447 2568 flags.go:64] FLAG: --application-metrics-count-limit="100" Jun 03 16:17:09.283160 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:09.281451 2568 flags.go:64] FLAG: --authentication-token-webhook="false" Jun 03 16:17:09.283160 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:09.281454 2568 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jun 03 16:17:09.283160 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:09.281458 2568 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jun 03 16:17:09.283160 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:09.281463 2568 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jun 03 16:17:09.283160 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:09.281466 2568 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jun 03 16:17:09.283160 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:09.281469 2568 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jun 03 16:17:09.283160 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:09.281473 2568 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jun 03 16:17:09.283160 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:09.281476 2568 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jun 03 16:17:09.283160 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:09.281479 2568 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jun 03 16:17:09.283160 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:09.281482 2568 flags.go:64] FLAG: --cgroup-root="" Jun 03 16:17:09.283160 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:09.281485 2568 flags.go:64] FLAG: --cgroups-per-qos="true" Jun 03 16:17:09.283160 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:09.281488 2568 flags.go:64] FLAG: --client-ca-file="" Jun 03 16:17:09.283160 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:09.281491 2568 flags.go:64] FLAG: --cloud-config="" Jun 03 16:17:09.283160 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:09.281493 2568 flags.go:64] FLAG: --cloud-provider="external" Jun 03 16:17:09.283804 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:09.281496 2568 flags.go:64] FLAG: --cluster-dns="[]" Jun 03 16:17:09.283804 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:09.281501 2568 flags.go:64] FLAG: --cluster-domain="" Jun 03 16:17:09.283804 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:09.281504 2568 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jun 03 16:17:09.283804 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:09.281507 2568 flags.go:64] FLAG: --config-dir="" Jun 03 16:17:09.283804 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:09.281510 2568 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jun 03 16:17:09.283804 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:09.281514 2568 flags.go:64] FLAG: --container-log-max-files="5" Jun 03 16:17:09.283804 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:09.281518 2568 flags.go:64] FLAG: --container-log-max-size="10Mi" Jun 03 16:17:09.283804 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:09.281521 2568 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jun 03 16:17:09.283804 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:09.281524 2568 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jun 03 16:17:09.283804 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:09.281528 2568 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jun 03 16:17:09.283804 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:09.281530 2568 flags.go:64] FLAG: --contention-profiling="false" Jun 03 16:17:09.283804 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:09.281549 2568 flags.go:64] FLAG: --cpu-cfs-quota="true" Jun 03 16:17:09.283804 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:09.281552 2568 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jun 03 16:17:09.283804 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:09.281555 2568 flags.go:64] FLAG: --cpu-manager-policy="none" Jun 03 16:17:09.283804 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:09.281558 2568 flags.go:64] FLAG: --cpu-manager-policy-options="" Jun 03 16:17:09.283804 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:09.281562 2568 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jun 03 16:17:09.283804 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:09.281565 2568 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jun 03 16:17:09.283804 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:09.281568 2568 flags.go:64] FLAG: --enable-debugging-handlers="true" Jun 03 16:17:09.283804 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:09.281572 2568 flags.go:64] FLAG: --enable-load-reader="false" Jun 03 16:17:09.283804 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:09.281575 2568 flags.go:64] FLAG: --enable-server="true" Jun 03 16:17:09.283804 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:09.281578 2568 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jun 03 16:17:09.283804 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:09.281582 2568 flags.go:64] FLAG: --event-burst="100" Jun 03 16:17:09.283804 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:09.281585 2568 flags.go:64] FLAG: --event-qps="50" Jun 03 16:17:09.283804 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:09.281588 2568 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jun 03 16:17:09.283804 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:09.281591 2568 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jun 03 16:17:09.284404 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:09.281594 2568 flags.go:64] FLAG: --eviction-hard="" Jun 03 16:17:09.284404 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:09.281597 2568 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jun 03 16:17:09.284404 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:09.281600 2568 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jun 03 16:17:09.284404 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:09.281603 2568 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jun 03 16:17:09.284404 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:09.281606 2568 flags.go:64] FLAG: --eviction-soft="" Jun 03 16:17:09.284404 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:09.281609 2568 flags.go:64] FLAG: --eviction-soft-grace-period="" Jun 03 16:17:09.284404 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:09.281612 2568 flags.go:64] FLAG: --exit-on-lock-contention="false" Jun 03 16:17:09.284404 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:09.281615 2568 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jun 03 16:17:09.284404 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:09.281618 2568 flags.go:64] FLAG: --experimental-mounter-path="" Jun 03 16:17:09.284404 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:09.281621 2568 flags.go:64] FLAG: --fail-cgroupv1="false" Jun 03 16:17:09.284404 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:09.281624 2568 flags.go:64] FLAG: --fail-swap-on="true" Jun 03 16:17:09.284404 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:09.281626 2568 flags.go:64] FLAG: --feature-gates="" Jun 03 16:17:09.284404 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:09.281630 2568 flags.go:64] FLAG: --file-check-frequency="20s" Jun 03 16:17:09.284404 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:09.281637 2568 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jun 03 16:17:09.284404 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:09.281640 2568 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jun 03 16:17:09.284404 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:09.281643 2568 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jun 03 16:17:09.284404 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:09.281646 2568 flags.go:64] FLAG: --healthz-port="10248" Jun 03 16:17:09.284404 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:09.281649 2568 flags.go:64] FLAG: --help="false" Jun 03 16:17:09.284404 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:09.281652 2568 flags.go:64] FLAG: --hostname-override="ip-10-0-128-88.ec2.internal" Jun 03 16:17:09.284404 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:09.281655 2568 flags.go:64] FLAG: --housekeeping-interval="10s" Jun 03 16:17:09.284404 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:09.281658 2568 flags.go:64] FLAG: --http-check-frequency="20s" Jun 03 16:17:09.284404 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:09.281661 2568 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jun 03 16:17:09.284404 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:09.281664 2568 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jun 03 16:17:09.284404 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:09.281667 2568 flags.go:64] FLAG: --image-gc-high-threshold="85" Jun 03 16:17:09.285035 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:09.281670 2568 flags.go:64] FLAG: --image-gc-low-threshold="80" Jun 03 16:17:09.285035 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:09.281673 2568 flags.go:64] FLAG: --image-service-endpoint="" Jun 03 16:17:09.285035 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:09.281676 2568 flags.go:64] FLAG: --kernel-memcg-notification="false" Jun 03 16:17:09.285035 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:09.281678 2568 flags.go:64] FLAG: --kube-api-burst="100" Jun 03 16:17:09.285035 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:09.281682 2568 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jun 03 16:17:09.285035 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:09.281687 2568 flags.go:64] FLAG: --kube-api-qps="50" Jun 03 16:17:09.285035 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:09.281690 2568 flags.go:64] FLAG: --kube-reserved="" Jun 03 16:17:09.285035 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:09.281693 2568 flags.go:64] FLAG: --kube-reserved-cgroup="" Jun 03 16:17:09.285035 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:09.281696 2568 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jun 03 16:17:09.285035 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:09.281699 2568 flags.go:64] FLAG: --kubelet-cgroups="" Jun 03 16:17:09.285035 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:09.281702 2568 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jun 03 16:17:09.285035 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:09.281705 2568 flags.go:64] FLAG: --lock-file="" Jun 03 16:17:09.285035 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:09.281707 2568 flags.go:64] FLAG: --log-cadvisor-usage="false" Jun 03 16:17:09.285035 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:09.281710 2568 flags.go:64] FLAG: --log-flush-frequency="5s" Jun 03 16:17:09.285035 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:09.281714 2568 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jun 03 16:17:09.285035 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:09.281719 2568 flags.go:64] FLAG: --log-json-split-stream="false" Jun 03 16:17:09.285035 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:09.281722 2568 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jun 03 16:17:09.285035 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:09.281725 2568 flags.go:64] FLAG: --log-text-split-stream="false" Jun 03 16:17:09.285035 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:09.281728 2568 flags.go:64] FLAG: --logging-format="text" Jun 03 16:17:09.285035 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:09.281730 2568 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jun 03 16:17:09.285035 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:09.281734 2568 flags.go:64] FLAG: --make-iptables-util-chains="true" Jun 03 16:17:09.285035 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:09.281737 2568 flags.go:64] FLAG: --manifest-url="" Jun 03 16:17:09.285035 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:09.281745 2568 flags.go:64] FLAG: --manifest-url-header="" Jun 03 16:17:09.285035 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:09.281750 2568 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jun 03 16:17:09.285035 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:09.281753 2568 flags.go:64] FLAG: --max-open-files="1000000" Jun 03 16:17:09.285643 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:09.281757 2568 flags.go:64] FLAG: --max-pods="110" Jun 03 16:17:09.285643 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:09.281760 2568 flags.go:64] FLAG: --maximum-dead-containers="-1" Jun 03 16:17:09.285643 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:09.281763 2568 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jun 03 16:17:09.285643 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:09.281766 2568 flags.go:64] FLAG: --memory-manager-policy="None" Jun 03 16:17:09.285643 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:09.281768 2568 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jun 03 16:17:09.285643 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:09.281771 2568 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jun 03 16:17:09.285643 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:09.281774 2568 flags.go:64] FLAG: --node-ip="0.0.0.0" Jun 03 16:17:09.285643 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:09.281777 2568 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jun 03 16:17:09.285643 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:09.281784 2568 flags.go:64] FLAG: --node-status-max-images="50" Jun 03 16:17:09.285643 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:09.281787 2568 flags.go:64] FLAG: --node-status-update-frequency="10s" Jun 03 16:17:09.285643 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:09.281790 2568 flags.go:64] FLAG: --oom-score-adj="-999" Jun 03 16:17:09.285643 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:09.281793 2568 flags.go:64] FLAG: --pod-cidr="" Jun 03 16:17:09.285643 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:09.281800 2568 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:f1083dc6abd9805f79797fa11c9accacc4a38caa073c94cb99097faffce05983" Jun 03 16:17:09.285643 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:09.281806 2568 flags.go:64] FLAG: --pod-manifest-path="" Jun 03 16:17:09.285643 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:09.281809 2568 flags.go:64] FLAG: --pod-max-pids="-1" Jun 03 16:17:09.285643 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:09.281812 2568 flags.go:64] FLAG: --pods-per-core="0" Jun 03 16:17:09.285643 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:09.281815 2568 flags.go:64] FLAG: --port="10250" Jun 03 16:17:09.285643 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:09.281821 2568 flags.go:64] FLAG: --protect-kernel-defaults="false" Jun 03 16:17:09.285643 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:09.281824 2568 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-02babd245024aca3f" Jun 03 16:17:09.285643 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:09.281827 2568 flags.go:64] FLAG: --qos-reserved="" Jun 03 16:17:09.285643 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:09.281830 2568 flags.go:64] FLAG: --read-only-port="10255" Jun 03 16:17:09.285643 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:09.281832 2568 flags.go:64] FLAG: --register-node="true" Jun 03 16:17:09.285643 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:09.281835 2568 flags.go:64] FLAG: --register-schedulable="true" Jun 03 16:17:09.285643 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:09.281838 2568 flags.go:64] FLAG: --register-with-taints="" Jun 03 16:17:09.286256 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:09.281842 2568 flags.go:64] FLAG: --registry-burst="10" Jun 03 16:17:09.286256 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:09.281845 2568 flags.go:64] FLAG: --registry-qps="5" Jun 03 16:17:09.286256 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:09.281848 2568 flags.go:64] FLAG: --reserved-cpus="" Jun 03 16:17:09.286256 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:09.281851 2568 flags.go:64] FLAG: --reserved-memory="" Jun 03 16:17:09.286256 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:09.281854 2568 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jun 03 16:17:09.286256 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:09.281857 2568 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jun 03 16:17:09.286256 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:09.281861 2568 flags.go:64] FLAG: --rotate-certificates="false" Jun 03 16:17:09.286256 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:09.281864 2568 flags.go:64] FLAG: --rotate-server-certificates="false" Jun 03 16:17:09.286256 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:09.281866 2568 flags.go:64] FLAG: --runonce="false" Jun 03 16:17:09.286256 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:09.281869 2568 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jun 03 16:17:09.286256 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:09.281872 2568 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jun 03 16:17:09.286256 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:09.281875 2568 flags.go:64] FLAG: --seccomp-default="false" Jun 03 16:17:09.286256 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:09.281878 2568 flags.go:64] FLAG: --serialize-image-pulls="true" Jun 03 16:17:09.286256 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:09.281881 2568 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jun 03 16:17:09.286256 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:09.281884 2568 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jun 03 16:17:09.286256 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:09.281887 2568 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jun 03 16:17:09.286256 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:09.281890 2568 flags.go:64] FLAG: --storage-driver-password="root" Jun 03 16:17:09.286256 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:09.281892 2568 flags.go:64] FLAG: --storage-driver-secure="false" Jun 03 16:17:09.286256 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:09.281895 2568 flags.go:64] FLAG: --storage-driver-table="stats" Jun 03 16:17:09.286256 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:09.281898 2568 flags.go:64] FLAG: --storage-driver-user="root" Jun 03 16:17:09.286256 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:09.281902 2568 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jun 03 16:17:09.286256 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:09.281905 2568 flags.go:64] FLAG: --sync-frequency="1m0s" Jun 03 16:17:09.286256 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:09.281907 2568 flags.go:64] FLAG: --system-cgroups="" Jun 03 16:17:09.286256 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:09.281910 2568 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jun 03 16:17:09.286256 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:09.281916 2568 flags.go:64] FLAG: --system-reserved-cgroup="" Jun 03 16:17:09.286862 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:09.281920 2568 flags.go:64] FLAG: --tls-cert-file="" Jun 03 16:17:09.286862 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:09.281923 2568 flags.go:64] FLAG: --tls-cipher-suites="[]" Jun 03 16:17:09.286862 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:09.281930 2568 flags.go:64] FLAG: --tls-min-version="" Jun 03 16:17:09.286862 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:09.281933 2568 flags.go:64] FLAG: --tls-private-key-file="" Jun 03 16:17:09.286862 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:09.281936 2568 flags.go:64] FLAG: --topology-manager-policy="none" Jun 03 16:17:09.286862 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:09.281939 2568 flags.go:64] FLAG: --topology-manager-policy-options="" Jun 03 16:17:09.286862 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:09.281942 2568 flags.go:64] FLAG: --topology-manager-scope="container" Jun 03 16:17:09.286862 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:09.281945 2568 flags.go:64] FLAG: --v="2" Jun 03 16:17:09.286862 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:09.281950 2568 flags.go:64] FLAG: --version="false" Jun 03 16:17:09.286862 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:09.281956 2568 flags.go:64] FLAG: --vmodule="" Jun 03 16:17:09.286862 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:09.281960 2568 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jun 03 16:17:09.286862 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:09.281963 2568 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jun 03 16:17:09.286862 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.282090 2568 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 03 16:17:09.286862 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.282094 2568 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 03 16:17:09.286862 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.282098 2568 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 03 16:17:09.286862 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.282100 2568 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 03 16:17:09.286862 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.282104 2568 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 03 16:17:09.286862 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.282107 2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 03 16:17:09.286862 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.282110 2568 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS Jun 03 16:17:09.286862 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.282113 2568 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 03 16:17:09.286862 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.282117 2568 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 03 16:17:09.286862 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.282120 2568 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 03 16:17:09.287384 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.282123 2568 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 03 16:17:09.287384 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.282126 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 03 16:17:09.287384 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.282129 2568 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 03 16:17:09.287384 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.282132 2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 03 16:17:09.287384 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.282135 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 03 16:17:09.287384 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.282139 2568 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 03 16:17:09.287384 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.282142 2568 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 03 16:17:09.287384 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.282146 2568 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 03 16:17:09.287384 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.282148 2568 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 03 16:17:09.287384 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.282151 2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 03 16:17:09.287384 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.282155 2568 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 03 16:17:09.287384 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.282158 2568 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 03 16:17:09.287384 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.282161 2568 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 03 16:17:09.287384 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.282163 2568 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 03 16:17:09.287384 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.282166 2568 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 03 16:17:09.287384 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.282168 2568 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 03 16:17:09.287384 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.282172 2568 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 03 16:17:09.287384 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.282176 2568 feature_gate.go:328] unrecognized feature gate: Example2 Jun 03 16:17:09.287384 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.282178 2568 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 03 16:17:09.287878 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.282181 2568 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 03 16:17:09.287878 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.282184 2568 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 03 16:17:09.287878 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.282186 2568 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 03 16:17:09.287878 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.282189 2568 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 03 16:17:09.287878 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.282191 2568 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 03 16:17:09.287878 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.282194 2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 03 16:17:09.287878 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.282197 2568 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 03 16:17:09.287878 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.282199 2568 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 03 16:17:09.287878 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.282202 2568 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 03 16:17:09.287878 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.282205 2568 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 03 16:17:09.287878 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.282207 2568 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 03 16:17:09.287878 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.282209 2568 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 03 16:17:09.287878 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.282212 2568 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 03 16:17:09.287878 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.282214 2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 03 16:17:09.287878 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.282217 2568 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 03 16:17:09.287878 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.282219 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 03 16:17:09.287878 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.282221 2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 03 16:17:09.287878 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.282224 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 03 16:17:09.287878 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.282227 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 03 16:17:09.287878 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.282229 2568 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 03 16:17:09.288370 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.282232 2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 03 16:17:09.288370 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.282234 2568 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 03 16:17:09.288370 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.282236 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 03 16:17:09.288370 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.282240 2568 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 03 16:17:09.288370 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.282243 2568 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 03 16:17:09.288370 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.282245 2568 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 03 16:17:09.288370 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.282248 2568 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 03 16:17:09.288370 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.282250 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 03 16:17:09.288370 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.282253 2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 03 16:17:09.288370 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.282255 2568 feature_gate.go:328] unrecognized feature gate: Example Jun 03 16:17:09.288370 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.282258 2568 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 03 16:17:09.288370 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.282260 2568 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 03 16:17:09.288370 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.282263 2568 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 03 16:17:09.288370 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.282265 2568 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 03 16:17:09.288370 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.282268 2568 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 03 16:17:09.288370 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.282270 2568 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 03 16:17:09.288370 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.282273 2568 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI Jun 03 16:17:09.288370 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.282275 2568 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 03 16:17:09.288370 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.282277 2568 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 03 16:17:09.288370 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.282280 2568 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 03 16:17:09.288911 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.282283 2568 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 03 16:17:09.288911 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.282285 2568 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 03 16:17:09.288911 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.282288 2568 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 03 16:17:09.288911 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.282290 2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 03 16:17:09.288911 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.282293 2568 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 03 16:17:09.288911 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.282295 2568 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 03 16:17:09.288911 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.282298 2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 03 16:17:09.288911 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.282300 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 03 16:17:09.288911 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.282303 2568 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 03 16:17:09.288911 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.282305 2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 03 16:17:09.288911 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.282308 2568 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 03 16:17:09.288911 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.282310 2568 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 03 16:17:09.288911 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.282313 2568 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 03 16:17:09.288911 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.282315 2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 03 16:17:09.288911 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.282318 2568 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations Jun 03 16:17:09.288911 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.282322 2568 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 03 16:17:09.288911 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.282325 2568 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController Jun 03 16:17:09.289660 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:09.283484 2568 feature_gate.go:384] feature gates: {map[DynamicResourceAllocation:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutatingAdmissionPolicy:false NodeSwap:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:false]} Jun 03 16:17:09.290499 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:09.290475 2568 server.go:530] "Kubelet version" kubeletVersion="v1.33.12" Jun 03 16:17:09.290499 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:09.290499 2568 server.go:532] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jun 03 16:17:09.290672 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.290601 2568 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 03 16:17:09.290672 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.290610 2568 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController Jun 03 16:17:09.290672 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.290615 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 03 16:17:09.290672 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.290620 2568 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 03 16:17:09.290672 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.290624 2568 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 03 16:17:09.290672 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.290628 2568 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations Jun 03 16:17:09.290672 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.290633 2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 03 16:17:09.290672 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.290637 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 03 16:17:09.290672 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.290642 2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 03 16:17:09.290672 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.290646 2568 feature_gate.go:328] unrecognized feature gate: Example2 Jun 03 16:17:09.290672 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.290651 2568 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 03 16:17:09.290672 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.290655 2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 03 16:17:09.290672 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.290659 2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 03 16:17:09.290672 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.290663 2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 03 16:17:09.290672 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.290668 2568 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 03 16:17:09.290672 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.290672 2568 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 03 16:17:09.290672 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.290678 2568 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 03 16:17:09.291385 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.290684 2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 03 16:17:09.291385 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.290689 2568 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 03 16:17:09.291385 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.290693 2568 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 03 16:17:09.291385 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.290697 2568 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 03 16:17:09.291385 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.290701 2568 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 03 16:17:09.291385 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.290706 2568 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 03 16:17:09.291385 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.290710 2568 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 03 16:17:09.291385 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.290714 2568 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 03 16:17:09.291385 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.290719 2568 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 03 16:17:09.291385 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.290723 2568 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 03 16:17:09.291385 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.290727 2568 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 03 16:17:09.291385 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.290731 2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 03 16:17:09.291385 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.290736 2568 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 03 16:17:09.291385 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.290740 2568 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 03 16:17:09.291385 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.290745 2568 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 03 16:17:09.291385 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.290750 2568 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 03 16:17:09.291385 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.290754 2568 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 03 16:17:09.291385 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.290758 2568 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 03 16:17:09.291385 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.290762 2568 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 03 16:17:09.291385 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.290766 2568 feature_gate.go:328] unrecognized feature gate: Example Jun 03 16:17:09.292219 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.290770 2568 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 03 16:17:09.292219 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.290774 2568 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 03 16:17:09.292219 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.290778 2568 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 03 16:17:09.292219 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.290783 2568 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 03 16:17:09.292219 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.290787 2568 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 03 16:17:09.292219 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.290791 2568 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 03 16:17:09.292219 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.290795 2568 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 03 16:17:09.292219 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.290799 2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 03 16:17:09.292219 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.290804 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 03 16:17:09.292219 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.290808 2568 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 03 16:17:09.292219 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.290812 2568 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 03 16:17:09.292219 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.290819 2568 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 03 16:17:09.292219 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.290826 2568 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 03 16:17:09.292219 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.290832 2568 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 03 16:17:09.292219 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.290837 2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 03 16:17:09.292219 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.290841 2568 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 03 16:17:09.292219 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.290846 2568 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS Jun 03 16:17:09.292219 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.290850 2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 03 16:17:09.292219 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.290855 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 03 16:17:09.292219 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.290860 2568 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 03 16:17:09.293026 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.290864 2568 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI Jun 03 16:17:09.293026 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.290868 2568 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 03 16:17:09.293026 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.290873 2568 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 03 16:17:09.293026 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.290878 2568 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 03 16:17:09.293026 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.290883 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 03 16:17:09.293026 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.290887 2568 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 03 16:17:09.293026 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.290892 2568 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 03 16:17:09.293026 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.290897 2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 03 16:17:09.293026 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.290902 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 03 16:17:09.293026 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.290907 2568 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 03 16:17:09.293026 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.290911 2568 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 03 16:17:09.293026 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.290916 2568 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 03 16:17:09.293026 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.290921 2568 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 03 16:17:09.293026 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.290925 2568 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 03 16:17:09.293026 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.290929 2568 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 03 16:17:09.293026 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.290933 2568 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 03 16:17:09.293026 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.290938 2568 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 03 16:17:09.293026 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.290942 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 03 16:17:09.293026 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.290946 2568 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 03 16:17:09.293641 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.290950 2568 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 03 16:17:09.293641 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.290955 2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 03 16:17:09.293641 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.290959 2568 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 03 16:17:09.293641 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.290963 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 03 16:17:09.293641 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.290968 2568 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 03 16:17:09.293641 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.290972 2568 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 03 16:17:09.293641 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.290977 2568 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 03 16:17:09.293641 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.290981 2568 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 03 16:17:09.293641 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.290986 2568 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 03 16:17:09.293641 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.290990 2568 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 03 16:17:09.293641 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:09.290998 2568 feature_gate.go:384] feature gates: {map[DynamicResourceAllocation:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutatingAdmissionPolicy:false NodeSwap:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:false]} Jun 03 16:17:09.293641 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.292393 2568 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 03 16:17:09.293641 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.292565 2568 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 03 16:17:09.293641 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.292572 2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 03 16:17:09.293641 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.292576 2568 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 03 16:17:09.294014 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.292582 2568 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 03 16:17:09.294014 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.292588 2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 03 16:17:09.294014 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.292593 2568 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 03 16:17:09.294014 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.292597 2568 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 03 16:17:09.294014 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.292602 2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 03 16:17:09.294014 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.292606 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 03 16:17:09.294014 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.292619 2568 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 03 16:17:09.294014 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.292624 2568 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations Jun 03 16:17:09.294014 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.292629 2568 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 03 16:17:09.294014 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.292633 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 03 16:17:09.294014 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.292638 2568 feature_gate.go:328] unrecognized feature gate: Example Jun 03 16:17:09.294014 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.292642 2568 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 03 16:17:09.294014 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.292647 2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 03 16:17:09.294014 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.292651 2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 03 16:17:09.294014 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.292656 2568 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 03 16:17:09.294014 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.292660 2568 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 03 16:17:09.294014 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.292665 2568 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 03 16:17:09.294014 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.292670 2568 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 03 16:17:09.294014 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.292674 2568 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 03 16:17:09.294014 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.292683 2568 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 03 16:17:09.294014 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.292688 2568 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 03 16:17:09.294548 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.292692 2568 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 03 16:17:09.294548 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.292697 2568 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 03 16:17:09.294548 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.292702 2568 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 03 16:17:09.294548 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.292707 2568 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 03 16:17:09.294548 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.292712 2568 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS Jun 03 16:17:09.294548 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.292717 2568 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 03 16:17:09.294548 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.292721 2568 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 03 16:17:09.294548 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.292727 2568 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 03 16:17:09.294548 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.292731 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 03 16:17:09.294548 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.292736 2568 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 03 16:17:09.294548 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.292740 2568 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 03 16:17:09.294548 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.292751 2568 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 03 16:17:09.294548 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.292756 2568 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 03 16:17:09.294548 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.292760 2568 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 03 16:17:09.294548 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.292765 2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 03 16:17:09.294548 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.292769 2568 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 03 16:17:09.294548 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.292774 2568 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 03 16:17:09.294548 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.292778 2568 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 03 16:17:09.294548 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.292784 2568 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 03 16:17:09.294548 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.292789 2568 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 03 16:17:09.295042 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.292794 2568 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 03 16:17:09.295042 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.292799 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 03 16:17:09.295042 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.292804 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 03 16:17:09.295042 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.292813 2568 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 03 16:17:09.295042 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.292818 2568 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 03 16:17:09.295042 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.292822 2568 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 03 16:17:09.295042 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.292827 2568 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI Jun 03 16:17:09.295042 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.292831 2568 feature_gate.go:328] unrecognized feature gate: Example2 Jun 03 16:17:09.295042 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.292836 2568 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 03 16:17:09.295042 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.292840 2568 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 03 16:17:09.295042 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.292845 2568 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 03 16:17:09.295042 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.292849 2568 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 03 16:17:09.295042 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.292854 2568 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 03 16:17:09.295042 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.292893 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 03 16:17:09.295042 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.292898 2568 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 03 16:17:09.295042 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.292916 2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 03 16:17:09.295042 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.292920 2568 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 03 16:17:09.295042 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.293074 2568 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 03 16:17:09.295042 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.293091 2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 03 16:17:09.295042 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.293096 2568 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 03 16:17:09.295551 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.293100 2568 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 03 16:17:09.295551 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.293103 2568 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 03 16:17:09.295551 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.293106 2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 03 16:17:09.295551 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.293109 2568 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 03 16:17:09.295551 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.293112 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 03 16:17:09.295551 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.293114 2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 03 16:17:09.295551 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.293117 2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 03 16:17:09.295551 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.293119 2568 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 03 16:17:09.295551 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.293124 2568 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 03 16:17:09.295551 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.293128 2568 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController Jun 03 16:17:09.295551 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.293132 2568 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 03 16:17:09.295551 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.293137 2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 03 16:17:09.295551 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.293140 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 03 16:17:09.295551 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.293143 2568 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 03 16:17:09.295551 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.293146 2568 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 03 16:17:09.295551 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.293150 2568 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 03 16:17:09.295551 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.293152 2568 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 03 16:17:09.295551 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.293155 2568 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 03 16:17:09.295551 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.293158 2568 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 03 16:17:09.296002 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.293160 2568 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 03 16:17:09.296002 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:09.293163 2568 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 03 16:17:09.296002 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:09.293169 2568 feature_gate.go:384] feature gates: {map[DynamicResourceAllocation:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutatingAdmissionPolicy:false NodeSwap:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:false]} Jun 03 16:17:09.296002 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:09.294114 2568 server.go:962] "Client rotation is on, will bootstrap in background" Jun 03 16:17:09.296109 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:09.296097 2568 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jun 03 16:17:09.297179 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:09.297166 2568 server.go:1019] "Starting client certificate rotation" Jun 03 16:17:09.297292 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:09.297275 2568 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jun 03 16:17:09.298000 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:09.297987 2568 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jun 03 16:17:09.325996 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:09.325972 2568 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jun 03 16:17:09.327739 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:09.327719 2568 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jun 03 16:17:09.344602 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:09.344586 2568 log.go:25] "Validated CRI v1 runtime API" Jun 03 16:17:09.353892 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:09.353876 2568 log.go:25] "Validated CRI v1 image API" Jun 03 16:17:09.356477 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:09.356456 2568 server.go:1452] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jun 03 16:17:09.359627 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:09.359607 2568 fs.go:135] Filesystem UUIDs: map[0b249614-ac12-480a-b91a-11e47930613a:/dev/nvme0n1p4 27bfc8d0-8e7c-4dc0-91a2-884b1edf2b0c:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2] Jun 03 16:17:09.359698 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:09.359626 2568 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jun 03 16:17:09.360039 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:09.360016 2568 reflector.go:430] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jun 03 16:17:09.363911 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:09.363764 2568 manager.go:217] Machine: {Timestamp:2026-06-03 16:17:09.36327449 +0000 UTC m=+0.451701071 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3099812 MemoryCapacity:33164488704 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec24ec1f76393d3992c562c7d99ec5ab SystemUUID:ec24ec1f-7639-3d39-92c5-62c7d99ec5ab BootID:505d7235-6d32-42c6-9c67-0bfddd1fe904 Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582242304 Type:vfs Inodes:4048399 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6103040 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582246400 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:d5:ed:a7:6f:63 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:d5:ed:a7:6f:63 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:b2:56:db:4d:47:be Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164488704 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jun 03 16:17:09.365262 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:09.365235 2568 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jun 03 16:17:09.365431 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:09.365413 2568 manager.go:233] Version: {KernelVersion:5.14.0-570.117.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260521-1 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jun 03 16:17:09.371082 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:09.371058 2568 container_manager_linux.go:270] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jun 03 16:17:09.371216 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:09.371083 2568 container_manager_linux.go:275] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-128-88.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jun 03 16:17:09.371264 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:09.371225 2568 topology_manager.go:138] "Creating topology manager with none policy" Jun 03 16:17:09.371264 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:09.371234 2568 container_manager_linux.go:306] "Creating device plugin manager" Jun 03 16:17:09.371264 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:09.371242 2568 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jun 03 16:17:09.372665 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:09.372654 2568 server.go:72] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jun 03 16:17:09.375197 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:09.375186 2568 state_mem.go:36] "Initialized new in-memory state store" Jun 03 16:17:09.375452 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:09.375443 2568 server.go:1267] "Using root directory" path="/var/lib/kubelet" Jun 03 16:17:09.378781 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:09.378771 2568 kubelet.go:491] "Attempting to sync node with API server" Jun 03 16:17:09.378829 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:09.378784 2568 kubelet.go:386] "Adding static pod path" path="/etc/kubernetes/manifests" Jun 03 16:17:09.378829 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:09.378797 2568 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jun 03 16:17:09.378829 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:09.378806 2568 kubelet.go:397] "Adding apiserver pod source" Jun 03 16:17:09.378829 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:09.378814 2568 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jun 03 16:17:09.380281 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:09.380266 2568 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jun 03 16:17:09.380329 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:09.380296 2568 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jun 03 16:17:09.380957 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:09.380938 2568 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-wsjx7" Jun 03 16:17:09.384166 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:09.384152 2568 kuberuntime_manager.go:279] "Container runtime initialized" containerRuntime="cri-o" version="1.33.12-3.rhaos4.20.git64c6a00.el9" apiVersion="v1" Jun 03 16:17:09.387220 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:09.387206 2568 kubelet.go:953] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jun 03 16:17:09.387745 ip-10-0-128-88 kubenswrapper[2568]: E0603 16:17:09.387724 2568 reflector.go:200] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-128-88.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jun 03 16:17:09.387814 ip-10-0-128-88 kubenswrapper[2568]: E0603 16:17:09.387728 2568 reflector.go:200] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jun 03 16:17:09.389469 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:09.389453 2568 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-wsjx7" Jun 03 16:17:09.390866 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:09.390853 2568 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jun 03 16:17:09.390921 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:09.390872 2568 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jun 03 16:17:09.390921 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:09.390878 2568 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jun 03 16:17:09.390921 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:09.390891 2568 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jun 03 16:17:09.390921 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:09.390900 2568 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jun 03 16:17:09.390921 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:09.390906 2568 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jun 03 16:17:09.390921 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:09.390913 2568 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jun 03 16:17:09.390921 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:09.390918 2568 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jun 03 16:17:09.391095 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:09.390925 2568 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jun 03 16:17:09.391095 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:09.390931 2568 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jun 03 16:17:09.391095 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:09.390946 2568 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jun 03 16:17:09.391095 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:09.390952 2568 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jun 03 16:17:09.392211 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:09.392200 2568 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jun 03 16:17:09.392211 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:09.392211 2568 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/image" Jun 03 16:17:09.395206 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:09.395193 2568 watchdog_linux.go:99] "Systemd watchdog is not enabled" Jun 03 16:17:09.395273 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:09.395229 2568 server.go:1295] "Started kubelet" Jun 03 16:17:09.395323 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:09.395285 2568 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jun 03 16:17:09.395368 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:09.395311 2568 ratelimit.go:55] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jun 03 16:17:09.395415 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:09.395401 2568 server_v1.go:47] "podresources" method="list" useActivePods=true Jun 03 16:17:09.396122 ip-10-0-128-88 systemd[1]: Started Kubernetes Kubelet. Jun 03 16:17:09.397203 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:09.397094 2568 server.go:255] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jun 03 16:17:09.399040 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:09.399028 2568 server.go:317] "Adding debug handlers to kubelet server" Jun 03 16:17:09.413937 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:09.413922 2568 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jun 03 16:17:09.416143 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:09.416127 2568 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jun 03 16:17:09.416258 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:09.416243 2568 volume_manager.go:295] "The desired_state_of_world populator starts" Jun 03 16:17:09.416318 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:09.416258 2568 volume_manager.go:297] "Starting Kubelet Volume Manager" Jun 03 16:17:09.416318 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:09.416305 2568 desired_state_of_world_populator.go:150] "Desired state populator starts to run" Jun 03 16:17:09.416409 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:09.416367 2568 reconstruct.go:97] "Volume reconstruction finished" Jun 03 16:17:09.416409 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:09.416377 2568 reconciler.go:26] "Reconciler: start to sync state" Jun 03 16:17:09.416866 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:09.416851 2568 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jun 03 16:17:09.416866 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:09.416866 2568 factory.go:55] Registering systemd factory Jun 03 16:17:09.416968 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:09.416872 2568 factory.go:223] Registration of the systemd container factory successfully Jun 03 16:17:09.417085 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:09.417072 2568 factory.go:153] Registering CRI-O factory Jun 03 16:17:09.417085 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:09.417086 2568 factory.go:223] Registration of the crio container factory successfully Jun 03 16:17:09.417164 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:09.417099 2568 factory.go:103] Registering Raw factory Jun 03 16:17:09.417164 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:09.417112 2568 manager.go:1196] Started watching for new ooms in manager Jun 03 16:17:09.418451 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:09.418433 2568 nodeinfomanager.go:417] Failed to publish CSINode: nodes "ip-10-0-128-88.ec2.internal" not found Jun 03 16:17:09.418728 ip-10-0-128-88 kubenswrapper[2568]: E0603 16:17:09.418708 2568 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-128-88.ec2.internal\" not found" Jun 03 16:17:09.418913 ip-10-0-128-88 kubenswrapper[2568]: E0603 16:17:09.418890 2568 kubelet.go:1618] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jun 03 16:17:09.420725 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:09.420709 2568 manager.go:319] Starting recovery of all containers Jun 03 16:17:09.421123 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:09.421106 2568 reflector.go:430] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jun 03 16:17:09.421854 ip-10-0-128-88 kubenswrapper[2568]: E0603 16:17:09.421838 2568 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-128-88.ec2.internal\" not found" node="ip-10-0-128-88.ec2.internal" Jun 03 16:17:09.430372 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:09.430351 2568 manager.go:324] Recovery completed Jun 03 16:17:09.434048 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:09.434035 2568 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach" Jun 03 16:17:09.435719 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:09.435702 2568 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-128-88.ec2.internal" event="NodeHasSufficientMemory" Jun 03 16:17:09.435791 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:09.435731 2568 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-128-88.ec2.internal" event="NodeHasNoDiskPressure" Jun 03 16:17:09.435791 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:09.435741 2568 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-128-88.ec2.internal" event="NodeHasSufficientPID" Jun 03 16:17:09.436170 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:09.436156 2568 cpu_manager.go:222] "Starting CPU manager" policy="none" Jun 03 16:17:09.436170 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:09.436166 2568 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jun 03 16:17:09.436274 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:09.436181 2568 state_mem.go:36] "Initialized new in-memory state store" Jun 03 16:17:09.436590 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:09.436577 2568 nodeinfomanager.go:417] Failed to publish CSINode: nodes "ip-10-0-128-88.ec2.internal" not found Jun 03 16:17:09.440363 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:09.440352 2568 policy_none.go:49] "None policy: Start" Jun 03 16:17:09.440401 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:09.440367 2568 memory_manager.go:186] "Starting memorymanager" policy="None" Jun 03 16:17:09.440401 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:09.440378 2568 state_mem.go:35] "Initializing new in-memory state store" Jun 03 16:17:09.478061 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:09.478042 2568 manager.go:341] "Starting Device Plugin manager" Jun 03 16:17:09.492911 ip-10-0-128-88 kubenswrapper[2568]: E0603 16:17:09.478093 2568 manager.go:517] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jun 03 16:17:09.492911 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:09.478102 2568 server.go:85] "Starting device plugin registration server" Jun 03 16:17:09.492911 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:09.478291 2568 eviction_manager.go:189] "Eviction manager: starting control loop" Jun 03 16:17:09.492911 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:09.478302 2568 container_log_manager.go:189] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jun 03 16:17:09.492911 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:09.478402 2568 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jun 03 16:17:09.492911 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:09.478485 2568 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jun 03 16:17:09.492911 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:09.478494 2568 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jun 03 16:17:09.492911 ip-10-0-128-88 kubenswrapper[2568]: E0603 16:17:09.479008 2568 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jun 03 16:17:09.492911 ip-10-0-128-88 kubenswrapper[2568]: E0603 16:17:09.479042 2568 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-128-88.ec2.internal\" not found" Jun 03 16:17:09.497513 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:09.497500 2568 nodeinfomanager.go:417] Failed to publish CSINode: nodes "ip-10-0-128-88.ec2.internal" not found Jun 03 16:17:09.558222 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:09.558160 2568 kubelet_network_linux.go:49] "Initialized iptables rules." protocol="IPv4" Jun 03 16:17:09.559362 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:09.559345 2568 kubelet_network_linux.go:49] "Initialized iptables rules." protocol="IPv6" Jun 03 16:17:09.559431 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:09.559373 2568 status_manager.go:230] "Starting to sync pod status with apiserver" Jun 03 16:17:09.559431 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:09.559394 2568 watchdog_linux.go:127] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jun 03 16:17:09.559431 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:09.559403 2568 kubelet.go:2451] "Starting kubelet main sync loop" Jun 03 16:17:09.559594 ip-10-0-128-88 kubenswrapper[2568]: E0603 16:17:09.559446 2568 kubelet.go:2475] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jun 03 16:17:09.562212 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:09.562195 2568 reflector.go:430] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jun 03 16:17:09.579398 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:09.579384 2568 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach" Jun 03 16:17:09.580173 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:09.580152 2568 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-128-88.ec2.internal" event="NodeHasSufficientMemory" Jun 03 16:17:09.580257 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:09.580183 2568 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-128-88.ec2.internal" event="NodeHasNoDiskPressure" Jun 03 16:17:09.580257 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:09.580193 2568 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-128-88.ec2.internal" event="NodeHasSufficientPID" Jun 03 16:17:09.580257 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:09.580213 2568 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-128-88.ec2.internal" Jun 03 16:17:09.588961 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:09.588945 2568 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-128-88.ec2.internal" Jun 03 16:17:09.589042 ip-10-0-128-88 kubenswrapper[2568]: E0603 16:17:09.588965 2568 kubelet_node_status.go:597] "Error updating node status, will retry" err="error getting node \"ip-10-0-128-88.ec2.internal\": node \"ip-10-0-128-88.ec2.internal\" not found" Jun 03 16:17:09.602848 ip-10-0-128-88 kubenswrapper[2568]: E0603 16:17:09.602826 2568 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-128-88.ec2.internal\" not found" Jun 03 16:17:09.660546 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:09.660504 2568 kubelet.go:2537] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-88.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-128-88.ec2.internal"] Jun 03 16:17:09.660620 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:09.660588 2568 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach" Jun 03 16:17:09.662068 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:09.662054 2568 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-128-88.ec2.internal" event="NodeHasSufficientMemory" Jun 03 16:17:09.662149 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:09.662081 2568 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-128-88.ec2.internal" event="NodeHasNoDiskPressure" Jun 03 16:17:09.662149 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:09.662093 2568 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-128-88.ec2.internal" event="NodeHasSufficientPID" Jun 03 16:17:09.663208 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:09.663197 2568 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach" Jun 03 16:17:09.663366 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:09.663351 2568 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-88.ec2.internal" Jun 03 16:17:09.663400 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:09.663383 2568 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach" Jun 03 16:17:09.663896 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:09.663880 2568 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-128-88.ec2.internal" event="NodeHasSufficientMemory" Jun 03 16:17:09.663987 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:09.663884 2568 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-128-88.ec2.internal" event="NodeHasSufficientMemory" Jun 03 16:17:09.663987 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:09.663933 2568 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-128-88.ec2.internal" event="NodeHasNoDiskPressure" Jun 03 16:17:09.663987 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:09.663948 2568 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-128-88.ec2.internal" event="NodeHasSufficientPID" Jun 03 16:17:09.663987 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:09.663909 2568 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-128-88.ec2.internal" event="NodeHasNoDiskPressure" Jun 03 16:17:09.664174 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:09.663994 2568 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-128-88.ec2.internal" event="NodeHasSufficientPID" Jun 03 16:17:09.665114 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:09.665099 2568 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-88.ec2.internal" Jun 03 16:17:09.665198 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:09.665125 2568 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach" Jun 03 16:17:09.665741 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:09.665726 2568 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-128-88.ec2.internal" event="NodeHasSufficientMemory" Jun 03 16:17:09.665805 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:09.665753 2568 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-128-88.ec2.internal" event="NodeHasNoDiskPressure" Jun 03 16:17:09.665805 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:09.665764 2568 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-128-88.ec2.internal" event="NodeHasSufficientPID" Jun 03 16:17:09.695100 ip-10-0-128-88 kubenswrapper[2568]: E0603 16:17:09.695077 2568 kubelet.go:3336] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-128-88.ec2.internal\" not found" node="ip-10-0-128-88.ec2.internal" Jun 03 16:17:09.699361 ip-10-0-128-88 kubenswrapper[2568]: E0603 16:17:09.699346 2568 kubelet.go:3336] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-128-88.ec2.internal\" not found" node="ip-10-0-128-88.ec2.internal" Jun 03 16:17:09.703406 ip-10-0-128-88 kubenswrapper[2568]: E0603 16:17:09.703391 2568 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-128-88.ec2.internal\" not found" Jun 03 16:17:09.804282 ip-10-0-128-88 kubenswrapper[2568]: E0603 16:17:09.804263 2568 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-128-88.ec2.internal\" not found" Jun 03 16:17:09.818132 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:09.818093 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/a6d060b8d198dadd30a1a5529869c2b5-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-88.ec2.internal\" (UID: \"a6d060b8d198dadd30a1a5529869c2b5\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-88.ec2.internal" Jun 03 16:17:09.818132 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:09.818117 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a6d060b8d198dadd30a1a5529869c2b5-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-88.ec2.internal\" (UID: \"a6d060b8d198dadd30a1a5529869c2b5\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-88.ec2.internal" Jun 03 16:17:09.818215 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:09.818134 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/e4c4dbf6c50859cd8172ab3fc4042545-config\") pod \"kube-apiserver-proxy-ip-10-0-128-88.ec2.internal\" (UID: \"e4c4dbf6c50859cd8172ab3fc4042545\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-88.ec2.internal" Jun 03 16:17:09.904471 ip-10-0-128-88 kubenswrapper[2568]: E0603 16:17:09.904448 2568 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-128-88.ec2.internal\" not found" Jun 03 16:17:09.918810 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:09.918787 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/a6d060b8d198dadd30a1a5529869c2b5-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-88.ec2.internal\" (UID: \"a6d060b8d198dadd30a1a5529869c2b5\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-88.ec2.internal" Jun 03 16:17:09.918882 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:09.918816 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a6d060b8d198dadd30a1a5529869c2b5-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-88.ec2.internal\" (UID: \"a6d060b8d198dadd30a1a5529869c2b5\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-88.ec2.internal" Jun 03 16:17:09.918882 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:09.918832 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/e4c4dbf6c50859cd8172ab3fc4042545-config\") pod \"kube-apiserver-proxy-ip-10-0-128-88.ec2.internal\" (UID: \"e4c4dbf6c50859cd8172ab3fc4042545\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-88.ec2.internal" Jun 03 16:17:09.918882 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:09.918863 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/e4c4dbf6c50859cd8172ab3fc4042545-config\") pod \"kube-apiserver-proxy-ip-10-0-128-88.ec2.internal\" (UID: \"e4c4dbf6c50859cd8172ab3fc4042545\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-88.ec2.internal" Jun 03 16:17:09.918990 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:09.918885 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a6d060b8d198dadd30a1a5529869c2b5-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-88.ec2.internal\" (UID: \"a6d060b8d198dadd30a1a5529869c2b5\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-88.ec2.internal" Jun 03 16:17:09.918990 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:09.918884 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/a6d060b8d198dadd30a1a5529869c2b5-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-88.ec2.internal\" (UID: \"a6d060b8d198dadd30a1a5529869c2b5\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-88.ec2.internal" Jun 03 16:17:09.996889 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:09.996853 2568 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-88.ec2.internal" Jun 03 16:17:10.002613 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:10.002597 2568 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-88.ec2.internal" Jun 03 16:17:10.005425 ip-10-0-128-88 kubenswrapper[2568]: E0603 16:17:10.005410 2568 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-128-88.ec2.internal\" not found" Jun 03 16:17:10.105957 ip-10-0-128-88 kubenswrapper[2568]: E0603 16:17:10.105885 2568 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-128-88.ec2.internal\" not found" Jun 03 16:17:10.206366 ip-10-0-128-88 kubenswrapper[2568]: E0603 16:17:10.206342 2568 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-128-88.ec2.internal\" not found" Jun 03 16:17:10.212853 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:10.212827 2568 reflector.go:430] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jun 03 16:17:10.216603 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:10.216590 2568 kubelet.go:3340] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-88.ec2.internal" Jun 03 16:17:10.227293 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:10.227277 2568 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jun 03 16:17:10.229054 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:10.229042 2568 kubelet.go:3340] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-88.ec2.internal" Jun 03 16:17:10.238818 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:10.238802 2568 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jun 03 16:17:10.297961 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:10.297941 2568 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jun 03 16:17:10.298521 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:10.298048 2568 reflector.go:556] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jun 03 16:17:10.298521 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:10.298048 2568 reflector.go:556] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jun 03 16:17:10.379138 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:10.379086 2568 apiserver.go:52] "Watching apiserver" Jun 03 16:17:10.387529 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:10.387511 2568 reflector.go:430] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jun 03 16:17:10.388476 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:10.388455 2568 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-multus/multus-additional-cni-plugins-wvl4d","openshift-multus/network-metrics-daemon-fhpnv","openshift-ovn-kubernetes/ovnkube-node-xz69h","kube-system/kube-apiserver-proxy-ip-10-0-128-88.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xvpwg","openshift-dns/node-resolver-2v2dn","openshift-image-registry/node-ca-6jcbf","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-88.ec2.internal","openshift-network-diagnostics/network-check-target-m7zkz","openshift-network-operator/iptables-alerter-rmdrz","kube-system/konnectivity-agent-5hlfw","openshift-cluster-node-tuning-operator/tuned-5fhsz","openshift-multus/multus-8s5gg"] Jun 03 16:17:10.391156 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:10.391131 2568 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-rmdrz" Jun 03 16:17:10.391233 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:10.391213 2568 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fhpnv" Jun 03 16:17:10.391303 ip-10-0-128-88 kubenswrapper[2568]: E0603 16:17:10.391284 2568 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fhpnv" podUID="d93510f8-5732-4611-8aca-02573c7e296e" Jun 03 16:17:10.391413 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:10.391383 2568 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-06-02 16:12:09 +0000 UTC" deadline="2028-01-27 04:57:02.660508666 +0000 UTC" Jun 03 16:17:10.391482 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:10.391414 2568 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14460h39m52.269098015s" Jun 03 16:17:10.392484 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:10.392467 2568 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-xz69h" Jun 03 16:17:10.393821 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:10.393802 2568 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xvpwg" Jun 03 16:17:10.395166 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:10.395151 2568 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-2v2dn" Jun 03 16:17:10.395552 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:10.395517 2568 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jun 03 16:17:10.395679 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:10.395564 2568 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-txncn\"" Jun 03 16:17:10.395745 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:10.395624 2568 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jun 03 16:17:10.395745 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:10.395698 2568 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-kgx7b\"" Jun 03 16:17:10.395745 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:10.395668 2568 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jun 03 16:17:10.395878 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:10.395755 2568 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jun 03 16:17:10.395878 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:10.395762 2568 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jun 03 16:17:10.395878 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:10.395652 2568 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jun 03 16:17:10.396014 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:10.395968 2568 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jun 03 16:17:10.396064 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:10.396022 2568 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jun 03 16:17:10.396064 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:10.396041 2568 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jun 03 16:17:10.396161 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:10.396085 2568 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jun 03 16:17:10.396240 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:10.396220 2568 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-bqxrb\"" Jun 03 16:17:10.396341 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:10.396249 2568 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jun 03 16:17:10.396842 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:10.396808 2568 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jun 03 16:17:10.396918 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:10.396887 2568 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jun 03 16:17:10.397200 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:10.397182 2568 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-6jcbf" Jun 03 16:17:10.397708 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:10.397690 2568 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-g7t87\"" Jun 03 16:17:10.397854 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:10.397841 2568 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jun 03 16:17:10.398707 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:10.398689 2568 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m7zkz" Jun 03 16:17:10.398787 ip-10-0-128-88 kubenswrapper[2568]: E0603 16:17:10.398767 2568 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m7zkz" podUID="b8482073-6356-4d6e-bddf-2c8152f6a4f6" Jun 03 16:17:10.399253 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:10.399231 2568 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jun 03 16:17:10.399435 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:10.399414 2568 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jun 03 16:17:10.399607 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:10.399581 2568 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jun 03 16:17:10.399975 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:10.399961 2568 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-wvl4d" Jun 03 16:17:10.400034 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:10.400010 2568 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-zjwzg\"" Jun 03 16:17:10.401586 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:10.401569 2568 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-5hlfw" Jun 03 16:17:10.401860 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:10.401835 2568 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jun 03 16:17:10.401936 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:10.401915 2568 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jun 03 16:17:10.401936 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:10.401926 2568 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jun 03 16:17:10.402017 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:10.401948 2568 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jun 03 16:17:10.402017 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:10.402010 2568 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-qtxqj\"" Jun 03 16:17:10.402140 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:10.402128 2568 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jun 03 16:17:10.403408 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:10.403394 2568 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jun 03 16:17:10.403483 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:10.403469 2568 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-5fhsz" Jun 03 16:17:10.403668 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:10.403630 2568 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jun 03 16:17:10.403878 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:10.403863 2568 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-mx2mz\"" Jun 03 16:17:10.405148 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:10.405132 2568 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-7lkwj\"" Jun 03 16:17:10.405435 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:10.405419 2568 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jun 03 16:17:10.405503 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:10.405471 2568 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-8s5gg" Jun 03 16:17:10.405654 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:10.405637 2568 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jun 03 16:17:10.407170 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:10.407153 2568 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-xvgj4\"" Jun 03 16:17:10.407294 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:10.407231 2568 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jun 03 16:17:10.414373 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:10.414358 2568 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jun 03 16:17:10.417153 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:10.417135 2568 desired_state_of_world_populator.go:158] "Finished populating initial desired state of world" Jun 03 16:17:10.422004 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:10.421979 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9xhts\" (UniqueName: \"kubernetes.io/projected/2d19a6a4-13e1-407c-a488-189a3651f578-kube-api-access-9xhts\") pod \"node-ca-6jcbf\" (UID: \"2d19a6a4-13e1-407c-a488-189a3651f578\") " pod="openshift-image-registry/node-ca-6jcbf" Jun 03 16:17:10.422123 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:10.422013 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tdcd7\" (UniqueName: \"kubernetes.io/projected/b8482073-6356-4d6e-bddf-2c8152f6a4f6-kube-api-access-tdcd7\") pod \"network-check-target-m7zkz\" (UID: \"b8482073-6356-4d6e-bddf-2c8152f6a4f6\") " pod="openshift-network-diagnostics/network-check-target-m7zkz" Jun 03 16:17:10.422123 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:10.422039 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/f601dafe-2b89-43a6-bb9e-2080b98aaa82-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-wvl4d\" (UID: \"f601dafe-2b89-43a6-bb9e-2080b98aaa82\") " pod="openshift-multus/multus-additional-cni-plugins-wvl4d" Jun 03 16:17:10.422123 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:10.422063 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/9c10c83c-4e52-4366-aef2-577fb3fa951f-etc-systemd\") pod \"tuned-5fhsz\" (UID: \"9c10c83c-4e52-4366-aef2-577fb3fa951f\") " pod="openshift-cluster-node-tuning-operator/tuned-5fhsz" Jun 03 16:17:10.422123 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:10.422086 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/9c10c83c-4e52-4366-aef2-577fb3fa951f-etc-tuned\") pod \"tuned-5fhsz\" (UID: \"9c10c83c-4e52-4366-aef2-577fb3fa951f\") " pod="openshift-cluster-node-tuning-operator/tuned-5fhsz" Jun 03 16:17:10.422123 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:10.422107 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/e926d1e6-fe73-4a66-9488-1bd8d711ea79-run-systemd\") pod \"ovnkube-node-xz69h\" (UID: \"e926d1e6-fe73-4a66-9488-1bd8d711ea79\") " pod="openshift-ovn-kubernetes/ovnkube-node-xz69h" Jun 03 16:17:10.422279 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:10.422139 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/2b06ddd4-c92a-45e5-9538-03fe00b0503f-kubelet-dir\") pod \"aws-ebs-csi-driver-node-xvpwg\" (UID: \"2b06ddd4-c92a-45e5-9538-03fe00b0503f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xvpwg" Jun 03 16:17:10.422279 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:10.422162 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/2b06ddd4-c92a-45e5-9538-03fe00b0503f-socket-dir\") pod \"aws-ebs-csi-driver-node-xvpwg\" (UID: \"2b06ddd4-c92a-45e5-9538-03fe00b0503f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xvpwg" Jun 03 16:17:10.422279 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:10.422199 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/2b06ddd4-c92a-45e5-9538-03fe00b0503f-registration-dir\") pod \"aws-ebs-csi-driver-node-xvpwg\" (UID: \"2b06ddd4-c92a-45e5-9538-03fe00b0503f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xvpwg" Jun 03 16:17:10.422279 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:10.422213 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/80818dcb-cac7-4ac4-a376-ea67605a5ff8-host-var-lib-cni-bin\") pod \"multus-8s5gg\" (UID: \"80818dcb-cac7-4ac4-a376-ea67605a5ff8\") " pod="openshift-multus/multus-8s5gg" Jun 03 16:17:10.422279 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:10.422227 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/e926d1e6-fe73-4a66-9488-1bd8d711ea79-run-ovn\") pod \"ovnkube-node-xz69h\" (UID: \"e926d1e6-fe73-4a66-9488-1bd8d711ea79\") " pod="openshift-ovn-kubernetes/ovnkube-node-xz69h" Jun 03 16:17:10.422279 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:10.422242 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e926d1e6-fe73-4a66-9488-1bd8d711ea79-host-run-ovn-kubernetes\") pod \"ovnkube-node-xz69h\" (UID: \"e926d1e6-fe73-4a66-9488-1bd8d711ea79\") " pod="openshift-ovn-kubernetes/ovnkube-node-xz69h" Jun 03 16:17:10.422279 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:10.422256 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/e926d1e6-fe73-4a66-9488-1bd8d711ea79-ovnkube-config\") pod \"ovnkube-node-xz69h\" (UID: \"e926d1e6-fe73-4a66-9488-1bd8d711ea79\") " pod="openshift-ovn-kubernetes/ovnkube-node-xz69h" Jun 03 16:17:10.422279 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:10.422269 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/2d19a6a4-13e1-407c-a488-189a3651f578-host\") pod \"node-ca-6jcbf\" (UID: \"2d19a6a4-13e1-407c-a488-189a3651f578\") " pod="openshift-image-registry/node-ca-6jcbf" Jun 03 16:17:10.422633 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:10.422285 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/9c10c83c-4e52-4366-aef2-577fb3fa951f-etc-modprobe-d\") pod \"tuned-5fhsz\" (UID: \"9c10c83c-4e52-4366-aef2-577fb3fa951f\") " pod="openshift-cluster-node-tuning-operator/tuned-5fhsz" Jun 03 16:17:10.422633 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:10.422316 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/9c10c83c-4e52-4366-aef2-577fb3fa951f-etc-sysctl-conf\") pod \"tuned-5fhsz\" (UID: \"9c10c83c-4e52-4366-aef2-577fb3fa951f\") " pod="openshift-cluster-node-tuning-operator/tuned-5fhsz" Jun 03 16:17:10.422633 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:10.422348 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/9c10c83c-4e52-4366-aef2-577fb3fa951f-run\") pod \"tuned-5fhsz\" (UID: \"9c10c83c-4e52-4366-aef2-577fb3fa951f\") " pod="openshift-cluster-node-tuning-operator/tuned-5fhsz" Jun 03 16:17:10.422633 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:10.422367 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/80818dcb-cac7-4ac4-a376-ea67605a5ff8-os-release\") pod \"multus-8s5gg\" (UID: \"80818dcb-cac7-4ac4-a376-ea67605a5ff8\") " pod="openshift-multus/multus-8s5gg" Jun 03 16:17:10.422633 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:10.422385 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/6bd7702f-6b77-4535-993e-d1e053d3771b-host-slash\") pod \"iptables-alerter-rmdrz\" (UID: \"6bd7702f-6b77-4535-993e-d1e053d3771b\") " pod="openshift-network-operator/iptables-alerter-rmdrz" Jun 03 16:17:10.422633 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:10.422404 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/80818dcb-cac7-4ac4-a376-ea67605a5ff8-cni-binary-copy\") pod \"multus-8s5gg\" (UID: \"80818dcb-cac7-4ac4-a376-ea67605a5ff8\") " pod="openshift-multus/multus-8s5gg" Jun 03 16:17:10.422633 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:10.422418 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/e926d1e6-fe73-4a66-9488-1bd8d711ea79-host-kubelet\") pod \"ovnkube-node-xz69h\" (UID: \"e926d1e6-fe73-4a66-9488-1bd8d711ea79\") " pod="openshift-ovn-kubernetes/ovnkube-node-xz69h" Jun 03 16:17:10.422633 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:10.422438 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e926d1e6-fe73-4a66-9488-1bd8d711ea79-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-xz69h\" (UID: \"e926d1e6-fe73-4a66-9488-1bd8d711ea79\") " pod="openshift-ovn-kubernetes/ovnkube-node-xz69h" Jun 03 16:17:10.422633 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:10.422454 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/130349f2-fa26-4e51-bf29-6819611104db-konnectivity-ca\") pod \"konnectivity-agent-5hlfw\" (UID: \"130349f2-fa26-4e51-bf29-6819611104db\") " pod="kube-system/konnectivity-agent-5hlfw" Jun 03 16:17:10.422633 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:10.422467 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f601dafe-2b89-43a6-bb9e-2080b98aaa82-cnibin\") pod \"multus-additional-cni-plugins-wvl4d\" (UID: \"f601dafe-2b89-43a6-bb9e-2080b98aaa82\") " pod="openshift-multus/multus-additional-cni-plugins-wvl4d" Jun 03 16:17:10.422633 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:10.422480 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/f601dafe-2b89-43a6-bb9e-2080b98aaa82-os-release\") pod \"multus-additional-cni-plugins-wvl4d\" (UID: \"f601dafe-2b89-43a6-bb9e-2080b98aaa82\") " pod="openshift-multus/multus-additional-cni-plugins-wvl4d" Jun 03 16:17:10.422633 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:10.422499 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f601dafe-2b89-43a6-bb9e-2080b98aaa82-cni-binary-copy\") pod \"multus-additional-cni-plugins-wvl4d\" (UID: \"f601dafe-2b89-43a6-bb9e-2080b98aaa82\") " pod="openshift-multus/multus-additional-cni-plugins-wvl4d" Jun 03 16:17:10.422633 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:10.422514 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/9c10c83c-4e52-4366-aef2-577fb3fa951f-etc-sysctl-d\") pod \"tuned-5fhsz\" (UID: \"9c10c83c-4e52-4366-aef2-577fb3fa951f\") " pod="openshift-cluster-node-tuning-operator/tuned-5fhsz" Jun 03 16:17:10.422633 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:10.422547 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/80818dcb-cac7-4ac4-a376-ea67605a5ff8-host-run-k8s-cni-cncf-io\") pod \"multus-8s5gg\" (UID: \"80818dcb-cac7-4ac4-a376-ea67605a5ff8\") " pod="openshift-multus/multus-8s5gg" Jun 03 16:17:10.422633 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:10.422583 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/80818dcb-cac7-4ac4-a376-ea67605a5ff8-host-run-netns\") pod \"multus-8s5gg\" (UID: \"80818dcb-cac7-4ac4-a376-ea67605a5ff8\") " pod="openshift-multus/multus-8s5gg" Jun 03 16:17:10.422633 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:10.422618 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e926d1e6-fe73-4a66-9488-1bd8d711ea79-var-lib-openvswitch\") pod \"ovnkube-node-xz69h\" (UID: \"e926d1e6-fe73-4a66-9488-1bd8d711ea79\") " pod="openshift-ovn-kubernetes/ovnkube-node-xz69h" Jun 03 16:17:10.423208 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:10.422661 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/f601dafe-2b89-43a6-bb9e-2080b98aaa82-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-wvl4d\" (UID: \"f601dafe-2b89-43a6-bb9e-2080b98aaa82\") " pod="openshift-multus/multus-additional-cni-plugins-wvl4d" Jun 03 16:17:10.423208 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:10.422680 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/80818dcb-cac7-4ac4-a376-ea67605a5ff8-multus-socket-dir-parent\") pod \"multus-8s5gg\" (UID: \"80818dcb-cac7-4ac4-a376-ea67605a5ff8\") " pod="openshift-multus/multus-8s5gg" Jun 03 16:17:10.423208 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:10.422700 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/80818dcb-cac7-4ac4-a376-ea67605a5ff8-host-var-lib-cni-multus\") pod \"multus-8s5gg\" (UID: \"80818dcb-cac7-4ac4-a376-ea67605a5ff8\") " pod="openshift-multus/multus-8s5gg" Jun 03 16:17:10.423208 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:10.422716 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/e926d1e6-fe73-4a66-9488-1bd8d711ea79-host-cni-netd\") pod \"ovnkube-node-xz69h\" (UID: \"e926d1e6-fe73-4a66-9488-1bd8d711ea79\") " pod="openshift-ovn-kubernetes/ovnkube-node-xz69h" Jun 03 16:17:10.423208 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:10.422729 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/130349f2-fa26-4e51-bf29-6819611104db-agent-certs\") pod \"konnectivity-agent-5hlfw\" (UID: \"130349f2-fa26-4e51-bf29-6819611104db\") " pod="kube-system/konnectivity-agent-5hlfw" Jun 03 16:17:10.423208 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:10.422751 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/6bd7702f-6b77-4535-993e-d1e053d3771b-iptables-alerter-script\") pod \"iptables-alerter-rmdrz\" (UID: \"6bd7702f-6b77-4535-993e-d1e053d3771b\") " pod="openshift-network-operator/iptables-alerter-rmdrz" Jun 03 16:17:10.423208 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:10.422774 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/80818dcb-cac7-4ac4-a376-ea67605a5ff8-system-cni-dir\") pod \"multus-8s5gg\" (UID: \"80818dcb-cac7-4ac4-a376-ea67605a5ff8\") " pod="openshift-multus/multus-8s5gg" Jun 03 16:17:10.423208 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:10.422789 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/80818dcb-cac7-4ac4-a376-ea67605a5ff8-cnibin\") pod \"multus-8s5gg\" (UID: \"80818dcb-cac7-4ac4-a376-ea67605a5ff8\") " pod="openshift-multus/multus-8s5gg" Jun 03 16:17:10.423208 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:10.422803 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/80818dcb-cac7-4ac4-a376-ea67605a5ff8-multus-conf-dir\") pod \"multus-8s5gg\" (UID: \"80818dcb-cac7-4ac4-a376-ea67605a5ff8\") " pod="openshift-multus/multus-8s5gg" Jun 03 16:17:10.423208 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:10.422818 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-77tzv\" (UniqueName: \"kubernetes.io/projected/d93510f8-5732-4611-8aca-02573c7e296e-kube-api-access-77tzv\") pod \"network-metrics-daemon-fhpnv\" (UID: \"d93510f8-5732-4611-8aca-02573c7e296e\") " pod="openshift-multus/network-metrics-daemon-fhpnv" Jun 03 16:17:10.423208 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:10.422838 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tzc6j\" (UniqueName: \"kubernetes.io/projected/2b06ddd4-c92a-45e5-9538-03fe00b0503f-kube-api-access-tzc6j\") pod \"aws-ebs-csi-driver-node-xvpwg\" (UID: \"2b06ddd4-c92a-45e5-9538-03fe00b0503f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xvpwg" Jun 03 16:17:10.423208 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:10.422853 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/b0a52397-ab84-4efe-a7e1-fb26e146dfbc-hosts-file\") pod \"node-resolver-2v2dn\" (UID: \"b0a52397-ab84-4efe-a7e1-fb26e146dfbc\") " pod="openshift-dns/node-resolver-2v2dn" Jun 03 16:17:10.423208 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:10.422877 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nlgqh\" (UniqueName: \"kubernetes.io/projected/f601dafe-2b89-43a6-bb9e-2080b98aaa82-kube-api-access-nlgqh\") pod \"multus-additional-cni-plugins-wvl4d\" (UID: \"f601dafe-2b89-43a6-bb9e-2080b98aaa82\") " pod="openshift-multus/multus-additional-cni-plugins-wvl4d" Jun 03 16:17:10.423208 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:10.422892 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9c10c83c-4e52-4366-aef2-577fb3fa951f-tmp\") pod \"tuned-5fhsz\" (UID: \"9c10c83c-4e52-4366-aef2-577fb3fa951f\") " pod="openshift-cluster-node-tuning-operator/tuned-5fhsz" Jun 03 16:17:10.423208 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:10.422908 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/2b06ddd4-c92a-45e5-9538-03fe00b0503f-sys-fs\") pod \"aws-ebs-csi-driver-node-xvpwg\" (UID: \"2b06ddd4-c92a-45e5-9538-03fe00b0503f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xvpwg" Jun 03 16:17:10.423208 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:10.422921 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b0a52397-ab84-4efe-a7e1-fb26e146dfbc-tmp-dir\") pod \"node-resolver-2v2dn\" (UID: \"b0a52397-ab84-4efe-a7e1-fb26e146dfbc\") " pod="openshift-dns/node-resolver-2v2dn" Jun 03 16:17:10.423929 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:10.422954 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/e926d1e6-fe73-4a66-9488-1bd8d711ea79-log-socket\") pod \"ovnkube-node-xz69h\" (UID: \"e926d1e6-fe73-4a66-9488-1bd8d711ea79\") " pod="openshift-ovn-kubernetes/ovnkube-node-xz69h" Jun 03 16:17:10.423929 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:10.422978 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/80818dcb-cac7-4ac4-a376-ea67605a5ff8-multus-daemon-config\") pod \"multus-8s5gg\" (UID: \"80818dcb-cac7-4ac4-a376-ea67605a5ff8\") " pod="openshift-multus/multus-8s5gg" Jun 03 16:17:10.423929 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:10.422995 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/80818dcb-cac7-4ac4-a376-ea67605a5ff8-host-run-multus-certs\") pod \"multus-8s5gg\" (UID: \"80818dcb-cac7-4ac4-a376-ea67605a5ff8\") " pod="openshift-multus/multus-8s5gg" Jun 03 16:17:10.423929 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:10.423019 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e926d1e6-fe73-4a66-9488-1bd8d711ea79-etc-openvswitch\") pod \"ovnkube-node-xz69h\" (UID: \"e926d1e6-fe73-4a66-9488-1bd8d711ea79\") " pod="openshift-ovn-kubernetes/ovnkube-node-xz69h" Jun 03 16:17:10.423929 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:10.423034 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e926d1e6-fe73-4a66-9488-1bd8d711ea79-run-openvswitch\") pod \"ovnkube-node-xz69h\" (UID: \"e926d1e6-fe73-4a66-9488-1bd8d711ea79\") " pod="openshift-ovn-kubernetes/ovnkube-node-xz69h" Jun 03 16:17:10.423929 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:10.423048 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/e926d1e6-fe73-4a66-9488-1bd8d711ea79-node-log\") pod \"ovnkube-node-xz69h\" (UID: \"e926d1e6-fe73-4a66-9488-1bd8d711ea79\") " pod="openshift-ovn-kubernetes/ovnkube-node-xz69h" Jun 03 16:17:10.423929 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:10.423060 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e926d1e6-fe73-4a66-9488-1bd8d711ea79-host-cni-bin\") pod \"ovnkube-node-xz69h\" (UID: \"e926d1e6-fe73-4a66-9488-1bd8d711ea79\") " pod="openshift-ovn-kubernetes/ovnkube-node-xz69h" Jun 03 16:17:10.423929 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:10.423074 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f601dafe-2b89-43a6-bb9e-2080b98aaa82-system-cni-dir\") pod \"multus-additional-cni-plugins-wvl4d\" (UID: \"f601dafe-2b89-43a6-bb9e-2080b98aaa82\") " pod="openshift-multus/multus-additional-cni-plugins-wvl4d" Jun 03 16:17:10.423929 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:10.423088 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f601dafe-2b89-43a6-bb9e-2080b98aaa82-tuning-conf-dir\") pod \"multus-additional-cni-plugins-wvl4d\" (UID: \"f601dafe-2b89-43a6-bb9e-2080b98aaa82\") " pod="openshift-multus/multus-additional-cni-plugins-wvl4d" Jun 03 16:17:10.423929 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:10.423102 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j527q\" (UniqueName: \"kubernetes.io/projected/80818dcb-cac7-4ac4-a376-ea67605a5ff8-kube-api-access-j527q\") pod \"multus-8s5gg\" (UID: \"80818dcb-cac7-4ac4-a376-ea67605a5ff8\") " pod="openshift-multus/multus-8s5gg" Jun 03 16:17:10.423929 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:10.423139 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e926d1e6-fe73-4a66-9488-1bd8d711ea79-host-run-netns\") pod \"ovnkube-node-xz69h\" (UID: \"e926d1e6-fe73-4a66-9488-1bd8d711ea79\") " pod="openshift-ovn-kubernetes/ovnkube-node-xz69h" Jun 03 16:17:10.423929 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:10.423170 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/e926d1e6-fe73-4a66-9488-1bd8d711ea79-ovnkube-script-lib\") pod \"ovnkube-node-xz69h\" (UID: \"e926d1e6-fe73-4a66-9488-1bd8d711ea79\") " pod="openshift-ovn-kubernetes/ovnkube-node-xz69h" Jun 03 16:17:10.423929 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:10.423194 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/2b06ddd4-c92a-45e5-9538-03fe00b0503f-etc-selinux\") pod \"aws-ebs-csi-driver-node-xvpwg\" (UID: \"2b06ddd4-c92a-45e5-9538-03fe00b0503f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xvpwg" Jun 03 16:17:10.423929 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:10.423217 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9c10c83c-4e52-4366-aef2-577fb3fa951f-etc-kubernetes\") pod \"tuned-5fhsz\" (UID: \"9c10c83c-4e52-4366-aef2-577fb3fa951f\") " pod="openshift-cluster-node-tuning-operator/tuned-5fhsz" Jun 03 16:17:10.423929 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:10.423260 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/9c10c83c-4e52-4366-aef2-577fb3fa951f-lib-modules\") pod \"tuned-5fhsz\" (UID: \"9c10c83c-4e52-4366-aef2-577fb3fa951f\") " pod="openshift-cluster-node-tuning-operator/tuned-5fhsz" Jun 03 16:17:10.423929 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:10.423286 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9c10c83c-4e52-4366-aef2-577fb3fa951f-var-lib-kubelet\") pod \"tuned-5fhsz\" (UID: \"9c10c83c-4e52-4366-aef2-577fb3fa951f\") " pod="openshift-cluster-node-tuning-operator/tuned-5fhsz" Jun 03 16:17:10.424359 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:10.423310 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/80818dcb-cac7-4ac4-a376-ea67605a5ff8-multus-cni-dir\") pod \"multus-8s5gg\" (UID: \"80818dcb-cac7-4ac4-a376-ea67605a5ff8\") " pod="openshift-multus/multus-8s5gg" Jun 03 16:17:10.424359 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:10.423332 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/9c10c83c-4e52-4366-aef2-577fb3fa951f-etc-sysconfig\") pod \"tuned-5fhsz\" (UID: \"9c10c83c-4e52-4366-aef2-577fb3fa951f\") " pod="openshift-cluster-node-tuning-operator/tuned-5fhsz" Jun 03 16:17:10.424359 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:10.423357 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/e926d1e6-fe73-4a66-9488-1bd8d711ea79-ovn-node-metrics-cert\") pod \"ovnkube-node-xz69h\" (UID: \"e926d1e6-fe73-4a66-9488-1bd8d711ea79\") " pod="openshift-ovn-kubernetes/ovnkube-node-xz69h" Jun 03 16:17:10.424359 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:10.423380 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d93510f8-5732-4611-8aca-02573c7e296e-metrics-certs\") pod \"network-metrics-daemon-fhpnv\" (UID: \"d93510f8-5732-4611-8aca-02573c7e296e\") " pod="openshift-multus/network-metrics-daemon-fhpnv" Jun 03 16:17:10.424359 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:10.423402 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/2b06ddd4-c92a-45e5-9538-03fe00b0503f-device-dir\") pod \"aws-ebs-csi-driver-node-xvpwg\" (UID: \"2b06ddd4-c92a-45e5-9538-03fe00b0503f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xvpwg" Jun 03 16:17:10.424359 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:10.423432 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j6d64\" (UniqueName: \"kubernetes.io/projected/b0a52397-ab84-4efe-a7e1-fb26e146dfbc-kube-api-access-j6d64\") pod \"node-resolver-2v2dn\" (UID: \"b0a52397-ab84-4efe-a7e1-fb26e146dfbc\") " pod="openshift-dns/node-resolver-2v2dn" Jun 03 16:17:10.424359 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:10.423457 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ggmwq\" (UniqueName: \"kubernetes.io/projected/9c10c83c-4e52-4366-aef2-577fb3fa951f-kube-api-access-ggmwq\") pod \"tuned-5fhsz\" (UID: \"9c10c83c-4e52-4366-aef2-577fb3fa951f\") " pod="openshift-cluster-node-tuning-operator/tuned-5fhsz" Jun 03 16:17:10.424359 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:10.423477 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/80818dcb-cac7-4ac4-a376-ea67605a5ff8-host-var-lib-kubelet\") pod \"multus-8s5gg\" (UID: \"80818dcb-cac7-4ac4-a376-ea67605a5ff8\") " pod="openshift-multus/multus-8s5gg" Jun 03 16:17:10.424359 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:10.423492 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/80818dcb-cac7-4ac4-a376-ea67605a5ff8-etc-kubernetes\") pod \"multus-8s5gg\" (UID: \"80818dcb-cac7-4ac4-a376-ea67605a5ff8\") " pod="openshift-multus/multus-8s5gg" Jun 03 16:17:10.424359 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:10.423513 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9sgdq\" (UniqueName: \"kubernetes.io/projected/e926d1e6-fe73-4a66-9488-1bd8d711ea79-kube-api-access-9sgdq\") pod \"ovnkube-node-xz69h\" (UID: \"e926d1e6-fe73-4a66-9488-1bd8d711ea79\") " pod="openshift-ovn-kubernetes/ovnkube-node-xz69h" Jun 03 16:17:10.424359 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:10.423526 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/2d19a6a4-13e1-407c-a488-189a3651f578-serviceca\") pod \"node-ca-6jcbf\" (UID: \"2d19a6a4-13e1-407c-a488-189a3651f578\") " pod="openshift-image-registry/node-ca-6jcbf" Jun 03 16:17:10.424359 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:10.423599 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9c10c83c-4e52-4366-aef2-577fb3fa951f-sys\") pod \"tuned-5fhsz\" (UID: \"9c10c83c-4e52-4366-aef2-577fb3fa951f\") " pod="openshift-cluster-node-tuning-operator/tuned-5fhsz" Jun 03 16:17:10.424359 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:10.423635 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/9c10c83c-4e52-4366-aef2-577fb3fa951f-host\") pod \"tuned-5fhsz\" (UID: \"9c10c83c-4e52-4366-aef2-577fb3fa951f\") " pod="openshift-cluster-node-tuning-operator/tuned-5fhsz" Jun 03 16:17:10.424359 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:10.423675 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/80818dcb-cac7-4ac4-a376-ea67605a5ff8-hostroot\") pod \"multus-8s5gg\" (UID: \"80818dcb-cac7-4ac4-a376-ea67605a5ff8\") " pod="openshift-multus/multus-8s5gg" Jun 03 16:17:10.424359 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:10.423698 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/e926d1e6-fe73-4a66-9488-1bd8d711ea79-host-slash\") pod \"ovnkube-node-xz69h\" (UID: \"e926d1e6-fe73-4a66-9488-1bd8d711ea79\") " pod="openshift-ovn-kubernetes/ovnkube-node-xz69h" Jun 03 16:17:10.424359 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:10.423720 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6b6n4\" (UniqueName: \"kubernetes.io/projected/6bd7702f-6b77-4535-993e-d1e053d3771b-kube-api-access-6b6n4\") pod \"iptables-alerter-rmdrz\" (UID: \"6bd7702f-6b77-4535-993e-d1e053d3771b\") " pod="openshift-network-operator/iptables-alerter-rmdrz" Jun 03 16:17:10.424359 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:10.423746 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/e926d1e6-fe73-4a66-9488-1bd8d711ea79-systemd-units\") pod \"ovnkube-node-xz69h\" (UID: \"e926d1e6-fe73-4a66-9488-1bd8d711ea79\") " pod="openshift-ovn-kubernetes/ovnkube-node-xz69h" Jun 03 16:17:10.424889 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:10.423767 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/e926d1e6-fe73-4a66-9488-1bd8d711ea79-env-overrides\") pod \"ovnkube-node-xz69h\" (UID: \"e926d1e6-fe73-4a66-9488-1bd8d711ea79\") " pod="openshift-ovn-kubernetes/ovnkube-node-xz69h" Jun 03 16:17:10.426380 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:10.426363 2568 reflector.go:430] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jun 03 16:17:10.457605 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:10.457583 2568 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-pvn5n" Jun 03 16:17:10.465450 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:10.465434 2568 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-pvn5n" Jun 03 16:17:10.524362 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:10.524342 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-9sgdq\" (UniqueName: \"kubernetes.io/projected/e926d1e6-fe73-4a66-9488-1bd8d711ea79-kube-api-access-9sgdq\") pod \"ovnkube-node-xz69h\" (UID: \"e926d1e6-fe73-4a66-9488-1bd8d711ea79\") " pod="openshift-ovn-kubernetes/ovnkube-node-xz69h" Jun 03 16:17:10.524443 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:10.524370 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/2d19a6a4-13e1-407c-a488-189a3651f578-serviceca\") pod \"node-ca-6jcbf\" (UID: \"2d19a6a4-13e1-407c-a488-189a3651f578\") " pod="openshift-image-registry/node-ca-6jcbf" Jun 03 16:17:10.524443 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:10.524387 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9c10c83c-4e52-4366-aef2-577fb3fa951f-sys\") pod \"tuned-5fhsz\" (UID: \"9c10c83c-4e52-4366-aef2-577fb3fa951f\") " pod="openshift-cluster-node-tuning-operator/tuned-5fhsz" Jun 03 16:17:10.524626 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:10.524444 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9c10c83c-4e52-4366-aef2-577fb3fa951f-sys\") pod \"tuned-5fhsz\" (UID: \"9c10c83c-4e52-4366-aef2-577fb3fa951f\") " pod="openshift-cluster-node-tuning-operator/tuned-5fhsz" Jun 03 16:17:10.524626 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:10.524443 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/9c10c83c-4e52-4366-aef2-577fb3fa951f-host\") pod \"tuned-5fhsz\" (UID: \"9c10c83c-4e52-4366-aef2-577fb3fa951f\") " pod="openshift-cluster-node-tuning-operator/tuned-5fhsz" Jun 03 16:17:10.524626 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:10.524482 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/80818dcb-cac7-4ac4-a376-ea67605a5ff8-hostroot\") pod \"multus-8s5gg\" (UID: \"80818dcb-cac7-4ac4-a376-ea67605a5ff8\") " pod="openshift-multus/multus-8s5gg" Jun 03 16:17:10.524626 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:10.524505 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/e926d1e6-fe73-4a66-9488-1bd8d711ea79-host-slash\") pod \"ovnkube-node-xz69h\" (UID: \"e926d1e6-fe73-4a66-9488-1bd8d711ea79\") " pod="openshift-ovn-kubernetes/ovnkube-node-xz69h" Jun 03 16:17:10.524626 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:10.524545 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-6b6n4\" (UniqueName: \"kubernetes.io/projected/6bd7702f-6b77-4535-993e-d1e053d3771b-kube-api-access-6b6n4\") pod \"iptables-alerter-rmdrz\" (UID: \"6bd7702f-6b77-4535-993e-d1e053d3771b\") " pod="openshift-network-operator/iptables-alerter-rmdrz" Jun 03 16:17:10.524626 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:10.524591 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/9c10c83c-4e52-4366-aef2-577fb3fa951f-host\") pod \"tuned-5fhsz\" (UID: \"9c10c83c-4e52-4366-aef2-577fb3fa951f\") " pod="openshift-cluster-node-tuning-operator/tuned-5fhsz" Jun 03 16:17:10.524626 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:10.524591 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/80818dcb-cac7-4ac4-a376-ea67605a5ff8-hostroot\") pod \"multus-8s5gg\" (UID: \"80818dcb-cac7-4ac4-a376-ea67605a5ff8\") " pod="openshift-multus/multus-8s5gg" Jun 03 16:17:10.524626 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:10.524591 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/e926d1e6-fe73-4a66-9488-1bd8d711ea79-host-slash\") pod \"ovnkube-node-xz69h\" (UID: \"e926d1e6-fe73-4a66-9488-1bd8d711ea79\") " pod="openshift-ovn-kubernetes/ovnkube-node-xz69h" Jun 03 16:17:10.524626 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:10.524614 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/e926d1e6-fe73-4a66-9488-1bd8d711ea79-systemd-units\") pod \"ovnkube-node-xz69h\" (UID: \"e926d1e6-fe73-4a66-9488-1bd8d711ea79\") " pod="openshift-ovn-kubernetes/ovnkube-node-xz69h" Jun 03 16:17:10.525390 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:10.524644 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/e926d1e6-fe73-4a66-9488-1bd8d711ea79-env-overrides\") pod \"ovnkube-node-xz69h\" (UID: \"e926d1e6-fe73-4a66-9488-1bd8d711ea79\") " pod="openshift-ovn-kubernetes/ovnkube-node-xz69h" Jun 03 16:17:10.525390 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:10.524688 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-9xhts\" (UniqueName: \"kubernetes.io/projected/2d19a6a4-13e1-407c-a488-189a3651f578-kube-api-access-9xhts\") pod \"node-ca-6jcbf\" (UID: \"2d19a6a4-13e1-407c-a488-189a3651f578\") " pod="openshift-image-registry/node-ca-6jcbf" Jun 03 16:17:10.525390 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:10.524705 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-tdcd7\" (UniqueName: \"kubernetes.io/projected/b8482073-6356-4d6e-bddf-2c8152f6a4f6-kube-api-access-tdcd7\") pod \"network-check-target-m7zkz\" (UID: \"b8482073-6356-4d6e-bddf-2c8152f6a4f6\") " pod="openshift-network-diagnostics/network-check-target-m7zkz" Jun 03 16:17:10.525390 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:10.524647 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/e926d1e6-fe73-4a66-9488-1bd8d711ea79-systemd-units\") pod \"ovnkube-node-xz69h\" (UID: \"e926d1e6-fe73-4a66-9488-1bd8d711ea79\") " pod="openshift-ovn-kubernetes/ovnkube-node-xz69h" Jun 03 16:17:10.525390 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:10.524727 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/f601dafe-2b89-43a6-bb9e-2080b98aaa82-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-wvl4d\" (UID: \"f601dafe-2b89-43a6-bb9e-2080b98aaa82\") " pod="openshift-multus/multus-additional-cni-plugins-wvl4d" Jun 03 16:17:10.525390 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:10.524750 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/9c10c83c-4e52-4366-aef2-577fb3fa951f-etc-systemd\") pod \"tuned-5fhsz\" (UID: \"9c10c83c-4e52-4366-aef2-577fb3fa951f\") " pod="openshift-cluster-node-tuning-operator/tuned-5fhsz" Jun 03 16:17:10.525390 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:10.524774 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/9c10c83c-4e52-4366-aef2-577fb3fa951f-etc-tuned\") pod \"tuned-5fhsz\" (UID: \"9c10c83c-4e52-4366-aef2-577fb3fa951f\") " pod="openshift-cluster-node-tuning-operator/tuned-5fhsz" Jun 03 16:17:10.525390 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:10.524797 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/e926d1e6-fe73-4a66-9488-1bd8d711ea79-run-systemd\") pod \"ovnkube-node-xz69h\" (UID: \"e926d1e6-fe73-4a66-9488-1bd8d711ea79\") " pod="openshift-ovn-kubernetes/ovnkube-node-xz69h" Jun 03 16:17:10.525390 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:10.524820 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/2b06ddd4-c92a-45e5-9538-03fe00b0503f-kubelet-dir\") pod \"aws-ebs-csi-driver-node-xvpwg\" (UID: \"2b06ddd4-c92a-45e5-9538-03fe00b0503f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xvpwg" Jun 03 16:17:10.525390 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:10.524843 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/2b06ddd4-c92a-45e5-9538-03fe00b0503f-socket-dir\") pod \"aws-ebs-csi-driver-node-xvpwg\" (UID: \"2b06ddd4-c92a-45e5-9538-03fe00b0503f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xvpwg" Jun 03 16:17:10.525390 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:10.524866 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/2b06ddd4-c92a-45e5-9538-03fe00b0503f-registration-dir\") pod \"aws-ebs-csi-driver-node-xvpwg\" (UID: \"2b06ddd4-c92a-45e5-9538-03fe00b0503f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xvpwg" Jun 03 16:17:10.525390 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:10.524900 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/80818dcb-cac7-4ac4-a376-ea67605a5ff8-host-var-lib-cni-bin\") pod \"multus-8s5gg\" (UID: \"80818dcb-cac7-4ac4-a376-ea67605a5ff8\") " pod="openshift-multus/multus-8s5gg" Jun 03 16:17:10.525390 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:10.524901 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/e926d1e6-fe73-4a66-9488-1bd8d711ea79-run-systemd\") pod \"ovnkube-node-xz69h\" (UID: \"e926d1e6-fe73-4a66-9488-1bd8d711ea79\") " pod="openshift-ovn-kubernetes/ovnkube-node-xz69h" Jun 03 16:17:10.525390 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:10.524938 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/e926d1e6-fe73-4a66-9488-1bd8d711ea79-run-ovn\") pod \"ovnkube-node-xz69h\" (UID: \"e926d1e6-fe73-4a66-9488-1bd8d711ea79\") " pod="openshift-ovn-kubernetes/ovnkube-node-xz69h" Jun 03 16:17:10.525390 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:10.524950 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/80818dcb-cac7-4ac4-a376-ea67605a5ff8-host-var-lib-cni-bin\") pod \"multus-8s5gg\" (UID: \"80818dcb-cac7-4ac4-a376-ea67605a5ff8\") " pod="openshift-multus/multus-8s5gg" Jun 03 16:17:10.525390 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:10.524972 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e926d1e6-fe73-4a66-9488-1bd8d711ea79-host-run-ovn-kubernetes\") pod \"ovnkube-node-xz69h\" (UID: \"e926d1e6-fe73-4a66-9488-1bd8d711ea79\") " pod="openshift-ovn-kubernetes/ovnkube-node-xz69h" Jun 03 16:17:10.525390 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:10.524999 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/9c10c83c-4e52-4366-aef2-577fb3fa951f-etc-systemd\") pod \"tuned-5fhsz\" (UID: \"9c10c83c-4e52-4366-aef2-577fb3fa951f\") " pod="openshift-cluster-node-tuning-operator/tuned-5fhsz" Jun 03 16:17:10.526172 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:10.525010 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/e926d1e6-fe73-4a66-9488-1bd8d711ea79-ovnkube-config\") pod \"ovnkube-node-xz69h\" (UID: \"e926d1e6-fe73-4a66-9488-1bd8d711ea79\") " pod="openshift-ovn-kubernetes/ovnkube-node-xz69h" Jun 03 16:17:10.526172 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:10.525018 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/2b06ddd4-c92a-45e5-9538-03fe00b0503f-kubelet-dir\") pod \"aws-ebs-csi-driver-node-xvpwg\" (UID: \"2b06ddd4-c92a-45e5-9538-03fe00b0503f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xvpwg" Jun 03 16:17:10.526172 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:10.525033 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/2d19a6a4-13e1-407c-a488-189a3651f578-host\") pod \"node-ca-6jcbf\" (UID: \"2d19a6a4-13e1-407c-a488-189a3651f578\") " pod="openshift-image-registry/node-ca-6jcbf" Jun 03 16:17:10.526172 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:10.525044 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/e926d1e6-fe73-4a66-9488-1bd8d711ea79-run-ovn\") pod \"ovnkube-node-xz69h\" (UID: \"e926d1e6-fe73-4a66-9488-1bd8d711ea79\") " pod="openshift-ovn-kubernetes/ovnkube-node-xz69h" Jun 03 16:17:10.526172 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:10.525057 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/9c10c83c-4e52-4366-aef2-577fb3fa951f-etc-modprobe-d\") pod \"tuned-5fhsz\" (UID: \"9c10c83c-4e52-4366-aef2-577fb3fa951f\") " pod="openshift-cluster-node-tuning-operator/tuned-5fhsz" Jun 03 16:17:10.526172 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:10.525082 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/9c10c83c-4e52-4366-aef2-577fb3fa951f-etc-sysctl-conf\") pod \"tuned-5fhsz\" (UID: \"9c10c83c-4e52-4366-aef2-577fb3fa951f\") " pod="openshift-cluster-node-tuning-operator/tuned-5fhsz" Jun 03 16:17:10.526172 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:10.525093 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/2d19a6a4-13e1-407c-a488-189a3651f578-serviceca\") pod \"node-ca-6jcbf\" (UID: \"2d19a6a4-13e1-407c-a488-189a3651f578\") " pod="openshift-image-registry/node-ca-6jcbf" Jun 03 16:17:10.526172 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:10.525105 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/9c10c83c-4e52-4366-aef2-577fb3fa951f-run\") pod \"tuned-5fhsz\" (UID: \"9c10c83c-4e52-4366-aef2-577fb3fa951f\") " pod="openshift-cluster-node-tuning-operator/tuned-5fhsz" Jun 03 16:17:10.526172 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:10.525127 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/2b06ddd4-c92a-45e5-9538-03fe00b0503f-socket-dir\") pod \"aws-ebs-csi-driver-node-xvpwg\" (UID: \"2b06ddd4-c92a-45e5-9538-03fe00b0503f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xvpwg" Jun 03 16:17:10.526172 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:10.525133 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/80818dcb-cac7-4ac4-a376-ea67605a5ff8-os-release\") pod \"multus-8s5gg\" (UID: \"80818dcb-cac7-4ac4-a376-ea67605a5ff8\") " pod="openshift-multus/multus-8s5gg" Jun 03 16:17:10.526172 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:10.525159 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/6bd7702f-6b77-4535-993e-d1e053d3771b-host-slash\") pod \"iptables-alerter-rmdrz\" (UID: \"6bd7702f-6b77-4535-993e-d1e053d3771b\") " pod="openshift-network-operator/iptables-alerter-rmdrz" Jun 03 16:17:10.526172 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:10.525179 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/2b06ddd4-c92a-45e5-9538-03fe00b0503f-registration-dir\") pod \"aws-ebs-csi-driver-node-xvpwg\" (UID: \"2b06ddd4-c92a-45e5-9538-03fe00b0503f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xvpwg" Jun 03 16:17:10.526172 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:10.525185 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/80818dcb-cac7-4ac4-a376-ea67605a5ff8-cni-binary-copy\") pod \"multus-8s5gg\" (UID: \"80818dcb-cac7-4ac4-a376-ea67605a5ff8\") " pod="openshift-multus/multus-8s5gg" Jun 03 16:17:10.526172 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:10.525192 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/9c10c83c-4e52-4366-aef2-577fb3fa951f-run\") pod \"tuned-5fhsz\" (UID: \"9c10c83c-4e52-4366-aef2-577fb3fa951f\") " pod="openshift-cluster-node-tuning-operator/tuned-5fhsz" Jun 03 16:17:10.526172 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:10.525115 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/e926d1e6-fe73-4a66-9488-1bd8d711ea79-env-overrides\") pod \"ovnkube-node-xz69h\" (UID: \"e926d1e6-fe73-4a66-9488-1bd8d711ea79\") " pod="openshift-ovn-kubernetes/ovnkube-node-xz69h" Jun 03 16:17:10.526172 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:10.525210 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/e926d1e6-fe73-4a66-9488-1bd8d711ea79-host-kubelet\") pod \"ovnkube-node-xz69h\" (UID: \"e926d1e6-fe73-4a66-9488-1bd8d711ea79\") " pod="openshift-ovn-kubernetes/ovnkube-node-xz69h" Jun 03 16:17:10.526172 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:10.525214 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/9c10c83c-4e52-4366-aef2-577fb3fa951f-etc-modprobe-d\") pod \"tuned-5fhsz\" (UID: \"9c10c83c-4e52-4366-aef2-577fb3fa951f\") " pod="openshift-cluster-node-tuning-operator/tuned-5fhsz" Jun 03 16:17:10.526172 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:10.525256 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/e926d1e6-fe73-4a66-9488-1bd8d711ea79-host-kubelet\") pod \"ovnkube-node-xz69h\" (UID: \"e926d1e6-fe73-4a66-9488-1bd8d711ea79\") " pod="openshift-ovn-kubernetes/ovnkube-node-xz69h" Jun 03 16:17:10.527087 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:10.525259 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e926d1e6-fe73-4a66-9488-1bd8d711ea79-host-run-ovn-kubernetes\") pod \"ovnkube-node-xz69h\" (UID: \"e926d1e6-fe73-4a66-9488-1bd8d711ea79\") " pod="openshift-ovn-kubernetes/ovnkube-node-xz69h" Jun 03 16:17:10.527087 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:10.525265 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/80818dcb-cac7-4ac4-a376-ea67605a5ff8-os-release\") pod \"multus-8s5gg\" (UID: \"80818dcb-cac7-4ac4-a376-ea67605a5ff8\") " pod="openshift-multus/multus-8s5gg" Jun 03 16:17:10.527087 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:10.525260 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/9c10c83c-4e52-4366-aef2-577fb3fa951f-etc-sysctl-conf\") pod \"tuned-5fhsz\" (UID: \"9c10c83c-4e52-4366-aef2-577fb3fa951f\") " pod="openshift-cluster-node-tuning-operator/tuned-5fhsz" Jun 03 16:17:10.527087 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:10.525304 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e926d1e6-fe73-4a66-9488-1bd8d711ea79-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-xz69h\" (UID: \"e926d1e6-fe73-4a66-9488-1bd8d711ea79\") " pod="openshift-ovn-kubernetes/ovnkube-node-xz69h" Jun 03 16:17:10.527087 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:10.525319 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/6bd7702f-6b77-4535-993e-d1e053d3771b-host-slash\") pod \"iptables-alerter-rmdrz\" (UID: \"6bd7702f-6b77-4535-993e-d1e053d3771b\") " pod="openshift-network-operator/iptables-alerter-rmdrz" Jun 03 16:17:10.527087 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:10.525327 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/2d19a6a4-13e1-407c-a488-189a3651f578-host\") pod \"node-ca-6jcbf\" (UID: \"2d19a6a4-13e1-407c-a488-189a3651f578\") " pod="openshift-image-registry/node-ca-6jcbf" Jun 03 16:17:10.527087 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:10.525338 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/130349f2-fa26-4e51-bf29-6819611104db-konnectivity-ca\") pod \"konnectivity-agent-5hlfw\" (UID: \"130349f2-fa26-4e51-bf29-6819611104db\") " pod="kube-system/konnectivity-agent-5hlfw" Jun 03 16:17:10.527087 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:10.525356 2568 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jun 03 16:17:10.527087 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:10.525363 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f601dafe-2b89-43a6-bb9e-2080b98aaa82-cnibin\") pod \"multus-additional-cni-plugins-wvl4d\" (UID: \"f601dafe-2b89-43a6-bb9e-2080b98aaa82\") " pod="openshift-multus/multus-additional-cni-plugins-wvl4d" Jun 03 16:17:10.527087 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:10.525386 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/f601dafe-2b89-43a6-bb9e-2080b98aaa82-os-release\") pod \"multus-additional-cni-plugins-wvl4d\" (UID: \"f601dafe-2b89-43a6-bb9e-2080b98aaa82\") " pod="openshift-multus/multus-additional-cni-plugins-wvl4d" Jun 03 16:17:10.527087 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:10.525384 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e926d1e6-fe73-4a66-9488-1bd8d711ea79-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-xz69h\" (UID: \"e926d1e6-fe73-4a66-9488-1bd8d711ea79\") " pod="openshift-ovn-kubernetes/ovnkube-node-xz69h" Jun 03 16:17:10.527087 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:10.525411 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f601dafe-2b89-43a6-bb9e-2080b98aaa82-cni-binary-copy\") pod \"multus-additional-cni-plugins-wvl4d\" (UID: \"f601dafe-2b89-43a6-bb9e-2080b98aaa82\") " pod="openshift-multus/multus-additional-cni-plugins-wvl4d" Jun 03 16:17:10.527087 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:10.525434 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/9c10c83c-4e52-4366-aef2-577fb3fa951f-etc-sysctl-d\") pod \"tuned-5fhsz\" (UID: \"9c10c83c-4e52-4366-aef2-577fb3fa951f\") " pod="openshift-cluster-node-tuning-operator/tuned-5fhsz" Jun 03 16:17:10.527087 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:10.525458 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/80818dcb-cac7-4ac4-a376-ea67605a5ff8-host-run-k8s-cni-cncf-io\") pod \"multus-8s5gg\" (UID: \"80818dcb-cac7-4ac4-a376-ea67605a5ff8\") " pod="openshift-multus/multus-8s5gg" Jun 03 16:17:10.527087 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:10.525463 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/f601dafe-2b89-43a6-bb9e-2080b98aaa82-os-release\") pod \"multus-additional-cni-plugins-wvl4d\" (UID: \"f601dafe-2b89-43a6-bb9e-2080b98aaa82\") " pod="openshift-multus/multus-additional-cni-plugins-wvl4d" Jun 03 16:17:10.527087 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:10.525483 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/80818dcb-cac7-4ac4-a376-ea67605a5ff8-host-run-netns\") pod \"multus-8s5gg\" (UID: \"80818dcb-cac7-4ac4-a376-ea67605a5ff8\") " pod="openshift-multus/multus-8s5gg" Jun 03 16:17:10.527087 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:10.525506 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e926d1e6-fe73-4a66-9488-1bd8d711ea79-var-lib-openvswitch\") pod \"ovnkube-node-xz69h\" (UID: \"e926d1e6-fe73-4a66-9488-1bd8d711ea79\") " pod="openshift-ovn-kubernetes/ovnkube-node-xz69h" Jun 03 16:17:10.527882 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:10.525545 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/f601dafe-2b89-43a6-bb9e-2080b98aaa82-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-wvl4d\" (UID: \"f601dafe-2b89-43a6-bb9e-2080b98aaa82\") " pod="openshift-multus/multus-additional-cni-plugins-wvl4d" Jun 03 16:17:10.527882 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:10.525570 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/9c10c83c-4e52-4366-aef2-577fb3fa951f-etc-sysctl-d\") pod \"tuned-5fhsz\" (UID: \"9c10c83c-4e52-4366-aef2-577fb3fa951f\") " pod="openshift-cluster-node-tuning-operator/tuned-5fhsz" Jun 03 16:17:10.527882 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:10.525573 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/80818dcb-cac7-4ac4-a376-ea67605a5ff8-multus-socket-dir-parent\") pod \"multus-8s5gg\" (UID: \"80818dcb-cac7-4ac4-a376-ea67605a5ff8\") " pod="openshift-multus/multus-8s5gg" Jun 03 16:17:10.527882 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:10.525621 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/80818dcb-cac7-4ac4-a376-ea67605a5ff8-host-run-k8s-cni-cncf-io\") pod \"multus-8s5gg\" (UID: \"80818dcb-cac7-4ac4-a376-ea67605a5ff8\") " pod="openshift-multus/multus-8s5gg" Jun 03 16:17:10.527882 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:10.525663 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/80818dcb-cac7-4ac4-a376-ea67605a5ff8-host-run-netns\") pod \"multus-8s5gg\" (UID: \"80818dcb-cac7-4ac4-a376-ea67605a5ff8\") " pod="openshift-multus/multus-8s5gg" Jun 03 16:17:10.527882 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:10.525702 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e926d1e6-fe73-4a66-9488-1bd8d711ea79-var-lib-openvswitch\") pod \"ovnkube-node-xz69h\" (UID: \"e926d1e6-fe73-4a66-9488-1bd8d711ea79\") " pod="openshift-ovn-kubernetes/ovnkube-node-xz69h" Jun 03 16:17:10.527882 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:10.525748 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/e926d1e6-fe73-4a66-9488-1bd8d711ea79-ovnkube-config\") pod \"ovnkube-node-xz69h\" (UID: \"e926d1e6-fe73-4a66-9488-1bd8d711ea79\") " pod="openshift-ovn-kubernetes/ovnkube-node-xz69h" Jun 03 16:17:10.527882 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:10.525824 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/80818dcb-cac7-4ac4-a376-ea67605a5ff8-multus-socket-dir-parent\") pod \"multus-8s5gg\" (UID: \"80818dcb-cac7-4ac4-a376-ea67605a5ff8\") " pod="openshift-multus/multus-8s5gg" Jun 03 16:17:10.527882 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:10.525877 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f601dafe-2b89-43a6-bb9e-2080b98aaa82-cnibin\") pod \"multus-additional-cni-plugins-wvl4d\" (UID: \"f601dafe-2b89-43a6-bb9e-2080b98aaa82\") " pod="openshift-multus/multus-additional-cni-plugins-wvl4d" Jun 03 16:17:10.527882 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:10.525908 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/80818dcb-cac7-4ac4-a376-ea67605a5ff8-host-var-lib-cni-multus\") pod \"multus-8s5gg\" (UID: \"80818dcb-cac7-4ac4-a376-ea67605a5ff8\") " pod="openshift-multus/multus-8s5gg" Jun 03 16:17:10.527882 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:10.525936 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/e926d1e6-fe73-4a66-9488-1bd8d711ea79-host-cni-netd\") pod \"ovnkube-node-xz69h\" (UID: \"e926d1e6-fe73-4a66-9488-1bd8d711ea79\") " pod="openshift-ovn-kubernetes/ovnkube-node-xz69h" Jun 03 16:17:10.527882 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:10.525972 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/130349f2-fa26-4e51-bf29-6819611104db-agent-certs\") pod \"konnectivity-agent-5hlfw\" (UID: \"130349f2-fa26-4e51-bf29-6819611104db\") " pod="kube-system/konnectivity-agent-5hlfw" Jun 03 16:17:10.527882 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:10.525988 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/6bd7702f-6b77-4535-993e-d1e053d3771b-iptables-alerter-script\") pod \"iptables-alerter-rmdrz\" (UID: \"6bd7702f-6b77-4535-993e-d1e053d3771b\") " pod="openshift-network-operator/iptables-alerter-rmdrz" Jun 03 16:17:10.527882 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:10.526004 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/80818dcb-cac7-4ac4-a376-ea67605a5ff8-system-cni-dir\") pod \"multus-8s5gg\" (UID: \"80818dcb-cac7-4ac4-a376-ea67605a5ff8\") " pod="openshift-multus/multus-8s5gg" Jun 03 16:17:10.527882 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:10.526007 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/80818dcb-cac7-4ac4-a376-ea67605a5ff8-host-var-lib-cni-multus\") pod \"multus-8s5gg\" (UID: \"80818dcb-cac7-4ac4-a376-ea67605a5ff8\") " pod="openshift-multus/multus-8s5gg" Jun 03 16:17:10.527882 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:10.526024 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/80818dcb-cac7-4ac4-a376-ea67605a5ff8-cnibin\") pod \"multus-8s5gg\" (UID: \"80818dcb-cac7-4ac4-a376-ea67605a5ff8\") " pod="openshift-multus/multus-8s5gg" Jun 03 16:17:10.527882 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:10.526064 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/80818dcb-cac7-4ac4-a376-ea67605a5ff8-multus-conf-dir\") pod \"multus-8s5gg\" (UID: \"80818dcb-cac7-4ac4-a376-ea67605a5ff8\") " pod="openshift-multus/multus-8s5gg" Jun 03 16:17:10.527882 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:10.526070 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/130349f2-fa26-4e51-bf29-6819611104db-konnectivity-ca\") pod \"konnectivity-agent-5hlfw\" (UID: \"130349f2-fa26-4e51-bf29-6819611104db\") " pod="kube-system/konnectivity-agent-5hlfw" Jun 03 16:17:10.528906 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:10.526092 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-77tzv\" (UniqueName: \"kubernetes.io/projected/d93510f8-5732-4611-8aca-02573c7e296e-kube-api-access-77tzv\") pod \"network-metrics-daemon-fhpnv\" (UID: \"d93510f8-5732-4611-8aca-02573c7e296e\") " pod="openshift-multus/network-metrics-daemon-fhpnv" Jun 03 16:17:10.528906 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:10.526067 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/80818dcb-cac7-4ac4-a376-ea67605a5ff8-cnibin\") pod \"multus-8s5gg\" (UID: \"80818dcb-cac7-4ac4-a376-ea67605a5ff8\") " pod="openshift-multus/multus-8s5gg" Jun 03 16:17:10.528906 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:10.526092 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/80818dcb-cac7-4ac4-a376-ea67605a5ff8-cni-binary-copy\") pod \"multus-8s5gg\" (UID: \"80818dcb-cac7-4ac4-a376-ea67605a5ff8\") " pod="openshift-multus/multus-8s5gg" Jun 03 16:17:10.528906 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:10.526121 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-tzc6j\" (UniqueName: \"kubernetes.io/projected/2b06ddd4-c92a-45e5-9538-03fe00b0503f-kube-api-access-tzc6j\") pod \"aws-ebs-csi-driver-node-xvpwg\" (UID: \"2b06ddd4-c92a-45e5-9538-03fe00b0503f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xvpwg" Jun 03 16:17:10.528906 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:10.526123 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/80818dcb-cac7-4ac4-a376-ea67605a5ff8-multus-conf-dir\") pod \"multus-8s5gg\" (UID: \"80818dcb-cac7-4ac4-a376-ea67605a5ff8\") " pod="openshift-multus/multus-8s5gg" Jun 03 16:17:10.528906 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:10.526094 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/e926d1e6-fe73-4a66-9488-1bd8d711ea79-host-cni-netd\") pod \"ovnkube-node-xz69h\" (UID: \"e926d1e6-fe73-4a66-9488-1bd8d711ea79\") " pod="openshift-ovn-kubernetes/ovnkube-node-xz69h" Jun 03 16:17:10.528906 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:10.526140 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/80818dcb-cac7-4ac4-a376-ea67605a5ff8-system-cni-dir\") pod \"multus-8s5gg\" (UID: \"80818dcb-cac7-4ac4-a376-ea67605a5ff8\") " pod="openshift-multus/multus-8s5gg" Jun 03 16:17:10.528906 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:10.526147 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/b0a52397-ab84-4efe-a7e1-fb26e146dfbc-hosts-file\") pod \"node-resolver-2v2dn\" (UID: \"b0a52397-ab84-4efe-a7e1-fb26e146dfbc\") " pod="openshift-dns/node-resolver-2v2dn" Jun 03 16:17:10.528906 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:10.526165 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f601dafe-2b89-43a6-bb9e-2080b98aaa82-cni-binary-copy\") pod \"multus-additional-cni-plugins-wvl4d\" (UID: \"f601dafe-2b89-43a6-bb9e-2080b98aaa82\") " pod="openshift-multus/multus-additional-cni-plugins-wvl4d" Jun 03 16:17:10.528906 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:10.526272 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-nlgqh\" (UniqueName: \"kubernetes.io/projected/f601dafe-2b89-43a6-bb9e-2080b98aaa82-kube-api-access-nlgqh\") pod \"multus-additional-cni-plugins-wvl4d\" (UID: \"f601dafe-2b89-43a6-bb9e-2080b98aaa82\") " pod="openshift-multus/multus-additional-cni-plugins-wvl4d" Jun 03 16:17:10.528906 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:10.526282 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/b0a52397-ab84-4efe-a7e1-fb26e146dfbc-hosts-file\") pod \"node-resolver-2v2dn\" (UID: \"b0a52397-ab84-4efe-a7e1-fb26e146dfbc\") " pod="openshift-dns/node-resolver-2v2dn" Jun 03 16:17:10.528906 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:10.526300 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9c10c83c-4e52-4366-aef2-577fb3fa951f-tmp\") pod \"tuned-5fhsz\" (UID: \"9c10c83c-4e52-4366-aef2-577fb3fa951f\") " pod="openshift-cluster-node-tuning-operator/tuned-5fhsz" Jun 03 16:17:10.528906 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:10.526325 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/2b06ddd4-c92a-45e5-9538-03fe00b0503f-sys-fs\") pod \"aws-ebs-csi-driver-node-xvpwg\" (UID: \"2b06ddd4-c92a-45e5-9538-03fe00b0503f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xvpwg" Jun 03 16:17:10.528906 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:10.526349 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b0a52397-ab84-4efe-a7e1-fb26e146dfbc-tmp-dir\") pod \"node-resolver-2v2dn\" (UID: \"b0a52397-ab84-4efe-a7e1-fb26e146dfbc\") " pod="openshift-dns/node-resolver-2v2dn" Jun 03 16:17:10.528906 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:10.526404 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/e926d1e6-fe73-4a66-9488-1bd8d711ea79-log-socket\") pod \"ovnkube-node-xz69h\" (UID: \"e926d1e6-fe73-4a66-9488-1bd8d711ea79\") " pod="openshift-ovn-kubernetes/ovnkube-node-xz69h" Jun 03 16:17:10.528906 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:10.526428 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/2b06ddd4-c92a-45e5-9538-03fe00b0503f-sys-fs\") pod \"aws-ebs-csi-driver-node-xvpwg\" (UID: \"2b06ddd4-c92a-45e5-9538-03fe00b0503f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xvpwg" Jun 03 16:17:10.528906 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:10.526431 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/80818dcb-cac7-4ac4-a376-ea67605a5ff8-multus-daemon-config\") pod \"multus-8s5gg\" (UID: \"80818dcb-cac7-4ac4-a376-ea67605a5ff8\") " pod="openshift-multus/multus-8s5gg" Jun 03 16:17:10.528906 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:10.526370 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/f601dafe-2b89-43a6-bb9e-2080b98aaa82-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-wvl4d\" (UID: \"f601dafe-2b89-43a6-bb9e-2080b98aaa82\") " pod="openshift-multus/multus-additional-cni-plugins-wvl4d" Jun 03 16:17:10.529784 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:10.526482 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/80818dcb-cac7-4ac4-a376-ea67605a5ff8-host-run-multus-certs\") pod \"multus-8s5gg\" (UID: \"80818dcb-cac7-4ac4-a376-ea67605a5ff8\") " pod="openshift-multus/multus-8s5gg" Jun 03 16:17:10.529784 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:10.526512 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e926d1e6-fe73-4a66-9488-1bd8d711ea79-etc-openvswitch\") pod \"ovnkube-node-xz69h\" (UID: \"e926d1e6-fe73-4a66-9488-1bd8d711ea79\") " pod="openshift-ovn-kubernetes/ovnkube-node-xz69h" Jun 03 16:17:10.529784 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:10.526561 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/e926d1e6-fe73-4a66-9488-1bd8d711ea79-log-socket\") pod \"ovnkube-node-xz69h\" (UID: \"e926d1e6-fe73-4a66-9488-1bd8d711ea79\") " pod="openshift-ovn-kubernetes/ovnkube-node-xz69h" Jun 03 16:17:10.529784 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:10.526556 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e926d1e6-fe73-4a66-9488-1bd8d711ea79-run-openvswitch\") pod \"ovnkube-node-xz69h\" (UID: \"e926d1e6-fe73-4a66-9488-1bd8d711ea79\") " pod="openshift-ovn-kubernetes/ovnkube-node-xz69h" Jun 03 16:17:10.529784 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:10.526620 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/e926d1e6-fe73-4a66-9488-1bd8d711ea79-node-log\") pod \"ovnkube-node-xz69h\" (UID: \"e926d1e6-fe73-4a66-9488-1bd8d711ea79\") " pod="openshift-ovn-kubernetes/ovnkube-node-xz69h" Jun 03 16:17:10.529784 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:10.526673 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e926d1e6-fe73-4a66-9488-1bd8d711ea79-run-openvswitch\") pod \"ovnkube-node-xz69h\" (UID: \"e926d1e6-fe73-4a66-9488-1bd8d711ea79\") " pod="openshift-ovn-kubernetes/ovnkube-node-xz69h" Jun 03 16:17:10.529784 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:10.526703 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e926d1e6-fe73-4a66-9488-1bd8d711ea79-host-cni-bin\") pod \"ovnkube-node-xz69h\" (UID: \"e926d1e6-fe73-4a66-9488-1bd8d711ea79\") " pod="openshift-ovn-kubernetes/ovnkube-node-xz69h" Jun 03 16:17:10.529784 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:10.526809 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b0a52397-ab84-4efe-a7e1-fb26e146dfbc-tmp-dir\") pod \"node-resolver-2v2dn\" (UID: \"b0a52397-ab84-4efe-a7e1-fb26e146dfbc\") " pod="openshift-dns/node-resolver-2v2dn" Jun 03 16:17:10.529784 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:10.526865 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/80818dcb-cac7-4ac4-a376-ea67605a5ff8-host-run-multus-certs\") pod \"multus-8s5gg\" (UID: \"80818dcb-cac7-4ac4-a376-ea67605a5ff8\") " pod="openshift-multus/multus-8s5gg" Jun 03 16:17:10.529784 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:10.526905 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e926d1e6-fe73-4a66-9488-1bd8d711ea79-etc-openvswitch\") pod \"ovnkube-node-xz69h\" (UID: \"e926d1e6-fe73-4a66-9488-1bd8d711ea79\") " pod="openshift-ovn-kubernetes/ovnkube-node-xz69h" Jun 03 16:17:10.529784 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:10.526944 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/e926d1e6-fe73-4a66-9488-1bd8d711ea79-node-log\") pod \"ovnkube-node-xz69h\" (UID: \"e926d1e6-fe73-4a66-9488-1bd8d711ea79\") " pod="openshift-ovn-kubernetes/ovnkube-node-xz69h" Jun 03 16:17:10.529784 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:10.526961 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/80818dcb-cac7-4ac4-a376-ea67605a5ff8-multus-daemon-config\") pod \"multus-8s5gg\" (UID: \"80818dcb-cac7-4ac4-a376-ea67605a5ff8\") " pod="openshift-multus/multus-8s5gg" Jun 03 16:17:10.529784 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:10.526997 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e926d1e6-fe73-4a66-9488-1bd8d711ea79-host-cni-bin\") pod \"ovnkube-node-xz69h\" (UID: \"e926d1e6-fe73-4a66-9488-1bd8d711ea79\") " pod="openshift-ovn-kubernetes/ovnkube-node-xz69h" Jun 03 16:17:10.529784 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:10.527100 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f601dafe-2b89-43a6-bb9e-2080b98aaa82-system-cni-dir\") pod \"multus-additional-cni-plugins-wvl4d\" (UID: \"f601dafe-2b89-43a6-bb9e-2080b98aaa82\") " pod="openshift-multus/multus-additional-cni-plugins-wvl4d" Jun 03 16:17:10.529784 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:10.527129 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/6bd7702f-6b77-4535-993e-d1e053d3771b-iptables-alerter-script\") pod \"iptables-alerter-rmdrz\" (UID: \"6bd7702f-6b77-4535-993e-d1e053d3771b\") " pod="openshift-network-operator/iptables-alerter-rmdrz" Jun 03 16:17:10.529784 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:10.527294 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f601dafe-2b89-43a6-bb9e-2080b98aaa82-system-cni-dir\") pod \"multus-additional-cni-plugins-wvl4d\" (UID: \"f601dafe-2b89-43a6-bb9e-2080b98aaa82\") " pod="openshift-multus/multus-additional-cni-plugins-wvl4d" Jun 03 16:17:10.529784 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:10.527321 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/f601dafe-2b89-43a6-bb9e-2080b98aaa82-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-wvl4d\" (UID: \"f601dafe-2b89-43a6-bb9e-2080b98aaa82\") " pod="openshift-multus/multus-additional-cni-plugins-wvl4d" Jun 03 16:17:10.530492 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:10.527350 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f601dafe-2b89-43a6-bb9e-2080b98aaa82-tuning-conf-dir\") pod \"multus-additional-cni-plugins-wvl4d\" (UID: \"f601dafe-2b89-43a6-bb9e-2080b98aaa82\") " pod="openshift-multus/multus-additional-cni-plugins-wvl4d" Jun 03 16:17:10.530492 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:10.527377 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-j527q\" (UniqueName: \"kubernetes.io/projected/80818dcb-cac7-4ac4-a376-ea67605a5ff8-kube-api-access-j527q\") pod \"multus-8s5gg\" (UID: \"80818dcb-cac7-4ac4-a376-ea67605a5ff8\") " pod="openshift-multus/multus-8s5gg" Jun 03 16:17:10.530492 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:10.527402 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e926d1e6-fe73-4a66-9488-1bd8d711ea79-host-run-netns\") pod \"ovnkube-node-xz69h\" (UID: \"e926d1e6-fe73-4a66-9488-1bd8d711ea79\") " pod="openshift-ovn-kubernetes/ovnkube-node-xz69h" Jun 03 16:17:10.530492 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:10.527427 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/e926d1e6-fe73-4a66-9488-1bd8d711ea79-ovnkube-script-lib\") pod \"ovnkube-node-xz69h\" (UID: \"e926d1e6-fe73-4a66-9488-1bd8d711ea79\") " pod="openshift-ovn-kubernetes/ovnkube-node-xz69h" Jun 03 16:17:10.530492 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:10.527453 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/2b06ddd4-c92a-45e5-9538-03fe00b0503f-etc-selinux\") pod \"aws-ebs-csi-driver-node-xvpwg\" (UID: \"2b06ddd4-c92a-45e5-9538-03fe00b0503f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xvpwg" Jun 03 16:17:10.530492 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:10.527478 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9c10c83c-4e52-4366-aef2-577fb3fa951f-etc-kubernetes\") pod \"tuned-5fhsz\" (UID: \"9c10c83c-4e52-4366-aef2-577fb3fa951f\") " pod="openshift-cluster-node-tuning-operator/tuned-5fhsz" Jun 03 16:17:10.530492 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:10.527503 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/9c10c83c-4e52-4366-aef2-577fb3fa951f-lib-modules\") pod \"tuned-5fhsz\" (UID: \"9c10c83c-4e52-4366-aef2-577fb3fa951f\") " pod="openshift-cluster-node-tuning-operator/tuned-5fhsz" Jun 03 16:17:10.530492 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:10.527527 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9c10c83c-4e52-4366-aef2-577fb3fa951f-var-lib-kubelet\") pod \"tuned-5fhsz\" (UID: \"9c10c83c-4e52-4366-aef2-577fb3fa951f\") " pod="openshift-cluster-node-tuning-operator/tuned-5fhsz" Jun 03 16:17:10.530492 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:10.527568 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/80818dcb-cac7-4ac4-a376-ea67605a5ff8-multus-cni-dir\") pod \"multus-8s5gg\" (UID: \"80818dcb-cac7-4ac4-a376-ea67605a5ff8\") " pod="openshift-multus/multus-8s5gg" Jun 03 16:17:10.530492 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:10.527592 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/9c10c83c-4e52-4366-aef2-577fb3fa951f-etc-sysconfig\") pod \"tuned-5fhsz\" (UID: \"9c10c83c-4e52-4366-aef2-577fb3fa951f\") " pod="openshift-cluster-node-tuning-operator/tuned-5fhsz" Jun 03 16:17:10.530492 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:10.527617 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/e926d1e6-fe73-4a66-9488-1bd8d711ea79-ovn-node-metrics-cert\") pod \"ovnkube-node-xz69h\" (UID: \"e926d1e6-fe73-4a66-9488-1bd8d711ea79\") " pod="openshift-ovn-kubernetes/ovnkube-node-xz69h" Jun 03 16:17:10.530492 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:10.527641 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d93510f8-5732-4611-8aca-02573c7e296e-metrics-certs\") pod \"network-metrics-daemon-fhpnv\" (UID: \"d93510f8-5732-4611-8aca-02573c7e296e\") " pod="openshift-multus/network-metrics-daemon-fhpnv" Jun 03 16:17:10.530492 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:10.527667 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/2b06ddd4-c92a-45e5-9538-03fe00b0503f-device-dir\") pod \"aws-ebs-csi-driver-node-xvpwg\" (UID: \"2b06ddd4-c92a-45e5-9538-03fe00b0503f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xvpwg" Jun 03 16:17:10.530492 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:10.527693 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-j6d64\" (UniqueName: \"kubernetes.io/projected/b0a52397-ab84-4efe-a7e1-fb26e146dfbc-kube-api-access-j6d64\") pod \"node-resolver-2v2dn\" (UID: \"b0a52397-ab84-4efe-a7e1-fb26e146dfbc\") " pod="openshift-dns/node-resolver-2v2dn" Jun 03 16:17:10.530492 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:10.527717 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-ggmwq\" (UniqueName: \"kubernetes.io/projected/9c10c83c-4e52-4366-aef2-577fb3fa951f-kube-api-access-ggmwq\") pod \"tuned-5fhsz\" (UID: \"9c10c83c-4e52-4366-aef2-577fb3fa951f\") " pod="openshift-cluster-node-tuning-operator/tuned-5fhsz" Jun 03 16:17:10.530492 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:10.527732 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f601dafe-2b89-43a6-bb9e-2080b98aaa82-tuning-conf-dir\") pod \"multus-additional-cni-plugins-wvl4d\" (UID: \"f601dafe-2b89-43a6-bb9e-2080b98aaa82\") " pod="openshift-multus/multus-additional-cni-plugins-wvl4d" Jun 03 16:17:10.530492 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:10.527742 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/80818dcb-cac7-4ac4-a376-ea67605a5ff8-host-var-lib-kubelet\") pod \"multus-8s5gg\" (UID: \"80818dcb-cac7-4ac4-a376-ea67605a5ff8\") " pod="openshift-multus/multus-8s5gg" Jun 03 16:17:10.531272 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:10.527766 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/80818dcb-cac7-4ac4-a376-ea67605a5ff8-etc-kubernetes\") pod \"multus-8s5gg\" (UID: \"80818dcb-cac7-4ac4-a376-ea67605a5ff8\") " pod="openshift-multus/multus-8s5gg" Jun 03 16:17:10.531272 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:10.527799 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/80818dcb-cac7-4ac4-a376-ea67605a5ff8-multus-cni-dir\") pod \"multus-8s5gg\" (UID: \"80818dcb-cac7-4ac4-a376-ea67605a5ff8\") " pod="openshift-multus/multus-8s5gg" Jun 03 16:17:10.531272 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:10.527836 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/80818dcb-cac7-4ac4-a376-ea67605a5ff8-etc-kubernetes\") pod \"multus-8s5gg\" (UID: \"80818dcb-cac7-4ac4-a376-ea67605a5ff8\") " pod="openshift-multus/multus-8s5gg" Jun 03 16:17:10.531272 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:10.527841 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e926d1e6-fe73-4a66-9488-1bd8d711ea79-host-run-netns\") pod \"ovnkube-node-xz69h\" (UID: \"e926d1e6-fe73-4a66-9488-1bd8d711ea79\") " pod="openshift-ovn-kubernetes/ovnkube-node-xz69h" Jun 03 16:17:10.531272 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:10.528081 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9c10c83c-4e52-4366-aef2-577fb3fa951f-etc-kubernetes\") pod \"tuned-5fhsz\" (UID: \"9c10c83c-4e52-4366-aef2-577fb3fa951f\") " pod="openshift-cluster-node-tuning-operator/tuned-5fhsz" Jun 03 16:17:10.531272 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:10.528150 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/2b06ddd4-c92a-45e5-9538-03fe00b0503f-etc-selinux\") pod \"aws-ebs-csi-driver-node-xvpwg\" (UID: \"2b06ddd4-c92a-45e5-9538-03fe00b0503f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xvpwg" Jun 03 16:17:10.531272 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:10.528318 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/e926d1e6-fe73-4a66-9488-1bd8d711ea79-ovnkube-script-lib\") pod \"ovnkube-node-xz69h\" (UID: \"e926d1e6-fe73-4a66-9488-1bd8d711ea79\") " pod="openshift-ovn-kubernetes/ovnkube-node-xz69h" Jun 03 16:17:10.531272 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:10.528379 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/9c10c83c-4e52-4366-aef2-577fb3fa951f-etc-sysconfig\") pod \"tuned-5fhsz\" (UID: \"9c10c83c-4e52-4366-aef2-577fb3fa951f\") " pod="openshift-cluster-node-tuning-operator/tuned-5fhsz" Jun 03 16:17:10.531272 ip-10-0-128-88 kubenswrapper[2568]: E0603 16:17:10.528411 2568 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 03 16:17:10.531272 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:10.528461 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/9c10c83c-4e52-4366-aef2-577fb3fa951f-lib-modules\") pod \"tuned-5fhsz\" (UID: \"9c10c83c-4e52-4366-aef2-577fb3fa951f\") " pod="openshift-cluster-node-tuning-operator/tuned-5fhsz" Jun 03 16:17:10.531272 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:10.528511 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9c10c83c-4e52-4366-aef2-577fb3fa951f-var-lib-kubelet\") pod \"tuned-5fhsz\" (UID: \"9c10c83c-4e52-4366-aef2-577fb3fa951f\") " pod="openshift-cluster-node-tuning-operator/tuned-5fhsz" Jun 03 16:17:10.531272 ip-10-0-128-88 kubenswrapper[2568]: E0603 16:17:10.528554 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d93510f8-5732-4611-8aca-02573c7e296e-metrics-certs podName:d93510f8-5732-4611-8aca-02573c7e296e nodeName:}" failed. No retries permitted until 2026-06-03 16:17:11.028452633 +0000 UTC m=+2.116879227 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d93510f8-5732-4611-8aca-02573c7e296e-metrics-certs") pod "network-metrics-daemon-fhpnv" (UID: "d93510f8-5732-4611-8aca-02573c7e296e") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 03 16:17:10.531272 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:10.528618 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/2b06ddd4-c92a-45e5-9538-03fe00b0503f-device-dir\") pod \"aws-ebs-csi-driver-node-xvpwg\" (UID: \"2b06ddd4-c92a-45e5-9538-03fe00b0503f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xvpwg" Jun 03 16:17:10.531272 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:10.528671 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/80818dcb-cac7-4ac4-a376-ea67605a5ff8-host-var-lib-kubelet\") pod \"multus-8s5gg\" (UID: \"80818dcb-cac7-4ac4-a376-ea67605a5ff8\") " pod="openshift-multus/multus-8s5gg" Jun 03 16:17:10.531272 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:10.529437 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9c10c83c-4e52-4366-aef2-577fb3fa951f-tmp\") pod \"tuned-5fhsz\" (UID: \"9c10c83c-4e52-4366-aef2-577fb3fa951f\") " pod="openshift-cluster-node-tuning-operator/tuned-5fhsz" Jun 03 16:17:10.531272 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:10.529458 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/130349f2-fa26-4e51-bf29-6819611104db-agent-certs\") pod \"konnectivity-agent-5hlfw\" (UID: \"130349f2-fa26-4e51-bf29-6819611104db\") " pod="kube-system/konnectivity-agent-5hlfw" Jun 03 16:17:10.531272 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:10.529627 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/9c10c83c-4e52-4366-aef2-577fb3fa951f-etc-tuned\") pod \"tuned-5fhsz\" (UID: \"9c10c83c-4e52-4366-aef2-577fb3fa951f\") " pod="openshift-cluster-node-tuning-operator/tuned-5fhsz" Jun 03 16:17:10.532362 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:10.532272 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/e926d1e6-fe73-4a66-9488-1bd8d711ea79-ovn-node-metrics-cert\") pod \"ovnkube-node-xz69h\" (UID: \"e926d1e6-fe73-4a66-9488-1bd8d711ea79\") " pod="openshift-ovn-kubernetes/ovnkube-node-xz69h" Jun 03 16:17:10.536270 ip-10-0-128-88 kubenswrapper[2568]: E0603 16:17:10.536248 2568 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 03 16:17:10.536365 ip-10-0-128-88 kubenswrapper[2568]: E0603 16:17:10.536297 2568 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 03 16:17:10.536365 ip-10-0-128-88 kubenswrapper[2568]: E0603 16:17:10.536312 2568 projected.go:194] Error preparing data for projected volume kube-api-access-tdcd7 for pod openshift-network-diagnostics/network-check-target-m7zkz: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 03 16:17:10.536477 ip-10-0-128-88 kubenswrapper[2568]: E0603 16:17:10.536380 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b8482073-6356-4d6e-bddf-2c8152f6a4f6-kube-api-access-tdcd7 podName:b8482073-6356-4d6e-bddf-2c8152f6a4f6 nodeName:}" failed. No retries permitted until 2026-06-03 16:17:11.036362318 +0000 UTC m=+2.124788907 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-tdcd7" (UniqueName: "kubernetes.io/projected/b8482073-6356-4d6e-bddf-2c8152f6a4f6-kube-api-access-tdcd7") pod "network-check-target-m7zkz" (UID: "b8482073-6356-4d6e-bddf-2c8152f6a4f6") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 03 16:17:10.537395 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:10.537368 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6b6n4\" (UniqueName: \"kubernetes.io/projected/6bd7702f-6b77-4535-993e-d1e053d3771b-kube-api-access-6b6n4\") pod \"iptables-alerter-rmdrz\" (UID: \"6bd7702f-6b77-4535-993e-d1e053d3771b\") " pod="openshift-network-operator/iptables-alerter-rmdrz" Jun 03 16:17:10.537470 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:10.537450 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tzc6j\" (UniqueName: \"kubernetes.io/projected/2b06ddd4-c92a-45e5-9538-03fe00b0503f-kube-api-access-tzc6j\") pod \"aws-ebs-csi-driver-node-xvpwg\" (UID: \"2b06ddd4-c92a-45e5-9538-03fe00b0503f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xvpwg" Jun 03 16:17:10.537601 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:10.537518 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode4c4dbf6c50859cd8172ab3fc4042545.slice/crio-1dbd30a71dd52e9d5d388d966ea953f6b24cd3edaa4b488c535ebddf96045f56 WatchSource:0}: Error finding container 1dbd30a71dd52e9d5d388d966ea953f6b24cd3edaa4b488c535ebddf96045f56: Status 404 returned error can't find the container with id 1dbd30a71dd52e9d5d388d966ea953f6b24cd3edaa4b488c535ebddf96045f56 Jun 03 16:17:10.537718 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:10.537697 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9xhts\" (UniqueName: \"kubernetes.io/projected/2d19a6a4-13e1-407c-a488-189a3651f578-kube-api-access-9xhts\") pod \"node-ca-6jcbf\" (UID: \"2d19a6a4-13e1-407c-a488-189a3651f578\") " pod="openshift-image-registry/node-ca-6jcbf" Jun 03 16:17:10.538710 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:10.538688 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j527q\" (UniqueName: \"kubernetes.io/projected/80818dcb-cac7-4ac4-a376-ea67605a5ff8-kube-api-access-j527q\") pod \"multus-8s5gg\" (UID: \"80818dcb-cac7-4ac4-a376-ea67605a5ff8\") " pod="openshift-multus/multus-8s5gg" Jun 03 16:17:10.538776 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:10.538738 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j6d64\" (UniqueName: \"kubernetes.io/projected/b0a52397-ab84-4efe-a7e1-fb26e146dfbc-kube-api-access-j6d64\") pod \"node-resolver-2v2dn\" (UID: \"b0a52397-ab84-4efe-a7e1-fb26e146dfbc\") " pod="openshift-dns/node-resolver-2v2dn" Jun 03 16:17:10.539718 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:10.539615 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ggmwq\" (UniqueName: \"kubernetes.io/projected/9c10c83c-4e52-4366-aef2-577fb3fa951f-kube-api-access-ggmwq\") pod \"tuned-5fhsz\" (UID: \"9c10c83c-4e52-4366-aef2-577fb3fa951f\") " pod="openshift-cluster-node-tuning-operator/tuned-5fhsz" Jun 03 16:17:10.539718 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:10.539669 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-77tzv\" (UniqueName: \"kubernetes.io/projected/d93510f8-5732-4611-8aca-02573c7e296e-kube-api-access-77tzv\") pod \"network-metrics-daemon-fhpnv\" (UID: \"d93510f8-5732-4611-8aca-02573c7e296e\") " pod="openshift-multus/network-metrics-daemon-fhpnv" Jun 03 16:17:10.539718 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:10.539685 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda6d060b8d198dadd30a1a5529869c2b5.slice/crio-21bc2b282bb782030665d605db4c591d9270c8d82ea4ce9eb18d5f91387bd9df WatchSource:0}: Error finding container 21bc2b282bb782030665d605db4c591d9270c8d82ea4ce9eb18d5f91387bd9df: Status 404 returned error can't find the container with id 21bc2b282bb782030665d605db4c591d9270c8d82ea4ce9eb18d5f91387bd9df Jun 03 16:17:10.541145 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:10.541080 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nlgqh\" (UniqueName: \"kubernetes.io/projected/f601dafe-2b89-43a6-bb9e-2080b98aaa82-kube-api-access-nlgqh\") pod \"multus-additional-cni-plugins-wvl4d\" (UID: \"f601dafe-2b89-43a6-bb9e-2080b98aaa82\") " pod="openshift-multus/multus-additional-cni-plugins-wvl4d" Jun 03 16:17:10.541241 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:10.541223 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9sgdq\" (UniqueName: \"kubernetes.io/projected/e926d1e6-fe73-4a66-9488-1bd8d711ea79-kube-api-access-9sgdq\") pod \"ovnkube-node-xz69h\" (UID: \"e926d1e6-fe73-4a66-9488-1bd8d711ea79\") " pod="openshift-ovn-kubernetes/ovnkube-node-xz69h" Jun 03 16:17:10.542262 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:10.542247 2568 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jun 03 16:17:10.543830 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:10.543817 2568 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-5fhsz" Jun 03 16:17:10.549448 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:10.549432 2568 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-8s5gg" Jun 03 16:17:10.552873 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:10.552853 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9c10c83c_4e52_4366_aef2_577fb3fa951f.slice/crio-472f277aabf7a69674b358f33dca883763298321e38fb4fd97783a18507b5570 WatchSource:0}: Error finding container 472f277aabf7a69674b358f33dca883763298321e38fb4fd97783a18507b5570: Status 404 returned error can't find the container with id 472f277aabf7a69674b358f33dca883763298321e38fb4fd97783a18507b5570 Jun 03 16:17:10.555669 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:10.555636 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod80818dcb_cac7_4ac4_a376_ea67605a5ff8.slice/crio-305eeb59077ce91aeae9baeb24b9da66afe50b28cdf1ea96e06017030e61b7d3 WatchSource:0}: Error finding container 305eeb59077ce91aeae9baeb24b9da66afe50b28cdf1ea96e06017030e61b7d3: Status 404 returned error can't find the container with id 305eeb59077ce91aeae9baeb24b9da66afe50b28cdf1ea96e06017030e61b7d3 Jun 03 16:17:10.561722 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:10.561685 2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-8s5gg" event={"ID":"80818dcb-cac7-4ac4-a376-ea67605a5ff8","Type":"ContainerStarted","Data":"305eeb59077ce91aeae9baeb24b9da66afe50b28cdf1ea96e06017030e61b7d3"} Jun 03 16:17:10.562458 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:10.562442 2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-5fhsz" event={"ID":"9c10c83c-4e52-4366-aef2-577fb3fa951f","Type":"ContainerStarted","Data":"472f277aabf7a69674b358f33dca883763298321e38fb4fd97783a18507b5570"} Jun 03 16:17:10.563214 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:10.563198 2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-88.ec2.internal" event={"ID":"a6d060b8d198dadd30a1a5529869c2b5","Type":"ContainerStarted","Data":"21bc2b282bb782030665d605db4c591d9270c8d82ea4ce9eb18d5f91387bd9df"} Jun 03 16:17:10.563919 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:10.563897 2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-88.ec2.internal" event={"ID":"e4c4dbf6c50859cd8172ab3fc4042545","Type":"ContainerStarted","Data":"1dbd30a71dd52e9d5d388d966ea953f6b24cd3edaa4b488c535ebddf96045f56"} Jun 03 16:17:10.708400 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:10.708346 2568 reflector.go:430] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jun 03 16:17:10.732928 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:10.732909 2568 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-rmdrz" Jun 03 16:17:10.738346 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:10.738325 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6bd7702f_6b77_4535_993e_d1e053d3771b.slice/crio-ba1048766416d2fdf15a550e1d317abd413062f4a0b9dd511c2a7b68e137abc1 WatchSource:0}: Error finding container ba1048766416d2fdf15a550e1d317abd413062f4a0b9dd511c2a7b68e137abc1: Status 404 returned error can't find the container with id ba1048766416d2fdf15a550e1d317abd413062f4a0b9dd511c2a7b68e137abc1 Jun 03 16:17:10.748225 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:10.748209 2568 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-xz69h" Jun 03 16:17:10.753303 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:10.753284 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode926d1e6_fe73_4a66_9488_1bd8d711ea79.slice/crio-4f02e77002616564d8dd586eecdcc9d886e5e96bd2a9e5cf8b0a208c209f7ea2 WatchSource:0}: Error finding container 4f02e77002616564d8dd586eecdcc9d886e5e96bd2a9e5cf8b0a208c209f7ea2: Status 404 returned error can't find the container with id 4f02e77002616564d8dd586eecdcc9d886e5e96bd2a9e5cf8b0a208c209f7ea2 Jun 03 16:17:10.763824 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:10.763800 2568 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xvpwg" Jun 03 16:17:10.769281 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:10.769257 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2b06ddd4_c92a_45e5_9538_03fe00b0503f.slice/crio-b3fb91aa74e2b0db66d0f37156cb5ad647e311286c9e31fe2238c06640aea9b1 WatchSource:0}: Error finding container b3fb91aa74e2b0db66d0f37156cb5ad647e311286c9e31fe2238c06640aea9b1: Status 404 returned error can't find the container with id b3fb91aa74e2b0db66d0f37156cb5ad647e311286c9e31fe2238c06640aea9b1 Jun 03 16:17:10.773188 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:10.773174 2568 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-2v2dn" Jun 03 16:17:10.779500 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:10.779480 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb0a52397_ab84_4efe_a7e1_fb26e146dfbc.slice/crio-9b6ef1948d4c53674a5faac241c97d471a1853c5c62a897d60818bc65e6c5607 WatchSource:0}: Error finding container 9b6ef1948d4c53674a5faac241c97d471a1853c5c62a897d60818bc65e6c5607: Status 404 returned error can't find the container with id 9b6ef1948d4c53674a5faac241c97d471a1853c5c62a897d60818bc65e6c5607 Jun 03 16:17:10.791259 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:10.791244 2568 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-6jcbf" Jun 03 16:17:10.796720 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:10.796701 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2d19a6a4_13e1_407c_a488_189a3651f578.slice/crio-d1748713765f3cd105cbde4eebce654b3223d7e5a3a295eab191879841f02e06 WatchSource:0}: Error finding container d1748713765f3cd105cbde4eebce654b3223d7e5a3a295eab191879841f02e06: Status 404 returned error can't find the container with id d1748713765f3cd105cbde4eebce654b3223d7e5a3a295eab191879841f02e06 Jun 03 16:17:10.806173 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:10.806158 2568 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-wvl4d" Jun 03 16:17:10.812550 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:10.812392 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf601dafe_2b89_43a6_bb9e_2080b98aaa82.slice/crio-37fa33503d050ad21e00f8b4e9af207f52c5cf7c1a67d70f809848de5fe6d53d WatchSource:0}: Error finding container 37fa33503d050ad21e00f8b4e9af207f52c5cf7c1a67d70f809848de5fe6d53d: Status 404 returned error can't find the container with id 37fa33503d050ad21e00f8b4e9af207f52c5cf7c1a67d70f809848de5fe6d53d Jun 03 16:17:10.825497 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:10.825480 2568 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-5hlfw" Jun 03 16:17:10.830767 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:10.830745 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod130349f2_fa26_4e51_bf29_6819611104db.slice/crio-1142c40cbe3f5e0600bfc1e901ad7234dfcd795640ad39369d58cfe38874ea74 WatchSource:0}: Error finding container 1142c40cbe3f5e0600bfc1e901ad7234dfcd795640ad39369d58cfe38874ea74: Status 404 returned error can't find the container with id 1142c40cbe3f5e0600bfc1e901ad7234dfcd795640ad39369d58cfe38874ea74 Jun 03 16:17:11.032158 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:11.032073 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d93510f8-5732-4611-8aca-02573c7e296e-metrics-certs\") pod \"network-metrics-daemon-fhpnv\" (UID: \"d93510f8-5732-4611-8aca-02573c7e296e\") " pod="openshift-multus/network-metrics-daemon-fhpnv" Jun 03 16:17:11.032302 ip-10-0-128-88 kubenswrapper[2568]: E0603 16:17:11.032243 2568 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 03 16:17:11.032366 ip-10-0-128-88 kubenswrapper[2568]: E0603 16:17:11.032309 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d93510f8-5732-4611-8aca-02573c7e296e-metrics-certs podName:d93510f8-5732-4611-8aca-02573c7e296e nodeName:}" failed. No retries permitted until 2026-06-03 16:17:12.032288502 +0000 UTC m=+3.120715074 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d93510f8-5732-4611-8aca-02573c7e296e-metrics-certs") pod "network-metrics-daemon-fhpnv" (UID: "d93510f8-5732-4611-8aca-02573c7e296e") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 03 16:17:11.133485 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:11.132887 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-tdcd7\" (UniqueName: \"kubernetes.io/projected/b8482073-6356-4d6e-bddf-2c8152f6a4f6-kube-api-access-tdcd7\") pod \"network-check-target-m7zkz\" (UID: \"b8482073-6356-4d6e-bddf-2c8152f6a4f6\") " pod="openshift-network-diagnostics/network-check-target-m7zkz" Jun 03 16:17:11.133485 ip-10-0-128-88 kubenswrapper[2568]: E0603 16:17:11.133064 2568 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 03 16:17:11.133485 ip-10-0-128-88 kubenswrapper[2568]: E0603 16:17:11.133082 2568 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 03 16:17:11.133485 ip-10-0-128-88 kubenswrapper[2568]: E0603 16:17:11.133094 2568 projected.go:194] Error preparing data for projected volume kube-api-access-tdcd7 for pod openshift-network-diagnostics/network-check-target-m7zkz: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 03 16:17:11.133485 ip-10-0-128-88 kubenswrapper[2568]: E0603 16:17:11.133147 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b8482073-6356-4d6e-bddf-2c8152f6a4f6-kube-api-access-tdcd7 podName:b8482073-6356-4d6e-bddf-2c8152f6a4f6 nodeName:}" failed. No retries permitted until 2026-06-03 16:17:12.133128941 +0000 UTC m=+3.221555509 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-tdcd7" (UniqueName: "kubernetes.io/projected/b8482073-6356-4d6e-bddf-2c8152f6a4f6-kube-api-access-tdcd7") pod "network-check-target-m7zkz" (UID: "b8482073-6356-4d6e-bddf-2c8152f6a4f6") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 03 16:17:11.352898 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:11.352790 2568 reflector.go:430] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jun 03 16:17:11.359676 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:11.359653 2568 reflector.go:430] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jun 03 16:17:11.467037 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:11.467000 2568 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-06-02 16:12:10 +0000 UTC" deadline="2028-01-05 14:01:43.238598695 +0000 UTC" Jun 03 16:17:11.467183 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:11.467036 2568 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13941h44m31.771566859s" Jun 03 16:17:11.581373 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:11.581323 2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wvl4d" event={"ID":"f601dafe-2b89-43a6-bb9e-2080b98aaa82","Type":"ContainerStarted","Data":"37fa33503d050ad21e00f8b4e9af207f52c5cf7c1a67d70f809848de5fe6d53d"} Jun 03 16:17:11.595560 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:11.595509 2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-6jcbf" event={"ID":"2d19a6a4-13e1-407c-a488-189a3651f578","Type":"ContainerStarted","Data":"d1748713765f3cd105cbde4eebce654b3223d7e5a3a295eab191879841f02e06"} Jun 03 16:17:11.610417 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:11.610301 2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-2v2dn" event={"ID":"b0a52397-ab84-4efe-a7e1-fb26e146dfbc","Type":"ContainerStarted","Data":"9b6ef1948d4c53674a5faac241c97d471a1853c5c62a897d60818bc65e6c5607"} Jun 03 16:17:11.613175 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:11.613148 2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-rmdrz" event={"ID":"6bd7702f-6b77-4535-993e-d1e053d3771b","Type":"ContainerStarted","Data":"ba1048766416d2fdf15a550e1d317abd413062f4a0b9dd511c2a7b68e137abc1"} Jun 03 16:17:11.621084 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:11.621037 2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-5hlfw" event={"ID":"130349f2-fa26-4e51-bf29-6819611104db","Type":"ContainerStarted","Data":"1142c40cbe3f5e0600bfc1e901ad7234dfcd795640ad39369d58cfe38874ea74"} Jun 03 16:17:11.640262 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:11.640232 2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xvpwg" event={"ID":"2b06ddd4-c92a-45e5-9538-03fe00b0503f","Type":"ContainerStarted","Data":"b3fb91aa74e2b0db66d0f37156cb5ad647e311286c9e31fe2238c06640aea9b1"} Jun 03 16:17:11.664074 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:11.664049 2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xz69h" event={"ID":"e926d1e6-fe73-4a66-9488-1bd8d711ea79","Type":"ContainerStarted","Data":"4f02e77002616564d8dd586eecdcc9d886e5e96bd2a9e5cf8b0a208c209f7ea2"} Jun 03 16:17:12.041968 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:12.041866 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d93510f8-5732-4611-8aca-02573c7e296e-metrics-certs\") pod \"network-metrics-daemon-fhpnv\" (UID: \"d93510f8-5732-4611-8aca-02573c7e296e\") " pod="openshift-multus/network-metrics-daemon-fhpnv" Jun 03 16:17:12.042142 ip-10-0-128-88 kubenswrapper[2568]: E0603 16:17:12.042034 2568 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 03 16:17:12.042142 ip-10-0-128-88 kubenswrapper[2568]: E0603 16:17:12.042098 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d93510f8-5732-4611-8aca-02573c7e296e-metrics-certs podName:d93510f8-5732-4611-8aca-02573c7e296e nodeName:}" failed. No retries permitted until 2026-06-03 16:17:14.042079471 +0000 UTC m=+5.130506046 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d93510f8-5732-4611-8aca-02573c7e296e-metrics-certs") pod "network-metrics-daemon-fhpnv" (UID: "d93510f8-5732-4611-8aca-02573c7e296e") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 03 16:17:12.142250 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:12.142213 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-tdcd7\" (UniqueName: \"kubernetes.io/projected/b8482073-6356-4d6e-bddf-2c8152f6a4f6-kube-api-access-tdcd7\") pod \"network-check-target-m7zkz\" (UID: \"b8482073-6356-4d6e-bddf-2c8152f6a4f6\") " pod="openshift-network-diagnostics/network-check-target-m7zkz" Jun 03 16:17:12.142435 ip-10-0-128-88 kubenswrapper[2568]: E0603 16:17:12.142392 2568 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 03 16:17:12.142435 ip-10-0-128-88 kubenswrapper[2568]: E0603 16:17:12.142417 2568 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 03 16:17:12.142435 ip-10-0-128-88 kubenswrapper[2568]: E0603 16:17:12.142431 2568 projected.go:194] Error preparing data for projected volume kube-api-access-tdcd7 for pod openshift-network-diagnostics/network-check-target-m7zkz: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 03 16:17:12.142614 ip-10-0-128-88 kubenswrapper[2568]: E0603 16:17:12.142492 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b8482073-6356-4d6e-bddf-2c8152f6a4f6-kube-api-access-tdcd7 podName:b8482073-6356-4d6e-bddf-2c8152f6a4f6 nodeName:}" failed. No retries permitted until 2026-06-03 16:17:14.142473443 +0000 UTC m=+5.230900026 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-tdcd7" (UniqueName: "kubernetes.io/projected/b8482073-6356-4d6e-bddf-2c8152f6a4f6-kube-api-access-tdcd7") pod "network-check-target-m7zkz" (UID: "b8482073-6356-4d6e-bddf-2c8152f6a4f6") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 03 16:17:12.467466 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:12.467386 2568 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-06-02 16:12:10 +0000 UTC" deadline="2028-01-17 10:47:08.08073444 +0000 UTC" Jun 03 16:17:12.467466 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:12.467417 2568 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14226h29m55.613319088s" Jun 03 16:17:12.560598 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:12.560476 2568 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m7zkz" Jun 03 16:17:12.560774 ip-10-0-128-88 kubenswrapper[2568]: E0603 16:17:12.560612 2568 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m7zkz" podUID="b8482073-6356-4d6e-bddf-2c8152f6a4f6" Jun 03 16:17:12.560774 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:12.560642 2568 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fhpnv" Jun 03 16:17:12.560774 ip-10-0-128-88 kubenswrapper[2568]: E0603 16:17:12.560755 2568 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fhpnv" podUID="d93510f8-5732-4611-8aca-02573c7e296e" Jun 03 16:17:14.057796 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:14.057205 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d93510f8-5732-4611-8aca-02573c7e296e-metrics-certs\") pod \"network-metrics-daemon-fhpnv\" (UID: \"d93510f8-5732-4611-8aca-02573c7e296e\") " pod="openshift-multus/network-metrics-daemon-fhpnv" Jun 03 16:17:14.057796 ip-10-0-128-88 kubenswrapper[2568]: E0603 16:17:14.057370 2568 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 03 16:17:14.057796 ip-10-0-128-88 kubenswrapper[2568]: E0603 16:17:14.057431 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d93510f8-5732-4611-8aca-02573c7e296e-metrics-certs podName:d93510f8-5732-4611-8aca-02573c7e296e nodeName:}" failed. No retries permitted until 2026-06-03 16:17:18.057413701 +0000 UTC m=+9.145840276 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d93510f8-5732-4611-8aca-02573c7e296e-metrics-certs") pod "network-metrics-daemon-fhpnv" (UID: "d93510f8-5732-4611-8aca-02573c7e296e") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 03 16:17:14.157870 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:14.157805 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-tdcd7\" (UniqueName: \"kubernetes.io/projected/b8482073-6356-4d6e-bddf-2c8152f6a4f6-kube-api-access-tdcd7\") pod \"network-check-target-m7zkz\" (UID: \"b8482073-6356-4d6e-bddf-2c8152f6a4f6\") " pod="openshift-network-diagnostics/network-check-target-m7zkz" Jun 03 16:17:14.158035 ip-10-0-128-88 kubenswrapper[2568]: E0603 16:17:14.157961 2568 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 03 16:17:14.158035 ip-10-0-128-88 kubenswrapper[2568]: E0603 16:17:14.157981 2568 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 03 16:17:14.158035 ip-10-0-128-88 kubenswrapper[2568]: E0603 16:17:14.157993 2568 projected.go:194] Error preparing data for projected volume kube-api-access-tdcd7 for pod openshift-network-diagnostics/network-check-target-m7zkz: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 03 16:17:14.158158 ip-10-0-128-88 kubenswrapper[2568]: E0603 16:17:14.158051 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b8482073-6356-4d6e-bddf-2c8152f6a4f6-kube-api-access-tdcd7 podName:b8482073-6356-4d6e-bddf-2c8152f6a4f6 nodeName:}" failed. No retries permitted until 2026-06-03 16:17:18.158033828 +0000 UTC m=+9.246460407 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-tdcd7" (UniqueName: "kubernetes.io/projected/b8482073-6356-4d6e-bddf-2c8152f6a4f6-kube-api-access-tdcd7") pod "network-check-target-m7zkz" (UID: "b8482073-6356-4d6e-bddf-2c8152f6a4f6") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 03 16:17:14.560629 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:14.560561 2568 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m7zkz" Jun 03 16:17:14.560858 ip-10-0-128-88 kubenswrapper[2568]: E0603 16:17:14.560702 2568 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m7zkz" podUID="b8482073-6356-4d6e-bddf-2c8152f6a4f6" Jun 03 16:17:14.560858 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:14.560776 2568 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fhpnv" Jun 03 16:17:14.560998 ip-10-0-128-88 kubenswrapper[2568]: E0603 16:17:14.560888 2568 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fhpnv" podUID="d93510f8-5732-4611-8aca-02573c7e296e" Jun 03 16:17:16.476187 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:16.476154 2568 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kube-system/global-pull-secret-syncer-6g497"] Jun 03 16:17:16.479320 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:16.479297 2568 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6g497" Jun 03 16:17:16.479468 ip-10-0-128-88 kubenswrapper[2568]: E0603 16:17:16.479380 2568 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6g497" podUID="778470c3-9be3-440f-b177-7b846a0c7d32" Jun 03 16:17:16.560418 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:16.560383 2568 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m7zkz" Jun 03 16:17:16.560647 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:16.560385 2568 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fhpnv" Jun 03 16:17:16.560647 ip-10-0-128-88 kubenswrapper[2568]: E0603 16:17:16.560512 2568 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m7zkz" podUID="b8482073-6356-4d6e-bddf-2c8152f6a4f6" Jun 03 16:17:16.560647 ip-10-0-128-88 kubenswrapper[2568]: E0603 16:17:16.560623 2568 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fhpnv" podUID="d93510f8-5732-4611-8aca-02573c7e296e" Jun 03 16:17:16.578527 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:16.578394 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/778470c3-9be3-440f-b177-7b846a0c7d32-original-pull-secret\") pod \"global-pull-secret-syncer-6g497\" (UID: \"778470c3-9be3-440f-b177-7b846a0c7d32\") " pod="kube-system/global-pull-secret-syncer-6g497" Jun 03 16:17:16.578527 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:16.578456 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/778470c3-9be3-440f-b177-7b846a0c7d32-dbus\") pod \"global-pull-secret-syncer-6g497\" (UID: \"778470c3-9be3-440f-b177-7b846a0c7d32\") " pod="kube-system/global-pull-secret-syncer-6g497" Jun 03 16:17:16.578527 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:16.578499 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/778470c3-9be3-440f-b177-7b846a0c7d32-kubelet-config\") pod \"global-pull-secret-syncer-6g497\" (UID: \"778470c3-9be3-440f-b177-7b846a0c7d32\") " pod="kube-system/global-pull-secret-syncer-6g497" Jun 03 16:17:16.679486 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:16.679452 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/778470c3-9be3-440f-b177-7b846a0c7d32-original-pull-secret\") pod \"global-pull-secret-syncer-6g497\" (UID: \"778470c3-9be3-440f-b177-7b846a0c7d32\") " pod="kube-system/global-pull-secret-syncer-6g497" Jun 03 16:17:16.679685 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:16.679521 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/778470c3-9be3-440f-b177-7b846a0c7d32-dbus\") pod \"global-pull-secret-syncer-6g497\" (UID: \"778470c3-9be3-440f-b177-7b846a0c7d32\") " pod="kube-system/global-pull-secret-syncer-6g497" Jun 03 16:17:16.679685 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:16.679583 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/778470c3-9be3-440f-b177-7b846a0c7d32-kubelet-config\") pod \"global-pull-secret-syncer-6g497\" (UID: \"778470c3-9be3-440f-b177-7b846a0c7d32\") " pod="kube-system/global-pull-secret-syncer-6g497" Jun 03 16:17:16.679685 ip-10-0-128-88 kubenswrapper[2568]: E0603 16:17:16.679625 2568 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 03 16:17:16.679685 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:16.679679 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/778470c3-9be3-440f-b177-7b846a0c7d32-kubelet-config\") pod \"global-pull-secret-syncer-6g497\" (UID: \"778470c3-9be3-440f-b177-7b846a0c7d32\") " pod="kube-system/global-pull-secret-syncer-6g497" Jun 03 16:17:16.679897 ip-10-0-128-88 kubenswrapper[2568]: E0603 16:17:16.679697 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/778470c3-9be3-440f-b177-7b846a0c7d32-original-pull-secret podName:778470c3-9be3-440f-b177-7b846a0c7d32 nodeName:}" failed. No retries permitted until 2026-06-03 16:17:17.179675753 +0000 UTC m=+8.268102329 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/778470c3-9be3-440f-b177-7b846a0c7d32-original-pull-secret") pod "global-pull-secret-syncer-6g497" (UID: "778470c3-9be3-440f-b177-7b846a0c7d32") : object "kube-system"/"original-pull-secret" not registered Jun 03 16:17:16.679897 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:16.679820 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/778470c3-9be3-440f-b177-7b846a0c7d32-dbus\") pod \"global-pull-secret-syncer-6g497\" (UID: \"778470c3-9be3-440f-b177-7b846a0c7d32\") " pod="kube-system/global-pull-secret-syncer-6g497" Jun 03 16:17:17.183773 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:17.183197 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/778470c3-9be3-440f-b177-7b846a0c7d32-original-pull-secret\") pod \"global-pull-secret-syncer-6g497\" (UID: \"778470c3-9be3-440f-b177-7b846a0c7d32\") " pod="kube-system/global-pull-secret-syncer-6g497" Jun 03 16:17:17.183773 ip-10-0-128-88 kubenswrapper[2568]: E0603 16:17:17.183348 2568 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 03 16:17:17.183773 ip-10-0-128-88 kubenswrapper[2568]: E0603 16:17:17.183406 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/778470c3-9be3-440f-b177-7b846a0c7d32-original-pull-secret podName:778470c3-9be3-440f-b177-7b846a0c7d32 nodeName:}" failed. No retries permitted until 2026-06-03 16:17:18.183387381 +0000 UTC m=+9.271813964 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/778470c3-9be3-440f-b177-7b846a0c7d32-original-pull-secret") pod "global-pull-secret-syncer-6g497" (UID: "778470c3-9be3-440f-b177-7b846a0c7d32") : object "kube-system"/"original-pull-secret" not registered Jun 03 16:17:18.091382 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:18.091348 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d93510f8-5732-4611-8aca-02573c7e296e-metrics-certs\") pod \"network-metrics-daemon-fhpnv\" (UID: \"d93510f8-5732-4611-8aca-02573c7e296e\") " pod="openshift-multus/network-metrics-daemon-fhpnv" Jun 03 16:17:18.091874 ip-10-0-128-88 kubenswrapper[2568]: E0603 16:17:18.091478 2568 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 03 16:17:18.091874 ip-10-0-128-88 kubenswrapper[2568]: E0603 16:17:18.091561 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d93510f8-5732-4611-8aca-02573c7e296e-metrics-certs podName:d93510f8-5732-4611-8aca-02573c7e296e nodeName:}" failed. No retries permitted until 2026-06-03 16:17:26.091526051 +0000 UTC m=+17.179952622 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d93510f8-5732-4611-8aca-02573c7e296e-metrics-certs") pod "network-metrics-daemon-fhpnv" (UID: "d93510f8-5732-4611-8aca-02573c7e296e") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 03 16:17:18.192644 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:18.192503 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-tdcd7\" (UniqueName: \"kubernetes.io/projected/b8482073-6356-4d6e-bddf-2c8152f6a4f6-kube-api-access-tdcd7\") pod \"network-check-target-m7zkz\" (UID: \"b8482073-6356-4d6e-bddf-2c8152f6a4f6\") " pod="openshift-network-diagnostics/network-check-target-m7zkz" Jun 03 16:17:18.192644 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:18.192611 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/778470c3-9be3-440f-b177-7b846a0c7d32-original-pull-secret\") pod \"global-pull-secret-syncer-6g497\" (UID: \"778470c3-9be3-440f-b177-7b846a0c7d32\") " pod="kube-system/global-pull-secret-syncer-6g497" Jun 03 16:17:18.192831 ip-10-0-128-88 kubenswrapper[2568]: E0603 16:17:18.192676 2568 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 03 16:17:18.192831 ip-10-0-128-88 kubenswrapper[2568]: E0603 16:17:18.192697 2568 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 03 16:17:18.192831 ip-10-0-128-88 kubenswrapper[2568]: E0603 16:17:18.192710 2568 projected.go:194] Error preparing data for projected volume kube-api-access-tdcd7 for pod openshift-network-diagnostics/network-check-target-m7zkz: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 03 16:17:18.192831 ip-10-0-128-88 kubenswrapper[2568]: E0603 16:17:18.192722 2568 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 03 16:17:18.192831 ip-10-0-128-88 kubenswrapper[2568]: E0603 16:17:18.192772 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/778470c3-9be3-440f-b177-7b846a0c7d32-original-pull-secret podName:778470c3-9be3-440f-b177-7b846a0c7d32 nodeName:}" failed. No retries permitted until 2026-06-03 16:17:20.192754533 +0000 UTC m=+11.281181102 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/778470c3-9be3-440f-b177-7b846a0c7d32-original-pull-secret") pod "global-pull-secret-syncer-6g497" (UID: "778470c3-9be3-440f-b177-7b846a0c7d32") : object "kube-system"/"original-pull-secret" not registered Jun 03 16:17:18.192831 ip-10-0-128-88 kubenswrapper[2568]: E0603 16:17:18.192790 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b8482073-6356-4d6e-bddf-2c8152f6a4f6-kube-api-access-tdcd7 podName:b8482073-6356-4d6e-bddf-2c8152f6a4f6 nodeName:}" failed. No retries permitted until 2026-06-03 16:17:26.192781429 +0000 UTC m=+17.281207998 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-tdcd7" (UniqueName: "kubernetes.io/projected/b8482073-6356-4d6e-bddf-2c8152f6a4f6-kube-api-access-tdcd7") pod "network-check-target-m7zkz" (UID: "b8482073-6356-4d6e-bddf-2c8152f6a4f6") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 03 16:17:18.559779 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:18.559742 2568 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6g497" Jun 03 16:17:18.560011 ip-10-0-128-88 kubenswrapper[2568]: E0603 16:17:18.559869 2568 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6g497" podUID="778470c3-9be3-440f-b177-7b846a0c7d32" Jun 03 16:17:18.560011 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:18.559742 2568 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fhpnv" Jun 03 16:17:18.560011 ip-10-0-128-88 kubenswrapper[2568]: E0603 16:17:18.559974 2568 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fhpnv" podUID="d93510f8-5732-4611-8aca-02573c7e296e" Jun 03 16:17:18.560011 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:18.559742 2568 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m7zkz" Jun 03 16:17:18.560209 ip-10-0-128-88 kubenswrapper[2568]: E0603 16:17:18.560041 2568 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m7zkz" podUID="b8482073-6356-4d6e-bddf-2c8152f6a4f6" Jun 03 16:17:20.204828 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:20.204790 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/778470c3-9be3-440f-b177-7b846a0c7d32-original-pull-secret\") pod \"global-pull-secret-syncer-6g497\" (UID: \"778470c3-9be3-440f-b177-7b846a0c7d32\") " pod="kube-system/global-pull-secret-syncer-6g497" Jun 03 16:17:20.205304 ip-10-0-128-88 kubenswrapper[2568]: E0603 16:17:20.204975 2568 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 03 16:17:20.205304 ip-10-0-128-88 kubenswrapper[2568]: E0603 16:17:20.205068 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/778470c3-9be3-440f-b177-7b846a0c7d32-original-pull-secret podName:778470c3-9be3-440f-b177-7b846a0c7d32 nodeName:}" failed. No retries permitted until 2026-06-03 16:17:24.205048195 +0000 UTC m=+15.293474772 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/778470c3-9be3-440f-b177-7b846a0c7d32-original-pull-secret") pod "global-pull-secret-syncer-6g497" (UID: "778470c3-9be3-440f-b177-7b846a0c7d32") : object "kube-system"/"original-pull-secret" not registered Jun 03 16:17:20.560190 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:20.560115 2568 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fhpnv" Jun 03 16:17:20.560349 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:20.560116 2568 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m7zkz" Jun 03 16:17:20.560349 ip-10-0-128-88 kubenswrapper[2568]: E0603 16:17:20.560263 2568 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fhpnv" podUID="d93510f8-5732-4611-8aca-02573c7e296e" Jun 03 16:17:20.560349 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:20.560116 2568 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6g497" Jun 03 16:17:20.560349 ip-10-0-128-88 kubenswrapper[2568]: E0603 16:17:20.560326 2568 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m7zkz" podUID="b8482073-6356-4d6e-bddf-2c8152f6a4f6" Jun 03 16:17:20.560550 ip-10-0-128-88 kubenswrapper[2568]: E0603 16:17:20.560390 2568 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6g497" podUID="778470c3-9be3-440f-b177-7b846a0c7d32" Jun 03 16:17:22.560354 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:22.560318 2568 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6g497" Jun 03 16:17:22.560802 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:22.560318 2568 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fhpnv" Jun 03 16:17:22.560802 ip-10-0-128-88 kubenswrapper[2568]: E0603 16:17:22.560439 2568 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6g497" podUID="778470c3-9be3-440f-b177-7b846a0c7d32" Jun 03 16:17:22.560802 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:22.560318 2568 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m7zkz" Jun 03 16:17:22.560802 ip-10-0-128-88 kubenswrapper[2568]: E0603 16:17:22.560551 2568 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fhpnv" podUID="d93510f8-5732-4611-8aca-02573c7e296e" Jun 03 16:17:22.560802 ip-10-0-128-88 kubenswrapper[2568]: E0603 16:17:22.560596 2568 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m7zkz" podUID="b8482073-6356-4d6e-bddf-2c8152f6a4f6" Jun 03 16:17:24.240447 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:24.240414 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/778470c3-9be3-440f-b177-7b846a0c7d32-original-pull-secret\") pod \"global-pull-secret-syncer-6g497\" (UID: \"778470c3-9be3-440f-b177-7b846a0c7d32\") " pod="kube-system/global-pull-secret-syncer-6g497" Jun 03 16:17:24.240859 ip-10-0-128-88 kubenswrapper[2568]: E0603 16:17:24.240572 2568 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 03 16:17:24.240859 ip-10-0-128-88 kubenswrapper[2568]: E0603 16:17:24.240647 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/778470c3-9be3-440f-b177-7b846a0c7d32-original-pull-secret podName:778470c3-9be3-440f-b177-7b846a0c7d32 nodeName:}" failed. No retries permitted until 2026-06-03 16:17:32.240628989 +0000 UTC m=+23.329055566 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/778470c3-9be3-440f-b177-7b846a0c7d32-original-pull-secret") pod "global-pull-secret-syncer-6g497" (UID: "778470c3-9be3-440f-b177-7b846a0c7d32") : object "kube-system"/"original-pull-secret" not registered Jun 03 16:17:24.560158 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:24.560084 2568 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m7zkz" Jun 03 16:17:24.560322 ip-10-0-128-88 kubenswrapper[2568]: E0603 16:17:24.560209 2568 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m7zkz" podUID="b8482073-6356-4d6e-bddf-2c8152f6a4f6" Jun 03 16:17:24.560322 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:24.560216 2568 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fhpnv" Jun 03 16:17:24.560322 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:24.560258 2568 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6g497" Jun 03 16:17:24.560322 ip-10-0-128-88 kubenswrapper[2568]: E0603 16:17:24.560315 2568 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fhpnv" podUID="d93510f8-5732-4611-8aca-02573c7e296e" Jun 03 16:17:24.560506 ip-10-0-128-88 kubenswrapper[2568]: E0603 16:17:24.560388 2568 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6g497" podUID="778470c3-9be3-440f-b177-7b846a0c7d32" Jun 03 16:17:26.157062 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:26.157029 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d93510f8-5732-4611-8aca-02573c7e296e-metrics-certs\") pod \"network-metrics-daemon-fhpnv\" (UID: \"d93510f8-5732-4611-8aca-02573c7e296e\") " pod="openshift-multus/network-metrics-daemon-fhpnv" Jun 03 16:17:26.157511 ip-10-0-128-88 kubenswrapper[2568]: E0603 16:17:26.157196 2568 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 03 16:17:26.157511 ip-10-0-128-88 kubenswrapper[2568]: E0603 16:17:26.157272 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d93510f8-5732-4611-8aca-02573c7e296e-metrics-certs podName:d93510f8-5732-4611-8aca-02573c7e296e nodeName:}" failed. No retries permitted until 2026-06-03 16:17:42.157255679 +0000 UTC m=+33.245682251 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d93510f8-5732-4611-8aca-02573c7e296e-metrics-certs") pod "network-metrics-daemon-fhpnv" (UID: "d93510f8-5732-4611-8aca-02573c7e296e") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 03 16:17:26.257433 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:26.257390 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-tdcd7\" (UniqueName: \"kubernetes.io/projected/b8482073-6356-4d6e-bddf-2c8152f6a4f6-kube-api-access-tdcd7\") pod \"network-check-target-m7zkz\" (UID: \"b8482073-6356-4d6e-bddf-2c8152f6a4f6\") " pod="openshift-network-diagnostics/network-check-target-m7zkz" Jun 03 16:17:26.257631 ip-10-0-128-88 kubenswrapper[2568]: E0603 16:17:26.257584 2568 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 03 16:17:26.257631 ip-10-0-128-88 kubenswrapper[2568]: E0603 16:17:26.257607 2568 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 03 16:17:26.257631 ip-10-0-128-88 kubenswrapper[2568]: E0603 16:17:26.257618 2568 projected.go:194] Error preparing data for projected volume kube-api-access-tdcd7 for pod openshift-network-diagnostics/network-check-target-m7zkz: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 03 16:17:26.257776 ip-10-0-128-88 kubenswrapper[2568]: E0603 16:17:26.257682 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b8482073-6356-4d6e-bddf-2c8152f6a4f6-kube-api-access-tdcd7 podName:b8482073-6356-4d6e-bddf-2c8152f6a4f6 nodeName:}" failed. No retries permitted until 2026-06-03 16:17:42.257663333 +0000 UTC m=+33.346089913 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-tdcd7" (UniqueName: "kubernetes.io/projected/b8482073-6356-4d6e-bddf-2c8152f6a4f6-kube-api-access-tdcd7") pod "network-check-target-m7zkz" (UID: "b8482073-6356-4d6e-bddf-2c8152f6a4f6") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 03 16:17:26.560446 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:26.560365 2568 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m7zkz" Jun 03 16:17:26.560446 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:26.560397 2568 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6g497" Jun 03 16:17:26.560682 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:26.560377 2568 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fhpnv" Jun 03 16:17:26.560682 ip-10-0-128-88 kubenswrapper[2568]: E0603 16:17:26.560486 2568 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m7zkz" podUID="b8482073-6356-4d6e-bddf-2c8152f6a4f6" Jun 03 16:17:26.560682 ip-10-0-128-88 kubenswrapper[2568]: E0603 16:17:26.560572 2568 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fhpnv" podUID="d93510f8-5732-4611-8aca-02573c7e296e" Jun 03 16:17:26.560682 ip-10-0-128-88 kubenswrapper[2568]: E0603 16:17:26.560623 2568 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6g497" podUID="778470c3-9be3-440f-b177-7b846a0c7d32" Jun 03 16:17:28.560482 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:28.560448 2568 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fhpnv" Jun 03 16:17:28.560922 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:28.560493 2568 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6g497" Jun 03 16:17:28.560922 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:28.560559 2568 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m7zkz" Jun 03 16:17:28.560922 ip-10-0-128-88 kubenswrapper[2568]: E0603 16:17:28.560674 2568 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fhpnv" podUID="d93510f8-5732-4611-8aca-02573c7e296e" Jun 03 16:17:28.560922 ip-10-0-128-88 kubenswrapper[2568]: E0603 16:17:28.560739 2568 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6g497" podUID="778470c3-9be3-440f-b177-7b846a0c7d32" Jun 03 16:17:28.560922 ip-10-0-128-88 kubenswrapper[2568]: E0603 16:17:28.560822 2568 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m7zkz" podUID="b8482073-6356-4d6e-bddf-2c8152f6a4f6" Jun 03 16:17:30.559984 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:30.559833 2568 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m7zkz" Jun 03 16:17:30.560597 ip-10-0-128-88 kubenswrapper[2568]: E0603 16:17:30.560052 2568 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m7zkz" podUID="b8482073-6356-4d6e-bddf-2c8152f6a4f6" Jun 03 16:17:30.560597 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:30.559833 2568 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fhpnv" Jun 03 16:17:30.560597 ip-10-0-128-88 kubenswrapper[2568]: E0603 16:17:30.560118 2568 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fhpnv" podUID="d93510f8-5732-4611-8aca-02573c7e296e" Jun 03 16:17:30.560597 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:30.559833 2568 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6g497" Jun 03 16:17:30.560597 ip-10-0-128-88 kubenswrapper[2568]: E0603 16:17:30.560227 2568 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6g497" podUID="778470c3-9be3-440f-b177-7b846a0c7d32" Jun 03 16:17:30.700018 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:30.699992 2568 generic.go:358] "Generic (PLEG): container finished" podID="a6d060b8d198dadd30a1a5529869c2b5" containerID="315c6a84f4cd65d81dc2992714b53ada6923d6dd12ebb7c9c2e040afe963df04" exitCode=0 Jun 03 16:17:30.700103 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:30.700068 2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-88.ec2.internal" event={"ID":"a6d060b8d198dadd30a1a5529869c2b5","Type":"ContainerDied","Data":"315c6a84f4cd65d81dc2992714b53ada6923d6dd12ebb7c9c2e040afe963df04"} Jun 03 16:17:30.701243 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:30.701224 2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-88.ec2.internal" event={"ID":"e4c4dbf6c50859cd8172ab3fc4042545","Type":"ContainerStarted","Data":"f4e46061ae51112caafdc65b28d135203b7ce419a02731b00b7e1d8ad862fcc3"} Jun 03 16:17:30.702416 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:30.702395 2568 generic.go:358] "Generic (PLEG): container finished" podID="f601dafe-2b89-43a6-bb9e-2080b98aaa82" containerID="80293d758de57cf19296b129937e6e0110535cc7ac16c7400d04de8e923da50d" exitCode=0 Jun 03 16:17:30.702523 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:30.702464 2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wvl4d" event={"ID":"f601dafe-2b89-43a6-bb9e-2080b98aaa82","Type":"ContainerDied","Data":"80293d758de57cf19296b129937e6e0110535cc7ac16c7400d04de8e923da50d"} Jun 03 16:17:30.703747 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:30.703665 2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-6jcbf" event={"ID":"2d19a6a4-13e1-407c-a488-189a3651f578","Type":"ContainerStarted","Data":"10ac8d650a9e0f68e87ce4ea84453c662e1f03d9102eb44507c429005631b5ba"} Jun 03 16:17:30.704829 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:30.704806 2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-2v2dn" event={"ID":"b0a52397-ab84-4efe-a7e1-fb26e146dfbc","Type":"ContainerStarted","Data":"78636301c768a2bbcdb3afd1ab94cc5ddb353ec2054a0d9ab880813fd7735380"} Jun 03 16:17:30.706022 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:30.706002 2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-8s5gg" event={"ID":"80818dcb-cac7-4ac4-a376-ea67605a5ff8","Type":"ContainerStarted","Data":"bcb326ab81a4f593942af823aa6e02c19d17a428ffda646acb1cdfa415f2d907"} Jun 03 16:17:30.707109 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:30.707092 2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-5fhsz" event={"ID":"9c10c83c-4e52-4366-aef2-577fb3fa951f","Type":"ContainerStarted","Data":"cdb1723078c03feb45b9ce648b74e863a28a7357912e669b6019d9b7bff17d09"} Jun 03 16:17:30.708123 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:30.708106 2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-5hlfw" event={"ID":"130349f2-fa26-4e51-bf29-6819611104db","Type":"ContainerStarted","Data":"7c8b588461b2b6c3b55e09f50bdf782078108e2fb067678571cd4e5a22ef081a"} Jun 03 16:17:30.710773 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:30.710744 2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xvpwg" event={"ID":"2b06ddd4-c92a-45e5-9538-03fe00b0503f","Type":"ContainerStarted","Data":"a8202ceac8ef5b4c3f4b07cf348c74f01c2ec10ec33260fa644e194c21d68986"} Jun 03 16:17:30.713580 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:30.713561 2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xz69h" event={"ID":"e926d1e6-fe73-4a66-9488-1bd8d711ea79","Type":"ContainerStarted","Data":"05f1bcd628724fa14af0abe11428ffb2cd33c05154aa9f62b139538dd3182a57"} Jun 03 16:17:30.713580 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:30.713582 2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xz69h" event={"ID":"e926d1e6-fe73-4a66-9488-1bd8d711ea79","Type":"ContainerStarted","Data":"8b79eef32f2d78befb91e11ab288fcb170bc657adc2a8ba43ed64b3a5ad73807"} Jun 03 16:17:30.713732 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:30.713591 2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xz69h" event={"ID":"e926d1e6-fe73-4a66-9488-1bd8d711ea79","Type":"ContainerStarted","Data":"4ac4071a50bc82c7f6f103d5771cc5332df28a02489adc39ba7e4569b9f0abaa"} Jun 03 16:17:30.713732 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:30.713599 2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xz69h" event={"ID":"e926d1e6-fe73-4a66-9488-1bd8d711ea79","Type":"ContainerStarted","Data":"704488f5d60c610792efee2f1ce84b8b879d52aef20195bc07895271aec00838"} Jun 03 16:17:30.728894 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:30.728859 2568 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-5fhsz" podStartSLOduration=2.702700792 podStartE2EDuration="21.728848213s" podCreationTimestamp="2026-06-03 16:17:09 +0000 UTC" firstStartedPulling="2026-06-03 16:17:10.554289348 +0000 UTC m=+1.642715916" lastFinishedPulling="2026-06-03 16:17:29.580436756 +0000 UTC m=+20.668863337" observedRunningTime="2026-06-03 16:17:30.728555816 +0000 UTC m=+21.816982402" watchObservedRunningTime="2026-06-03 16:17:30.728848213 +0000 UTC m=+21.817274820" Jun 03 16:17:30.754288 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:30.754248 2568 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-88.ec2.internal" podStartSLOduration=20.754233514 podStartE2EDuration="20.754233514s" podCreationTimestamp="2026-06-03 16:17:10 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-06-03 16:17:30.741168625 +0000 UTC m=+21.829595216" watchObservedRunningTime="2026-06-03 16:17:30.754233514 +0000 UTC m=+21.842660106" Jun 03 16:17:30.754412 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:30.754384 2568 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-dns/node-resolver-2v2dn" podStartSLOduration=2.956763787 podStartE2EDuration="21.754377496s" podCreationTimestamp="2026-06-03 16:17:09 +0000 UTC" firstStartedPulling="2026-06-03 16:17:10.780997429 +0000 UTC m=+1.869423997" lastFinishedPulling="2026-06-03 16:17:29.578611123 +0000 UTC m=+20.667037706" observedRunningTime="2026-06-03 16:17:30.75436753 +0000 UTC m=+21.842794121" watchObservedRunningTime="2026-06-03 16:17:30.754377496 +0000 UTC m=+21.842804086" Jun 03 16:17:30.789513 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:30.789478 2568 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-image-registry/node-ca-6jcbf" podStartSLOduration=3.02664598 podStartE2EDuration="21.789463644s" podCreationTimestamp="2026-06-03 16:17:09 +0000 UTC" firstStartedPulling="2026-06-03 16:17:10.798054066 +0000 UTC m=+1.886480634" lastFinishedPulling="2026-06-03 16:17:29.560871717 +0000 UTC m=+20.649298298" observedRunningTime="2026-06-03 16:17:30.768235677 +0000 UTC m=+21.856662268" watchObservedRunningTime="2026-06-03 16:17:30.789463644 +0000 UTC m=+21.877890234" Jun 03 16:17:30.801398 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:30.801363 2568 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kube-system/konnectivity-agent-5hlfw" podStartSLOduration=3.054471844 podStartE2EDuration="21.801354853s" podCreationTimestamp="2026-06-03 16:17:09 +0000 UTC" firstStartedPulling="2026-06-03 16:17:10.832033216 +0000 UTC m=+1.920459785" lastFinishedPulling="2026-06-03 16:17:29.578916214 +0000 UTC m=+20.667342794" observedRunningTime="2026-06-03 16:17:30.801007759 +0000 UTC m=+21.889434350" watchObservedRunningTime="2026-06-03 16:17:30.801354853 +0000 UTC m=+21.889781459" Jun 03 16:17:30.818876 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:30.818842 2568 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-multus/multus-8s5gg" podStartSLOduration=2.7665357310000003 podStartE2EDuration="21.818832859s" podCreationTimestamp="2026-06-03 16:17:09 +0000 UTC" firstStartedPulling="2026-06-03 16:17:10.556967574 +0000 UTC m=+1.645394142" lastFinishedPulling="2026-06-03 16:17:29.609264698 +0000 UTC m=+20.697691270" observedRunningTime="2026-06-03 16:17:30.818572621 +0000 UTC m=+21.906999211" watchObservedRunningTime="2026-06-03 16:17:30.818832859 +0000 UTC m=+21.907259448" Jun 03 16:17:31.652040 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:31.652012 2568 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jun 03 16:17:31.716908 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:31.716872 2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xvpwg" event={"ID":"2b06ddd4-c92a-45e5-9538-03fe00b0503f","Type":"ContainerStarted","Data":"b9e3e2261a1eaaf0f3e5c38428da8575981a5a8578c88b7a9c3cee2b25327628"} Jun 03 16:17:31.719552 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:31.719518 2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xz69h" event={"ID":"e926d1e6-fe73-4a66-9488-1bd8d711ea79","Type":"ContainerStarted","Data":"844c302249f96817b5e4642670f01eadf3cca08aa8bc18299122a6fa31715fe5"} Jun 03 16:17:31.719645 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:31.719564 2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xz69h" event={"ID":"e926d1e6-fe73-4a66-9488-1bd8d711ea79","Type":"ContainerStarted","Data":"af3343888daaf59ef9070eefbeb61bf652e85c12713d8eb47910000e49839a18"} Jun 03 16:17:31.721348 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:31.721327 2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-88.ec2.internal" event={"ID":"a6d060b8d198dadd30a1a5529869c2b5","Type":"ContainerStarted","Data":"ce65fe3f31036c940b236a577eba67f32513ed055f760565d4838ee060098725"} Jun 03 16:17:31.722732 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:31.722707 2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-rmdrz" event={"ID":"6bd7702f-6b77-4535-993e-d1e053d3771b","Type":"ContainerStarted","Data":"50b3aa09b61c2fab00fa157d279ae1417981afcb2826ae114a4028ec60b07a07"} Jun 03 16:17:31.737600 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:31.737554 2568 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-88.ec2.internal" podStartSLOduration=21.737519276 podStartE2EDuration="21.737519276s" podCreationTimestamp="2026-06-03 16:17:10 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-06-03 16:17:31.737139411 +0000 UTC m=+22.825566000" watchObservedRunningTime="2026-06-03 16:17:31.737519276 +0000 UTC m=+22.825945868" Jun 03 16:17:31.752642 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:31.752597 2568 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-rmdrz" podStartSLOduration=3.912334682 podStartE2EDuration="22.752584973s" podCreationTimestamp="2026-06-03 16:17:09 +0000 UTC" firstStartedPulling="2026-06-03 16:17:10.739805028 +0000 UTC m=+1.828231597" lastFinishedPulling="2026-06-03 16:17:29.580055314 +0000 UTC m=+20.668481888" observedRunningTime="2026-06-03 16:17:31.752407507 +0000 UTC m=+22.840834097" watchObservedRunningTime="2026-06-03 16:17:31.752584973 +0000 UTC m=+22.841011611" Jun 03 16:17:32.301206 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:32.301168 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/778470c3-9be3-440f-b177-7b846a0c7d32-original-pull-secret\") pod \"global-pull-secret-syncer-6g497\" (UID: \"778470c3-9be3-440f-b177-7b846a0c7d32\") " pod="kube-system/global-pull-secret-syncer-6g497" Jun 03 16:17:32.301475 ip-10-0-128-88 kubenswrapper[2568]: E0603 16:17:32.301284 2568 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 03 16:17:32.301475 ip-10-0-128-88 kubenswrapper[2568]: E0603 16:17:32.301347 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/778470c3-9be3-440f-b177-7b846a0c7d32-original-pull-secret podName:778470c3-9be3-440f-b177-7b846a0c7d32 nodeName:}" failed. No retries permitted until 2026-06-03 16:17:48.301327691 +0000 UTC m=+39.389754259 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/778470c3-9be3-440f-b177-7b846a0c7d32-original-pull-secret") pod "global-pull-secret-syncer-6g497" (UID: "778470c3-9be3-440f-b177-7b846a0c7d32") : object "kube-system"/"original-pull-secret" not registered Jun 03 16:17:32.492344 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:32.492239 2568 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-06-03T16:17:31.652036172Z","UUID":"4d96f82f-0300-4aa0-a9ae-60a26e43837a","Handler":null,"Name":"","Endpoint":""} Jun 03 16:17:32.496003 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:32.495972 2568 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jun 03 16:17:32.496003 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:32.496004 2568 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jun 03 16:17:32.560345 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:32.560267 2568 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m7zkz" Jun 03 16:17:32.560345 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:32.560335 2568 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6g497" Jun 03 16:17:32.560565 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:32.560346 2568 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fhpnv" Jun 03 16:17:32.560565 ip-10-0-128-88 kubenswrapper[2568]: E0603 16:17:32.560431 2568 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m7zkz" podUID="b8482073-6356-4d6e-bddf-2c8152f6a4f6" Jun 03 16:17:32.560754 ip-10-0-128-88 kubenswrapper[2568]: E0603 16:17:32.560729 2568 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fhpnv" podUID="d93510f8-5732-4611-8aca-02573c7e296e" Jun 03 16:17:32.560869 ip-10-0-128-88 kubenswrapper[2568]: E0603 16:17:32.560837 2568 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6g497" podUID="778470c3-9be3-440f-b177-7b846a0c7d32" Jun 03 16:17:33.729082 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:33.728773 2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xvpwg" event={"ID":"2b06ddd4-c92a-45e5-9538-03fe00b0503f","Type":"ContainerStarted","Data":"07f0132fa997bca552ad0cbc0acdf921194f96e26b5ad71c7c07338ab66b128e"} Jun 03 16:17:33.732280 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:33.732245 2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xz69h" event={"ID":"e926d1e6-fe73-4a66-9488-1bd8d711ea79","Type":"ContainerStarted","Data":"638b6f78478332977db6ff2d01e4e7852062b51d27d4681ae4cb95a2adc2a87e"} Jun 03 16:17:33.847761 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:33.847733 2568 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-5hlfw" Jun 03 16:17:33.848211 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:33.848193 2568 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-5hlfw" Jun 03 16:17:33.862271 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:33.862232 2568 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xvpwg" podStartSLOduration=2.635660475 podStartE2EDuration="24.862217226s" podCreationTimestamp="2026-06-03 16:17:09 +0000 UTC" firstStartedPulling="2026-06-03 16:17:10.770913785 +0000 UTC m=+1.859340354" lastFinishedPulling="2026-06-03 16:17:32.997470534 +0000 UTC m=+24.085897105" observedRunningTime="2026-06-03 16:17:33.745397697 +0000 UTC m=+24.833824289" watchObservedRunningTime="2026-06-03 16:17:33.862217226 +0000 UTC m=+24.950643817" Jun 03 16:17:34.559989 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:34.559956 2568 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m7zkz" Jun 03 16:17:34.560195 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:34.559956 2568 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6g497" Jun 03 16:17:34.560195 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:34.559961 2568 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fhpnv" Jun 03 16:17:34.560195 ip-10-0-128-88 kubenswrapper[2568]: E0603 16:17:34.560149 2568 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6g497" podUID="778470c3-9be3-440f-b177-7b846a0c7d32" Jun 03 16:17:34.560195 ip-10-0-128-88 kubenswrapper[2568]: E0603 16:17:34.560052 2568 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m7zkz" podUID="b8482073-6356-4d6e-bddf-2c8152f6a4f6" Jun 03 16:17:34.560366 ip-10-0-128-88 kubenswrapper[2568]: E0603 16:17:34.560229 2568 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fhpnv" podUID="d93510f8-5732-4611-8aca-02573c7e296e" Jun 03 16:17:35.740498 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:35.740430 2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-xz69h" event={"ID":"e926d1e6-fe73-4a66-9488-1bd8d711ea79","Type":"ContainerStarted","Data":"9140f71452cdd02c842762d9dbcf328062d7665fa7bc164213f6cf97e5a2f2e2"} Jun 03 16:17:35.741715 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:35.740669 2568 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-xz69h" Jun 03 16:17:35.741715 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:35.740751 2568 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-xz69h" Jun 03 16:17:35.741715 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:35.740779 2568 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-xz69h" Jun 03 16:17:35.757708 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:35.757570 2568 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-xz69h" Jun 03 16:17:35.758545 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:35.758447 2568 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-xz69h" Jun 03 16:17:35.768186 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:35.768104 2568 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-xz69h" podStartSLOduration=7.365242457 podStartE2EDuration="26.768086803s" podCreationTimestamp="2026-06-03 16:17:09 +0000 UTC" firstStartedPulling="2026-06-03 16:17:10.754849079 +0000 UTC m=+1.843275646" lastFinishedPulling="2026-06-03 16:17:30.157693407 +0000 UTC m=+21.246119992" observedRunningTime="2026-06-03 16:17:35.76704696 +0000 UTC m=+26.855473579" watchObservedRunningTime="2026-06-03 16:17:35.768086803 +0000 UTC m=+26.856513394" Jun 03 16:17:36.559990 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:36.559962 2568 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m7zkz" Jun 03 16:17:36.559990 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:36.559975 2568 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fhpnv" Jun 03 16:17:36.559990 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:36.559975 2568 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6g497" Jun 03 16:17:36.560190 ip-10-0-128-88 kubenswrapper[2568]: E0603 16:17:36.560042 2568 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m7zkz" podUID="b8482073-6356-4d6e-bddf-2c8152f6a4f6" Jun 03 16:17:36.560190 ip-10-0-128-88 kubenswrapper[2568]: E0603 16:17:36.560133 2568 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fhpnv" podUID="d93510f8-5732-4611-8aca-02573c7e296e" Jun 03 16:17:36.560255 ip-10-0-128-88 kubenswrapper[2568]: E0603 16:17:36.560208 2568 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6g497" podUID="778470c3-9be3-440f-b177-7b846a0c7d32" Jun 03 16:17:36.743646 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:36.743612 2568 generic.go:358] "Generic (PLEG): container finished" podID="f601dafe-2b89-43a6-bb9e-2080b98aaa82" containerID="6dcf1d8e55db74608ce2ffb2788ab79fa190e06c8434d8ce7901e55b99efc3f3" exitCode=0 Jun 03 16:17:36.744109 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:36.743672 2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wvl4d" event={"ID":"f601dafe-2b89-43a6-bb9e-2080b98aaa82","Type":"ContainerDied","Data":"6dcf1d8e55db74608ce2ffb2788ab79fa190e06c8434d8ce7901e55b99efc3f3"} Jun 03 16:17:37.417860 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:37.417834 2568 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-5hlfw" Jun 03 16:17:37.418006 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:37.417987 2568 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Jun 03 16:17:37.418509 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:37.418488 2568 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-5hlfw" Jun 03 16:17:37.629549 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:37.629490 2568 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-6g497"] Jun 03 16:17:37.629961 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:37.629933 2568 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6g497" Jun 03 16:17:37.630068 ip-10-0-128-88 kubenswrapper[2568]: E0603 16:17:37.630046 2568 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6g497" podUID="778470c3-9be3-440f-b177-7b846a0c7d32" Jun 03 16:17:37.630367 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:37.630350 2568 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-m7zkz"] Jun 03 16:17:37.630427 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:37.630418 2568 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m7zkz" Jun 03 16:17:37.630503 ip-10-0-128-88 kubenswrapper[2568]: E0603 16:17:37.630486 2568 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m7zkz" podUID="b8482073-6356-4d6e-bddf-2c8152f6a4f6" Jun 03 16:17:37.631835 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:37.631813 2568 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-fhpnv"] Jun 03 16:17:37.631941 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:37.631886 2568 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fhpnv" Jun 03 16:17:37.631999 ip-10-0-128-88 kubenswrapper[2568]: E0603 16:17:37.631956 2568 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fhpnv" podUID="d93510f8-5732-4611-8aca-02573c7e296e" Jun 03 16:17:37.747327 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:37.747267 2568 generic.go:358] "Generic (PLEG): container finished" podID="f601dafe-2b89-43a6-bb9e-2080b98aaa82" containerID="c55c083f3ff1329d56f949dca92e1ee7f36e8a03d9bf5f6bd066bfaa7abc5b7d" exitCode=0 Jun 03 16:17:37.747656 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:37.747341 2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wvl4d" event={"ID":"f601dafe-2b89-43a6-bb9e-2080b98aaa82","Type":"ContainerDied","Data":"c55c083f3ff1329d56f949dca92e1ee7f36e8a03d9bf5f6bd066bfaa7abc5b7d"} Jun 03 16:17:38.750639 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:38.750604 2568 generic.go:358] "Generic (PLEG): container finished" podID="f601dafe-2b89-43a6-bb9e-2080b98aaa82" containerID="28f4cbf8aeaedadf674a7499a161540c9a743ce9183594e436b9118c8e1a7e6e" exitCode=0 Jun 03 16:17:38.750930 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:38.750659 2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wvl4d" event={"ID":"f601dafe-2b89-43a6-bb9e-2080b98aaa82","Type":"ContainerDied","Data":"28f4cbf8aeaedadf674a7499a161540c9a743ce9183594e436b9118c8e1a7e6e"} Jun 03 16:17:39.561352 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:39.561106 2568 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m7zkz" Jun 03 16:17:39.561490 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:39.561167 2568 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fhpnv" Jun 03 16:17:39.561490 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:39.561190 2568 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6g497" Jun 03 16:17:39.561616 ip-10-0-128-88 kubenswrapper[2568]: E0603 16:17:39.561484 2568 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m7zkz" podUID="b8482073-6356-4d6e-bddf-2c8152f6a4f6" Jun 03 16:17:39.561662 ip-10-0-128-88 kubenswrapper[2568]: E0603 16:17:39.561638 2568 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fhpnv" podUID="d93510f8-5732-4611-8aca-02573c7e296e" Jun 03 16:17:39.561751 ip-10-0-128-88 kubenswrapper[2568]: E0603 16:17:39.561731 2568 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6g497" podUID="778470c3-9be3-440f-b177-7b846a0c7d32" Jun 03 16:17:41.560162 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:41.560127 2568 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m7zkz" Jun 03 16:17:41.560162 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:41.560146 2568 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6g497" Jun 03 16:17:41.560818 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:41.560145 2568 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fhpnv" Jun 03 16:17:41.560818 ip-10-0-128-88 kubenswrapper[2568]: E0603 16:17:41.560243 2568 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-m7zkz" podUID="b8482073-6356-4d6e-bddf-2c8152f6a4f6" Jun 03 16:17:41.560818 ip-10-0-128-88 kubenswrapper[2568]: E0603 16:17:41.560338 2568 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6g497" podUID="778470c3-9be3-440f-b177-7b846a0c7d32" Jun 03 16:17:41.560818 ip-10-0-128-88 kubenswrapper[2568]: E0603 16:17:41.560428 2568 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-fhpnv" podUID="d93510f8-5732-4611-8aca-02573c7e296e" Jun 03 16:17:42.182712 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:42.182669 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d93510f8-5732-4611-8aca-02573c7e296e-metrics-certs\") pod \"network-metrics-daemon-fhpnv\" (UID: \"d93510f8-5732-4611-8aca-02573c7e296e\") " pod="openshift-multus/network-metrics-daemon-fhpnv" Jun 03 16:17:42.182905 ip-10-0-128-88 kubenswrapper[2568]: E0603 16:17:42.182841 2568 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 03 16:17:42.182960 ip-10-0-128-88 kubenswrapper[2568]: E0603 16:17:42.182915 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d93510f8-5732-4611-8aca-02573c7e296e-metrics-certs podName:d93510f8-5732-4611-8aca-02573c7e296e nodeName:}" failed. No retries permitted until 2026-06-03 16:18:14.182893267 +0000 UTC m=+65.271319849 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d93510f8-5732-4611-8aca-02573c7e296e-metrics-certs") pod "network-metrics-daemon-fhpnv" (UID: "d93510f8-5732-4611-8aca-02573c7e296e") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 03 16:17:42.283605 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:42.283569 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-tdcd7\" (UniqueName: \"kubernetes.io/projected/b8482073-6356-4d6e-bddf-2c8152f6a4f6-kube-api-access-tdcd7\") pod \"network-check-target-m7zkz\" (UID: \"b8482073-6356-4d6e-bddf-2c8152f6a4f6\") " pod="openshift-network-diagnostics/network-check-target-m7zkz" Jun 03 16:17:42.283764 ip-10-0-128-88 kubenswrapper[2568]: E0603 16:17:42.283724 2568 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 03 16:17:42.283764 ip-10-0-128-88 kubenswrapper[2568]: E0603 16:17:42.283740 2568 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 03 16:17:42.283764 ip-10-0-128-88 kubenswrapper[2568]: E0603 16:17:42.283749 2568 projected.go:194] Error preparing data for projected volume kube-api-access-tdcd7 for pod openshift-network-diagnostics/network-check-target-m7zkz: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 03 16:17:42.283899 ip-10-0-128-88 kubenswrapper[2568]: E0603 16:17:42.283808 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b8482073-6356-4d6e-bddf-2c8152f6a4f6-kube-api-access-tdcd7 podName:b8482073-6356-4d6e-bddf-2c8152f6a4f6 nodeName:}" failed. No retries permitted until 2026-06-03 16:18:14.283793454 +0000 UTC m=+65.372220022 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-tdcd7" (UniqueName: "kubernetes.io/projected/b8482073-6356-4d6e-bddf-2c8152f6a4f6-kube-api-access-tdcd7") pod "network-check-target-m7zkz" (UID: "b8482073-6356-4d6e-bddf-2c8152f6a4f6") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 03 16:17:42.730847 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:42.730815 2568 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-128-88.ec2.internal" event="NodeReady" Jun 03 16:17:42.731351 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:42.730933 2568 kubelet_node_status.go:550] "Fast updating node status as it just became ready" Jun 03 16:17:42.764673 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:42.764595 2568 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-5644dfb6d8-ttv56"] Jun 03 16:17:42.785916 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:42.785884 2568 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-lrqwv"] Jun 03 16:17:42.786324 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:42.786296 2568 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5644dfb6d8-ttv56" Jun 03 16:17:42.788873 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:42.788672 2568 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jun 03 16:17:42.788873 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:42.788678 2568 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jun 03 16:17:42.790658 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:42.789979 2568 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jun 03 16:17:42.790658 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:42.790390 2568 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-9hn4z\"" Jun 03 16:17:42.798632 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:42.798061 2568 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jun 03 16:17:42.799189 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:42.799162 2568 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-9bfn5"] Jun 03 16:17:42.800020 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:42.799614 2568 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-lrqwv" Jun 03 16:17:42.802231 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:42.802087 2568 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jun 03 16:17:42.802231 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:42.802124 2568 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jun 03 16:17:42.802381 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:42.802243 2568 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-h928k\"" Jun 03 16:17:42.813485 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:42.813462 2568 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-5644dfb6d8-ttv56"] Jun 03 16:17:42.813592 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:42.813558 2568 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-lrqwv"] Jun 03 16:17:42.813592 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:42.813583 2568 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-9bfn5"] Jun 03 16:17:42.813686 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:42.813606 2568 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-9bfn5" Jun 03 16:17:42.816721 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:42.816702 2568 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jun 03 16:17:42.816820 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:42.816702 2568 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jun 03 16:17:42.816820 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:42.816730 2568 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jun 03 16:17:42.816990 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:42.816969 2568 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-nmtcn\"" Jun 03 16:17:42.889416 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:42.889386 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/783221aa-e7e0-4a24-934b-f6bf8890149b-ca-trust-extracted\") pod \"image-registry-5644dfb6d8-ttv56\" (UID: \"783221aa-e7e0-4a24-934b-f6bf8890149b\") " pod="openshift-image-registry/image-registry-5644dfb6d8-ttv56" Jun 03 16:17:42.889416 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:42.889420 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/783221aa-e7e0-4a24-934b-f6bf8890149b-bound-sa-token\") pod \"image-registry-5644dfb6d8-ttv56\" (UID: \"783221aa-e7e0-4a24-934b-f6bf8890149b\") " pod="openshift-image-registry/image-registry-5644dfb6d8-ttv56" Jun 03 16:17:42.889656 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:42.889443 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/783221aa-e7e0-4a24-934b-f6bf8890149b-trusted-ca\") pod \"image-registry-5644dfb6d8-ttv56\" (UID: \"783221aa-e7e0-4a24-934b-f6bf8890149b\") " pod="openshift-image-registry/image-registry-5644dfb6d8-ttv56" Jun 03 16:17:42.889656 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:42.889482 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/0692610f-5dcf-4120-be04-37201b315bf6-config-volume\") pod \"dns-default-lrqwv\" (UID: \"0692610f-5dcf-4120-be04-37201b315bf6\") " pod="openshift-dns/dns-default-lrqwv" Jun 03 16:17:42.889656 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:42.889564 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0692610f-5dcf-4120-be04-37201b315bf6-metrics-tls\") pod \"dns-default-lrqwv\" (UID: \"0692610f-5dcf-4120-be04-37201b315bf6\") " pod="openshift-dns/dns-default-lrqwv" Jun 03 16:17:42.889656 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:42.889584 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0692610f-5dcf-4120-be04-37201b315bf6-tmp-dir\") pod \"dns-default-lrqwv\" (UID: \"0692610f-5dcf-4120-be04-37201b315bf6\") " pod="openshift-dns/dns-default-lrqwv" Jun 03 16:17:42.889656 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:42.889611 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qs7kb\" (UniqueName: \"kubernetes.io/projected/0692610f-5dcf-4120-be04-37201b315bf6-kube-api-access-qs7kb\") pod \"dns-default-lrqwv\" (UID: \"0692610f-5dcf-4120-be04-37201b315bf6\") " pod="openshift-dns/dns-default-lrqwv" Jun 03 16:17:42.889656 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:42.889649 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/783221aa-e7e0-4a24-934b-f6bf8890149b-registry-certificates\") pod \"image-registry-5644dfb6d8-ttv56\" (UID: \"783221aa-e7e0-4a24-934b-f6bf8890149b\") " pod="openshift-image-registry/image-registry-5644dfb6d8-ttv56" Jun 03 16:17:42.889912 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:42.889677 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/783221aa-e7e0-4a24-934b-f6bf8890149b-registry-tls\") pod \"image-registry-5644dfb6d8-ttv56\" (UID: \"783221aa-e7e0-4a24-934b-f6bf8890149b\") " pod="openshift-image-registry/image-registry-5644dfb6d8-ttv56" Jun 03 16:17:42.889912 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:42.889698 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/783221aa-e7e0-4a24-934b-f6bf8890149b-installation-pull-secrets\") pod \"image-registry-5644dfb6d8-ttv56\" (UID: \"783221aa-e7e0-4a24-934b-f6bf8890149b\") " pod="openshift-image-registry/image-registry-5644dfb6d8-ttv56" Jun 03 16:17:42.889912 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:42.889722 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9796n\" (UniqueName: \"kubernetes.io/projected/783221aa-e7e0-4a24-934b-f6bf8890149b-kube-api-access-9796n\") pod \"image-registry-5644dfb6d8-ttv56\" (UID: \"783221aa-e7e0-4a24-934b-f6bf8890149b\") " pod="openshift-image-registry/image-registry-5644dfb6d8-ttv56" Jun 03 16:17:42.889912 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:42.889755 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/783221aa-e7e0-4a24-934b-f6bf8890149b-image-registry-private-configuration\") pod \"image-registry-5644dfb6d8-ttv56\" (UID: \"783221aa-e7e0-4a24-934b-f6bf8890149b\") " pod="openshift-image-registry/image-registry-5644dfb6d8-ttv56" Jun 03 16:17:42.990230 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:42.990195 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/783221aa-e7e0-4a24-934b-f6bf8890149b-registry-certificates\") pod \"image-registry-5644dfb6d8-ttv56\" (UID: \"783221aa-e7e0-4a24-934b-f6bf8890149b\") " pod="openshift-image-registry/image-registry-5644dfb6d8-ttv56" Jun 03 16:17:42.990415 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:42.990245 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/783221aa-e7e0-4a24-934b-f6bf8890149b-registry-tls\") pod \"image-registry-5644dfb6d8-ttv56\" (UID: \"783221aa-e7e0-4a24-934b-f6bf8890149b\") " pod="openshift-image-registry/image-registry-5644dfb6d8-ttv56" Jun 03 16:17:42.990415 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:42.990277 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/783221aa-e7e0-4a24-934b-f6bf8890149b-installation-pull-secrets\") pod \"image-registry-5644dfb6d8-ttv56\" (UID: \"783221aa-e7e0-4a24-934b-f6bf8890149b\") " pod="openshift-image-registry/image-registry-5644dfb6d8-ttv56" Jun 03 16:17:42.990415 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:42.990301 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-9796n\" (UniqueName: \"kubernetes.io/projected/783221aa-e7e0-4a24-934b-f6bf8890149b-kube-api-access-9796n\") pod \"image-registry-5644dfb6d8-ttv56\" (UID: \"783221aa-e7e0-4a24-934b-f6bf8890149b\") " pod="openshift-image-registry/image-registry-5644dfb6d8-ttv56" Jun 03 16:17:42.990415 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:42.990331 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/783221aa-e7e0-4a24-934b-f6bf8890149b-image-registry-private-configuration\") pod \"image-registry-5644dfb6d8-ttv56\" (UID: \"783221aa-e7e0-4a24-934b-f6bf8890149b\") " pod="openshift-image-registry/image-registry-5644dfb6d8-ttv56" Jun 03 16:17:42.990415 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:42.990371 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/469cfe56-169a-4b98-b437-38f4ea41f481-cert\") pod \"ingress-canary-9bfn5\" (UID: \"469cfe56-169a-4b98-b437-38f4ea41f481\") " pod="openshift-ingress-canary/ingress-canary-9bfn5" Jun 03 16:17:42.990415 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:42.990411 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4pgpb\" (UniqueName: \"kubernetes.io/projected/469cfe56-169a-4b98-b437-38f4ea41f481-kube-api-access-4pgpb\") pod \"ingress-canary-9bfn5\" (UID: \"469cfe56-169a-4b98-b437-38f4ea41f481\") " pod="openshift-ingress-canary/ingress-canary-9bfn5" Jun 03 16:17:42.990720 ip-10-0-128-88 kubenswrapper[2568]: E0603 16:17:42.990437 2568 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 03 16:17:42.990720 ip-10-0-128-88 kubenswrapper[2568]: E0603 16:17:42.990460 2568 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5644dfb6d8-ttv56: secret "image-registry-tls" not found Jun 03 16:17:42.990720 ip-10-0-128-88 kubenswrapper[2568]: E0603 16:17:42.990507 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/783221aa-e7e0-4a24-934b-f6bf8890149b-registry-tls podName:783221aa-e7e0-4a24-934b-f6bf8890149b nodeName:}" failed. No retries permitted until 2026-06-03 16:17:43.490492845 +0000 UTC m=+34.578919414 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/783221aa-e7e0-4a24-934b-f6bf8890149b-registry-tls") pod "image-registry-5644dfb6d8-ttv56" (UID: "783221aa-e7e0-4a24-934b-f6bf8890149b") : secret "image-registry-tls" not found Jun 03 16:17:42.990720 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:42.990441 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/783221aa-e7e0-4a24-934b-f6bf8890149b-ca-trust-extracted\") pod \"image-registry-5644dfb6d8-ttv56\" (UID: \"783221aa-e7e0-4a24-934b-f6bf8890149b\") " pod="openshift-image-registry/image-registry-5644dfb6d8-ttv56" Jun 03 16:17:42.990720 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:42.990688 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/783221aa-e7e0-4a24-934b-f6bf8890149b-bound-sa-token\") pod \"image-registry-5644dfb6d8-ttv56\" (UID: \"783221aa-e7e0-4a24-934b-f6bf8890149b\") " pod="openshift-image-registry/image-registry-5644dfb6d8-ttv56" Jun 03 16:17:42.990968 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:42.990728 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/783221aa-e7e0-4a24-934b-f6bf8890149b-trusted-ca\") pod \"image-registry-5644dfb6d8-ttv56\" (UID: \"783221aa-e7e0-4a24-934b-f6bf8890149b\") " pod="openshift-image-registry/image-registry-5644dfb6d8-ttv56" Jun 03 16:17:42.990968 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:42.990754 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/0692610f-5dcf-4120-be04-37201b315bf6-config-volume\") pod \"dns-default-lrqwv\" (UID: \"0692610f-5dcf-4120-be04-37201b315bf6\") " pod="openshift-dns/dns-default-lrqwv" Jun 03 16:17:42.990968 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:42.990772 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/783221aa-e7e0-4a24-934b-f6bf8890149b-ca-trust-extracted\") pod \"image-registry-5644dfb6d8-ttv56\" (UID: \"783221aa-e7e0-4a24-934b-f6bf8890149b\") " pod="openshift-image-registry/image-registry-5644dfb6d8-ttv56" Jun 03 16:17:42.990968 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:42.990807 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0692610f-5dcf-4120-be04-37201b315bf6-metrics-tls\") pod \"dns-default-lrqwv\" (UID: \"0692610f-5dcf-4120-be04-37201b315bf6\") " pod="openshift-dns/dns-default-lrqwv" Jun 03 16:17:42.990968 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:42.990835 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0692610f-5dcf-4120-be04-37201b315bf6-tmp-dir\") pod \"dns-default-lrqwv\" (UID: \"0692610f-5dcf-4120-be04-37201b315bf6\") " pod="openshift-dns/dns-default-lrqwv" Jun 03 16:17:42.990968 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:42.990859 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-qs7kb\" (UniqueName: \"kubernetes.io/projected/0692610f-5dcf-4120-be04-37201b315bf6-kube-api-access-qs7kb\") pod \"dns-default-lrqwv\" (UID: \"0692610f-5dcf-4120-be04-37201b315bf6\") " pod="openshift-dns/dns-default-lrqwv" Jun 03 16:17:42.990968 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:42.990960 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/783221aa-e7e0-4a24-934b-f6bf8890149b-registry-certificates\") pod \"image-registry-5644dfb6d8-ttv56\" (UID: \"783221aa-e7e0-4a24-934b-f6bf8890149b\") " pod="openshift-image-registry/image-registry-5644dfb6d8-ttv56" Jun 03 16:17:42.991296 ip-10-0-128-88 kubenswrapper[2568]: E0603 16:17:42.991046 2568 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 03 16:17:42.991296 ip-10-0-128-88 kubenswrapper[2568]: E0603 16:17:42.991094 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0692610f-5dcf-4120-be04-37201b315bf6-metrics-tls podName:0692610f-5dcf-4120-be04-37201b315bf6 nodeName:}" failed. No retries permitted until 2026-06-03 16:17:43.491077312 +0000 UTC m=+34.579503885 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/0692610f-5dcf-4120-be04-37201b315bf6-metrics-tls") pod "dns-default-lrqwv" (UID: "0692610f-5dcf-4120-be04-37201b315bf6") : secret "dns-default-metrics-tls" not found Jun 03 16:17:42.991413 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:42.991347 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/0692610f-5dcf-4120-be04-37201b315bf6-config-volume\") pod \"dns-default-lrqwv\" (UID: \"0692610f-5dcf-4120-be04-37201b315bf6\") " pod="openshift-dns/dns-default-lrqwv" Jun 03 16:17:42.991413 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:42.991347 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0692610f-5dcf-4120-be04-37201b315bf6-tmp-dir\") pod \"dns-default-lrqwv\" (UID: \"0692610f-5dcf-4120-be04-37201b315bf6\") " pod="openshift-dns/dns-default-lrqwv" Jun 03 16:17:42.991734 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:42.991649 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/783221aa-e7e0-4a24-934b-f6bf8890149b-trusted-ca\") pod \"image-registry-5644dfb6d8-ttv56\" (UID: \"783221aa-e7e0-4a24-934b-f6bf8890149b\") " pod="openshift-image-registry/image-registry-5644dfb6d8-ttv56" Jun 03 16:17:42.995366 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:42.995321 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/783221aa-e7e0-4a24-934b-f6bf8890149b-image-registry-private-configuration\") pod \"image-registry-5644dfb6d8-ttv56\" (UID: \"783221aa-e7e0-4a24-934b-f6bf8890149b\") " pod="openshift-image-registry/image-registry-5644dfb6d8-ttv56" Jun 03 16:17:42.995469 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:42.995395 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/783221aa-e7e0-4a24-934b-f6bf8890149b-installation-pull-secrets\") pod \"image-registry-5644dfb6d8-ttv56\" (UID: \"783221aa-e7e0-4a24-934b-f6bf8890149b\") " pod="openshift-image-registry/image-registry-5644dfb6d8-ttv56" Jun 03 16:17:43.000093 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:43.000047 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9796n\" (UniqueName: \"kubernetes.io/projected/783221aa-e7e0-4a24-934b-f6bf8890149b-kube-api-access-9796n\") pod \"image-registry-5644dfb6d8-ttv56\" (UID: \"783221aa-e7e0-4a24-934b-f6bf8890149b\") " pod="openshift-image-registry/image-registry-5644dfb6d8-ttv56" Jun 03 16:17:43.000093 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:43.000078 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/783221aa-e7e0-4a24-934b-f6bf8890149b-bound-sa-token\") pod \"image-registry-5644dfb6d8-ttv56\" (UID: \"783221aa-e7e0-4a24-934b-f6bf8890149b\") " pod="openshift-image-registry/image-registry-5644dfb6d8-ttv56" Jun 03 16:17:43.000711 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:43.000689 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qs7kb\" (UniqueName: \"kubernetes.io/projected/0692610f-5dcf-4120-be04-37201b315bf6-kube-api-access-qs7kb\") pod \"dns-default-lrqwv\" (UID: \"0692610f-5dcf-4120-be04-37201b315bf6\") " pod="openshift-dns/dns-default-lrqwv" Jun 03 16:17:43.091864 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:43.091792 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/469cfe56-169a-4b98-b437-38f4ea41f481-cert\") pod \"ingress-canary-9bfn5\" (UID: \"469cfe56-169a-4b98-b437-38f4ea41f481\") " pod="openshift-ingress-canary/ingress-canary-9bfn5" Jun 03 16:17:43.091864 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:43.091840 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-4pgpb\" (UniqueName: \"kubernetes.io/projected/469cfe56-169a-4b98-b437-38f4ea41f481-kube-api-access-4pgpb\") pod \"ingress-canary-9bfn5\" (UID: \"469cfe56-169a-4b98-b437-38f4ea41f481\") " pod="openshift-ingress-canary/ingress-canary-9bfn5" Jun 03 16:17:43.092092 ip-10-0-128-88 kubenswrapper[2568]: E0603 16:17:43.091952 2568 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found Jun 03 16:17:43.092092 ip-10-0-128-88 kubenswrapper[2568]: E0603 16:17:43.092014 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/469cfe56-169a-4b98-b437-38f4ea41f481-cert podName:469cfe56-169a-4b98-b437-38f4ea41f481 nodeName:}" failed. No retries permitted until 2026-06-03 16:17:43.591994485 +0000 UTC m=+34.680421056 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/469cfe56-169a-4b98-b437-38f4ea41f481-cert") pod "ingress-canary-9bfn5" (UID: "469cfe56-169a-4b98-b437-38f4ea41f481") : secret "canary-serving-cert" not found Jun 03 16:17:43.106454 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:43.106428 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4pgpb\" (UniqueName: \"kubernetes.io/projected/469cfe56-169a-4b98-b437-38f4ea41f481-kube-api-access-4pgpb\") pod \"ingress-canary-9bfn5\" (UID: \"469cfe56-169a-4b98-b437-38f4ea41f481\") " pod="openshift-ingress-canary/ingress-canary-9bfn5" Jun 03 16:17:43.495816 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:43.495777 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0692610f-5dcf-4120-be04-37201b315bf6-metrics-tls\") pod \"dns-default-lrqwv\" (UID: \"0692610f-5dcf-4120-be04-37201b315bf6\") " pod="openshift-dns/dns-default-lrqwv" Jun 03 16:17:43.496103 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:43.495830 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/783221aa-e7e0-4a24-934b-f6bf8890149b-registry-tls\") pod \"image-registry-5644dfb6d8-ttv56\" (UID: \"783221aa-e7e0-4a24-934b-f6bf8890149b\") " pod="openshift-image-registry/image-registry-5644dfb6d8-ttv56" Jun 03 16:17:43.496103 ip-10-0-128-88 kubenswrapper[2568]: E0603 16:17:43.495938 2568 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 03 16:17:43.496103 ip-10-0-128-88 kubenswrapper[2568]: E0603 16:17:43.495949 2568 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 03 16:17:43.496103 ip-10-0-128-88 kubenswrapper[2568]: E0603 16:17:43.495963 2568 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5644dfb6d8-ttv56: secret "image-registry-tls" not found Jun 03 16:17:43.496103 ip-10-0-128-88 kubenswrapper[2568]: E0603 16:17:43.496002 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0692610f-5dcf-4120-be04-37201b315bf6-metrics-tls podName:0692610f-5dcf-4120-be04-37201b315bf6 nodeName:}" failed. No retries permitted until 2026-06-03 16:17:44.495986551 +0000 UTC m=+35.584413120 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/0692610f-5dcf-4120-be04-37201b315bf6-metrics-tls") pod "dns-default-lrqwv" (UID: "0692610f-5dcf-4120-be04-37201b315bf6") : secret "dns-default-metrics-tls" not found Jun 03 16:17:43.496103 ip-10-0-128-88 kubenswrapper[2568]: E0603 16:17:43.496015 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/783221aa-e7e0-4a24-934b-f6bf8890149b-registry-tls podName:783221aa-e7e0-4a24-934b-f6bf8890149b nodeName:}" failed. No retries permitted until 2026-06-03 16:17:44.496009388 +0000 UTC m=+35.584435955 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/783221aa-e7e0-4a24-934b-f6bf8890149b-registry-tls") pod "image-registry-5644dfb6d8-ttv56" (UID: "783221aa-e7e0-4a24-934b-f6bf8890149b") : secret "image-registry-tls" not found Jun 03 16:17:43.560643 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:43.560602 2568 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fhpnv" Jun 03 16:17:43.560815 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:43.560602 2568 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6g497" Jun 03 16:17:43.560815 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:43.560600 2568 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m7zkz" Jun 03 16:17:43.564014 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:43.563864 2568 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jun 03 16:17:43.564014 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:43.563891 2568 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jun 03 16:17:43.564200 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:43.564094 2568 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-f9ccm\"" Jun 03 16:17:43.564200 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:43.564188 2568 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-cchx6\"" Jun 03 16:17:43.564301 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:43.564284 2568 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jun 03 16:17:43.564439 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:43.564423 2568 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jun 03 16:17:43.597076 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:43.597054 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/469cfe56-169a-4b98-b437-38f4ea41f481-cert\") pod \"ingress-canary-9bfn5\" (UID: \"469cfe56-169a-4b98-b437-38f4ea41f481\") " pod="openshift-ingress-canary/ingress-canary-9bfn5" Jun 03 16:17:43.597198 ip-10-0-128-88 kubenswrapper[2568]: E0603 16:17:43.597179 2568 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found Jun 03 16:17:43.597256 ip-10-0-128-88 kubenswrapper[2568]: E0603 16:17:43.597242 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/469cfe56-169a-4b98-b437-38f4ea41f481-cert podName:469cfe56-169a-4b98-b437-38f4ea41f481 nodeName:}" failed. No retries permitted until 2026-06-03 16:17:44.597227086 +0000 UTC m=+35.685653673 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/469cfe56-169a-4b98-b437-38f4ea41f481-cert") pod "ingress-canary-9bfn5" (UID: "469cfe56-169a-4b98-b437-38f4ea41f481") : secret "canary-serving-cert" not found Jun 03 16:17:44.505136 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:44.505105 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0692610f-5dcf-4120-be04-37201b315bf6-metrics-tls\") pod \"dns-default-lrqwv\" (UID: \"0692610f-5dcf-4120-be04-37201b315bf6\") " pod="openshift-dns/dns-default-lrqwv" Jun 03 16:17:44.505768 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:44.505154 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/783221aa-e7e0-4a24-934b-f6bf8890149b-registry-tls\") pod \"image-registry-5644dfb6d8-ttv56\" (UID: \"783221aa-e7e0-4a24-934b-f6bf8890149b\") " pod="openshift-image-registry/image-registry-5644dfb6d8-ttv56" Jun 03 16:17:44.505768 ip-10-0-128-88 kubenswrapper[2568]: E0603 16:17:44.505245 2568 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 03 16:17:44.505768 ip-10-0-128-88 kubenswrapper[2568]: E0603 16:17:44.505307 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0692610f-5dcf-4120-be04-37201b315bf6-metrics-tls podName:0692610f-5dcf-4120-be04-37201b315bf6 nodeName:}" failed. No retries permitted until 2026-06-03 16:17:46.505288096 +0000 UTC m=+37.593714669 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/0692610f-5dcf-4120-be04-37201b315bf6-metrics-tls") pod "dns-default-lrqwv" (UID: "0692610f-5dcf-4120-be04-37201b315bf6") : secret "dns-default-metrics-tls" not found Jun 03 16:17:44.505768 ip-10-0-128-88 kubenswrapper[2568]: E0603 16:17:44.505247 2568 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 03 16:17:44.505768 ip-10-0-128-88 kubenswrapper[2568]: E0603 16:17:44.505350 2568 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5644dfb6d8-ttv56: secret "image-registry-tls" not found Jun 03 16:17:44.505768 ip-10-0-128-88 kubenswrapper[2568]: E0603 16:17:44.505389 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/783221aa-e7e0-4a24-934b-f6bf8890149b-registry-tls podName:783221aa-e7e0-4a24-934b-f6bf8890149b nodeName:}" failed. No retries permitted until 2026-06-03 16:17:46.505377274 +0000 UTC m=+37.593803849 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/783221aa-e7e0-4a24-934b-f6bf8890149b-registry-tls") pod "image-registry-5644dfb6d8-ttv56" (UID: "783221aa-e7e0-4a24-934b-f6bf8890149b") : secret "image-registry-tls" not found Jun 03 16:17:44.606045 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:44.606014 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/469cfe56-169a-4b98-b437-38f4ea41f481-cert\") pod \"ingress-canary-9bfn5\" (UID: \"469cfe56-169a-4b98-b437-38f4ea41f481\") " pod="openshift-ingress-canary/ingress-canary-9bfn5" Jun 03 16:17:44.606198 ip-10-0-128-88 kubenswrapper[2568]: E0603 16:17:44.606120 2568 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found Jun 03 16:17:44.606198 ip-10-0-128-88 kubenswrapper[2568]: E0603 16:17:44.606170 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/469cfe56-169a-4b98-b437-38f4ea41f481-cert podName:469cfe56-169a-4b98-b437-38f4ea41f481 nodeName:}" failed. No retries permitted until 2026-06-03 16:17:46.606156188 +0000 UTC m=+37.694582757 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/469cfe56-169a-4b98-b437-38f4ea41f481-cert") pod "ingress-canary-9bfn5" (UID: "469cfe56-169a-4b98-b437-38f4ea41f481") : secret "canary-serving-cert" not found Jun 03 16:17:45.767392 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:45.767362 2568 generic.go:358] "Generic (PLEG): container finished" podID="f601dafe-2b89-43a6-bb9e-2080b98aaa82" containerID="e0b9aa8ddba3c9a9e48c7da10f8c501bc63bb272b0d16dd5985808dffe18cea9" exitCode=0 Jun 03 16:17:45.767890 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:45.767407 2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wvl4d" event={"ID":"f601dafe-2b89-43a6-bb9e-2080b98aaa82","Type":"ContainerDied","Data":"e0b9aa8ddba3c9a9e48c7da10f8c501bc63bb272b0d16dd5985808dffe18cea9"} Jun 03 16:17:46.519880 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:46.519847 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0692610f-5dcf-4120-be04-37201b315bf6-metrics-tls\") pod \"dns-default-lrqwv\" (UID: \"0692610f-5dcf-4120-be04-37201b315bf6\") " pod="openshift-dns/dns-default-lrqwv" Jun 03 16:17:46.520036 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:46.519898 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/783221aa-e7e0-4a24-934b-f6bf8890149b-registry-tls\") pod \"image-registry-5644dfb6d8-ttv56\" (UID: \"783221aa-e7e0-4a24-934b-f6bf8890149b\") " pod="openshift-image-registry/image-registry-5644dfb6d8-ttv56" Jun 03 16:17:46.520036 ip-10-0-128-88 kubenswrapper[2568]: E0603 16:17:46.519996 2568 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 03 16:17:46.520125 ip-10-0-128-88 kubenswrapper[2568]: E0603 16:17:46.520063 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0692610f-5dcf-4120-be04-37201b315bf6-metrics-tls podName:0692610f-5dcf-4120-be04-37201b315bf6 nodeName:}" failed. No retries permitted until 2026-06-03 16:17:50.52004468 +0000 UTC m=+41.608471248 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/0692610f-5dcf-4120-be04-37201b315bf6-metrics-tls") pod "dns-default-lrqwv" (UID: "0692610f-5dcf-4120-be04-37201b315bf6") : secret "dns-default-metrics-tls" not found Jun 03 16:17:46.520125 ip-10-0-128-88 kubenswrapper[2568]: E0603 16:17:46.520004 2568 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 03 16:17:46.520125 ip-10-0-128-88 kubenswrapper[2568]: E0603 16:17:46.520094 2568 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5644dfb6d8-ttv56: secret "image-registry-tls" not found Jun 03 16:17:46.520250 ip-10-0-128-88 kubenswrapper[2568]: E0603 16:17:46.520133 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/783221aa-e7e0-4a24-934b-f6bf8890149b-registry-tls podName:783221aa-e7e0-4a24-934b-f6bf8890149b nodeName:}" failed. No retries permitted until 2026-06-03 16:17:50.520122371 +0000 UTC m=+41.608548939 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/783221aa-e7e0-4a24-934b-f6bf8890149b-registry-tls") pod "image-registry-5644dfb6d8-ttv56" (UID: "783221aa-e7e0-4a24-934b-f6bf8890149b") : secret "image-registry-tls" not found Jun 03 16:17:46.620839 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:46.620805 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/469cfe56-169a-4b98-b437-38f4ea41f481-cert\") pod \"ingress-canary-9bfn5\" (UID: \"469cfe56-169a-4b98-b437-38f4ea41f481\") " pod="openshift-ingress-canary/ingress-canary-9bfn5" Jun 03 16:17:46.621020 ip-10-0-128-88 kubenswrapper[2568]: E0603 16:17:46.620975 2568 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found Jun 03 16:17:46.621081 ip-10-0-128-88 kubenswrapper[2568]: E0603 16:17:46.621053 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/469cfe56-169a-4b98-b437-38f4ea41f481-cert podName:469cfe56-169a-4b98-b437-38f4ea41f481 nodeName:}" failed. No retries permitted until 2026-06-03 16:17:50.621032521 +0000 UTC m=+41.709459093 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/469cfe56-169a-4b98-b437-38f4ea41f481-cert") pod "ingress-canary-9bfn5" (UID: "469cfe56-169a-4b98-b437-38f4ea41f481") : secret "canary-serving-cert" not found Jun 03 16:17:46.773052 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:46.772969 2568 generic.go:358] "Generic (PLEG): container finished" podID="f601dafe-2b89-43a6-bb9e-2080b98aaa82" containerID="77afb77124df2d37fc272255729d6e14344fd8e41b20f496f4866371487f3d78" exitCode=0 Jun 03 16:17:46.773052 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:46.773017 2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wvl4d" event={"ID":"f601dafe-2b89-43a6-bb9e-2080b98aaa82","Type":"ContainerDied","Data":"77afb77124df2d37fc272255729d6e14344fd8e41b20f496f4866371487f3d78"} Jun 03 16:17:47.777991 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:47.777813 2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wvl4d" event={"ID":"f601dafe-2b89-43a6-bb9e-2080b98aaa82","Type":"ContainerStarted","Data":"92fffb68772a777a00b9b686eff91283bec206b926db3a6c2952f86d1ca0b3c6"} Jun 03 16:17:47.799488 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:47.799441 2568 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-wvl4d" podStartSLOduration=4.574866203 podStartE2EDuration="38.799428215s" podCreationTimestamp="2026-06-03 16:17:09 +0000 UTC" firstStartedPulling="2026-06-03 16:17:10.813764949 +0000 UTC m=+1.902191521" lastFinishedPulling="2026-06-03 16:17:45.038326961 +0000 UTC m=+36.126753533" observedRunningTime="2026-06-03 16:17:47.79822782 +0000 UTC m=+38.886654409" watchObservedRunningTime="2026-06-03 16:17:47.799428215 +0000 UTC m=+38.887854804" Jun 03 16:17:48.334133 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:48.334100 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/778470c3-9be3-440f-b177-7b846a0c7d32-original-pull-secret\") pod \"global-pull-secret-syncer-6g497\" (UID: \"778470c3-9be3-440f-b177-7b846a0c7d32\") " pod="kube-system/global-pull-secret-syncer-6g497" Jun 03 16:17:48.337095 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:48.337073 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/778470c3-9be3-440f-b177-7b846a0c7d32-original-pull-secret\") pod \"global-pull-secret-syncer-6g497\" (UID: \"778470c3-9be3-440f-b177-7b846a0c7d32\") " pod="kube-system/global-pull-secret-syncer-6g497" Jun 03 16:17:48.379580 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:48.379551 2568 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6g497" Jun 03 16:17:48.528830 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:48.528775 2568 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-6g497"] Jun 03 16:17:48.533000 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:48.532961 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod778470c3_9be3_440f_b177_7b846a0c7d32.slice/crio-b1348012011050cc2c3f8f690acbf73d4ad85f42d87e7ace5ab01fe0509d0ded WatchSource:0}: Error finding container b1348012011050cc2c3f8f690acbf73d4ad85f42d87e7ace5ab01fe0509d0ded: Status 404 returned error can't find the container with id b1348012011050cc2c3f8f690acbf73d4ad85f42d87e7ace5ab01fe0509d0ded Jun 03 16:17:48.780353 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:48.780316 2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-6g497" event={"ID":"778470c3-9be3-440f-b177-7b846a0c7d32","Type":"ContainerStarted","Data":"b1348012011050cc2c3f8f690acbf73d4ad85f42d87e7ace5ab01fe0509d0ded"} Jun 03 16:17:49.329183 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:49.329153 2568 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-96c6f749-klgt4"] Jun 03 16:17:49.345632 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:49.345577 2568 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-96c6f749-klgt4"] Jun 03 16:17:49.345778 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:49.345724 2568 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-96c6f749-klgt4" Jun 03 16:17:49.348214 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:49.348190 2568 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\"" Jun 03 16:17:49.349067 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:49.348876 2568 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-l4mjn\"" Jun 03 16:17:49.349067 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:49.348884 2568 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\"" Jun 03 16:17:49.442974 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:49.442943 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lg46j\" (UniqueName: \"kubernetes.io/projected/24cc0ece-f688-422f-a4cd-e2bd862e0ffd-kube-api-access-lg46j\") pod \"migrator-96c6f749-klgt4\" (UID: \"24cc0ece-f688-422f-a4cd-e2bd862e0ffd\") " pod="openshift-kube-storage-version-migrator/migrator-96c6f749-klgt4" Jun 03 16:17:49.543348 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:49.543319 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-lg46j\" (UniqueName: \"kubernetes.io/projected/24cc0ece-f688-422f-a4cd-e2bd862e0ffd-kube-api-access-lg46j\") pod \"migrator-96c6f749-klgt4\" (UID: \"24cc0ece-f688-422f-a4cd-e2bd862e0ffd\") " pod="openshift-kube-storage-version-migrator/migrator-96c6f749-klgt4" Jun 03 16:17:49.552345 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:49.552320 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lg46j\" (UniqueName: \"kubernetes.io/projected/24cc0ece-f688-422f-a4cd-e2bd862e0ffd-kube-api-access-lg46j\") pod \"migrator-96c6f749-klgt4\" (UID: \"24cc0ece-f688-422f-a4cd-e2bd862e0ffd\") " pod="openshift-kube-storage-version-migrator/migrator-96c6f749-klgt4" Jun 03 16:17:49.656969 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:49.656946 2568 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-96c6f749-klgt4" Jun 03 16:17:49.810615 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:49.810575 2568 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-96c6f749-klgt4"] Jun 03 16:17:49.814987 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:17:49.814956 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod24cc0ece_f688_422f_a4cd_e2bd862e0ffd.slice/crio-981a34a225d9f5d25e6e0458fea754bf0b7a6773133af8535d93e0ed0224f553 WatchSource:0}: Error finding container 981a34a225d9f5d25e6e0458fea754bf0b7a6773133af8535d93e0ed0224f553: Status 404 returned error can't find the container with id 981a34a225d9f5d25e6e0458fea754bf0b7a6773133af8535d93e0ed0224f553 Jun 03 16:17:49.850508 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:49.850482 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-2v2dn_b0a52397-ab84-4efe-a7e1-fb26e146dfbc/dns-node-resolver/0.log" Jun 03 16:17:50.553759 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:50.553721 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0692610f-5dcf-4120-be04-37201b315bf6-metrics-tls\") pod \"dns-default-lrqwv\" (UID: \"0692610f-5dcf-4120-be04-37201b315bf6\") " pod="openshift-dns/dns-default-lrqwv" Jun 03 16:17:50.554031 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:50.553792 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/783221aa-e7e0-4a24-934b-f6bf8890149b-registry-tls\") pod \"image-registry-5644dfb6d8-ttv56\" (UID: \"783221aa-e7e0-4a24-934b-f6bf8890149b\") " pod="openshift-image-registry/image-registry-5644dfb6d8-ttv56" Jun 03 16:17:50.554031 ip-10-0-128-88 kubenswrapper[2568]: E0603 16:17:50.553907 2568 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 03 16:17:50.554031 ip-10-0-128-88 kubenswrapper[2568]: E0603 16:17:50.553961 2568 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 03 16:17:50.554031 ip-10-0-128-88 kubenswrapper[2568]: E0603 16:17:50.553982 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0692610f-5dcf-4120-be04-37201b315bf6-metrics-tls podName:0692610f-5dcf-4120-be04-37201b315bf6 nodeName:}" failed. No retries permitted until 2026-06-03 16:17:58.553958071 +0000 UTC m=+49.642384643 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/0692610f-5dcf-4120-be04-37201b315bf6-metrics-tls") pod "dns-default-lrqwv" (UID: "0692610f-5dcf-4120-be04-37201b315bf6") : secret "dns-default-metrics-tls" not found Jun 03 16:17:50.554031 ip-10-0-128-88 kubenswrapper[2568]: E0603 16:17:50.553984 2568 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5644dfb6d8-ttv56: secret "image-registry-tls" not found Jun 03 16:17:50.554273 ip-10-0-128-88 kubenswrapper[2568]: E0603 16:17:50.554040 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/783221aa-e7e0-4a24-934b-f6bf8890149b-registry-tls podName:783221aa-e7e0-4a24-934b-f6bf8890149b nodeName:}" failed. No retries permitted until 2026-06-03 16:17:58.55402307 +0000 UTC m=+49.642449657 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/783221aa-e7e0-4a24-934b-f6bf8890149b-registry-tls") pod "image-registry-5644dfb6d8-ttv56" (UID: "783221aa-e7e0-4a24-934b-f6bf8890149b") : secret "image-registry-tls" not found Jun 03 16:17:50.655130 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:50.655091 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/469cfe56-169a-4b98-b437-38f4ea41f481-cert\") pod \"ingress-canary-9bfn5\" (UID: \"469cfe56-169a-4b98-b437-38f4ea41f481\") " pod="openshift-ingress-canary/ingress-canary-9bfn5" Jun 03 16:17:50.655281 ip-10-0-128-88 kubenswrapper[2568]: E0603 16:17:50.655233 2568 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found Jun 03 16:17:50.655411 ip-10-0-128-88 kubenswrapper[2568]: E0603 16:17:50.655326 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/469cfe56-169a-4b98-b437-38f4ea41f481-cert podName:469cfe56-169a-4b98-b437-38f4ea41f481 nodeName:}" failed. No retries permitted until 2026-06-03 16:17:58.655305088 +0000 UTC m=+49.743731661 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/469cfe56-169a-4b98-b437-38f4ea41f481-cert") pod "ingress-canary-9bfn5" (UID: "469cfe56-169a-4b98-b437-38f4ea41f481") : secret "canary-serving-cert" not found Jun 03 16:17:50.786310 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:50.786268 2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-96c6f749-klgt4" event={"ID":"24cc0ece-f688-422f-a4cd-e2bd862e0ffd","Type":"ContainerStarted","Data":"981a34a225d9f5d25e6e0458fea754bf0b7a6773133af8535d93e0ed0224f553"} Jun 03 16:17:51.050466 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:51.050438 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-6jcbf_2d19a6a4-13e1-407c-a488-189a3651f578/node-ca/0.log" Jun 03 16:17:53.794114 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:53.794082 2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-96c6f749-klgt4" event={"ID":"24cc0ece-f688-422f-a4cd-e2bd862e0ffd","Type":"ContainerStarted","Data":"72eb20644950d6611c35bc4b0030d4329fc526f0225a9adbc138b1aa2b5f8fcc"} Jun 03 16:17:53.794547 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:53.794121 2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-96c6f749-klgt4" event={"ID":"24cc0ece-f688-422f-a4cd-e2bd862e0ffd","Type":"ContainerStarted","Data":"ab2e52d0d0b7937a7d92c91190ed49febe53c0100c1ba2ff2aed7b63edede3dc"} Jun 03 16:17:53.795356 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:53.795333 2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-6g497" event={"ID":"778470c3-9be3-440f-b177-7b846a0c7d32","Type":"ContainerStarted","Data":"7f4c4b26e6972f49219d425ce1b1349b4141d4f6192caf0c1683ae8d22e75521"} Jun 03 16:17:53.809440 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:53.809399 2568 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-96c6f749-klgt4" podStartSLOduration=1.5454133300000001 podStartE2EDuration="4.809387701s" podCreationTimestamp="2026-06-03 16:17:49 +0000 UTC" firstStartedPulling="2026-06-03 16:17:49.817170775 +0000 UTC m=+40.905597348" lastFinishedPulling="2026-06-03 16:17:53.081145136 +0000 UTC m=+44.169571719" observedRunningTime="2026-06-03 16:17:53.808865979 +0000 UTC m=+44.897292590" watchObservedRunningTime="2026-06-03 16:17:53.809387701 +0000 UTC m=+44.897814291" Jun 03 16:17:53.825268 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:53.825230 2568 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-6g497" podStartSLOduration=33.267261646 podStartE2EDuration="37.825219791s" podCreationTimestamp="2026-06-03 16:17:16 +0000 UTC" firstStartedPulling="2026-06-03 16:17:48.534624793 +0000 UTC m=+39.623051361" lastFinishedPulling="2026-06-03 16:17:53.092582933 +0000 UTC m=+44.181009506" observedRunningTime="2026-06-03 16:17:53.825142766 +0000 UTC m=+44.913569358" watchObservedRunningTime="2026-06-03 16:17:53.825219791 +0000 UTC m=+44.913646382" Jun 03 16:17:58.610122 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:58.610084 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/783221aa-e7e0-4a24-934b-f6bf8890149b-registry-tls\") pod \"image-registry-5644dfb6d8-ttv56\" (UID: \"783221aa-e7e0-4a24-934b-f6bf8890149b\") " pod="openshift-image-registry/image-registry-5644dfb6d8-ttv56" Jun 03 16:17:58.610495 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:58.610164 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0692610f-5dcf-4120-be04-37201b315bf6-metrics-tls\") pod \"dns-default-lrqwv\" (UID: \"0692610f-5dcf-4120-be04-37201b315bf6\") " pod="openshift-dns/dns-default-lrqwv" Jun 03 16:17:58.610495 ip-10-0-128-88 kubenswrapper[2568]: E0603 16:17:58.610224 2568 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 03 16:17:58.610495 ip-10-0-128-88 kubenswrapper[2568]: E0603 16:17:58.610243 2568 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5644dfb6d8-ttv56: secret "image-registry-tls" not found Jun 03 16:17:58.610495 ip-10-0-128-88 kubenswrapper[2568]: E0603 16:17:58.610246 2568 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 03 16:17:58.610495 ip-10-0-128-88 kubenswrapper[2568]: E0603 16:17:58.610305 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0692610f-5dcf-4120-be04-37201b315bf6-metrics-tls podName:0692610f-5dcf-4120-be04-37201b315bf6 nodeName:}" failed. No retries permitted until 2026-06-03 16:18:14.610291762 +0000 UTC m=+65.698718330 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/0692610f-5dcf-4120-be04-37201b315bf6-metrics-tls") pod "dns-default-lrqwv" (UID: "0692610f-5dcf-4120-be04-37201b315bf6") : secret "dns-default-metrics-tls" not found Jun 03 16:17:58.610495 ip-10-0-128-88 kubenswrapper[2568]: E0603 16:17:58.610318 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/783221aa-e7e0-4a24-934b-f6bf8890149b-registry-tls podName:783221aa-e7e0-4a24-934b-f6bf8890149b nodeName:}" failed. No retries permitted until 2026-06-03 16:18:14.610312348 +0000 UTC m=+65.698738917 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/783221aa-e7e0-4a24-934b-f6bf8890149b-registry-tls") pod "image-registry-5644dfb6d8-ttv56" (UID: "783221aa-e7e0-4a24-934b-f6bf8890149b") : secret "image-registry-tls" not found Jun 03 16:17:58.710461 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:17:58.710432 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/469cfe56-169a-4b98-b437-38f4ea41f481-cert\") pod \"ingress-canary-9bfn5\" (UID: \"469cfe56-169a-4b98-b437-38f4ea41f481\") " pod="openshift-ingress-canary/ingress-canary-9bfn5" Jun 03 16:17:58.710648 ip-10-0-128-88 kubenswrapper[2568]: E0603 16:17:58.710598 2568 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found Jun 03 16:17:58.710709 ip-10-0-128-88 kubenswrapper[2568]: E0603 16:17:58.710681 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/469cfe56-169a-4b98-b437-38f4ea41f481-cert podName:469cfe56-169a-4b98-b437-38f4ea41f481 nodeName:}" failed. No retries permitted until 2026-06-03 16:18:14.710665056 +0000 UTC m=+65.799091629 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/469cfe56-169a-4b98-b437-38f4ea41f481-cert") pod "ingress-canary-9bfn5" (UID: "469cfe56-169a-4b98-b437-38f4ea41f481") : secret "canary-serving-cert" not found Jun 03 16:18:07.758362 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:07.758334 2568 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-xz69h" Jun 03 16:18:13.251934 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:13.251902 2568 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-5644dfb6d8-ttv56"] Jun 03 16:18:13.252399 ip-10-0-128-88 kubenswrapper[2568]: E0603 16:18:13.252081 2568 pod_workers.go:1301] "Error syncing pod, skipping" err="unmounted volumes=[registry-tls], unattached volumes=[], failed to process volumes=[]: context canceled" pod="openshift-image-registry/image-registry-5644dfb6d8-ttv56" podUID="783221aa-e7e0-4a24-934b-f6bf8890149b" Jun 03 16:18:13.366312 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:13.366282 2568 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-4264d"] Jun 03 16:18:13.370193 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:13.370171 2568 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-4264d" Jun 03 16:18:13.372431 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:13.372405 2568 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jun 03 16:18:13.372557 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:13.372488 2568 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jun 03 16:18:13.372557 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:13.372494 2568 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jun 03 16:18:13.372718 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:13.372580 2568 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jun 03 16:18:13.373042 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:13.373029 2568 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-hq6wm\"" Jun 03 16:18:13.382081 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:13.382061 2568 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-4264d"] Jun 03 16:18:13.418948 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:13.418921 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/6cb85c51-522f-4c5d-b982-8d4039519c70-crio-socket\") pod \"insights-runtime-extractor-4264d\" (UID: \"6cb85c51-522f-4c5d-b982-8d4039519c70\") " pod="openshift-insights/insights-runtime-extractor-4264d" Jun 03 16:18:13.419071 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:13.418955 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/6cb85c51-522f-4c5d-b982-8d4039519c70-data-volume\") pod \"insights-runtime-extractor-4264d\" (UID: \"6cb85c51-522f-4c5d-b982-8d4039519c70\") " pod="openshift-insights/insights-runtime-extractor-4264d" Jun 03 16:18:13.419071 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:13.419019 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6cb85c51-522f-4c5d-b982-8d4039519c70-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-4264d\" (UID: \"6cb85c51-522f-4c5d-b982-8d4039519c70\") " pod="openshift-insights/insights-runtime-extractor-4264d" Jun 03 16:18:13.419071 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:13.419036 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6lglc\" (UniqueName: \"kubernetes.io/projected/6cb85c51-522f-4c5d-b982-8d4039519c70-kube-api-access-6lglc\") pod \"insights-runtime-extractor-4264d\" (UID: \"6cb85c51-522f-4c5d-b982-8d4039519c70\") " pod="openshift-insights/insights-runtime-extractor-4264d" Jun 03 16:18:13.419172 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:13.419078 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6cb85c51-522f-4c5d-b982-8d4039519c70-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-4264d\" (UID: \"6cb85c51-522f-4c5d-b982-8d4039519c70\") " pod="openshift-insights/insights-runtime-extractor-4264d" Jun 03 16:18:13.519804 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:13.519733 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6cb85c51-522f-4c5d-b982-8d4039519c70-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-4264d\" (UID: \"6cb85c51-522f-4c5d-b982-8d4039519c70\") " pod="openshift-insights/insights-runtime-extractor-4264d" Jun 03 16:18:13.519804 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:13.519787 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/6cb85c51-522f-4c5d-b982-8d4039519c70-crio-socket\") pod \"insights-runtime-extractor-4264d\" (UID: \"6cb85c51-522f-4c5d-b982-8d4039519c70\") " pod="openshift-insights/insights-runtime-extractor-4264d" Jun 03 16:18:13.519994 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:13.519816 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/6cb85c51-522f-4c5d-b982-8d4039519c70-data-volume\") pod \"insights-runtime-extractor-4264d\" (UID: \"6cb85c51-522f-4c5d-b982-8d4039519c70\") " pod="openshift-insights/insights-runtime-extractor-4264d" Jun 03 16:18:13.519994 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:13.519851 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6cb85c51-522f-4c5d-b982-8d4039519c70-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-4264d\" (UID: \"6cb85c51-522f-4c5d-b982-8d4039519c70\") " pod="openshift-insights/insights-runtime-extractor-4264d" Jun 03 16:18:13.519994 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:13.519869 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-6lglc\" (UniqueName: \"kubernetes.io/projected/6cb85c51-522f-4c5d-b982-8d4039519c70-kube-api-access-6lglc\") pod \"insights-runtime-extractor-4264d\" (UID: \"6cb85c51-522f-4c5d-b982-8d4039519c70\") " pod="openshift-insights/insights-runtime-extractor-4264d" Jun 03 16:18:13.519994 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:13.519979 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/6cb85c51-522f-4c5d-b982-8d4039519c70-crio-socket\") pod \"insights-runtime-extractor-4264d\" (UID: \"6cb85c51-522f-4c5d-b982-8d4039519c70\") " pod="openshift-insights/insights-runtime-extractor-4264d" Jun 03 16:18:13.520174 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:13.520159 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/6cb85c51-522f-4c5d-b982-8d4039519c70-data-volume\") pod \"insights-runtime-extractor-4264d\" (UID: \"6cb85c51-522f-4c5d-b982-8d4039519c70\") " pod="openshift-insights/insights-runtime-extractor-4264d" Jun 03 16:18:13.520412 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:13.520396 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6cb85c51-522f-4c5d-b982-8d4039519c70-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-4264d\" (UID: \"6cb85c51-522f-4c5d-b982-8d4039519c70\") " pod="openshift-insights/insights-runtime-extractor-4264d" Jun 03 16:18:13.522017 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:13.521989 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6cb85c51-522f-4c5d-b982-8d4039519c70-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-4264d\" (UID: \"6cb85c51-522f-4c5d-b982-8d4039519c70\") " pod="openshift-insights/insights-runtime-extractor-4264d" Jun 03 16:18:13.532334 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:13.532306 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6lglc\" (UniqueName: \"kubernetes.io/projected/6cb85c51-522f-4c5d-b982-8d4039519c70-kube-api-access-6lglc\") pod \"insights-runtime-extractor-4264d\" (UID: \"6cb85c51-522f-4c5d-b982-8d4039519c70\") " pod="openshift-insights/insights-runtime-extractor-4264d" Jun 03 16:18:13.679552 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:13.679512 2568 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-4264d" Jun 03 16:18:13.796643 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:13.796569 2568 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-4264d"] Jun 03 16:18:13.800040 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:18:13.800008 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6cb85c51_522f_4c5d_b982_8d4039519c70.slice/crio-73da89a07d8745a34a937add04e5bda75ac80cb4839a614fa58aa3cbe57fc5ed WatchSource:0}: Error finding container 73da89a07d8745a34a937add04e5bda75ac80cb4839a614fa58aa3cbe57fc5ed: Status 404 returned error can't find the container with id 73da89a07d8745a34a937add04e5bda75ac80cb4839a614fa58aa3cbe57fc5ed Jun 03 16:18:13.830469 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:13.830436 2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-4264d" event={"ID":"6cb85c51-522f-4c5d-b982-8d4039519c70","Type":"ContainerStarted","Data":"73da89a07d8745a34a937add04e5bda75ac80cb4839a614fa58aa3cbe57fc5ed"} Jun 03 16:18:13.830606 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:13.830475 2568 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5644dfb6d8-ttv56" Jun 03 16:18:13.835124 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:13.835110 2568 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5644dfb6d8-ttv56" Jun 03 16:18:13.922095 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:13.922072 2568 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9796n\" (UniqueName: \"kubernetes.io/projected/783221aa-e7e0-4a24-934b-f6bf8890149b-kube-api-access-9796n\") pod \"783221aa-e7e0-4a24-934b-f6bf8890149b\" (UID: \"783221aa-e7e0-4a24-934b-f6bf8890149b\") " Jun 03 16:18:13.922251 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:13.922103 2568 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/783221aa-e7e0-4a24-934b-f6bf8890149b-image-registry-private-configuration\") pod \"783221aa-e7e0-4a24-934b-f6bf8890149b\" (UID: \"783221aa-e7e0-4a24-934b-f6bf8890149b\") " Jun 03 16:18:13.922251 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:13.922127 2568 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/783221aa-e7e0-4a24-934b-f6bf8890149b-trusted-ca\") pod \"783221aa-e7e0-4a24-934b-f6bf8890149b\" (UID: \"783221aa-e7e0-4a24-934b-f6bf8890149b\") " Jun 03 16:18:13.922251 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:13.922166 2568 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/783221aa-e7e0-4a24-934b-f6bf8890149b-bound-sa-token\") pod \"783221aa-e7e0-4a24-934b-f6bf8890149b\" (UID: \"783221aa-e7e0-4a24-934b-f6bf8890149b\") " Jun 03 16:18:13.922251 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:13.922207 2568 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/783221aa-e7e0-4a24-934b-f6bf8890149b-installation-pull-secrets\") pod \"783221aa-e7e0-4a24-934b-f6bf8890149b\" (UID: \"783221aa-e7e0-4a24-934b-f6bf8890149b\") " Jun 03 16:18:13.922251 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:13.922246 2568 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/783221aa-e7e0-4a24-934b-f6bf8890149b-registry-certificates\") pod \"783221aa-e7e0-4a24-934b-f6bf8890149b\" (UID: \"783221aa-e7e0-4a24-934b-f6bf8890149b\") " Jun 03 16:18:13.922505 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:13.922278 2568 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/783221aa-e7e0-4a24-934b-f6bf8890149b-ca-trust-extracted\") pod \"783221aa-e7e0-4a24-934b-f6bf8890149b\" (UID: \"783221aa-e7e0-4a24-934b-f6bf8890149b\") " Jun 03 16:18:13.922693 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:13.922664 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/783221aa-e7e0-4a24-934b-f6bf8890149b-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "783221aa-e7e0-4a24-934b-f6bf8890149b" (UID: "783221aa-e7e0-4a24-934b-f6bf8890149b"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 03 16:18:13.922769 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:13.922599 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/783221aa-e7e0-4a24-934b-f6bf8890149b-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "783221aa-e7e0-4a24-934b-f6bf8890149b" (UID: "783221aa-e7e0-4a24-934b-f6bf8890149b"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 03 16:18:13.922769 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:13.922675 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/783221aa-e7e0-4a24-934b-f6bf8890149b-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "783221aa-e7e0-4a24-934b-f6bf8890149b" (UID: "783221aa-e7e0-4a24-934b-f6bf8890149b"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 03 16:18:13.924340 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:13.924314 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/783221aa-e7e0-4a24-934b-f6bf8890149b-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "783221aa-e7e0-4a24-934b-f6bf8890149b" (UID: "783221aa-e7e0-4a24-934b-f6bf8890149b"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 03 16:18:13.924419 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:13.924320 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/783221aa-e7e0-4a24-934b-f6bf8890149b-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "783221aa-e7e0-4a24-934b-f6bf8890149b" (UID: "783221aa-e7e0-4a24-934b-f6bf8890149b"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 03 16:18:13.924675 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:13.924648 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/783221aa-e7e0-4a24-934b-f6bf8890149b-kube-api-access-9796n" (OuterVolumeSpecName: "kube-api-access-9796n") pod "783221aa-e7e0-4a24-934b-f6bf8890149b" (UID: "783221aa-e7e0-4a24-934b-f6bf8890149b"). InnerVolumeSpecName "kube-api-access-9796n". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 03 16:18:13.924675 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:13.924660 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/783221aa-e7e0-4a24-934b-f6bf8890149b-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "783221aa-e7e0-4a24-934b-f6bf8890149b" (UID: "783221aa-e7e0-4a24-934b-f6bf8890149b"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 03 16:18:14.023112 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:14.023085 2568 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/783221aa-e7e0-4a24-934b-f6bf8890149b-installation-pull-secrets\") on node \"ip-10-0-128-88.ec2.internal\" DevicePath \"\"" Jun 03 16:18:14.023112 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:14.023108 2568 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/783221aa-e7e0-4a24-934b-f6bf8890149b-registry-certificates\") on node \"ip-10-0-128-88.ec2.internal\" DevicePath \"\"" Jun 03 16:18:14.023112 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:14.023118 2568 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/783221aa-e7e0-4a24-934b-f6bf8890149b-ca-trust-extracted\") on node \"ip-10-0-128-88.ec2.internal\" DevicePath \"\"" Jun 03 16:18:14.023292 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:14.023127 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9796n\" (UniqueName: \"kubernetes.io/projected/783221aa-e7e0-4a24-934b-f6bf8890149b-kube-api-access-9796n\") on node \"ip-10-0-128-88.ec2.internal\" DevicePath \"\"" Jun 03 16:18:14.023292 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:14.023136 2568 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/783221aa-e7e0-4a24-934b-f6bf8890149b-image-registry-private-configuration\") on node \"ip-10-0-128-88.ec2.internal\" DevicePath \"\"" Jun 03 16:18:14.023292 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:14.023146 2568 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/783221aa-e7e0-4a24-934b-f6bf8890149b-trusted-ca\") on node \"ip-10-0-128-88.ec2.internal\" DevicePath \"\"" Jun 03 16:18:14.023292 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:14.023155 2568 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/783221aa-e7e0-4a24-934b-f6bf8890149b-bound-sa-token\") on node \"ip-10-0-128-88.ec2.internal\" DevicePath \"\"" Jun 03 16:18:14.224937 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:14.224899 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d93510f8-5732-4611-8aca-02573c7e296e-metrics-certs\") pod \"network-metrics-daemon-fhpnv\" (UID: \"d93510f8-5732-4611-8aca-02573c7e296e\") " pod="openshift-multus/network-metrics-daemon-fhpnv" Jun 03 16:18:14.227173 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:14.227155 2568 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jun 03 16:18:14.237863 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:14.237839 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d93510f8-5732-4611-8aca-02573c7e296e-metrics-certs\") pod \"network-metrics-daemon-fhpnv\" (UID: \"d93510f8-5732-4611-8aca-02573c7e296e\") " pod="openshift-multus/network-metrics-daemon-fhpnv" Jun 03 16:18:14.325708 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:14.325672 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-tdcd7\" (UniqueName: \"kubernetes.io/projected/b8482073-6356-4d6e-bddf-2c8152f6a4f6-kube-api-access-tdcd7\") pod \"network-check-target-m7zkz\" (UID: \"b8482073-6356-4d6e-bddf-2c8152f6a4f6\") " pod="openshift-network-diagnostics/network-check-target-m7zkz" Jun 03 16:18:14.327764 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:14.327748 2568 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jun 03 16:18:14.338215 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:14.338197 2568 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jun 03 16:18:14.349022 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:14.348997 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tdcd7\" (UniqueName: \"kubernetes.io/projected/b8482073-6356-4d6e-bddf-2c8152f6a4f6-kube-api-access-tdcd7\") pod \"network-check-target-m7zkz\" (UID: \"b8482073-6356-4d6e-bddf-2c8152f6a4f6\") " pod="openshift-network-diagnostics/network-check-target-m7zkz" Jun 03 16:18:14.474775 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:14.474745 2568 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-cchx6\"" Jun 03 16:18:14.483503 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:14.483429 2568 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-fhpnv" Jun 03 16:18:14.486907 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:14.486882 2568 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-f9ccm\"" Jun 03 16:18:14.495524 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:14.495493 2568 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-m7zkz" Jun 03 16:18:14.628141 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:14.628111 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0692610f-5dcf-4120-be04-37201b315bf6-metrics-tls\") pod \"dns-default-lrqwv\" (UID: \"0692610f-5dcf-4120-be04-37201b315bf6\") " pod="openshift-dns/dns-default-lrqwv" Jun 03 16:18:14.628267 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:14.628156 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/783221aa-e7e0-4a24-934b-f6bf8890149b-registry-tls\") pod \"image-registry-5644dfb6d8-ttv56\" (UID: \"783221aa-e7e0-4a24-934b-f6bf8890149b\") " pod="openshift-image-registry/image-registry-5644dfb6d8-ttv56" Jun 03 16:18:14.630620 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:14.630593 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0692610f-5dcf-4120-be04-37201b315bf6-metrics-tls\") pod \"dns-default-lrqwv\" (UID: \"0692610f-5dcf-4120-be04-37201b315bf6\") " pod="openshift-dns/dns-default-lrqwv" Jun 03 16:18:14.630807 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:14.630784 2568 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-fhpnv"] Jun 03 16:18:14.631505 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:14.631485 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/783221aa-e7e0-4a24-934b-f6bf8890149b-registry-tls\") pod \"image-registry-5644dfb6d8-ttv56\" (UID: \"783221aa-e7e0-4a24-934b-f6bf8890149b\") " pod="openshift-image-registry/image-registry-5644dfb6d8-ttv56" Jun 03 16:18:14.633578 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:18:14.633556 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd93510f8_5732_4611_8aca_02573c7e296e.slice/crio-ee474e3fa21adbb70d2a60f6ddb81ad4bdab19c1d16c8ae4f51abe86fad4535d WatchSource:0}: Error finding container ee474e3fa21adbb70d2a60f6ddb81ad4bdab19c1d16c8ae4f51abe86fad4535d: Status 404 returned error can't find the container with id ee474e3fa21adbb70d2a60f6ddb81ad4bdab19c1d16c8ae4f51abe86fad4535d Jun 03 16:18:14.646730 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:14.646711 2568 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-m7zkz"] Jun 03 16:18:14.649489 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:18:14.649465 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb8482073_6356_4d6e_bddf_2c8152f6a4f6.slice/crio-deef39d50aaac02f284bab30856f0fa2ba41edf9cbbf4bf8b8547ce57e7fe0ab WatchSource:0}: Error finding container deef39d50aaac02f284bab30856f0fa2ba41edf9cbbf4bf8b8547ce57e7fe0ab: Status 404 returned error can't find the container with id deef39d50aaac02f284bab30856f0fa2ba41edf9cbbf4bf8b8547ce57e7fe0ab Jun 03 16:18:14.729423 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:14.729375 2568 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/783221aa-e7e0-4a24-934b-f6bf8890149b-registry-tls\") pod \"783221aa-e7e0-4a24-934b-f6bf8890149b\" (UID: \"783221aa-e7e0-4a24-934b-f6bf8890149b\") " Jun 03 16:18:14.729588 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:14.729471 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/469cfe56-169a-4b98-b437-38f4ea41f481-cert\") pod \"ingress-canary-9bfn5\" (UID: \"469cfe56-169a-4b98-b437-38f4ea41f481\") " pod="openshift-ingress-canary/ingress-canary-9bfn5" Jun 03 16:18:14.731347 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:14.731319 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/783221aa-e7e0-4a24-934b-f6bf8890149b-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "783221aa-e7e0-4a24-934b-f6bf8890149b" (UID: "783221aa-e7e0-4a24-934b-f6bf8890149b"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 03 16:18:14.731788 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:14.731772 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/469cfe56-169a-4b98-b437-38f4ea41f481-cert\") pod \"ingress-canary-9bfn5\" (UID: \"469cfe56-169a-4b98-b437-38f4ea41f481\") " pod="openshift-ingress-canary/ingress-canary-9bfn5" Jun 03 16:18:14.830155 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:14.830079 2568 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/783221aa-e7e0-4a24-934b-f6bf8890149b-registry-tls\") on node \"ip-10-0-128-88.ec2.internal\" DevicePath \"\"" Jun 03 16:18:14.833290 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:14.833262 2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-m7zkz" event={"ID":"b8482073-6356-4d6e-bddf-2c8152f6a4f6","Type":"ContainerStarted","Data":"deef39d50aaac02f284bab30856f0fa2ba41edf9cbbf4bf8b8547ce57e7fe0ab"} Jun 03 16:18:14.834154 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:14.834129 2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-fhpnv" event={"ID":"d93510f8-5732-4611-8aca-02573c7e296e","Type":"ContainerStarted","Data":"ee474e3fa21adbb70d2a60f6ddb81ad4bdab19c1d16c8ae4f51abe86fad4535d"} Jun 03 16:18:14.835571 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:14.835554 2568 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5644dfb6d8-ttv56" Jun 03 16:18:14.835571 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:14.835560 2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-4264d" event={"ID":"6cb85c51-522f-4c5d-b982-8d4039519c70","Type":"ContainerStarted","Data":"efe0b708c628b6e414aa7ba40e294a8b12cff63dfc23d2f8bdf69416d798e8f9"} Jun 03 16:18:14.835704 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:14.835586 2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-4264d" event={"ID":"6cb85c51-522f-4c5d-b982-8d4039519c70","Type":"ContainerStarted","Data":"5dfad649d8d10fea05cf53765c97e6b33d1f46cccbfbbe5333b64a9ff729c047"} Jun 03 16:18:14.874832 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:14.874808 2568 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-5644dfb6d8-ttv56"] Jun 03 16:18:14.880764 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:14.880743 2568 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-5644dfb6d8-ttv56"] Jun 03 16:18:14.914766 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:14.912331 2568 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-h928k\"" Jun 03 16:18:14.920432 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:14.920415 2568 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-lrqwv" Jun 03 16:18:14.924854 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:14.924838 2568 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-nmtcn\"" Jun 03 16:18:14.933795 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:14.933773 2568 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-9bfn5" Jun 03 16:18:15.048940 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:15.048912 2568 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-lrqwv"] Jun 03 16:18:15.053870 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:18:15.053845 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0692610f_5dcf_4120_be04_37201b315bf6.slice/crio-b6a7703aaea12e38fce8d54f5c9d1f7d13924816edb41c121d324aef0851357e WatchSource:0}: Error finding container b6a7703aaea12e38fce8d54f5c9d1f7d13924816edb41c121d324aef0851357e: Status 404 returned error can't find the container with id b6a7703aaea12e38fce8d54f5c9d1f7d13924816edb41c121d324aef0851357e Jun 03 16:18:15.067128 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:15.067103 2568 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-9bfn5"] Jun 03 16:18:15.070513 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:18:15.070490 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod469cfe56_169a_4b98_b437_38f4ea41f481.slice/crio-a8f1b5225980e0f560a3ddbca0f7e6103a286dfdfbd3aaf35a34cd61391c9789 WatchSource:0}: Error finding container a8f1b5225980e0f560a3ddbca0f7e6103a286dfdfbd3aaf35a34cd61391c9789: Status 404 returned error can't find the container with id a8f1b5225980e0f560a3ddbca0f7e6103a286dfdfbd3aaf35a34cd61391c9789 Jun 03 16:18:15.102641 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:15.102586 2568 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-59b7f8756b-jpmb7"] Jun 03 16:18:15.104645 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:15.104622 2568 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-59b7f8756b-jpmb7" Jun 03 16:18:15.106789 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:15.106768 2568 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\"" Jun 03 16:18:15.106900 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:15.106827 2568 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-wfznd\"" Jun 03 16:18:15.115203 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:15.115153 2568 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-59b7f8756b-jpmb7"] Jun 03 16:18:15.133004 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:15.132974 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/7b9b2d8d-9463-4a44-9477-faa61cc8c0d1-tls-certificates\") pod \"prometheus-operator-admission-webhook-59b7f8756b-jpmb7\" (UID: \"7b9b2d8d-9463-4a44-9477-faa61cc8c0d1\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-59b7f8756b-jpmb7" Jun 03 16:18:15.233548 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:15.233499 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/7b9b2d8d-9463-4a44-9477-faa61cc8c0d1-tls-certificates\") pod \"prometheus-operator-admission-webhook-59b7f8756b-jpmb7\" (UID: \"7b9b2d8d-9463-4a44-9477-faa61cc8c0d1\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-59b7f8756b-jpmb7" Jun 03 16:18:15.233718 ip-10-0-128-88 kubenswrapper[2568]: E0603 16:18:15.233678 2568 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-admission-webhook-tls: secret "prometheus-operator-admission-webhook-tls" not found Jun 03 16:18:15.233781 ip-10-0-128-88 kubenswrapper[2568]: E0603 16:18:15.233749 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7b9b2d8d-9463-4a44-9477-faa61cc8c0d1-tls-certificates podName:7b9b2d8d-9463-4a44-9477-faa61cc8c0d1 nodeName:}" failed. No retries permitted until 2026-06-03 16:18:15.733728249 +0000 UTC m=+66.822154821 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certificates" (UniqueName: "kubernetes.io/secret/7b9b2d8d-9463-4a44-9477-faa61cc8c0d1-tls-certificates") pod "prometheus-operator-admission-webhook-59b7f8756b-jpmb7" (UID: "7b9b2d8d-9463-4a44-9477-faa61cc8c0d1") : secret "prometheus-operator-admission-webhook-tls" not found Jun 03 16:18:15.565238 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:15.565206 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="783221aa-e7e0-4a24-934b-f6bf8890149b" path="/var/lib/kubelet/pods/783221aa-e7e0-4a24-934b-f6bf8890149b/volumes" Jun 03 16:18:15.740513 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:15.740475 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/7b9b2d8d-9463-4a44-9477-faa61cc8c0d1-tls-certificates\") pod \"prometheus-operator-admission-webhook-59b7f8756b-jpmb7\" (UID: \"7b9b2d8d-9463-4a44-9477-faa61cc8c0d1\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-59b7f8756b-jpmb7" Jun 03 16:18:15.749508 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:15.749443 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/7b9b2d8d-9463-4a44-9477-faa61cc8c0d1-tls-certificates\") pod \"prometheus-operator-admission-webhook-59b7f8756b-jpmb7\" (UID: \"7b9b2d8d-9463-4a44-9477-faa61cc8c0d1\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-59b7f8756b-jpmb7" Jun 03 16:18:15.839604 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:15.839491 2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-9bfn5" event={"ID":"469cfe56-169a-4b98-b437-38f4ea41f481","Type":"ContainerStarted","Data":"a8f1b5225980e0f560a3ddbca0f7e6103a286dfdfbd3aaf35a34cd61391c9789"} Jun 03 16:18:15.841695 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:15.841648 2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-lrqwv" event={"ID":"0692610f-5dcf-4120-be04-37201b315bf6","Type":"ContainerStarted","Data":"b6a7703aaea12e38fce8d54f5c9d1f7d13924816edb41c121d324aef0851357e"} Jun 03 16:18:16.018738 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:16.018302 2568 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-59b7f8756b-jpmb7" Jun 03 16:18:18.677561 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:18.676309 2568 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-59b7f8756b-jpmb7"] Jun 03 16:18:18.686906 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:18:18.686874 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7b9b2d8d_9463_4a44_9477_faa61cc8c0d1.slice/crio-81a99180a87d8b58ff73a1d875ff6e8f434661e554673bdbe8dca47e2c2221b0 WatchSource:0}: Error finding container 81a99180a87d8b58ff73a1d875ff6e8f434661e554673bdbe8dca47e2c2221b0: Status 404 returned error can't find the container with id 81a99180a87d8b58ff73a1d875ff6e8f434661e554673bdbe8dca47e2c2221b0 Jun 03 16:18:18.853313 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:18.853185 2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-59b7f8756b-jpmb7" event={"ID":"7b9b2d8d-9463-4a44-9477-faa61cc8c0d1","Type":"ContainerStarted","Data":"81a99180a87d8b58ff73a1d875ff6e8f434661e554673bdbe8dca47e2c2221b0"} Jun 03 16:18:18.854847 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:18.854818 2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-lrqwv" event={"ID":"0692610f-5dcf-4120-be04-37201b315bf6","Type":"ContainerStarted","Data":"6e71b79e9a5aa98ecec59903cb9763b00fb6e7df7303e0d1e1d83c35fd04251d"} Jun 03 16:18:18.854847 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:18.854850 2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-lrqwv" event={"ID":"0692610f-5dcf-4120-be04-37201b315bf6","Type":"ContainerStarted","Data":"364b9065c182b1d2da691639b902d2c8ed129588712f0b8323b2391e5cd42e73"} Jun 03 16:18:18.855063 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:18.855042 2568 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-lrqwv" Jun 03 16:18:18.856845 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:18.856811 2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-m7zkz" event={"ID":"b8482073-6356-4d6e-bddf-2c8152f6a4f6","Type":"ContainerStarted","Data":"c48224c008e3787a645f7cab0390be3e8c3622a7c52c11dcac66397ed8ab2803"} Jun 03 16:18:18.857066 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:18.857034 2568 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-m7zkz" Jun 03 16:18:18.858744 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:18.858715 2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-fhpnv" event={"ID":"d93510f8-5732-4611-8aca-02573c7e296e","Type":"ContainerStarted","Data":"d2feeb1cbcc48835d912c50df7b4dabe6f87e9e0968ad9f1af427813948ab778"} Jun 03 16:18:18.858855 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:18.858745 2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-fhpnv" event={"ID":"d93510f8-5732-4611-8aca-02573c7e296e","Type":"ContainerStarted","Data":"20278aef8ed64d61594e9db3d4d2d059240ca7ecf7b9c51ee51a7a944161b9cb"} Jun 03 16:18:18.860427 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:18.860409 2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-4264d" event={"ID":"6cb85c51-522f-4c5d-b982-8d4039519c70","Type":"ContainerStarted","Data":"191f88e5734483d5f5016b9d3429343434ff62c7c3579392165961ab6dcab483"} Jun 03 16:18:18.861739 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:18.861722 2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-9bfn5" event={"ID":"469cfe56-169a-4b98-b437-38f4ea41f481","Type":"ContainerStarted","Data":"79f04d5702eb2e1361d69cc15121d2a4290b0db7e028eab1e1a9571367552532"} Jun 03 16:18:18.873363 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:18.873314 2568 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-dns/dns-default-lrqwv" podStartSLOduration=33.399116473 podStartE2EDuration="36.873301086s" podCreationTimestamp="2026-06-03 16:17:42 +0000 UTC" firstStartedPulling="2026-06-03 16:18:15.056648478 +0000 UTC m=+66.145075045" lastFinishedPulling="2026-06-03 16:18:18.530833086 +0000 UTC m=+69.619259658" observedRunningTime="2026-06-03 16:18:18.872906553 +0000 UTC m=+69.961333143" watchObservedRunningTime="2026-06-03 16:18:18.873301086 +0000 UTC m=+69.961727673" Jun 03 16:18:18.889842 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:18.889798 2568 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-9bfn5" podStartSLOduration=33.42678204 podStartE2EDuration="36.889784263s" podCreationTimestamp="2026-06-03 16:17:42 +0000 UTC" firstStartedPulling="2026-06-03 16:18:15.072343897 +0000 UTC m=+66.160770478" lastFinishedPulling="2026-06-03 16:18:18.535346129 +0000 UTC m=+69.623772701" observedRunningTime="2026-06-03 16:18:18.889612012 +0000 UTC m=+69.978038599" watchObservedRunningTime="2026-06-03 16:18:18.889784263 +0000 UTC m=+69.978210855" Jun 03 16:18:18.918954 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:18.918910 2568 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-4264d" podStartSLOduration=1.244806182 podStartE2EDuration="5.918896685s" podCreationTimestamp="2026-06-03 16:18:13 +0000 UTC" firstStartedPulling="2026-06-03 16:18:13.854272642 +0000 UTC m=+64.942699223" lastFinishedPulling="2026-06-03 16:18:18.528363147 +0000 UTC m=+69.616789726" observedRunningTime="2026-06-03 16:18:18.918273338 +0000 UTC m=+70.006699929" watchObservedRunningTime="2026-06-03 16:18:18.918896685 +0000 UTC m=+70.007323272" Jun 03 16:18:18.935474 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:18.935433 2568 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-m7zkz" podStartSLOduration=65.927597358 podStartE2EDuration="1m9.93542168s" podCreationTimestamp="2026-06-03 16:17:09 +0000 UTC" firstStartedPulling="2026-06-03 16:18:14.651271152 +0000 UTC m=+65.739697723" lastFinishedPulling="2026-06-03 16:18:18.659095464 +0000 UTC m=+69.747522045" observedRunningTime="2026-06-03 16:18:18.934190916 +0000 UTC m=+70.022617506" watchObservedRunningTime="2026-06-03 16:18:18.93542168 +0000 UTC m=+70.023848270" Jun 03 16:18:18.950500 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:18.950458 2568 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-fhpnv" podStartSLOduration=66.055920895 podStartE2EDuration="1m9.950444311s" podCreationTimestamp="2026-06-03 16:17:09 +0000 UTC" firstStartedPulling="2026-06-03 16:18:14.635561973 +0000 UTC m=+65.723988559" lastFinishedPulling="2026-06-03 16:18:18.530085399 +0000 UTC m=+69.618511975" observedRunningTime="2026-06-03 16:18:18.949795903 +0000 UTC m=+70.038222493" watchObservedRunningTime="2026-06-03 16:18:18.950444311 +0000 UTC m=+70.038870901" Jun 03 16:18:20.867940 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:20.867903 2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-59b7f8756b-jpmb7" event={"ID":"7b9b2d8d-9463-4a44-9477-faa61cc8c0d1","Type":"ContainerStarted","Data":"7a9bbf0b3c0584bd9c37b15fa016dd2730bfe29da2ad8157bfaada6e569dd251"} Jun 03 16:18:20.868409 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:20.868153 2568 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-59b7f8756b-jpmb7" Jun 03 16:18:20.872891 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:20.872866 2568 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-59b7f8756b-jpmb7" Jun 03 16:18:20.882287 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:20.882250 2568 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-59b7f8756b-jpmb7" podStartSLOduration=4.472649861 podStartE2EDuration="5.882238767s" podCreationTimestamp="2026-06-03 16:18:15 +0000 UTC" firstStartedPulling="2026-06-03 16:18:18.692647487 +0000 UTC m=+69.781074059" lastFinishedPulling="2026-06-03 16:18:20.102236384 +0000 UTC m=+71.190662965" observedRunningTime="2026-06-03 16:18:20.881693365 +0000 UTC m=+71.970119956" watchObservedRunningTime="2026-06-03 16:18:20.882238767 +0000 UTC m=+71.970665357" Jun 03 16:18:21.154448 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:21.154367 2568 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-658bcd8bdf-cfg8g"] Jun 03 16:18:21.179147 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:21.179116 2568 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-658bcd8bdf-cfg8g"] Jun 03 16:18:21.179289 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:21.179245 2568 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-658bcd8bdf-cfg8g" Jun 03 16:18:21.181551 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:21.181515 2568 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\"" Jun 03 16:18:21.181693 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:21.181516 2568 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\"" Jun 03 16:18:21.181693 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:21.181521 2568 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jun 03 16:18:21.181693 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:21.181521 2568 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jun 03 16:18:21.182150 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:21.182123 2568 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jun 03 16:18:21.182258 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:21.182176 2568 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-9j9w6\"" Jun 03 16:18:21.277930 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:21.277908 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/eed996ff-81a2-4df1-b751-be50acf655e0-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-658bcd8bdf-cfg8g\" (UID: \"eed996ff-81a2-4df1-b751-be50acf655e0\") " pod="openshift-monitoring/prometheus-operator-658bcd8bdf-cfg8g" Jun 03 16:18:21.278055 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:21.277950 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/eed996ff-81a2-4df1-b751-be50acf655e0-prometheus-operator-tls\") pod \"prometheus-operator-658bcd8bdf-cfg8g\" (UID: \"eed996ff-81a2-4df1-b751-be50acf655e0\") " pod="openshift-monitoring/prometheus-operator-658bcd8bdf-cfg8g" Jun 03 16:18:21.278055 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:21.277991 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/eed996ff-81a2-4df1-b751-be50acf655e0-metrics-client-ca\") pod \"prometheus-operator-658bcd8bdf-cfg8g\" (UID: \"eed996ff-81a2-4df1-b751-be50acf655e0\") " pod="openshift-monitoring/prometheus-operator-658bcd8bdf-cfg8g" Jun 03 16:18:21.278055 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:21.278018 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jv76c\" (UniqueName: \"kubernetes.io/projected/eed996ff-81a2-4df1-b751-be50acf655e0-kube-api-access-jv76c\") pod \"prometheus-operator-658bcd8bdf-cfg8g\" (UID: \"eed996ff-81a2-4df1-b751-be50acf655e0\") " pod="openshift-monitoring/prometheus-operator-658bcd8bdf-cfg8g" Jun 03 16:18:21.378958 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:21.378920 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/eed996ff-81a2-4df1-b751-be50acf655e0-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-658bcd8bdf-cfg8g\" (UID: \"eed996ff-81a2-4df1-b751-be50acf655e0\") " pod="openshift-monitoring/prometheus-operator-658bcd8bdf-cfg8g" Jun 03 16:18:21.378958 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:21.378966 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/eed996ff-81a2-4df1-b751-be50acf655e0-prometheus-operator-tls\") pod \"prometheus-operator-658bcd8bdf-cfg8g\" (UID: \"eed996ff-81a2-4df1-b751-be50acf655e0\") " pod="openshift-monitoring/prometheus-operator-658bcd8bdf-cfg8g" Jun 03 16:18:21.379190 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:21.379000 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/eed996ff-81a2-4df1-b751-be50acf655e0-metrics-client-ca\") pod \"prometheus-operator-658bcd8bdf-cfg8g\" (UID: \"eed996ff-81a2-4df1-b751-be50acf655e0\") " pod="openshift-monitoring/prometheus-operator-658bcd8bdf-cfg8g" Jun 03 16:18:21.379190 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:21.379020 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-jv76c\" (UniqueName: \"kubernetes.io/projected/eed996ff-81a2-4df1-b751-be50acf655e0-kube-api-access-jv76c\") pod \"prometheus-operator-658bcd8bdf-cfg8g\" (UID: \"eed996ff-81a2-4df1-b751-be50acf655e0\") " pod="openshift-monitoring/prometheus-operator-658bcd8bdf-cfg8g" Jun 03 16:18:21.379283 ip-10-0-128-88 kubenswrapper[2568]: E0603 16:18:21.379191 2568 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-tls: secret "prometheus-operator-tls" not found Jun 03 16:18:21.379283 ip-10-0-128-88 kubenswrapper[2568]: E0603 16:18:21.379262 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/eed996ff-81a2-4df1-b751-be50acf655e0-prometheus-operator-tls podName:eed996ff-81a2-4df1-b751-be50acf655e0 nodeName:}" failed. No retries permitted until 2026-06-03 16:18:21.879241808 +0000 UTC m=+72.967668378 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "prometheus-operator-tls" (UniqueName: "kubernetes.io/secret/eed996ff-81a2-4df1-b751-be50acf655e0-prometheus-operator-tls") pod "prometheus-operator-658bcd8bdf-cfg8g" (UID: "eed996ff-81a2-4df1-b751-be50acf655e0") : secret "prometheus-operator-tls" not found Jun 03 16:18:21.379828 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:21.379808 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/eed996ff-81a2-4df1-b751-be50acf655e0-metrics-client-ca\") pod \"prometheus-operator-658bcd8bdf-cfg8g\" (UID: \"eed996ff-81a2-4df1-b751-be50acf655e0\") " pod="openshift-monitoring/prometheus-operator-658bcd8bdf-cfg8g" Jun 03 16:18:21.381421 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:21.381395 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/eed996ff-81a2-4df1-b751-be50acf655e0-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-658bcd8bdf-cfg8g\" (UID: \"eed996ff-81a2-4df1-b751-be50acf655e0\") " pod="openshift-monitoring/prometheus-operator-658bcd8bdf-cfg8g" Jun 03 16:18:21.387438 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:21.387419 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jv76c\" (UniqueName: \"kubernetes.io/projected/eed996ff-81a2-4df1-b751-be50acf655e0-kube-api-access-jv76c\") pod \"prometheus-operator-658bcd8bdf-cfg8g\" (UID: \"eed996ff-81a2-4df1-b751-be50acf655e0\") " pod="openshift-monitoring/prometheus-operator-658bcd8bdf-cfg8g" Jun 03 16:18:21.571651 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:21.571571 2568 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-console/console-78bcf7ccbc-vs8xn"] Jun 03 16:18:21.596634 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:21.596611 2568 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-78bcf7ccbc-vs8xn"] Jun 03 16:18:21.596757 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:21.596712 2568 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-78bcf7ccbc-vs8xn" Jun 03 16:18:21.599252 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:21.599229 2568 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jun 03 16:18:21.599359 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:21.599232 2568 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jun 03 16:18:21.599520 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:21.599502 2568 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jun 03 16:18:21.599650 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:21.599624 2568 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jun 03 16:18:21.599713 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:21.599661 2568 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jun 03 16:18:21.599772 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:21.599735 2568 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-dsghr\"" Jun 03 16:18:21.599815 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:21.599783 2568 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jun 03 16:18:21.599970 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:21.599956 2568 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jun 03 16:18:21.680559 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:21.680523 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/99d39038-e9d1-45f4-916b-e8f846c1b36f-console-serving-cert\") pod \"console-78bcf7ccbc-vs8xn\" (UID: \"99d39038-e9d1-45f4-916b-e8f846c1b36f\") " pod="openshift-console/console-78bcf7ccbc-vs8xn" Jun 03 16:18:21.680671 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:21.680564 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/99d39038-e9d1-45f4-916b-e8f846c1b36f-console-oauth-config\") pod \"console-78bcf7ccbc-vs8xn\" (UID: \"99d39038-e9d1-45f4-916b-e8f846c1b36f\") " pod="openshift-console/console-78bcf7ccbc-vs8xn" Jun 03 16:18:21.680671 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:21.680588 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/99d39038-e9d1-45f4-916b-e8f846c1b36f-console-config\") pod \"console-78bcf7ccbc-vs8xn\" (UID: \"99d39038-e9d1-45f4-916b-e8f846c1b36f\") " pod="openshift-console/console-78bcf7ccbc-vs8xn" Jun 03 16:18:21.680671 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:21.680657 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vrh95\" (UniqueName: \"kubernetes.io/projected/99d39038-e9d1-45f4-916b-e8f846c1b36f-kube-api-access-vrh95\") pod \"console-78bcf7ccbc-vs8xn\" (UID: \"99d39038-e9d1-45f4-916b-e8f846c1b36f\") " pod="openshift-console/console-78bcf7ccbc-vs8xn" Jun 03 16:18:21.680782 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:21.680703 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/99d39038-e9d1-45f4-916b-e8f846c1b36f-service-ca\") pod \"console-78bcf7ccbc-vs8xn\" (UID: \"99d39038-e9d1-45f4-916b-e8f846c1b36f\") " pod="openshift-console/console-78bcf7ccbc-vs8xn" Jun 03 16:18:21.680782 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:21.680731 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/99d39038-e9d1-45f4-916b-e8f846c1b36f-oauth-serving-cert\") pod \"console-78bcf7ccbc-vs8xn\" (UID: \"99d39038-e9d1-45f4-916b-e8f846c1b36f\") " pod="openshift-console/console-78bcf7ccbc-vs8xn" Jun 03 16:18:21.781839 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:21.781809 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/99d39038-e9d1-45f4-916b-e8f846c1b36f-service-ca\") pod \"console-78bcf7ccbc-vs8xn\" (UID: \"99d39038-e9d1-45f4-916b-e8f846c1b36f\") " pod="openshift-console/console-78bcf7ccbc-vs8xn" Jun 03 16:18:21.781968 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:21.781856 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/99d39038-e9d1-45f4-916b-e8f846c1b36f-oauth-serving-cert\") pod \"console-78bcf7ccbc-vs8xn\" (UID: \"99d39038-e9d1-45f4-916b-e8f846c1b36f\") " pod="openshift-console/console-78bcf7ccbc-vs8xn" Jun 03 16:18:21.781968 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:21.781907 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/99d39038-e9d1-45f4-916b-e8f846c1b36f-console-serving-cert\") pod \"console-78bcf7ccbc-vs8xn\" (UID: \"99d39038-e9d1-45f4-916b-e8f846c1b36f\") " pod="openshift-console/console-78bcf7ccbc-vs8xn" Jun 03 16:18:21.781968 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:21.781929 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/99d39038-e9d1-45f4-916b-e8f846c1b36f-console-oauth-config\") pod \"console-78bcf7ccbc-vs8xn\" (UID: \"99d39038-e9d1-45f4-916b-e8f846c1b36f\") " pod="openshift-console/console-78bcf7ccbc-vs8xn" Jun 03 16:18:21.781968 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:21.781952 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/99d39038-e9d1-45f4-916b-e8f846c1b36f-console-config\") pod \"console-78bcf7ccbc-vs8xn\" (UID: \"99d39038-e9d1-45f4-916b-e8f846c1b36f\") " pod="openshift-console/console-78bcf7ccbc-vs8xn" Jun 03 16:18:21.782159 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:21.781982 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-vrh95\" (UniqueName: \"kubernetes.io/projected/99d39038-e9d1-45f4-916b-e8f846c1b36f-kube-api-access-vrh95\") pod \"console-78bcf7ccbc-vs8xn\" (UID: \"99d39038-e9d1-45f4-916b-e8f846c1b36f\") " pod="openshift-console/console-78bcf7ccbc-vs8xn" Jun 03 16:18:21.782616 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:21.782588 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/99d39038-e9d1-45f4-916b-e8f846c1b36f-service-ca\") pod \"console-78bcf7ccbc-vs8xn\" (UID: \"99d39038-e9d1-45f4-916b-e8f846c1b36f\") " pod="openshift-console/console-78bcf7ccbc-vs8xn" Jun 03 16:18:21.782713 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:21.782621 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/99d39038-e9d1-45f4-916b-e8f846c1b36f-oauth-serving-cert\") pod \"console-78bcf7ccbc-vs8xn\" (UID: \"99d39038-e9d1-45f4-916b-e8f846c1b36f\") " pod="openshift-console/console-78bcf7ccbc-vs8xn" Jun 03 16:18:21.782713 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:21.782677 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/99d39038-e9d1-45f4-916b-e8f846c1b36f-console-config\") pod \"console-78bcf7ccbc-vs8xn\" (UID: \"99d39038-e9d1-45f4-916b-e8f846c1b36f\") " pod="openshift-console/console-78bcf7ccbc-vs8xn" Jun 03 16:18:21.784354 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:21.784337 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/99d39038-e9d1-45f4-916b-e8f846c1b36f-console-oauth-config\") pod \"console-78bcf7ccbc-vs8xn\" (UID: \"99d39038-e9d1-45f4-916b-e8f846c1b36f\") " pod="openshift-console/console-78bcf7ccbc-vs8xn" Jun 03 16:18:21.784439 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:21.784344 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/99d39038-e9d1-45f4-916b-e8f846c1b36f-console-serving-cert\") pod \"console-78bcf7ccbc-vs8xn\" (UID: \"99d39038-e9d1-45f4-916b-e8f846c1b36f\") " pod="openshift-console/console-78bcf7ccbc-vs8xn" Jun 03 16:18:21.789864 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:21.789843 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vrh95\" (UniqueName: \"kubernetes.io/projected/99d39038-e9d1-45f4-916b-e8f846c1b36f-kube-api-access-vrh95\") pod \"console-78bcf7ccbc-vs8xn\" (UID: \"99d39038-e9d1-45f4-916b-e8f846c1b36f\") " pod="openshift-console/console-78bcf7ccbc-vs8xn" Jun 03 16:18:21.883091 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:21.883069 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/eed996ff-81a2-4df1-b751-be50acf655e0-prometheus-operator-tls\") pod \"prometheus-operator-658bcd8bdf-cfg8g\" (UID: \"eed996ff-81a2-4df1-b751-be50acf655e0\") " pod="openshift-monitoring/prometheus-operator-658bcd8bdf-cfg8g" Jun 03 16:18:21.885078 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:21.885061 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/eed996ff-81a2-4df1-b751-be50acf655e0-prometheus-operator-tls\") pod \"prometheus-operator-658bcd8bdf-cfg8g\" (UID: \"eed996ff-81a2-4df1-b751-be50acf655e0\") " pod="openshift-monitoring/prometheus-operator-658bcd8bdf-cfg8g" Jun 03 16:18:21.905472 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:21.905454 2568 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-78bcf7ccbc-vs8xn" Jun 03 16:18:22.026498 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:22.026466 2568 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-78bcf7ccbc-vs8xn"] Jun 03 16:18:22.030571 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:18:22.030526 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod99d39038_e9d1_45f4_916b_e8f846c1b36f.slice/crio-9b63b9bdfca298cf2196568bdb67b9ef3b453600a7ddedc7759dce179e9a3370 WatchSource:0}: Error finding container 9b63b9bdfca298cf2196568bdb67b9ef3b453600a7ddedc7759dce179e9a3370: Status 404 returned error can't find the container with id 9b63b9bdfca298cf2196568bdb67b9ef3b453600a7ddedc7759dce179e9a3370 Jun 03 16:18:22.088005 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:22.087975 2568 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-658bcd8bdf-cfg8g" Jun 03 16:18:22.203073 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:22.203037 2568 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-658bcd8bdf-cfg8g"] Jun 03 16:18:22.206224 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:18:22.206195 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeed996ff_81a2_4df1_b751_be50acf655e0.slice/crio-da7f037a85f94a5007bc531a76c318803c5a540583885074832bfab12e4c6754 WatchSource:0}: Error finding container da7f037a85f94a5007bc531a76c318803c5a540583885074832bfab12e4c6754: Status 404 returned error can't find the container with id da7f037a85f94a5007bc531a76c318803c5a540583885074832bfab12e4c6754 Jun 03 16:18:22.875245 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:22.875201 2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-658bcd8bdf-cfg8g" event={"ID":"eed996ff-81a2-4df1-b751-be50acf655e0","Type":"ContainerStarted","Data":"da7f037a85f94a5007bc531a76c318803c5a540583885074832bfab12e4c6754"} Jun 03 16:18:22.876859 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:22.876810 2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-78bcf7ccbc-vs8xn" event={"ID":"99d39038-e9d1-45f4-916b-e8f846c1b36f","Type":"ContainerStarted","Data":"9b63b9bdfca298cf2196568bdb67b9ef3b453600a7ddedc7759dce179e9a3370"} Jun 03 16:18:23.881078 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:23.881049 2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-658bcd8bdf-cfg8g" event={"ID":"eed996ff-81a2-4df1-b751-be50acf655e0","Type":"ContainerStarted","Data":"14c8d9aac988ee099ce2e21360c678096dd33256b50358223e1db8b831f5b654"} Jun 03 16:18:24.389405 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:24.389368 2568 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-console/console-767bd5547d-b88vd"] Jun 03 16:18:24.391446 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:24.391424 2568 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-767bd5547d-b88vd" Jun 03 16:18:24.399437 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:24.399412 2568 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jun 03 16:18:24.409221 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:24.409181 2568 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-767bd5547d-b88vd"] Jun 03 16:18:24.501740 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:24.501704 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/36276610-0bf3-4055-87e8-7370c9e78c8a-console-oauth-config\") pod \"console-767bd5547d-b88vd\" (UID: \"36276610-0bf3-4055-87e8-7370c9e78c8a\") " pod="openshift-console/console-767bd5547d-b88vd" Jun 03 16:18:24.501921 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:24.501749 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jr8mw\" (UniqueName: \"kubernetes.io/projected/36276610-0bf3-4055-87e8-7370c9e78c8a-kube-api-access-jr8mw\") pod \"console-767bd5547d-b88vd\" (UID: \"36276610-0bf3-4055-87e8-7370c9e78c8a\") " pod="openshift-console/console-767bd5547d-b88vd" Jun 03 16:18:24.501921 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:24.501816 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/36276610-0bf3-4055-87e8-7370c9e78c8a-console-config\") pod \"console-767bd5547d-b88vd\" (UID: \"36276610-0bf3-4055-87e8-7370c9e78c8a\") " pod="openshift-console/console-767bd5547d-b88vd" Jun 03 16:18:24.501921 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:24.501864 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/36276610-0bf3-4055-87e8-7370c9e78c8a-service-ca\") pod \"console-767bd5547d-b88vd\" (UID: \"36276610-0bf3-4055-87e8-7370c9e78c8a\") " pod="openshift-console/console-767bd5547d-b88vd" Jun 03 16:18:24.501921 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:24.501896 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/36276610-0bf3-4055-87e8-7370c9e78c8a-trusted-ca-bundle\") pod \"console-767bd5547d-b88vd\" (UID: \"36276610-0bf3-4055-87e8-7370c9e78c8a\") " pod="openshift-console/console-767bd5547d-b88vd" Jun 03 16:18:24.502111 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:24.501940 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/36276610-0bf3-4055-87e8-7370c9e78c8a-console-serving-cert\") pod \"console-767bd5547d-b88vd\" (UID: \"36276610-0bf3-4055-87e8-7370c9e78c8a\") " pod="openshift-console/console-767bd5547d-b88vd" Jun 03 16:18:24.502111 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:24.501974 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/36276610-0bf3-4055-87e8-7370c9e78c8a-oauth-serving-cert\") pod \"console-767bd5547d-b88vd\" (UID: \"36276610-0bf3-4055-87e8-7370c9e78c8a\") " pod="openshift-console/console-767bd5547d-b88vd" Jun 03 16:18:24.602433 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:24.602390 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/36276610-0bf3-4055-87e8-7370c9e78c8a-console-serving-cert\") pod \"console-767bd5547d-b88vd\" (UID: \"36276610-0bf3-4055-87e8-7370c9e78c8a\") " pod="openshift-console/console-767bd5547d-b88vd" Jun 03 16:18:24.602634 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:24.602445 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/36276610-0bf3-4055-87e8-7370c9e78c8a-oauth-serving-cert\") pod \"console-767bd5547d-b88vd\" (UID: \"36276610-0bf3-4055-87e8-7370c9e78c8a\") " pod="openshift-console/console-767bd5547d-b88vd" Jun 03 16:18:24.602634 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:24.602490 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/36276610-0bf3-4055-87e8-7370c9e78c8a-console-oauth-config\") pod \"console-767bd5547d-b88vd\" (UID: \"36276610-0bf3-4055-87e8-7370c9e78c8a\") " pod="openshift-console/console-767bd5547d-b88vd" Jun 03 16:18:24.602634 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:24.602506 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-jr8mw\" (UniqueName: \"kubernetes.io/projected/36276610-0bf3-4055-87e8-7370c9e78c8a-kube-api-access-jr8mw\") pod \"console-767bd5547d-b88vd\" (UID: \"36276610-0bf3-4055-87e8-7370c9e78c8a\") " pod="openshift-console/console-767bd5547d-b88vd" Jun 03 16:18:24.602634 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:24.602566 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/36276610-0bf3-4055-87e8-7370c9e78c8a-console-config\") pod \"console-767bd5547d-b88vd\" (UID: \"36276610-0bf3-4055-87e8-7370c9e78c8a\") " pod="openshift-console/console-767bd5547d-b88vd" Jun 03 16:18:24.602634 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:24.602594 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/36276610-0bf3-4055-87e8-7370c9e78c8a-service-ca\") pod \"console-767bd5547d-b88vd\" (UID: \"36276610-0bf3-4055-87e8-7370c9e78c8a\") " pod="openshift-console/console-767bd5547d-b88vd" Jun 03 16:18:24.602634 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:24.602618 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/36276610-0bf3-4055-87e8-7370c9e78c8a-trusted-ca-bundle\") pod \"console-767bd5547d-b88vd\" (UID: \"36276610-0bf3-4055-87e8-7370c9e78c8a\") " pod="openshift-console/console-767bd5547d-b88vd" Jun 03 16:18:24.603347 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:24.603296 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/36276610-0bf3-4055-87e8-7370c9e78c8a-oauth-serving-cert\") pod \"console-767bd5547d-b88vd\" (UID: \"36276610-0bf3-4055-87e8-7370c9e78c8a\") " pod="openshift-console/console-767bd5547d-b88vd" Jun 03 16:18:24.603347 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:24.603333 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/36276610-0bf3-4055-87e8-7370c9e78c8a-console-config\") pod \"console-767bd5547d-b88vd\" (UID: \"36276610-0bf3-4055-87e8-7370c9e78c8a\") " pod="openshift-console/console-767bd5547d-b88vd" Jun 03 16:18:24.603517 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:24.603437 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/36276610-0bf3-4055-87e8-7370c9e78c8a-service-ca\") pod \"console-767bd5547d-b88vd\" (UID: \"36276610-0bf3-4055-87e8-7370c9e78c8a\") " pod="openshift-console/console-767bd5547d-b88vd" Jun 03 16:18:24.603671 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:24.603649 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/36276610-0bf3-4055-87e8-7370c9e78c8a-trusted-ca-bundle\") pod \"console-767bd5547d-b88vd\" (UID: \"36276610-0bf3-4055-87e8-7370c9e78c8a\") " pod="openshift-console/console-767bd5547d-b88vd" Jun 03 16:18:24.605180 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:24.605159 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/36276610-0bf3-4055-87e8-7370c9e78c8a-console-oauth-config\") pod \"console-767bd5547d-b88vd\" (UID: \"36276610-0bf3-4055-87e8-7370c9e78c8a\") " pod="openshift-console/console-767bd5547d-b88vd" Jun 03 16:18:24.605417 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:24.605400 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/36276610-0bf3-4055-87e8-7370c9e78c8a-console-serving-cert\") pod \"console-767bd5547d-b88vd\" (UID: \"36276610-0bf3-4055-87e8-7370c9e78c8a\") " pod="openshift-console/console-767bd5547d-b88vd" Jun 03 16:18:24.610166 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:24.610146 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jr8mw\" (UniqueName: \"kubernetes.io/projected/36276610-0bf3-4055-87e8-7370c9e78c8a-kube-api-access-jr8mw\") pod \"console-767bd5547d-b88vd\" (UID: \"36276610-0bf3-4055-87e8-7370c9e78c8a\") " pod="openshift-console/console-767bd5547d-b88vd" Jun 03 16:18:24.702201 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:24.702122 2568 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-767bd5547d-b88vd" Jun 03 16:18:24.885404 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:24.885366 2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-658bcd8bdf-cfg8g" event={"ID":"eed996ff-81a2-4df1-b751-be50acf655e0","Type":"ContainerStarted","Data":"13a4b210f3c52bb20b06ab4d5c86f96f1f8950ba8dab769a69daf9a9894f8580"} Jun 03 16:18:24.901181 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:24.901129 2568 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-658bcd8bdf-cfg8g" podStartSLOduration=2.346936691 podStartE2EDuration="3.901112208s" podCreationTimestamp="2026-06-03 16:18:21 +0000 UTC" firstStartedPulling="2026-06-03 16:18:22.208071897 +0000 UTC m=+73.296498465" lastFinishedPulling="2026-06-03 16:18:23.76224741 +0000 UTC m=+74.850673982" observedRunningTime="2026-06-03 16:18:24.9005704 +0000 UTC m=+75.988996991" watchObservedRunningTime="2026-06-03 16:18:24.901112208 +0000 UTC m=+75.989538799" Jun 03 16:18:25.427111 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:25.427077 2568 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-767bd5547d-b88vd"] Jun 03 16:18:25.430548 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:18:25.430497 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod36276610_0bf3_4055_87e8_7370c9e78c8a.slice/crio-e34d9e2018b00bfead9dd03bbe3fcc1d71ae022ff107e65144e5e7032f6871a5 WatchSource:0}: Error finding container e34d9e2018b00bfead9dd03bbe3fcc1d71ae022ff107e65144e5e7032f6871a5: Status 404 returned error can't find the container with id e34d9e2018b00bfead9dd03bbe3fcc1d71ae022ff107e65144e5e7032f6871a5 Jun 03 16:18:25.889587 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:25.889528 2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-78bcf7ccbc-vs8xn" event={"ID":"99d39038-e9d1-45f4-916b-e8f846c1b36f","Type":"ContainerStarted","Data":"8d27cb740bb5c39de1c8ade3a3a919d65f11fb770149095794b83ce7ae4ad73e"} Jun 03 16:18:25.890924 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:25.890898 2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-767bd5547d-b88vd" event={"ID":"36276610-0bf3-4055-87e8-7370c9e78c8a","Type":"ContainerStarted","Data":"614a924acd13127dde47515a6cb27803289a128c36f80bb217124616cd2bcb82"} Jun 03 16:18:25.891024 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:25.890932 2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-767bd5547d-b88vd" event={"ID":"36276610-0bf3-4055-87e8-7370c9e78c8a","Type":"ContainerStarted","Data":"e34d9e2018b00bfead9dd03bbe3fcc1d71ae022ff107e65144e5e7032f6871a5"} Jun 03 16:18:25.907151 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:25.907106 2568 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-console/console-78bcf7ccbc-vs8xn" podStartSLOduration=1.612197341 podStartE2EDuration="4.907092819s" podCreationTimestamp="2026-06-03 16:18:21 +0000 UTC" firstStartedPulling="2026-06-03 16:18:22.032339485 +0000 UTC m=+73.120766052" lastFinishedPulling="2026-06-03 16:18:25.327234958 +0000 UTC m=+76.415661530" observedRunningTime="2026-06-03 16:18:25.90591506 +0000 UTC m=+76.994341650" watchObservedRunningTime="2026-06-03 16:18:25.907092819 +0000 UTC m=+76.995519408" Jun 03 16:18:25.921792 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:25.921751 2568 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-console/console-767bd5547d-b88vd" podStartSLOduration=1.921735432 podStartE2EDuration="1.921735432s" podCreationTimestamp="2026-06-03 16:18:24 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-06-03 16:18:25.92106467 +0000 UTC m=+77.009491251" watchObservedRunningTime="2026-06-03 16:18:25.921735432 +0000 UTC m=+77.010162031" Jun 03 16:18:26.527141 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:26.527104 2568 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-6fhtj"] Jun 03 16:18:26.529661 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:26.529637 2568 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-6fhtj" Jun 03 16:18:26.531761 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:26.531734 2568 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jun 03 16:18:26.531892 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:26.531804 2568 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jun 03 16:18:26.531892 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:26.531882 2568 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-8kl4l\"" Jun 03 16:18:26.531982 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:26.531961 2568 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jun 03 16:18:26.617197 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:26.617165 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/f1e80c68-33ac-44f8-88d2-b8abf7ecd943-root\") pod \"node-exporter-6fhtj\" (UID: \"f1e80c68-33ac-44f8-88d2-b8abf7ecd943\") " pod="openshift-monitoring/node-exporter-6fhtj" Jun 03 16:18:26.617378 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:26.617207 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f1e80c68-33ac-44f8-88d2-b8abf7ecd943-metrics-client-ca\") pod \"node-exporter-6fhtj\" (UID: \"f1e80c68-33ac-44f8-88d2-b8abf7ecd943\") " pod="openshift-monitoring/node-exporter-6fhtj" Jun 03 16:18:26.617378 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:26.617233 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/f1e80c68-33ac-44f8-88d2-b8abf7ecd943-node-exporter-wtmp\") pod \"node-exporter-6fhtj\" (UID: \"f1e80c68-33ac-44f8-88d2-b8abf7ecd943\") " pod="openshift-monitoring/node-exporter-6fhtj" Jun 03 16:18:26.617378 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:26.617291 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4grpn\" (UniqueName: \"kubernetes.io/projected/f1e80c68-33ac-44f8-88d2-b8abf7ecd943-kube-api-access-4grpn\") pod \"node-exporter-6fhtj\" (UID: \"f1e80c68-33ac-44f8-88d2-b8abf7ecd943\") " pod="openshift-monitoring/node-exporter-6fhtj" Jun 03 16:18:26.617378 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:26.617372 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f1e80c68-33ac-44f8-88d2-b8abf7ecd943-sys\") pod \"node-exporter-6fhtj\" (UID: \"f1e80c68-33ac-44f8-88d2-b8abf7ecd943\") " pod="openshift-monitoring/node-exporter-6fhtj" Jun 03 16:18:26.617621 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:26.617417 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/f1e80c68-33ac-44f8-88d2-b8abf7ecd943-node-exporter-accelerators-collector-config\") pod \"node-exporter-6fhtj\" (UID: \"f1e80c68-33ac-44f8-88d2-b8abf7ecd943\") " pod="openshift-monitoring/node-exporter-6fhtj" Jun 03 16:18:26.617621 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:26.617451 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/f1e80c68-33ac-44f8-88d2-b8abf7ecd943-node-exporter-textfile\") pod \"node-exporter-6fhtj\" (UID: \"f1e80c68-33ac-44f8-88d2-b8abf7ecd943\") " pod="openshift-monitoring/node-exporter-6fhtj" Jun 03 16:18:26.617621 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:26.617476 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/f1e80c68-33ac-44f8-88d2-b8abf7ecd943-node-exporter-tls\") pod \"node-exporter-6fhtj\" (UID: \"f1e80c68-33ac-44f8-88d2-b8abf7ecd943\") " pod="openshift-monitoring/node-exporter-6fhtj" Jun 03 16:18:26.617621 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:26.617521 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f1e80c68-33ac-44f8-88d2-b8abf7ecd943-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-6fhtj\" (UID: \"f1e80c68-33ac-44f8-88d2-b8abf7ecd943\") " pod="openshift-monitoring/node-exporter-6fhtj" Jun 03 16:18:26.718354 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:26.718329 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f1e80c68-33ac-44f8-88d2-b8abf7ecd943-metrics-client-ca\") pod \"node-exporter-6fhtj\" (UID: \"f1e80c68-33ac-44f8-88d2-b8abf7ecd943\") " pod="openshift-monitoring/node-exporter-6fhtj" Jun 03 16:18:26.718485 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:26.718360 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/f1e80c68-33ac-44f8-88d2-b8abf7ecd943-node-exporter-wtmp\") pod \"node-exporter-6fhtj\" (UID: \"f1e80c68-33ac-44f8-88d2-b8abf7ecd943\") " pod="openshift-monitoring/node-exporter-6fhtj" Jun 03 16:18:26.718485 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:26.718378 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-4grpn\" (UniqueName: \"kubernetes.io/projected/f1e80c68-33ac-44f8-88d2-b8abf7ecd943-kube-api-access-4grpn\") pod \"node-exporter-6fhtj\" (UID: \"f1e80c68-33ac-44f8-88d2-b8abf7ecd943\") " pod="openshift-monitoring/node-exporter-6fhtj" Jun 03 16:18:26.718485 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:26.718395 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f1e80c68-33ac-44f8-88d2-b8abf7ecd943-sys\") pod \"node-exporter-6fhtj\" (UID: \"f1e80c68-33ac-44f8-88d2-b8abf7ecd943\") " pod="openshift-monitoring/node-exporter-6fhtj" Jun 03 16:18:26.718485 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:26.718426 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/f1e80c68-33ac-44f8-88d2-b8abf7ecd943-node-exporter-accelerators-collector-config\") pod \"node-exporter-6fhtj\" (UID: \"f1e80c68-33ac-44f8-88d2-b8abf7ecd943\") " pod="openshift-monitoring/node-exporter-6fhtj" Jun 03 16:18:26.718485 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:26.718452 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/f1e80c68-33ac-44f8-88d2-b8abf7ecd943-node-exporter-textfile\") pod \"node-exporter-6fhtj\" (UID: \"f1e80c68-33ac-44f8-88d2-b8abf7ecd943\") " pod="openshift-monitoring/node-exporter-6fhtj" Jun 03 16:18:26.718485 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:26.718474 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/f1e80c68-33ac-44f8-88d2-b8abf7ecd943-node-exporter-tls\") pod \"node-exporter-6fhtj\" (UID: \"f1e80c68-33ac-44f8-88d2-b8abf7ecd943\") " pod="openshift-monitoring/node-exporter-6fhtj" Jun 03 16:18:26.718788 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:26.718494 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f1e80c68-33ac-44f8-88d2-b8abf7ecd943-sys\") pod \"node-exporter-6fhtj\" (UID: \"f1e80c68-33ac-44f8-88d2-b8abf7ecd943\") " pod="openshift-monitoring/node-exporter-6fhtj" Jun 03 16:18:26.718788 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:26.718515 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f1e80c68-33ac-44f8-88d2-b8abf7ecd943-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-6fhtj\" (UID: \"f1e80c68-33ac-44f8-88d2-b8abf7ecd943\") " pod="openshift-monitoring/node-exporter-6fhtj" Jun 03 16:18:26.718788 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:26.718525 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/f1e80c68-33ac-44f8-88d2-b8abf7ecd943-node-exporter-wtmp\") pod \"node-exporter-6fhtj\" (UID: \"f1e80c68-33ac-44f8-88d2-b8abf7ecd943\") " pod="openshift-monitoring/node-exporter-6fhtj" Jun 03 16:18:26.718788 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:26.718578 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/f1e80c68-33ac-44f8-88d2-b8abf7ecd943-root\") pod \"node-exporter-6fhtj\" (UID: \"f1e80c68-33ac-44f8-88d2-b8abf7ecd943\") " pod="openshift-monitoring/node-exporter-6fhtj" Jun 03 16:18:26.718788 ip-10-0-128-88 kubenswrapper[2568]: E0603 16:18:26.718621 2568 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Jun 03 16:18:26.718788 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:26.718650 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/f1e80c68-33ac-44f8-88d2-b8abf7ecd943-root\") pod \"node-exporter-6fhtj\" (UID: \"f1e80c68-33ac-44f8-88d2-b8abf7ecd943\") " pod="openshift-monitoring/node-exporter-6fhtj" Jun 03 16:18:26.718788 ip-10-0-128-88 kubenswrapper[2568]: E0603 16:18:26.718681 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f1e80c68-33ac-44f8-88d2-b8abf7ecd943-node-exporter-tls podName:f1e80c68-33ac-44f8-88d2-b8abf7ecd943 nodeName:}" failed. No retries permitted until 2026-06-03 16:18:27.218660836 +0000 UTC m=+78.307087412 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/f1e80c68-33ac-44f8-88d2-b8abf7ecd943-node-exporter-tls") pod "node-exporter-6fhtj" (UID: "f1e80c68-33ac-44f8-88d2-b8abf7ecd943") : secret "node-exporter-tls" not found Jun 03 16:18:26.719003 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:26.718839 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/f1e80c68-33ac-44f8-88d2-b8abf7ecd943-node-exporter-textfile\") pod \"node-exporter-6fhtj\" (UID: \"f1e80c68-33ac-44f8-88d2-b8abf7ecd943\") " pod="openshift-monitoring/node-exporter-6fhtj" Jun 03 16:18:26.719038 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:26.719015 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f1e80c68-33ac-44f8-88d2-b8abf7ecd943-metrics-client-ca\") pod \"node-exporter-6fhtj\" (UID: \"f1e80c68-33ac-44f8-88d2-b8abf7ecd943\") " pod="openshift-monitoring/node-exporter-6fhtj" Jun 03 16:18:26.719091 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:26.719074 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/f1e80c68-33ac-44f8-88d2-b8abf7ecd943-node-exporter-accelerators-collector-config\") pod \"node-exporter-6fhtj\" (UID: \"f1e80c68-33ac-44f8-88d2-b8abf7ecd943\") " pod="openshift-monitoring/node-exporter-6fhtj" Jun 03 16:18:26.720786 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:26.720770 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f1e80c68-33ac-44f8-88d2-b8abf7ecd943-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-6fhtj\" (UID: \"f1e80c68-33ac-44f8-88d2-b8abf7ecd943\") " pod="openshift-monitoring/node-exporter-6fhtj" Jun 03 16:18:26.726105 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:26.726082 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4grpn\" (UniqueName: \"kubernetes.io/projected/f1e80c68-33ac-44f8-88d2-b8abf7ecd943-kube-api-access-4grpn\") pod \"node-exporter-6fhtj\" (UID: \"f1e80c68-33ac-44f8-88d2-b8abf7ecd943\") " pod="openshift-monitoring/node-exporter-6fhtj" Jun 03 16:18:27.223611 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:27.223576 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/f1e80c68-33ac-44f8-88d2-b8abf7ecd943-node-exporter-tls\") pod \"node-exporter-6fhtj\" (UID: \"f1e80c68-33ac-44f8-88d2-b8abf7ecd943\") " pod="openshift-monitoring/node-exporter-6fhtj" Jun 03 16:18:27.225791 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:27.225770 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/f1e80c68-33ac-44f8-88d2-b8abf7ecd943-node-exporter-tls\") pod \"node-exporter-6fhtj\" (UID: \"f1e80c68-33ac-44f8-88d2-b8abf7ecd943\") " pod="openshift-monitoring/node-exporter-6fhtj" Jun 03 16:18:27.438647 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:27.438609 2568 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-6fhtj" Jun 03 16:18:27.448215 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:18:27.448147 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf1e80c68_33ac_44f8_88d2_b8abf7ecd943.slice/crio-824a466e4607fe63a469f745f954f936ca584c0cd0a12b56d35ee5646302ef25 WatchSource:0}: Error finding container 824a466e4607fe63a469f745f954f936ca584c0cd0a12b56d35ee5646302ef25: Status 404 returned error can't find the container with id 824a466e4607fe63a469f745f954f936ca584c0cd0a12b56d35ee5646302ef25 Jun 03 16:18:27.622334 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:27.622309 2568 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jun 03 16:18:27.625684 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:27.625664 2568 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jun 03 16:18:27.628056 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:27.628029 2568 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jun 03 16:18:27.628669 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:27.628643 2568 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jun 03 16:18:27.628762 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:27.628718 2568 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jun 03 16:18:27.628762 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:27.628759 2568 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jun 03 16:18:27.628877 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:27.628798 2568 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jun 03 16:18:27.628877 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:27.628798 2568 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-g8nnm\"" Jun 03 16:18:27.628877 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:27.628799 2568 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jun 03 16:18:27.629144 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:27.629122 2568 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jun 03 16:18:27.629144 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:27.629122 2568 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jun 03 16:18:27.629300 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:27.629188 2568 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jun 03 16:18:27.645630 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:27.645605 2568 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jun 03 16:18:27.727275 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:27.727237 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/41ddc394-b5da-48e1-ba7b-fc2e9fdfff0b-config-volume\") pod \"alertmanager-main-0\" (UID: \"41ddc394-b5da-48e1-ba7b-fc2e9fdfff0b\") " pod="openshift-monitoring/alertmanager-main-0" Jun 03 16:18:27.727435 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:27.727281 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/41ddc394-b5da-48e1-ba7b-fc2e9fdfff0b-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"41ddc394-b5da-48e1-ba7b-fc2e9fdfff0b\") " pod="openshift-monitoring/alertmanager-main-0" Jun 03 16:18:27.727435 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:27.727304 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/41ddc394-b5da-48e1-ba7b-fc2e9fdfff0b-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"41ddc394-b5da-48e1-ba7b-fc2e9fdfff0b\") " pod="openshift-monitoring/alertmanager-main-0" Jun 03 16:18:27.727435 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:27.727358 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/41ddc394-b5da-48e1-ba7b-fc2e9fdfff0b-tls-assets\") pod \"alertmanager-main-0\" (UID: \"41ddc394-b5da-48e1-ba7b-fc2e9fdfff0b\") " pod="openshift-monitoring/alertmanager-main-0" Jun 03 16:18:27.727568 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:27.727421 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/41ddc394-b5da-48e1-ba7b-fc2e9fdfff0b-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"41ddc394-b5da-48e1-ba7b-fc2e9fdfff0b\") " pod="openshift-monitoring/alertmanager-main-0" Jun 03 16:18:27.727568 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:27.727457 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/41ddc394-b5da-48e1-ba7b-fc2e9fdfff0b-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"41ddc394-b5da-48e1-ba7b-fc2e9fdfff0b\") " pod="openshift-monitoring/alertmanager-main-0" Jun 03 16:18:27.727568 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:27.727501 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/41ddc394-b5da-48e1-ba7b-fc2e9fdfff0b-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"41ddc394-b5da-48e1-ba7b-fc2e9fdfff0b\") " pod="openshift-monitoring/alertmanager-main-0" Jun 03 16:18:27.727671 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:27.727581 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/41ddc394-b5da-48e1-ba7b-fc2e9fdfff0b-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"41ddc394-b5da-48e1-ba7b-fc2e9fdfff0b\") " pod="openshift-monitoring/alertmanager-main-0" Jun 03 16:18:27.727671 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:27.727610 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/41ddc394-b5da-48e1-ba7b-fc2e9fdfff0b-web-config\") pod \"alertmanager-main-0\" (UID: \"41ddc394-b5da-48e1-ba7b-fc2e9fdfff0b\") " pod="openshift-monitoring/alertmanager-main-0" Jun 03 16:18:27.727671 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:27.727635 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/41ddc394-b5da-48e1-ba7b-fc2e9fdfff0b-config-out\") pod \"alertmanager-main-0\" (UID: \"41ddc394-b5da-48e1-ba7b-fc2e9fdfff0b\") " pod="openshift-monitoring/alertmanager-main-0" Jun 03 16:18:27.727671 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:27.727652 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/41ddc394-b5da-48e1-ba7b-fc2e9fdfff0b-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"41ddc394-b5da-48e1-ba7b-fc2e9fdfff0b\") " pod="openshift-monitoring/alertmanager-main-0" Jun 03 16:18:27.727813 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:27.727674 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s2l7l\" (UniqueName: \"kubernetes.io/projected/41ddc394-b5da-48e1-ba7b-fc2e9fdfff0b-kube-api-access-s2l7l\") pod \"alertmanager-main-0\" (UID: \"41ddc394-b5da-48e1-ba7b-fc2e9fdfff0b\") " pod="openshift-monitoring/alertmanager-main-0" Jun 03 16:18:27.727813 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:27.727696 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/41ddc394-b5da-48e1-ba7b-fc2e9fdfff0b-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"41ddc394-b5da-48e1-ba7b-fc2e9fdfff0b\") " pod="openshift-monitoring/alertmanager-main-0" Jun 03 16:18:27.828689 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:27.828658 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/41ddc394-b5da-48e1-ba7b-fc2e9fdfff0b-config-volume\") pod \"alertmanager-main-0\" (UID: \"41ddc394-b5da-48e1-ba7b-fc2e9fdfff0b\") " pod="openshift-monitoring/alertmanager-main-0" Jun 03 16:18:27.828848 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:27.828695 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/41ddc394-b5da-48e1-ba7b-fc2e9fdfff0b-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"41ddc394-b5da-48e1-ba7b-fc2e9fdfff0b\") " pod="openshift-monitoring/alertmanager-main-0" Jun 03 16:18:27.828848 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:27.828714 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/41ddc394-b5da-48e1-ba7b-fc2e9fdfff0b-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"41ddc394-b5da-48e1-ba7b-fc2e9fdfff0b\") " pod="openshift-monitoring/alertmanager-main-0" Jun 03 16:18:27.828848 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:27.828736 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/41ddc394-b5da-48e1-ba7b-fc2e9fdfff0b-tls-assets\") pod \"alertmanager-main-0\" (UID: \"41ddc394-b5da-48e1-ba7b-fc2e9fdfff0b\") " pod="openshift-monitoring/alertmanager-main-0" Jun 03 16:18:27.828848 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:27.828760 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/41ddc394-b5da-48e1-ba7b-fc2e9fdfff0b-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"41ddc394-b5da-48e1-ba7b-fc2e9fdfff0b\") " pod="openshift-monitoring/alertmanager-main-0" Jun 03 16:18:27.828848 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:27.828776 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/41ddc394-b5da-48e1-ba7b-fc2e9fdfff0b-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"41ddc394-b5da-48e1-ba7b-fc2e9fdfff0b\") " pod="openshift-monitoring/alertmanager-main-0" Jun 03 16:18:27.828848 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:27.828793 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/41ddc394-b5da-48e1-ba7b-fc2e9fdfff0b-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"41ddc394-b5da-48e1-ba7b-fc2e9fdfff0b\") " pod="openshift-monitoring/alertmanager-main-0" Jun 03 16:18:27.828848 ip-10-0-128-88 kubenswrapper[2568]: E0603 16:18:27.828832 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/41ddc394-b5da-48e1-ba7b-fc2e9fdfff0b-alertmanager-trusted-ca-bundle podName:41ddc394-b5da-48e1-ba7b-fc2e9fdfff0b nodeName:}" failed. No retries permitted until 2026-06-03 16:18:28.328811263 +0000 UTC m=+79.417237846 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "alertmanager-trusted-ca-bundle" (UniqueName: "kubernetes.io/configmap/41ddc394-b5da-48e1-ba7b-fc2e9fdfff0b-alertmanager-trusted-ca-bundle") pod "alertmanager-main-0" (UID: "41ddc394-b5da-48e1-ba7b-fc2e9fdfff0b") : configmap references non-existent config key: ca-bundle.crt Jun 03 16:18:27.829273 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:27.828864 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/41ddc394-b5da-48e1-ba7b-fc2e9fdfff0b-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"41ddc394-b5da-48e1-ba7b-fc2e9fdfff0b\") " pod="openshift-monitoring/alertmanager-main-0" Jun 03 16:18:27.829273 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:27.828899 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/41ddc394-b5da-48e1-ba7b-fc2e9fdfff0b-web-config\") pod \"alertmanager-main-0\" (UID: \"41ddc394-b5da-48e1-ba7b-fc2e9fdfff0b\") " pod="openshift-monitoring/alertmanager-main-0" Jun 03 16:18:27.829273 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:27.828932 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/41ddc394-b5da-48e1-ba7b-fc2e9fdfff0b-config-out\") pod \"alertmanager-main-0\" (UID: \"41ddc394-b5da-48e1-ba7b-fc2e9fdfff0b\") " pod="openshift-monitoring/alertmanager-main-0" Jun 03 16:18:27.829273 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:27.828969 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/41ddc394-b5da-48e1-ba7b-fc2e9fdfff0b-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"41ddc394-b5da-48e1-ba7b-fc2e9fdfff0b\") " pod="openshift-monitoring/alertmanager-main-0" Jun 03 16:18:27.829273 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:27.829004 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-s2l7l\" (UniqueName: \"kubernetes.io/projected/41ddc394-b5da-48e1-ba7b-fc2e9fdfff0b-kube-api-access-s2l7l\") pod \"alertmanager-main-0\" (UID: \"41ddc394-b5da-48e1-ba7b-fc2e9fdfff0b\") " pod="openshift-monitoring/alertmanager-main-0" Jun 03 16:18:27.829273 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:27.829032 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/41ddc394-b5da-48e1-ba7b-fc2e9fdfff0b-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"41ddc394-b5da-48e1-ba7b-fc2e9fdfff0b\") " pod="openshift-monitoring/alertmanager-main-0" Jun 03 16:18:27.830506 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:27.829922 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/41ddc394-b5da-48e1-ba7b-fc2e9fdfff0b-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"41ddc394-b5da-48e1-ba7b-fc2e9fdfff0b\") " pod="openshift-monitoring/alertmanager-main-0" Jun 03 16:18:27.830506 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:27.830191 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/41ddc394-b5da-48e1-ba7b-fc2e9fdfff0b-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"41ddc394-b5da-48e1-ba7b-fc2e9fdfff0b\") " pod="openshift-monitoring/alertmanager-main-0" Jun 03 16:18:27.831833 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:27.831812 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/41ddc394-b5da-48e1-ba7b-fc2e9fdfff0b-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"41ddc394-b5da-48e1-ba7b-fc2e9fdfff0b\") " pod="openshift-monitoring/alertmanager-main-0" Jun 03 16:18:27.831944 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:27.831887 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/41ddc394-b5da-48e1-ba7b-fc2e9fdfff0b-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"41ddc394-b5da-48e1-ba7b-fc2e9fdfff0b\") " pod="openshift-monitoring/alertmanager-main-0" Jun 03 16:18:27.832007 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:27.831939 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/41ddc394-b5da-48e1-ba7b-fc2e9fdfff0b-config-volume\") pod \"alertmanager-main-0\" (UID: \"41ddc394-b5da-48e1-ba7b-fc2e9fdfff0b\") " pod="openshift-monitoring/alertmanager-main-0" Jun 03 16:18:27.832007 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:27.831953 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/41ddc394-b5da-48e1-ba7b-fc2e9fdfff0b-config-out\") pod \"alertmanager-main-0\" (UID: \"41ddc394-b5da-48e1-ba7b-fc2e9fdfff0b\") " pod="openshift-monitoring/alertmanager-main-0" Jun 03 16:18:27.832215 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:27.832197 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/41ddc394-b5da-48e1-ba7b-fc2e9fdfff0b-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"41ddc394-b5da-48e1-ba7b-fc2e9fdfff0b\") " pod="openshift-monitoring/alertmanager-main-0" Jun 03 16:18:27.832308 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:27.832288 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/41ddc394-b5da-48e1-ba7b-fc2e9fdfff0b-tls-assets\") pod \"alertmanager-main-0\" (UID: \"41ddc394-b5da-48e1-ba7b-fc2e9fdfff0b\") " pod="openshift-monitoring/alertmanager-main-0" Jun 03 16:18:27.832507 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:27.832486 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/41ddc394-b5da-48e1-ba7b-fc2e9fdfff0b-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"41ddc394-b5da-48e1-ba7b-fc2e9fdfff0b\") " pod="openshift-monitoring/alertmanager-main-0" Jun 03 16:18:27.832917 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:27.832893 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/41ddc394-b5da-48e1-ba7b-fc2e9fdfff0b-web-config\") pod \"alertmanager-main-0\" (UID: \"41ddc394-b5da-48e1-ba7b-fc2e9fdfff0b\") " pod="openshift-monitoring/alertmanager-main-0" Jun 03 16:18:27.833895 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:27.833874 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/41ddc394-b5da-48e1-ba7b-fc2e9fdfff0b-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"41ddc394-b5da-48e1-ba7b-fc2e9fdfff0b\") " pod="openshift-monitoring/alertmanager-main-0" Jun 03 16:18:27.845611 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:27.845590 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-s2l7l\" (UniqueName: \"kubernetes.io/projected/41ddc394-b5da-48e1-ba7b-fc2e9fdfff0b-kube-api-access-s2l7l\") pod \"alertmanager-main-0\" (UID: \"41ddc394-b5da-48e1-ba7b-fc2e9fdfff0b\") " pod="openshift-monitoring/alertmanager-main-0" Jun 03 16:18:27.897151 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:27.897084 2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-6fhtj" event={"ID":"f1e80c68-33ac-44f8-88d2-b8abf7ecd943","Type":"ContainerStarted","Data":"824a466e4607fe63a469f745f954f936ca584c0cd0a12b56d35ee5646302ef25"} Jun 03 16:18:28.334697 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:28.334611 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/41ddc394-b5da-48e1-ba7b-fc2e9fdfff0b-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"41ddc394-b5da-48e1-ba7b-fc2e9fdfff0b\") " pod="openshift-monitoring/alertmanager-main-0" Jun 03 16:18:28.335429 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:28.335409 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/41ddc394-b5da-48e1-ba7b-fc2e9fdfff0b-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"41ddc394-b5da-48e1-ba7b-fc2e9fdfff0b\") " pod="openshift-monitoring/alertmanager-main-0" Jun 03 16:18:28.534913 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:28.534881 2568 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jun 03 16:18:28.609273 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:28.609240 2568 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-755bbc489c-bptwf"] Jun 03 16:18:28.616798 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:28.616422 2568 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-755bbc489c-bptwf" Jun 03 16:18:28.619382 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:28.618779 2568 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\"" Jun 03 16:18:28.619576 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:28.619552 2568 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\"" Jun 03 16:18:28.619702 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:28.619685 2568 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\"" Jun 03 16:18:28.620132 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:28.619786 2568 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-2d70iu39a0vd7\"" Jun 03 16:18:28.620132 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:28.619736 2568 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\"" Jun 03 16:18:28.620132 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:28.619790 2568 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-6vwhj\"" Jun 03 16:18:28.620132 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:28.619966 2568 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\"" Jun 03 16:18:28.625019 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:28.624997 2568 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-755bbc489c-bptwf"] Jun 03 16:18:28.677754 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:28.677725 2568 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jun 03 16:18:28.680839 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:18:28.680811 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod41ddc394_b5da_48e1_ba7b_fc2e9fdfff0b.slice/crio-f5878740a47bacbade13f76b972ad1e8aadf4e21547f050bd81f5754d9f0f85a WatchSource:0}: Error finding container f5878740a47bacbade13f76b972ad1e8aadf4e21547f050bd81f5754d9f0f85a: Status 404 returned error can't find the container with id f5878740a47bacbade13f76b972ad1e8aadf4e21547f050bd81f5754d9f0f85a Jun 03 16:18:28.738837 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:28.738814 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/1885968b-3bc3-437f-bfb9-6777652fb84c-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-755bbc489c-bptwf\" (UID: \"1885968b-3bc3-437f-bfb9-6777652fb84c\") " pod="openshift-monitoring/thanos-querier-755bbc489c-bptwf" Jun 03 16:18:28.738929 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:28.738848 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/1885968b-3bc3-437f-bfb9-6777652fb84c-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-755bbc489c-bptwf\" (UID: \"1885968b-3bc3-437f-bfb9-6777652fb84c\") " pod="openshift-monitoring/thanos-querier-755bbc489c-bptwf" Jun 03 16:18:28.738929 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:28.738879 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/1885968b-3bc3-437f-bfb9-6777652fb84c-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-755bbc489c-bptwf\" (UID: \"1885968b-3bc3-437f-bfb9-6777652fb84c\") " pod="openshift-monitoring/thanos-querier-755bbc489c-bptwf" Jun 03 16:18:28.738929 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:28.738908 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/1885968b-3bc3-437f-bfb9-6777652fb84c-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-755bbc489c-bptwf\" (UID: \"1885968b-3bc3-437f-bfb9-6777652fb84c\") " pod="openshift-monitoring/thanos-querier-755bbc489c-bptwf" Jun 03 16:18:28.739053 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:28.738957 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hz25j\" (UniqueName: \"kubernetes.io/projected/1885968b-3bc3-437f-bfb9-6777652fb84c-kube-api-access-hz25j\") pod \"thanos-querier-755bbc489c-bptwf\" (UID: \"1885968b-3bc3-437f-bfb9-6777652fb84c\") " pod="openshift-monitoring/thanos-querier-755bbc489c-bptwf" Jun 03 16:18:28.739053 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:28.738984 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/1885968b-3bc3-437f-bfb9-6777652fb84c-secret-thanos-querier-tls\") pod \"thanos-querier-755bbc489c-bptwf\" (UID: \"1885968b-3bc3-437f-bfb9-6777652fb84c\") " pod="openshift-monitoring/thanos-querier-755bbc489c-bptwf" Jun 03 16:18:28.739053 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:28.739002 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1885968b-3bc3-437f-bfb9-6777652fb84c-metrics-client-ca\") pod \"thanos-querier-755bbc489c-bptwf\" (UID: \"1885968b-3bc3-437f-bfb9-6777652fb84c\") " pod="openshift-monitoring/thanos-querier-755bbc489c-bptwf" Jun 03 16:18:28.739053 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:28.739017 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/1885968b-3bc3-437f-bfb9-6777652fb84c-secret-grpc-tls\") pod \"thanos-querier-755bbc489c-bptwf\" (UID: \"1885968b-3bc3-437f-bfb9-6777652fb84c\") " pod="openshift-monitoring/thanos-querier-755bbc489c-bptwf" Jun 03 16:18:28.840268 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:28.840212 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-hz25j\" (UniqueName: \"kubernetes.io/projected/1885968b-3bc3-437f-bfb9-6777652fb84c-kube-api-access-hz25j\") pod \"thanos-querier-755bbc489c-bptwf\" (UID: \"1885968b-3bc3-437f-bfb9-6777652fb84c\") " pod="openshift-monitoring/thanos-querier-755bbc489c-bptwf" Jun 03 16:18:28.840268 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:28.840248 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/1885968b-3bc3-437f-bfb9-6777652fb84c-secret-thanos-querier-tls\") pod \"thanos-querier-755bbc489c-bptwf\" (UID: \"1885968b-3bc3-437f-bfb9-6777652fb84c\") " pod="openshift-monitoring/thanos-querier-755bbc489c-bptwf" Jun 03 16:18:28.840422 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:28.840270 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1885968b-3bc3-437f-bfb9-6777652fb84c-metrics-client-ca\") pod \"thanos-querier-755bbc489c-bptwf\" (UID: \"1885968b-3bc3-437f-bfb9-6777652fb84c\") " pod="openshift-monitoring/thanos-querier-755bbc489c-bptwf" Jun 03 16:18:28.840422 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:28.840389 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/1885968b-3bc3-437f-bfb9-6777652fb84c-secret-grpc-tls\") pod \"thanos-querier-755bbc489c-bptwf\" (UID: \"1885968b-3bc3-437f-bfb9-6777652fb84c\") " pod="openshift-monitoring/thanos-querier-755bbc489c-bptwf" Jun 03 16:18:28.840519 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:28.840450 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/1885968b-3bc3-437f-bfb9-6777652fb84c-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-755bbc489c-bptwf\" (UID: \"1885968b-3bc3-437f-bfb9-6777652fb84c\") " pod="openshift-monitoring/thanos-querier-755bbc489c-bptwf" Jun 03 16:18:28.840607 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:28.840589 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/1885968b-3bc3-437f-bfb9-6777652fb84c-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-755bbc489c-bptwf\" (UID: \"1885968b-3bc3-437f-bfb9-6777652fb84c\") " pod="openshift-monitoring/thanos-querier-755bbc489c-bptwf" Jun 03 16:18:28.840655 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:28.840633 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/1885968b-3bc3-437f-bfb9-6777652fb84c-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-755bbc489c-bptwf\" (UID: \"1885968b-3bc3-437f-bfb9-6777652fb84c\") " pod="openshift-monitoring/thanos-querier-755bbc489c-bptwf" Jun 03 16:18:28.840700 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:28.840675 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/1885968b-3bc3-437f-bfb9-6777652fb84c-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-755bbc489c-bptwf\" (UID: \"1885968b-3bc3-437f-bfb9-6777652fb84c\") " pod="openshift-monitoring/thanos-querier-755bbc489c-bptwf" Jun 03 16:18:28.840949 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:28.840911 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1885968b-3bc3-437f-bfb9-6777652fb84c-metrics-client-ca\") pod \"thanos-querier-755bbc489c-bptwf\" (UID: \"1885968b-3bc3-437f-bfb9-6777652fb84c\") " pod="openshift-monitoring/thanos-querier-755bbc489c-bptwf" Jun 03 16:18:28.842964 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:28.842940 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/1885968b-3bc3-437f-bfb9-6777652fb84c-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-755bbc489c-bptwf\" (UID: \"1885968b-3bc3-437f-bfb9-6777652fb84c\") " pod="openshift-monitoring/thanos-querier-755bbc489c-bptwf" Jun 03 16:18:28.843067 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:28.842947 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/1885968b-3bc3-437f-bfb9-6777652fb84c-secret-thanos-querier-tls\") pod \"thanos-querier-755bbc489c-bptwf\" (UID: \"1885968b-3bc3-437f-bfb9-6777652fb84c\") " pod="openshift-monitoring/thanos-querier-755bbc489c-bptwf" Jun 03 16:18:28.843168 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:28.843148 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/1885968b-3bc3-437f-bfb9-6777652fb84c-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-755bbc489c-bptwf\" (UID: \"1885968b-3bc3-437f-bfb9-6777652fb84c\") " pod="openshift-monitoring/thanos-querier-755bbc489c-bptwf" Jun 03 16:18:28.843270 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:28.843244 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/1885968b-3bc3-437f-bfb9-6777652fb84c-secret-grpc-tls\") pod \"thanos-querier-755bbc489c-bptwf\" (UID: \"1885968b-3bc3-437f-bfb9-6777652fb84c\") " pod="openshift-monitoring/thanos-querier-755bbc489c-bptwf" Jun 03 16:18:28.843360 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:28.843251 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/1885968b-3bc3-437f-bfb9-6777652fb84c-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-755bbc489c-bptwf\" (UID: \"1885968b-3bc3-437f-bfb9-6777652fb84c\") " pod="openshift-monitoring/thanos-querier-755bbc489c-bptwf" Jun 03 16:18:28.843360 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:28.843295 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/1885968b-3bc3-437f-bfb9-6777652fb84c-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-755bbc489c-bptwf\" (UID: \"1885968b-3bc3-437f-bfb9-6777652fb84c\") " pod="openshift-monitoring/thanos-querier-755bbc489c-bptwf" Jun 03 16:18:28.847564 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:28.847522 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hz25j\" (UniqueName: \"kubernetes.io/projected/1885968b-3bc3-437f-bfb9-6777652fb84c-kube-api-access-hz25j\") pod \"thanos-querier-755bbc489c-bptwf\" (UID: \"1885968b-3bc3-437f-bfb9-6777652fb84c\") " pod="openshift-monitoring/thanos-querier-755bbc489c-bptwf" Jun 03 16:18:28.867181 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:28.867163 2568 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-lrqwv" Jun 03 16:18:28.900913 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:28.900882 2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"41ddc394-b5da-48e1-ba7b-fc2e9fdfff0b","Type":"ContainerStarted","Data":"f5878740a47bacbade13f76b972ad1e8aadf4e21547f050bd81f5754d9f0f85a"} Jun 03 16:18:28.902318 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:28.902285 2568 generic.go:358] "Generic (PLEG): container finished" podID="f1e80c68-33ac-44f8-88d2-b8abf7ecd943" containerID="cc5712dc2779dec95dd16a8c860d4288d132ccda4d544717ef592ce49bcd1ba4" exitCode=0 Jun 03 16:18:28.902428 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:28.902357 2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-6fhtj" event={"ID":"f1e80c68-33ac-44f8-88d2-b8abf7ecd943","Type":"ContainerDied","Data":"cc5712dc2779dec95dd16a8c860d4288d132ccda4d544717ef592ce49bcd1ba4"} Jun 03 16:18:28.932644 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:28.932623 2568 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-755bbc489c-bptwf" Jun 03 16:18:29.063372 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:29.063333 2568 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-755bbc489c-bptwf"] Jun 03 16:18:29.067287 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:18:29.067260 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1885968b_3bc3_437f_bfb9_6777652fb84c.slice/crio-6d185c5625fe33d34d8162fc7fef45c010e653483de85e84a3383b3c263ca4ee WatchSource:0}: Error finding container 6d185c5625fe33d34d8162fc7fef45c010e653483de85e84a3383b3c263ca4ee: Status 404 returned error can't find the container with id 6d185c5625fe33d34d8162fc7fef45c010e653483de85e84a3383b3c263ca4ee Jun 03 16:18:29.907398 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:29.907372 2568 generic.go:358] "Generic (PLEG): container finished" podID="41ddc394-b5da-48e1-ba7b-fc2e9fdfff0b" containerID="112ca06f598b29066261cc7cc7d7fa82fcb5421295d1686a55b04a3b930e4a0a" exitCode=0 Jun 03 16:18:29.907788 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:29.907437 2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"41ddc394-b5da-48e1-ba7b-fc2e9fdfff0b","Type":"ContainerDied","Data":"112ca06f598b29066261cc7cc7d7fa82fcb5421295d1686a55b04a3b930e4a0a"} Jun 03 16:18:29.909877 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:29.909852 2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-6fhtj" event={"ID":"f1e80c68-33ac-44f8-88d2-b8abf7ecd943","Type":"ContainerStarted","Data":"2e3d36b2cde6d122663200cf516cbf002c74573e781196d1857fb6cec890b48a"} Jun 03 16:18:29.909979 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:29.909886 2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-6fhtj" event={"ID":"f1e80c68-33ac-44f8-88d2-b8abf7ecd943","Type":"ContainerStarted","Data":"b032d127d0fa04babac7112c594aaddb0d6ed3bd213be115b3981a4450571b90"} Jun 03 16:18:29.911154 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:29.911131 2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-755bbc489c-bptwf" event={"ID":"1885968b-3bc3-437f-bfb9-6777652fb84c","Type":"ContainerStarted","Data":"6d185c5625fe33d34d8162fc7fef45c010e653483de85e84a3383b3c263ca4ee"} Jun 03 16:18:29.954201 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:29.954146 2568 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-6fhtj" podStartSLOduration=2.839461214 podStartE2EDuration="3.954132606s" podCreationTimestamp="2026-06-03 16:18:26 +0000 UTC" firstStartedPulling="2026-06-03 16:18:27.450453486 +0000 UTC m=+78.538880070" lastFinishedPulling="2026-06-03 16:18:28.565124891 +0000 UTC m=+79.653551462" observedRunningTime="2026-06-03 16:18:29.952848152 +0000 UTC m=+81.041274746" watchObservedRunningTime="2026-06-03 16:18:29.954132606 +0000 UTC m=+81.042559195" Jun 03 16:18:31.330313 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:31.330289 2568 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-5768fcc9dd-vhfbl"] Jun 03 16:18:31.333070 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:31.333045 2568 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-5768fcc9dd-vhfbl" Jun 03 16:18:31.336231 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:31.336210 2568 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"default-dockercfg-gchdc\"" Jun 03 16:18:31.336745 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:31.336727 2568 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\"" Jun 03 16:18:31.356615 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:31.356589 2568 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-5768fcc9dd-vhfbl"] Jun 03 16:18:31.460989 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:31.460918 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/2d77eba0-7912-4d36-9647-f1b651f01299-monitoring-plugin-cert\") pod \"monitoring-plugin-5768fcc9dd-vhfbl\" (UID: \"2d77eba0-7912-4d36-9647-f1b651f01299\") " pod="openshift-monitoring/monitoring-plugin-5768fcc9dd-vhfbl" Jun 03 16:18:31.561996 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:31.561876 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/2d77eba0-7912-4d36-9647-f1b651f01299-monitoring-plugin-cert\") pod \"monitoring-plugin-5768fcc9dd-vhfbl\" (UID: \"2d77eba0-7912-4d36-9647-f1b651f01299\") " pod="openshift-monitoring/monitoring-plugin-5768fcc9dd-vhfbl" Jun 03 16:18:31.562104 ip-10-0-128-88 kubenswrapper[2568]: E0603 16:18:31.562082 2568 secret.go:189] Couldn't get secret openshift-monitoring/monitoring-plugin-cert: secret "monitoring-plugin-cert" not found Jun 03 16:18:31.562184 ip-10-0-128-88 kubenswrapper[2568]: E0603 16:18:31.562156 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2d77eba0-7912-4d36-9647-f1b651f01299-monitoring-plugin-cert podName:2d77eba0-7912-4d36-9647-f1b651f01299 nodeName:}" failed. No retries permitted until 2026-06-03 16:18:32.062125593 +0000 UTC m=+83.150552175 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "monitoring-plugin-cert" (UniqueName: "kubernetes.io/secret/2d77eba0-7912-4d36-9647-f1b651f01299-monitoring-plugin-cert") pod "monitoring-plugin-5768fcc9dd-vhfbl" (UID: "2d77eba0-7912-4d36-9647-f1b651f01299") : secret "monitoring-plugin-cert" not found Jun 03 16:18:31.905998 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:31.905965 2568 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-78bcf7ccbc-vs8xn" Jun 03 16:18:31.906155 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:31.906015 2568 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-78bcf7ccbc-vs8xn" Jun 03 16:18:31.911739 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:31.911704 2568 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-78bcf7ccbc-vs8xn" Jun 03 16:18:31.920028 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:31.919998 2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-755bbc489c-bptwf" event={"ID":"1885968b-3bc3-437f-bfb9-6777652fb84c","Type":"ContainerStarted","Data":"f7b902164b85686288e123a96b799ba5e96d47d463b3c107e527b77fa68b72e2"} Jun 03 16:18:31.920141 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:31.920037 2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-755bbc489c-bptwf" event={"ID":"1885968b-3bc3-437f-bfb9-6777652fb84c","Type":"ContainerStarted","Data":"a317d6f54e2b24b551eff429e574b3a672a60879ff7fccf4910804e4003d4fdb"} Jun 03 16:18:31.920141 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:31.920053 2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-755bbc489c-bptwf" event={"ID":"1885968b-3bc3-437f-bfb9-6777652fb84c","Type":"ContainerStarted","Data":"b5eb6dfd9c0c94318a20df0da9255469c6d4cb9b7b0f112ecefb22fc6f634697"} Jun 03 16:18:31.923023 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:31.922990 2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"41ddc394-b5da-48e1-ba7b-fc2e9fdfff0b","Type":"ContainerStarted","Data":"cc813697325144e8acdbd14f256bd11f04caf2e5d66263b822d371eb8d33db21"} Jun 03 16:18:31.923023 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:31.923026 2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"41ddc394-b5da-48e1-ba7b-fc2e9fdfff0b","Type":"ContainerStarted","Data":"98201fad1a5377afe29f80fab06a809aff12b52ef5fd2d8e84f6fe8ba2e261f6"} Jun 03 16:18:31.923197 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:31.923035 2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"41ddc394-b5da-48e1-ba7b-fc2e9fdfff0b","Type":"ContainerStarted","Data":"b1fd20363429a00d7322b356960f9bdd4facd2fea63da352b1a92f6052a09198"} Jun 03 16:18:31.923197 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:31.923046 2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"41ddc394-b5da-48e1-ba7b-fc2e9fdfff0b","Type":"ContainerStarted","Data":"d8206bb31f5a97b4ed2a0c5f50a6348ffc71cad3a1169786a12294068d5cbcc8"} Jun 03 16:18:31.923197 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:31.923058 2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"41ddc394-b5da-48e1-ba7b-fc2e9fdfff0b","Type":"ContainerStarted","Data":"d10ad4283d97f9ab77109c3e6f19adf23cf33e23c923a002073adf3fceb93cdc"} Jun 03 16:18:31.928523 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:31.928496 2568 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-78bcf7ccbc-vs8xn" Jun 03 16:18:32.065357 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:32.065316 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/2d77eba0-7912-4d36-9647-f1b651f01299-monitoring-plugin-cert\") pod \"monitoring-plugin-5768fcc9dd-vhfbl\" (UID: \"2d77eba0-7912-4d36-9647-f1b651f01299\") " pod="openshift-monitoring/monitoring-plugin-5768fcc9dd-vhfbl" Jun 03 16:18:32.068133 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:32.068108 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/2d77eba0-7912-4d36-9647-f1b651f01299-monitoring-plugin-cert\") pod \"monitoring-plugin-5768fcc9dd-vhfbl\" (UID: \"2d77eba0-7912-4d36-9647-f1b651f01299\") " pod="openshift-monitoring/monitoring-plugin-5768fcc9dd-vhfbl" Jun 03 16:18:32.247144 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:32.247115 2568 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-5768fcc9dd-vhfbl" Jun 03 16:18:32.384783 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:32.384751 2568 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-5768fcc9dd-vhfbl"] Jun 03 16:18:32.389032 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:18:32.388995 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2d77eba0_7912_4d36_9647_f1b651f01299.slice/crio-ad4a7bd8a90f9fbf5d9aeb687cfb1afe7895f40bf33b75219ca7f47b61fc8002 WatchSource:0}: Error finding container ad4a7bd8a90f9fbf5d9aeb687cfb1afe7895f40bf33b75219ca7f47b61fc8002: Status 404 returned error can't find the container with id ad4a7bd8a90f9fbf5d9aeb687cfb1afe7895f40bf33b75219ca7f47b61fc8002 Jun 03 16:18:32.929332 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:32.929278 2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"41ddc394-b5da-48e1-ba7b-fc2e9fdfff0b","Type":"ContainerStarted","Data":"44abc33b3df91c51120125e57978399aa3e6d92f82e470f7304191882a8d765a"} Jun 03 16:18:32.930553 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:32.930511 2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-5768fcc9dd-vhfbl" event={"ID":"2d77eba0-7912-4d36-9647-f1b651f01299","Type":"ContainerStarted","Data":"ad4a7bd8a90f9fbf5d9aeb687cfb1afe7895f40bf33b75219ca7f47b61fc8002"} Jun 03 16:18:32.933359 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:32.933286 2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-755bbc489c-bptwf" event={"ID":"1885968b-3bc3-437f-bfb9-6777652fb84c","Type":"ContainerStarted","Data":"c5268f1f0c0f59dde60cda937d73a174ae6a6105ff21a63b6c2b324329a1735d"} Jun 03 16:18:32.933359 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:32.933313 2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-755bbc489c-bptwf" event={"ID":"1885968b-3bc3-437f-bfb9-6777652fb84c","Type":"ContainerStarted","Data":"e39f6841fde06416403fc7bd7e2ae230fd5c4dc7607a6b7d365594c0f1e7945a"} Jun 03 16:18:32.933359 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:32.933337 2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-755bbc489c-bptwf" event={"ID":"1885968b-3bc3-437f-bfb9-6777652fb84c","Type":"ContainerStarted","Data":"9b3e188ef3d8d428852121a5026765520a0fa632ac4a3139f72d6a2a6ca43ac0"} Jun 03 16:18:32.933572 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:32.933501 2568 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-755bbc489c-bptwf" Jun 03 16:18:32.966514 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:32.966453 2568 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.471070159 podStartE2EDuration="5.966440421s" podCreationTimestamp="2026-06-03 16:18:27 +0000 UTC" firstStartedPulling="2026-06-03 16:18:28.682792626 +0000 UTC m=+79.771219197" lastFinishedPulling="2026-06-03 16:18:32.178162886 +0000 UTC m=+83.266589459" observedRunningTime="2026-06-03 16:18:32.964974864 +0000 UTC m=+84.053401449" watchObservedRunningTime="2026-06-03 16:18:32.966440421 +0000 UTC m=+84.054867010" Jun 03 16:18:32.995935 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:32.995874 2568 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-755bbc489c-bptwf" podStartSLOduration=1.8941498719999998 podStartE2EDuration="4.995855103s" podCreationTimestamp="2026-06-03 16:18:28 +0000 UTC" firstStartedPulling="2026-06-03 16:18:29.069585214 +0000 UTC m=+80.158011795" lastFinishedPulling="2026-06-03 16:18:32.171290443 +0000 UTC m=+83.259717026" observedRunningTime="2026-06-03 16:18:32.994819756 +0000 UTC m=+84.083246369" watchObservedRunningTime="2026-06-03 16:18:32.995855103 +0000 UTC m=+84.084281694" Jun 03 16:18:33.941297 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:33.941256 2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-5768fcc9dd-vhfbl" event={"ID":"2d77eba0-7912-4d36-9647-f1b651f01299","Type":"ContainerStarted","Data":"109bb39d7b1b54f5bc4b9690eba654ac210d1f9404ed2cbea48aa8eebd7fe756"} Jun 03 16:18:33.941984 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:33.941965 2568 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-5768fcc9dd-vhfbl" Jun 03 16:18:33.946622 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:33.946601 2568 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-5768fcc9dd-vhfbl" Jun 03 16:18:33.957049 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:33.957011 2568 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-5768fcc9dd-vhfbl" podStartSLOduration=1.7674692680000001 podStartE2EDuration="2.957001036s" podCreationTimestamp="2026-06-03 16:18:31 +0000 UTC" firstStartedPulling="2026-06-03 16:18:32.391382567 +0000 UTC m=+83.479809135" lastFinishedPulling="2026-06-03 16:18:33.58091433 +0000 UTC m=+84.669340903" observedRunningTime="2026-06-03 16:18:33.955613798 +0000 UTC m=+85.044040388" watchObservedRunningTime="2026-06-03 16:18:33.957001036 +0000 UTC m=+85.045427623" Jun 03 16:18:34.702940 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:34.702908 2568 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-767bd5547d-b88vd" Jun 03 16:18:34.703186 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:34.703151 2568 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-767bd5547d-b88vd" Jun 03 16:18:34.707556 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:34.707524 2568 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-767bd5547d-b88vd" Jun 03 16:18:34.947690 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:34.947665 2568 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-767bd5547d-b88vd" Jun 03 16:18:34.991986 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:34.991907 2568 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-console/console-78bcf7ccbc-vs8xn"] Jun 03 16:18:36.996334 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:36.996300 2568 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-console/console-6c7fc6d78c-h7rmn"] Jun 03 16:18:36.999833 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:36.999812 2568 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6c7fc6d78c-h7rmn" Jun 03 16:18:37.020427 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:37.020403 2568 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6c7fc6d78c-h7rmn"] Jun 03 16:18:37.109202 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:37.109168 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c77z6\" (UniqueName: \"kubernetes.io/projected/1f4b8d97-0852-40af-a85b-601aebc7c56f-kube-api-access-c77z6\") pod \"console-6c7fc6d78c-h7rmn\" (UID: \"1f4b8d97-0852-40af-a85b-601aebc7c56f\") " pod="openshift-console/console-6c7fc6d78c-h7rmn" Jun 03 16:18:37.109202 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:37.109204 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1f4b8d97-0852-40af-a85b-601aebc7c56f-service-ca\") pod \"console-6c7fc6d78c-h7rmn\" (UID: \"1f4b8d97-0852-40af-a85b-601aebc7c56f\") " pod="openshift-console/console-6c7fc6d78c-h7rmn" Jun 03 16:18:37.109436 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:37.109257 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1f4b8d97-0852-40af-a85b-601aebc7c56f-console-serving-cert\") pod \"console-6c7fc6d78c-h7rmn\" (UID: \"1f4b8d97-0852-40af-a85b-601aebc7c56f\") " pod="openshift-console/console-6c7fc6d78c-h7rmn" Jun 03 16:18:37.109436 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:37.109325 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1f4b8d97-0852-40af-a85b-601aebc7c56f-oauth-serving-cert\") pod \"console-6c7fc6d78c-h7rmn\" (UID: \"1f4b8d97-0852-40af-a85b-601aebc7c56f\") " pod="openshift-console/console-6c7fc6d78c-h7rmn" Jun 03 16:18:37.109436 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:37.109373 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1f4b8d97-0852-40af-a85b-601aebc7c56f-trusted-ca-bundle\") pod \"console-6c7fc6d78c-h7rmn\" (UID: \"1f4b8d97-0852-40af-a85b-601aebc7c56f\") " pod="openshift-console/console-6c7fc6d78c-h7rmn" Jun 03 16:18:37.109631 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:37.109431 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1f4b8d97-0852-40af-a85b-601aebc7c56f-console-oauth-config\") pod \"console-6c7fc6d78c-h7rmn\" (UID: \"1f4b8d97-0852-40af-a85b-601aebc7c56f\") " pod="openshift-console/console-6c7fc6d78c-h7rmn" Jun 03 16:18:37.109631 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:37.109506 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1f4b8d97-0852-40af-a85b-601aebc7c56f-console-config\") pod \"console-6c7fc6d78c-h7rmn\" (UID: \"1f4b8d97-0852-40af-a85b-601aebc7c56f\") " pod="openshift-console/console-6c7fc6d78c-h7rmn" Jun 03 16:18:37.210391 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:37.210356 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1f4b8d97-0852-40af-a85b-601aebc7c56f-oauth-serving-cert\") pod \"console-6c7fc6d78c-h7rmn\" (UID: \"1f4b8d97-0852-40af-a85b-601aebc7c56f\") " pod="openshift-console/console-6c7fc6d78c-h7rmn" Jun 03 16:18:37.210576 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:37.210399 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1f4b8d97-0852-40af-a85b-601aebc7c56f-trusted-ca-bundle\") pod \"console-6c7fc6d78c-h7rmn\" (UID: \"1f4b8d97-0852-40af-a85b-601aebc7c56f\") " pod="openshift-console/console-6c7fc6d78c-h7rmn" Jun 03 16:18:37.210576 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:37.210443 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1f4b8d97-0852-40af-a85b-601aebc7c56f-console-oauth-config\") pod \"console-6c7fc6d78c-h7rmn\" (UID: \"1f4b8d97-0852-40af-a85b-601aebc7c56f\") " pod="openshift-console/console-6c7fc6d78c-h7rmn" Jun 03 16:18:37.210576 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:37.210481 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1f4b8d97-0852-40af-a85b-601aebc7c56f-console-config\") pod \"console-6c7fc6d78c-h7rmn\" (UID: \"1f4b8d97-0852-40af-a85b-601aebc7c56f\") " pod="openshift-console/console-6c7fc6d78c-h7rmn" Jun 03 16:18:37.210576 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:37.210514 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-c77z6\" (UniqueName: \"kubernetes.io/projected/1f4b8d97-0852-40af-a85b-601aebc7c56f-kube-api-access-c77z6\") pod \"console-6c7fc6d78c-h7rmn\" (UID: \"1f4b8d97-0852-40af-a85b-601aebc7c56f\") " pod="openshift-console/console-6c7fc6d78c-h7rmn" Jun 03 16:18:37.210576 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:37.210561 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1f4b8d97-0852-40af-a85b-601aebc7c56f-service-ca\") pod \"console-6c7fc6d78c-h7rmn\" (UID: \"1f4b8d97-0852-40af-a85b-601aebc7c56f\") " pod="openshift-console/console-6c7fc6d78c-h7rmn" Jun 03 16:18:37.210845 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:37.210596 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1f4b8d97-0852-40af-a85b-601aebc7c56f-console-serving-cert\") pod \"console-6c7fc6d78c-h7rmn\" (UID: \"1f4b8d97-0852-40af-a85b-601aebc7c56f\") " pod="openshift-console/console-6c7fc6d78c-h7rmn" Jun 03 16:18:37.211144 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:37.211117 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1f4b8d97-0852-40af-a85b-601aebc7c56f-console-config\") pod \"console-6c7fc6d78c-h7rmn\" (UID: \"1f4b8d97-0852-40af-a85b-601aebc7c56f\") " pod="openshift-console/console-6c7fc6d78c-h7rmn" Jun 03 16:18:37.211283 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:37.211255 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1f4b8d97-0852-40af-a85b-601aebc7c56f-service-ca\") pod \"console-6c7fc6d78c-h7rmn\" (UID: \"1f4b8d97-0852-40af-a85b-601aebc7c56f\") " pod="openshift-console/console-6c7fc6d78c-h7rmn" Jun 03 16:18:37.211386 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:37.211310 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1f4b8d97-0852-40af-a85b-601aebc7c56f-oauth-serving-cert\") pod \"console-6c7fc6d78c-h7rmn\" (UID: \"1f4b8d97-0852-40af-a85b-601aebc7c56f\") " pod="openshift-console/console-6c7fc6d78c-h7rmn" Jun 03 16:18:37.211386 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:37.211358 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1f4b8d97-0852-40af-a85b-601aebc7c56f-trusted-ca-bundle\") pod \"console-6c7fc6d78c-h7rmn\" (UID: \"1f4b8d97-0852-40af-a85b-601aebc7c56f\") " pod="openshift-console/console-6c7fc6d78c-h7rmn" Jun 03 16:18:37.212958 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:37.212935 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1f4b8d97-0852-40af-a85b-601aebc7c56f-console-oauth-config\") pod \"console-6c7fc6d78c-h7rmn\" (UID: \"1f4b8d97-0852-40af-a85b-601aebc7c56f\") " pod="openshift-console/console-6c7fc6d78c-h7rmn" Jun 03 16:18:37.213054 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:37.212960 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1f4b8d97-0852-40af-a85b-601aebc7c56f-console-serving-cert\") pod \"console-6c7fc6d78c-h7rmn\" (UID: \"1f4b8d97-0852-40af-a85b-601aebc7c56f\") " pod="openshift-console/console-6c7fc6d78c-h7rmn" Jun 03 16:18:37.220690 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:37.220672 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c77z6\" (UniqueName: \"kubernetes.io/projected/1f4b8d97-0852-40af-a85b-601aebc7c56f-kube-api-access-c77z6\") pod \"console-6c7fc6d78c-h7rmn\" (UID: \"1f4b8d97-0852-40af-a85b-601aebc7c56f\") " pod="openshift-console/console-6c7fc6d78c-h7rmn" Jun 03 16:18:37.308737 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:37.308663 2568 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6c7fc6d78c-h7rmn" Jun 03 16:18:37.434906 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:37.434877 2568 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6c7fc6d78c-h7rmn"] Jun 03 16:18:37.438055 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:18:37.438026 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1f4b8d97_0852_40af_a85b_601aebc7c56f.slice/crio-d1500fc0c486c8924261fce07e9b7a342fc51086c232095b7cd70b1d7d47c12a WatchSource:0}: Error finding container d1500fc0c486c8924261fce07e9b7a342fc51086c232095b7cd70b1d7d47c12a: Status 404 returned error can't find the container with id d1500fc0c486c8924261fce07e9b7a342fc51086c232095b7cd70b1d7d47c12a Jun 03 16:18:37.959383 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:37.959354 2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6c7fc6d78c-h7rmn" event={"ID":"1f4b8d97-0852-40af-a85b-601aebc7c56f","Type":"ContainerStarted","Data":"78305170c18904e9bb4903ddbd25e7b0c49142e926c7eecb1f5695cd90becad1"} Jun 03 16:18:37.959383 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:37.959387 2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6c7fc6d78c-h7rmn" event={"ID":"1f4b8d97-0852-40af-a85b-601aebc7c56f","Type":"ContainerStarted","Data":"d1500fc0c486c8924261fce07e9b7a342fc51086c232095b7cd70b1d7d47c12a"} Jun 03 16:18:37.979552 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:37.979489 2568 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-console/console-6c7fc6d78c-h7rmn" podStartSLOduration=1.9794760679999999 podStartE2EDuration="1.979476068s" podCreationTimestamp="2026-06-03 16:18:36 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-06-03 16:18:37.977843675 +0000 UTC m=+89.066270264" watchObservedRunningTime="2026-06-03 16:18:37.979476068 +0000 UTC m=+89.067902659" Jun 03 16:18:38.947004 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:38.946975 2568 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-755bbc489c-bptwf" Jun 03 16:18:47.308808 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:47.308771 2568 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6c7fc6d78c-h7rmn" Jun 03 16:18:47.308808 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:47.308818 2568 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-6c7fc6d78c-h7rmn" Jun 03 16:18:47.313394 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:47.313371 2568 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-6c7fc6d78c-h7rmn" Jun 03 16:18:47.992128 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:47.992099 2568 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-6c7fc6d78c-h7rmn" Jun 03 16:18:48.038701 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:48.038669 2568 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-console/console-767bd5547d-b88vd"] Jun 03 16:18:49.867249 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:18:49.867218 2568 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-m7zkz" Jun 03 16:19:00.017479 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:00.017431 2568 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-console/console-78bcf7ccbc-vs8xn" podUID="99d39038-e9d1-45f4-916b-e8f846c1b36f" containerName="console" containerID="cri-o://8d27cb740bb5c39de1c8ade3a3a919d65f11fb770149095794b83ce7ae4ad73e" gracePeriod=15 Jun 03 16:19:00.287591 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:00.287569 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-78bcf7ccbc-vs8xn_99d39038-e9d1-45f4-916b-e8f846c1b36f/console/0.log" Jun 03 16:19:00.287712 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:00.287641 2568 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-78bcf7ccbc-vs8xn" Jun 03 16:19:00.297750 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:00.297730 2568 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/99d39038-e9d1-45f4-916b-e8f846c1b36f-console-oauth-config\") pod \"99d39038-e9d1-45f4-916b-e8f846c1b36f\" (UID: \"99d39038-e9d1-45f4-916b-e8f846c1b36f\") " Jun 03 16:19:00.297829 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:00.297780 2568 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/99d39038-e9d1-45f4-916b-e8f846c1b36f-console-serving-cert\") pod \"99d39038-e9d1-45f4-916b-e8f846c1b36f\" (UID: \"99d39038-e9d1-45f4-916b-e8f846c1b36f\") " Jun 03 16:19:00.297829 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:00.297804 2568 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/99d39038-e9d1-45f4-916b-e8f846c1b36f-oauth-serving-cert\") pod \"99d39038-e9d1-45f4-916b-e8f846c1b36f\" (UID: \"99d39038-e9d1-45f4-916b-e8f846c1b36f\") " Jun 03 16:19:00.297918 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:00.297901 2568 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/99d39038-e9d1-45f4-916b-e8f846c1b36f-console-config\") pod \"99d39038-e9d1-45f4-916b-e8f846c1b36f\" (UID: \"99d39038-e9d1-45f4-916b-e8f846c1b36f\") " Jun 03 16:19:00.297958 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:00.297937 2568 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vrh95\" (UniqueName: \"kubernetes.io/projected/99d39038-e9d1-45f4-916b-e8f846c1b36f-kube-api-access-vrh95\") pod \"99d39038-e9d1-45f4-916b-e8f846c1b36f\" (UID: \"99d39038-e9d1-45f4-916b-e8f846c1b36f\") " Jun 03 16:19:00.298030 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:00.298012 2568 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/99d39038-e9d1-45f4-916b-e8f846c1b36f-service-ca\") pod \"99d39038-e9d1-45f4-916b-e8f846c1b36f\" (UID: \"99d39038-e9d1-45f4-916b-e8f846c1b36f\") " Jun 03 16:19:00.298207 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:00.298159 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/99d39038-e9d1-45f4-916b-e8f846c1b36f-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "99d39038-e9d1-45f4-916b-e8f846c1b36f" (UID: "99d39038-e9d1-45f4-916b-e8f846c1b36f"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 03 16:19:00.298330 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:00.298252 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/99d39038-e9d1-45f4-916b-e8f846c1b36f-console-config" (OuterVolumeSpecName: "console-config") pod "99d39038-e9d1-45f4-916b-e8f846c1b36f" (UID: "99d39038-e9d1-45f4-916b-e8f846c1b36f"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 03 16:19:00.298330 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:00.298277 2568 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/99d39038-e9d1-45f4-916b-e8f846c1b36f-oauth-serving-cert\") on node \"ip-10-0-128-88.ec2.internal\" DevicePath \"\"" Jun 03 16:19:00.298505 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:00.298483 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/99d39038-e9d1-45f4-916b-e8f846c1b36f-service-ca" (OuterVolumeSpecName: "service-ca") pod "99d39038-e9d1-45f4-916b-e8f846c1b36f" (UID: "99d39038-e9d1-45f4-916b-e8f846c1b36f"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 03 16:19:00.299961 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:00.299936 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/99d39038-e9d1-45f4-916b-e8f846c1b36f-kube-api-access-vrh95" (OuterVolumeSpecName: "kube-api-access-vrh95") pod "99d39038-e9d1-45f4-916b-e8f846c1b36f" (UID: "99d39038-e9d1-45f4-916b-e8f846c1b36f"). InnerVolumeSpecName "kube-api-access-vrh95". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 03 16:19:00.300044 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:00.299964 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/99d39038-e9d1-45f4-916b-e8f846c1b36f-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "99d39038-e9d1-45f4-916b-e8f846c1b36f" (UID: "99d39038-e9d1-45f4-916b-e8f846c1b36f"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 03 16:19:00.300044 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:00.299983 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/99d39038-e9d1-45f4-916b-e8f846c1b36f-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "99d39038-e9d1-45f4-916b-e8f846c1b36f" (UID: "99d39038-e9d1-45f4-916b-e8f846c1b36f"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 03 16:19:00.398847 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:00.398816 2568 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/99d39038-e9d1-45f4-916b-e8f846c1b36f-service-ca\") on node \"ip-10-0-128-88.ec2.internal\" DevicePath \"\"" Jun 03 16:19:00.398847 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:00.398843 2568 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/99d39038-e9d1-45f4-916b-e8f846c1b36f-console-oauth-config\") on node \"ip-10-0-128-88.ec2.internal\" DevicePath \"\"" Jun 03 16:19:00.398847 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:00.398853 2568 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/99d39038-e9d1-45f4-916b-e8f846c1b36f-console-serving-cert\") on node \"ip-10-0-128-88.ec2.internal\" DevicePath \"\"" Jun 03 16:19:00.399052 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:00.398862 2568 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/99d39038-e9d1-45f4-916b-e8f846c1b36f-console-config\") on node \"ip-10-0-128-88.ec2.internal\" DevicePath \"\"" Jun 03 16:19:00.399052 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:00.398874 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vrh95\" (UniqueName: \"kubernetes.io/projected/99d39038-e9d1-45f4-916b-e8f846c1b36f-kube-api-access-vrh95\") on node \"ip-10-0-128-88.ec2.internal\" DevicePath \"\"" Jun 03 16:19:01.028572 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:01.028529 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-78bcf7ccbc-vs8xn_99d39038-e9d1-45f4-916b-e8f846c1b36f/console/0.log" Jun 03 16:19:01.028959 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:01.028582 2568 generic.go:358] "Generic (PLEG): container finished" podID="99d39038-e9d1-45f4-916b-e8f846c1b36f" containerID="8d27cb740bb5c39de1c8ade3a3a919d65f11fb770149095794b83ce7ae4ad73e" exitCode=2 Jun 03 16:19:01.028959 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:01.028651 2568 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-78bcf7ccbc-vs8xn" Jun 03 16:19:01.028959 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:01.028673 2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-78bcf7ccbc-vs8xn" event={"ID":"99d39038-e9d1-45f4-916b-e8f846c1b36f","Type":"ContainerDied","Data":"8d27cb740bb5c39de1c8ade3a3a919d65f11fb770149095794b83ce7ae4ad73e"} Jun 03 16:19:01.028959 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:01.028713 2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-78bcf7ccbc-vs8xn" event={"ID":"99d39038-e9d1-45f4-916b-e8f846c1b36f","Type":"ContainerDied","Data":"9b63b9bdfca298cf2196568bdb67b9ef3b453600a7ddedc7759dce179e9a3370"} Jun 03 16:19:01.028959 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:01.028733 2568 scope.go:117] "RemoveContainer" containerID="8d27cb740bb5c39de1c8ade3a3a919d65f11fb770149095794b83ce7ae4ad73e" Jun 03 16:19:01.037485 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:01.037470 2568 scope.go:117] "RemoveContainer" containerID="8d27cb740bb5c39de1c8ade3a3a919d65f11fb770149095794b83ce7ae4ad73e" Jun 03 16:19:01.037785 ip-10-0-128-88 kubenswrapper[2568]: E0603 16:19:01.037764 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8d27cb740bb5c39de1c8ade3a3a919d65f11fb770149095794b83ce7ae4ad73e\": container with ID starting with 8d27cb740bb5c39de1c8ade3a3a919d65f11fb770149095794b83ce7ae4ad73e not found: ID does not exist" containerID="8d27cb740bb5c39de1c8ade3a3a919d65f11fb770149095794b83ce7ae4ad73e" Jun 03 16:19:01.037840 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:01.037795 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8d27cb740bb5c39de1c8ade3a3a919d65f11fb770149095794b83ce7ae4ad73e"} err="failed to get container status \"8d27cb740bb5c39de1c8ade3a3a919d65f11fb770149095794b83ce7ae4ad73e\": rpc error: code = NotFound desc = could not find container \"8d27cb740bb5c39de1c8ade3a3a919d65f11fb770149095794b83ce7ae4ad73e\": container with ID starting with 8d27cb740bb5c39de1c8ade3a3a919d65f11fb770149095794b83ce7ae4ad73e not found: ID does not exist" Jun 03 16:19:01.049152 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:01.049134 2568 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-console/console-78bcf7ccbc-vs8xn"] Jun 03 16:19:01.053797 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:01.053776 2568 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-78bcf7ccbc-vs8xn"] Jun 03 16:19:01.563551 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:01.563499 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="99d39038-e9d1-45f4-916b-e8f846c1b36f" path="/var/lib/kubelet/pods/99d39038-e9d1-45f4-916b-e8f846c1b36f/volumes" Jun 03 16:19:13.058052 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:13.057993 2568 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-console/console-767bd5547d-b88vd" podUID="36276610-0bf3-4055-87e8-7370c9e78c8a" containerName="console" containerID="cri-o://614a924acd13127dde47515a6cb27803289a128c36f80bb217124616cd2bcb82" gracePeriod=15 Jun 03 16:19:13.303647 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:13.303624 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-767bd5547d-b88vd_36276610-0bf3-4055-87e8-7370c9e78c8a/console/0.log" Jun 03 16:19:13.303756 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:13.303684 2568 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-767bd5547d-b88vd" Jun 03 16:19:13.401096 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:13.401060 2568 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/36276610-0bf3-4055-87e8-7370c9e78c8a-console-serving-cert\") pod \"36276610-0bf3-4055-87e8-7370c9e78c8a\" (UID: \"36276610-0bf3-4055-87e8-7370c9e78c8a\") " Jun 03 16:19:13.401096 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:13.401098 2568 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jr8mw\" (UniqueName: \"kubernetes.io/projected/36276610-0bf3-4055-87e8-7370c9e78c8a-kube-api-access-jr8mw\") pod \"36276610-0bf3-4055-87e8-7370c9e78c8a\" (UID: \"36276610-0bf3-4055-87e8-7370c9e78c8a\") " Jun 03 16:19:13.401320 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:13.401120 2568 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/36276610-0bf3-4055-87e8-7370c9e78c8a-trusted-ca-bundle\") pod \"36276610-0bf3-4055-87e8-7370c9e78c8a\" (UID: \"36276610-0bf3-4055-87e8-7370c9e78c8a\") " Jun 03 16:19:13.401320 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:13.401146 2568 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/36276610-0bf3-4055-87e8-7370c9e78c8a-console-oauth-config\") pod \"36276610-0bf3-4055-87e8-7370c9e78c8a\" (UID: \"36276610-0bf3-4055-87e8-7370c9e78c8a\") " Jun 03 16:19:13.401320 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:13.401304 2568 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/36276610-0bf3-4055-87e8-7370c9e78c8a-console-config\") pod \"36276610-0bf3-4055-87e8-7370c9e78c8a\" (UID: \"36276610-0bf3-4055-87e8-7370c9e78c8a\") " Jun 03 16:19:13.401473 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:13.401418 2568 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/36276610-0bf3-4055-87e8-7370c9e78c8a-service-ca\") pod \"36276610-0bf3-4055-87e8-7370c9e78c8a\" (UID: \"36276610-0bf3-4055-87e8-7370c9e78c8a\") " Jun 03 16:19:13.401473 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:13.401463 2568 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/36276610-0bf3-4055-87e8-7370c9e78c8a-oauth-serving-cert\") pod \"36276610-0bf3-4055-87e8-7370c9e78c8a\" (UID: \"36276610-0bf3-4055-87e8-7370c9e78c8a\") " Jun 03 16:19:13.401683 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:13.401643 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/36276610-0bf3-4055-87e8-7370c9e78c8a-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "36276610-0bf3-4055-87e8-7370c9e78c8a" (UID: "36276610-0bf3-4055-87e8-7370c9e78c8a"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 03 16:19:13.401890 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:13.401667 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/36276610-0bf3-4055-87e8-7370c9e78c8a-console-config" (OuterVolumeSpecName: "console-config") pod "36276610-0bf3-4055-87e8-7370c9e78c8a" (UID: "36276610-0bf3-4055-87e8-7370c9e78c8a"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 03 16:19:13.401890 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:13.401869 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/36276610-0bf3-4055-87e8-7370c9e78c8a-service-ca" (OuterVolumeSpecName: "service-ca") pod "36276610-0bf3-4055-87e8-7370c9e78c8a" (UID: "36276610-0bf3-4055-87e8-7370c9e78c8a"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 03 16:19:13.402016 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:13.401920 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/36276610-0bf3-4055-87e8-7370c9e78c8a-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "36276610-0bf3-4055-87e8-7370c9e78c8a" (UID: "36276610-0bf3-4055-87e8-7370c9e78c8a"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 03 16:19:13.403306 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:13.403282 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/36276610-0bf3-4055-87e8-7370c9e78c8a-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "36276610-0bf3-4055-87e8-7370c9e78c8a" (UID: "36276610-0bf3-4055-87e8-7370c9e78c8a"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 03 16:19:13.403393 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:13.403305 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/36276610-0bf3-4055-87e8-7370c9e78c8a-kube-api-access-jr8mw" (OuterVolumeSpecName: "kube-api-access-jr8mw") pod "36276610-0bf3-4055-87e8-7370c9e78c8a" (UID: "36276610-0bf3-4055-87e8-7370c9e78c8a"). InnerVolumeSpecName "kube-api-access-jr8mw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 03 16:19:13.403393 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:13.403358 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/36276610-0bf3-4055-87e8-7370c9e78c8a-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "36276610-0bf3-4055-87e8-7370c9e78c8a" (UID: "36276610-0bf3-4055-87e8-7370c9e78c8a"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 03 16:19:13.502473 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:13.502439 2568 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/36276610-0bf3-4055-87e8-7370c9e78c8a-service-ca\") on node \"ip-10-0-128-88.ec2.internal\" DevicePath \"\"" Jun 03 16:19:13.502473 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:13.502469 2568 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/36276610-0bf3-4055-87e8-7370c9e78c8a-oauth-serving-cert\") on node \"ip-10-0-128-88.ec2.internal\" DevicePath \"\"" Jun 03 16:19:13.502473 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:13.502479 2568 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/36276610-0bf3-4055-87e8-7370c9e78c8a-console-serving-cert\") on node \"ip-10-0-128-88.ec2.internal\" DevicePath \"\"" Jun 03 16:19:13.502707 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:13.502490 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jr8mw\" (UniqueName: \"kubernetes.io/projected/36276610-0bf3-4055-87e8-7370c9e78c8a-kube-api-access-jr8mw\") on node \"ip-10-0-128-88.ec2.internal\" DevicePath \"\"" Jun 03 16:19:13.502707 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:13.502499 2568 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/36276610-0bf3-4055-87e8-7370c9e78c8a-trusted-ca-bundle\") on node \"ip-10-0-128-88.ec2.internal\" DevicePath \"\"" Jun 03 16:19:13.502707 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:13.502509 2568 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/36276610-0bf3-4055-87e8-7370c9e78c8a-console-oauth-config\") on node \"ip-10-0-128-88.ec2.internal\" DevicePath \"\"" Jun 03 16:19:13.502707 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:13.502519 2568 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/36276610-0bf3-4055-87e8-7370c9e78c8a-console-config\") on node \"ip-10-0-128-88.ec2.internal\" DevicePath \"\"" Jun 03 16:19:14.066295 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:14.066271 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-767bd5547d-b88vd_36276610-0bf3-4055-87e8-7370c9e78c8a/console/0.log" Jun 03 16:19:14.066795 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:14.066308 2568 generic.go:358] "Generic (PLEG): container finished" podID="36276610-0bf3-4055-87e8-7370c9e78c8a" containerID="614a924acd13127dde47515a6cb27803289a128c36f80bb217124616cd2bcb82" exitCode=2 Jun 03 16:19:14.066795 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:14.066384 2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-767bd5547d-b88vd" event={"ID":"36276610-0bf3-4055-87e8-7370c9e78c8a","Type":"ContainerDied","Data":"614a924acd13127dde47515a6cb27803289a128c36f80bb217124616cd2bcb82"} Jun 03 16:19:14.066795 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:14.066391 2568 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-767bd5547d-b88vd" Jun 03 16:19:14.066795 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:14.066408 2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-767bd5547d-b88vd" event={"ID":"36276610-0bf3-4055-87e8-7370c9e78c8a","Type":"ContainerDied","Data":"e34d9e2018b00bfead9dd03bbe3fcc1d71ae022ff107e65144e5e7032f6871a5"} Jun 03 16:19:14.066795 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:14.066423 2568 scope.go:117] "RemoveContainer" containerID="614a924acd13127dde47515a6cb27803289a128c36f80bb217124616cd2bcb82" Jun 03 16:19:14.075421 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:14.075390 2568 scope.go:117] "RemoveContainer" containerID="614a924acd13127dde47515a6cb27803289a128c36f80bb217124616cd2bcb82" Jun 03 16:19:14.075771 ip-10-0-128-88 kubenswrapper[2568]: E0603 16:19:14.075751 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"614a924acd13127dde47515a6cb27803289a128c36f80bb217124616cd2bcb82\": container with ID starting with 614a924acd13127dde47515a6cb27803289a128c36f80bb217124616cd2bcb82 not found: ID does not exist" containerID="614a924acd13127dde47515a6cb27803289a128c36f80bb217124616cd2bcb82" Jun 03 16:19:14.075836 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:14.075779 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"614a924acd13127dde47515a6cb27803289a128c36f80bb217124616cd2bcb82"} err="failed to get container status \"614a924acd13127dde47515a6cb27803289a128c36f80bb217124616cd2bcb82\": rpc error: code = NotFound desc = could not find container \"614a924acd13127dde47515a6cb27803289a128c36f80bb217124616cd2bcb82\": container with ID starting with 614a924acd13127dde47515a6cb27803289a128c36f80bb217124616cd2bcb82 not found: ID does not exist" Jun 03 16:19:14.089361 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:14.089336 2568 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-console/console-767bd5547d-b88vd"] Jun 03 16:19:14.097080 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:14.097056 2568 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-767bd5547d-b88vd"] Jun 03 16:19:15.566429 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:15.566393 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="36276610-0bf3-4055-87e8-7370c9e78c8a" path="/var/lib/kubelet/pods/36276610-0bf3-4055-87e8-7370c9e78c8a/volumes" Jun 03 16:19:46.856562 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:46.856512 2568 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jun 03 16:19:46.857835 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:46.857799 2568 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="41ddc394-b5da-48e1-ba7b-fc2e9fdfff0b" containerName="alertmanager" containerID="cri-o://d10ad4283d97f9ab77109c3e6f19adf23cf33e23c923a002073adf3fceb93cdc" gracePeriod=120 Jun 03 16:19:46.857984 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:46.857881 2568 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="41ddc394-b5da-48e1-ba7b-fc2e9fdfff0b" containerName="prom-label-proxy" containerID="cri-o://44abc33b3df91c51120125e57978399aa3e6d92f82e470f7304191882a8d765a" gracePeriod=120 Jun 03 16:19:46.857984 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:46.857936 2568 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="41ddc394-b5da-48e1-ba7b-fc2e9fdfff0b" containerName="config-reloader" containerID="cri-o://d8206bb31f5a97b4ed2a0c5f50a6348ffc71cad3a1169786a12294068d5cbcc8" gracePeriod=120 Jun 03 16:19:46.858092 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:46.857881 2568 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="41ddc394-b5da-48e1-ba7b-fc2e9fdfff0b" containerName="kube-rbac-proxy-metric" containerID="cri-o://cc813697325144e8acdbd14f256bd11f04caf2e5d66263b822d371eb8d33db21" gracePeriod=120 Jun 03 16:19:46.858092 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:46.857881 2568 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="41ddc394-b5da-48e1-ba7b-fc2e9fdfff0b" containerName="kube-rbac-proxy-web" containerID="cri-o://b1fd20363429a00d7322b356960f9bdd4facd2fea63da352b1a92f6052a09198" gracePeriod=120 Jun 03 16:19:46.858197 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:46.858101 2568 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="41ddc394-b5da-48e1-ba7b-fc2e9fdfff0b" containerName="kube-rbac-proxy" containerID="cri-o://98201fad1a5377afe29f80fab06a809aff12b52ef5fd2d8e84f6fe8ba2e261f6" gracePeriod=120 Jun 03 16:19:47.158900 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:47.158816 2568 generic.go:358] "Generic (PLEG): container finished" podID="41ddc394-b5da-48e1-ba7b-fc2e9fdfff0b" containerID="44abc33b3df91c51120125e57978399aa3e6d92f82e470f7304191882a8d765a" exitCode=0 Jun 03 16:19:47.158900 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:47.158841 2568 generic.go:358] "Generic (PLEG): container finished" podID="41ddc394-b5da-48e1-ba7b-fc2e9fdfff0b" containerID="98201fad1a5377afe29f80fab06a809aff12b52ef5fd2d8e84f6fe8ba2e261f6" exitCode=0 Jun 03 16:19:47.158900 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:47.158849 2568 generic.go:358] "Generic (PLEG): container finished" podID="41ddc394-b5da-48e1-ba7b-fc2e9fdfff0b" containerID="d8206bb31f5a97b4ed2a0c5f50a6348ffc71cad3a1169786a12294068d5cbcc8" exitCode=0 Jun 03 16:19:47.158900 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:47.158856 2568 generic.go:358] "Generic (PLEG): container finished" podID="41ddc394-b5da-48e1-ba7b-fc2e9fdfff0b" containerID="d10ad4283d97f9ab77109c3e6f19adf23cf33e23c923a002073adf3fceb93cdc" exitCode=0 Jun 03 16:19:47.158900 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:47.158888 2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"41ddc394-b5da-48e1-ba7b-fc2e9fdfff0b","Type":"ContainerDied","Data":"44abc33b3df91c51120125e57978399aa3e6d92f82e470f7304191882a8d765a"} Jun 03 16:19:47.159161 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:47.158920 2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"41ddc394-b5da-48e1-ba7b-fc2e9fdfff0b","Type":"ContainerDied","Data":"98201fad1a5377afe29f80fab06a809aff12b52ef5fd2d8e84f6fe8ba2e261f6"} Jun 03 16:19:47.159161 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:47.158931 2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"41ddc394-b5da-48e1-ba7b-fc2e9fdfff0b","Type":"ContainerDied","Data":"d8206bb31f5a97b4ed2a0c5f50a6348ffc71cad3a1169786a12294068d5cbcc8"} Jun 03 16:19:47.159161 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:47.158940 2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"41ddc394-b5da-48e1-ba7b-fc2e9fdfff0b","Type":"ContainerDied","Data":"d10ad4283d97f9ab77109c3e6f19adf23cf33e23c923a002073adf3fceb93cdc"} Jun 03 16:19:48.093569 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:48.093525 2568 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jun 03 16:19:48.165564 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:48.165519 2568 generic.go:358] "Generic (PLEG): container finished" podID="41ddc394-b5da-48e1-ba7b-fc2e9fdfff0b" containerID="cc813697325144e8acdbd14f256bd11f04caf2e5d66263b822d371eb8d33db21" exitCode=0 Jun 03 16:19:48.165564 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:48.165560 2568 generic.go:358] "Generic (PLEG): container finished" podID="41ddc394-b5da-48e1-ba7b-fc2e9fdfff0b" containerID="b1fd20363429a00d7322b356960f9bdd4facd2fea63da352b1a92f6052a09198" exitCode=0 Jun 03 16:19:48.165810 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:48.165612 2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"41ddc394-b5da-48e1-ba7b-fc2e9fdfff0b","Type":"ContainerDied","Data":"cc813697325144e8acdbd14f256bd11f04caf2e5d66263b822d371eb8d33db21"} Jun 03 16:19:48.165810 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:48.165643 2568 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jun 03 16:19:48.165810 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:48.165661 2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"41ddc394-b5da-48e1-ba7b-fc2e9fdfff0b","Type":"ContainerDied","Data":"b1fd20363429a00d7322b356960f9bdd4facd2fea63da352b1a92f6052a09198"} Jun 03 16:19:48.165810 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:48.165676 2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"41ddc394-b5da-48e1-ba7b-fc2e9fdfff0b","Type":"ContainerDied","Data":"f5878740a47bacbade13f76b972ad1e8aadf4e21547f050bd81f5754d9f0f85a"} Jun 03 16:19:48.165810 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:48.165697 2568 scope.go:117] "RemoveContainer" containerID="44abc33b3df91c51120125e57978399aa3e6d92f82e470f7304191882a8d765a" Jun 03 16:19:48.173591 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:48.173556 2568 scope.go:117] "RemoveContainer" containerID="cc813697325144e8acdbd14f256bd11f04caf2e5d66263b822d371eb8d33db21" Jun 03 16:19:48.180076 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:48.180052 2568 scope.go:117] "RemoveContainer" containerID="98201fad1a5377afe29f80fab06a809aff12b52ef5fd2d8e84f6fe8ba2e261f6" Jun 03 16:19:48.186257 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:48.186241 2568 scope.go:117] "RemoveContainer" containerID="b1fd20363429a00d7322b356960f9bdd4facd2fea63da352b1a92f6052a09198" Jun 03 16:19:48.192224 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:48.192204 2568 scope.go:117] "RemoveContainer" containerID="d8206bb31f5a97b4ed2a0c5f50a6348ffc71cad3a1169786a12294068d5cbcc8" Jun 03 16:19:48.198277 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:48.198261 2568 scope.go:117] "RemoveContainer" containerID="d10ad4283d97f9ab77109c3e6f19adf23cf33e23c923a002073adf3fceb93cdc" Jun 03 16:19:48.204384 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:48.204369 2568 scope.go:117] "RemoveContainer" containerID="112ca06f598b29066261cc7cc7d7fa82fcb5421295d1686a55b04a3b930e4a0a" Jun 03 16:19:48.210496 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:48.210481 2568 scope.go:117] "RemoveContainer" containerID="44abc33b3df91c51120125e57978399aa3e6d92f82e470f7304191882a8d765a" Jun 03 16:19:48.210741 ip-10-0-128-88 kubenswrapper[2568]: E0603 16:19:48.210724 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"44abc33b3df91c51120125e57978399aa3e6d92f82e470f7304191882a8d765a\": container with ID starting with 44abc33b3df91c51120125e57978399aa3e6d92f82e470f7304191882a8d765a not found: ID does not exist" containerID="44abc33b3df91c51120125e57978399aa3e6d92f82e470f7304191882a8d765a" Jun 03 16:19:48.210795 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:48.210750 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"44abc33b3df91c51120125e57978399aa3e6d92f82e470f7304191882a8d765a"} err="failed to get container status \"44abc33b3df91c51120125e57978399aa3e6d92f82e470f7304191882a8d765a\": rpc error: code = NotFound desc = could not find container \"44abc33b3df91c51120125e57978399aa3e6d92f82e470f7304191882a8d765a\": container with ID starting with 44abc33b3df91c51120125e57978399aa3e6d92f82e470f7304191882a8d765a not found: ID does not exist" Jun 03 16:19:48.210795 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:48.210767 2568 scope.go:117] "RemoveContainer" containerID="cc813697325144e8acdbd14f256bd11f04caf2e5d66263b822d371eb8d33db21" Jun 03 16:19:48.210997 ip-10-0-128-88 kubenswrapper[2568]: E0603 16:19:48.210975 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cc813697325144e8acdbd14f256bd11f04caf2e5d66263b822d371eb8d33db21\": container with ID starting with cc813697325144e8acdbd14f256bd11f04caf2e5d66263b822d371eb8d33db21 not found: ID does not exist" containerID="cc813697325144e8acdbd14f256bd11f04caf2e5d66263b822d371eb8d33db21" Jun 03 16:19:48.211036 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:48.211006 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cc813697325144e8acdbd14f256bd11f04caf2e5d66263b822d371eb8d33db21"} err="failed to get container status \"cc813697325144e8acdbd14f256bd11f04caf2e5d66263b822d371eb8d33db21\": rpc error: code = NotFound desc = could not find container \"cc813697325144e8acdbd14f256bd11f04caf2e5d66263b822d371eb8d33db21\": container with ID starting with cc813697325144e8acdbd14f256bd11f04caf2e5d66263b822d371eb8d33db21 not found: ID does not exist" Jun 03 16:19:48.211036 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:48.211026 2568 scope.go:117] "RemoveContainer" containerID="98201fad1a5377afe29f80fab06a809aff12b52ef5fd2d8e84f6fe8ba2e261f6" Jun 03 16:19:48.211233 ip-10-0-128-88 kubenswrapper[2568]: E0603 16:19:48.211219 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"98201fad1a5377afe29f80fab06a809aff12b52ef5fd2d8e84f6fe8ba2e261f6\": container with ID starting with 98201fad1a5377afe29f80fab06a809aff12b52ef5fd2d8e84f6fe8ba2e261f6 not found: ID does not exist" containerID="98201fad1a5377afe29f80fab06a809aff12b52ef5fd2d8e84f6fe8ba2e261f6" Jun 03 16:19:48.211280 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:48.211239 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"98201fad1a5377afe29f80fab06a809aff12b52ef5fd2d8e84f6fe8ba2e261f6"} err="failed to get container status \"98201fad1a5377afe29f80fab06a809aff12b52ef5fd2d8e84f6fe8ba2e261f6\": rpc error: code = NotFound desc = could not find container \"98201fad1a5377afe29f80fab06a809aff12b52ef5fd2d8e84f6fe8ba2e261f6\": container with ID starting with 98201fad1a5377afe29f80fab06a809aff12b52ef5fd2d8e84f6fe8ba2e261f6 not found: ID does not exist" Jun 03 16:19:48.211280 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:48.211256 2568 scope.go:117] "RemoveContainer" containerID="b1fd20363429a00d7322b356960f9bdd4facd2fea63da352b1a92f6052a09198" Jun 03 16:19:48.211444 ip-10-0-128-88 kubenswrapper[2568]: E0603 16:19:48.211430 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b1fd20363429a00d7322b356960f9bdd4facd2fea63da352b1a92f6052a09198\": container with ID starting with b1fd20363429a00d7322b356960f9bdd4facd2fea63da352b1a92f6052a09198 not found: ID does not exist" containerID="b1fd20363429a00d7322b356960f9bdd4facd2fea63da352b1a92f6052a09198" Jun 03 16:19:48.211483 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:48.211447 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b1fd20363429a00d7322b356960f9bdd4facd2fea63da352b1a92f6052a09198"} err="failed to get container status \"b1fd20363429a00d7322b356960f9bdd4facd2fea63da352b1a92f6052a09198\": rpc error: code = NotFound desc = could not find container \"b1fd20363429a00d7322b356960f9bdd4facd2fea63da352b1a92f6052a09198\": container with ID starting with b1fd20363429a00d7322b356960f9bdd4facd2fea63da352b1a92f6052a09198 not found: ID does not exist" Jun 03 16:19:48.211483 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:48.211459 2568 scope.go:117] "RemoveContainer" containerID="d8206bb31f5a97b4ed2a0c5f50a6348ffc71cad3a1169786a12294068d5cbcc8" Jun 03 16:19:48.211728 ip-10-0-128-88 kubenswrapper[2568]: E0603 16:19:48.211712 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d8206bb31f5a97b4ed2a0c5f50a6348ffc71cad3a1169786a12294068d5cbcc8\": container with ID starting with d8206bb31f5a97b4ed2a0c5f50a6348ffc71cad3a1169786a12294068d5cbcc8 not found: ID does not exist" containerID="d8206bb31f5a97b4ed2a0c5f50a6348ffc71cad3a1169786a12294068d5cbcc8" Jun 03 16:19:48.211783 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:48.211730 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d8206bb31f5a97b4ed2a0c5f50a6348ffc71cad3a1169786a12294068d5cbcc8"} err="failed to get container status \"d8206bb31f5a97b4ed2a0c5f50a6348ffc71cad3a1169786a12294068d5cbcc8\": rpc error: code = NotFound desc = could not find container \"d8206bb31f5a97b4ed2a0c5f50a6348ffc71cad3a1169786a12294068d5cbcc8\": container with ID starting with d8206bb31f5a97b4ed2a0c5f50a6348ffc71cad3a1169786a12294068d5cbcc8 not found: ID does not exist" Jun 03 16:19:48.211783 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:48.211743 2568 scope.go:117] "RemoveContainer" containerID="d10ad4283d97f9ab77109c3e6f19adf23cf33e23c923a002073adf3fceb93cdc" Jun 03 16:19:48.211947 ip-10-0-128-88 kubenswrapper[2568]: E0603 16:19:48.211933 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d10ad4283d97f9ab77109c3e6f19adf23cf33e23c923a002073adf3fceb93cdc\": container with ID starting with d10ad4283d97f9ab77109c3e6f19adf23cf33e23c923a002073adf3fceb93cdc not found: ID does not exist" containerID="d10ad4283d97f9ab77109c3e6f19adf23cf33e23c923a002073adf3fceb93cdc" Jun 03 16:19:48.211992 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:48.211950 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d10ad4283d97f9ab77109c3e6f19adf23cf33e23c923a002073adf3fceb93cdc"} err="failed to get container status \"d10ad4283d97f9ab77109c3e6f19adf23cf33e23c923a002073adf3fceb93cdc\": rpc error: code = NotFound desc = could not find container \"d10ad4283d97f9ab77109c3e6f19adf23cf33e23c923a002073adf3fceb93cdc\": container with ID starting with d10ad4283d97f9ab77109c3e6f19adf23cf33e23c923a002073adf3fceb93cdc not found: ID does not exist" Jun 03 16:19:48.211992 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:48.211961 2568 scope.go:117] "RemoveContainer" containerID="112ca06f598b29066261cc7cc7d7fa82fcb5421295d1686a55b04a3b930e4a0a" Jun 03 16:19:48.212175 ip-10-0-128-88 kubenswrapper[2568]: E0603 16:19:48.212158 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"112ca06f598b29066261cc7cc7d7fa82fcb5421295d1686a55b04a3b930e4a0a\": container with ID starting with 112ca06f598b29066261cc7cc7d7fa82fcb5421295d1686a55b04a3b930e4a0a not found: ID does not exist" containerID="112ca06f598b29066261cc7cc7d7fa82fcb5421295d1686a55b04a3b930e4a0a" Jun 03 16:19:48.212213 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:48.212180 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"112ca06f598b29066261cc7cc7d7fa82fcb5421295d1686a55b04a3b930e4a0a"} err="failed to get container status \"112ca06f598b29066261cc7cc7d7fa82fcb5421295d1686a55b04a3b930e4a0a\": rpc error: code = NotFound desc = could not find container \"112ca06f598b29066261cc7cc7d7fa82fcb5421295d1686a55b04a3b930e4a0a\": container with ID starting with 112ca06f598b29066261cc7cc7d7fa82fcb5421295d1686a55b04a3b930e4a0a not found: ID does not exist" Jun 03 16:19:48.212213 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:48.212196 2568 scope.go:117] "RemoveContainer" containerID="44abc33b3df91c51120125e57978399aa3e6d92f82e470f7304191882a8d765a" Jun 03 16:19:48.212374 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:48.212358 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"44abc33b3df91c51120125e57978399aa3e6d92f82e470f7304191882a8d765a"} err="failed to get container status \"44abc33b3df91c51120125e57978399aa3e6d92f82e470f7304191882a8d765a\": rpc error: code = NotFound desc = could not find container \"44abc33b3df91c51120125e57978399aa3e6d92f82e470f7304191882a8d765a\": container with ID starting with 44abc33b3df91c51120125e57978399aa3e6d92f82e470f7304191882a8d765a not found: ID does not exist" Jun 03 16:19:48.212416 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:48.212375 2568 scope.go:117] "RemoveContainer" containerID="cc813697325144e8acdbd14f256bd11f04caf2e5d66263b822d371eb8d33db21" Jun 03 16:19:48.212585 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:48.212571 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cc813697325144e8acdbd14f256bd11f04caf2e5d66263b822d371eb8d33db21"} err="failed to get container status \"cc813697325144e8acdbd14f256bd11f04caf2e5d66263b822d371eb8d33db21\": rpc error: code = NotFound desc = could not find container \"cc813697325144e8acdbd14f256bd11f04caf2e5d66263b822d371eb8d33db21\": container with ID starting with cc813697325144e8acdbd14f256bd11f04caf2e5d66263b822d371eb8d33db21 not found: ID does not exist" Jun 03 16:19:48.212585 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:48.212585 2568 scope.go:117] "RemoveContainer" containerID="98201fad1a5377afe29f80fab06a809aff12b52ef5fd2d8e84f6fe8ba2e261f6" Jun 03 16:19:48.212779 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:48.212763 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"98201fad1a5377afe29f80fab06a809aff12b52ef5fd2d8e84f6fe8ba2e261f6"} err="failed to get container status \"98201fad1a5377afe29f80fab06a809aff12b52ef5fd2d8e84f6fe8ba2e261f6\": rpc error: code = NotFound desc = could not find container \"98201fad1a5377afe29f80fab06a809aff12b52ef5fd2d8e84f6fe8ba2e261f6\": container with ID starting with 98201fad1a5377afe29f80fab06a809aff12b52ef5fd2d8e84f6fe8ba2e261f6 not found: ID does not exist" Jun 03 16:19:48.212821 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:48.212781 2568 scope.go:117] "RemoveContainer" containerID="b1fd20363429a00d7322b356960f9bdd4facd2fea63da352b1a92f6052a09198" Jun 03 16:19:48.212963 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:48.212947 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b1fd20363429a00d7322b356960f9bdd4facd2fea63da352b1a92f6052a09198"} err="failed to get container status \"b1fd20363429a00d7322b356960f9bdd4facd2fea63da352b1a92f6052a09198\": rpc error: code = NotFound desc = could not find container \"b1fd20363429a00d7322b356960f9bdd4facd2fea63da352b1a92f6052a09198\": container with ID starting with b1fd20363429a00d7322b356960f9bdd4facd2fea63da352b1a92f6052a09198 not found: ID does not exist" Jun 03 16:19:48.213010 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:48.212964 2568 scope.go:117] "RemoveContainer" containerID="d8206bb31f5a97b4ed2a0c5f50a6348ffc71cad3a1169786a12294068d5cbcc8" Jun 03 16:19:48.213126 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:48.213111 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d8206bb31f5a97b4ed2a0c5f50a6348ffc71cad3a1169786a12294068d5cbcc8"} err="failed to get container status \"d8206bb31f5a97b4ed2a0c5f50a6348ffc71cad3a1169786a12294068d5cbcc8\": rpc error: code = NotFound desc = could not find container \"d8206bb31f5a97b4ed2a0c5f50a6348ffc71cad3a1169786a12294068d5cbcc8\": container with ID starting with d8206bb31f5a97b4ed2a0c5f50a6348ffc71cad3a1169786a12294068d5cbcc8 not found: ID does not exist" Jun 03 16:19:48.213185 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:48.213126 2568 scope.go:117] "RemoveContainer" containerID="d10ad4283d97f9ab77109c3e6f19adf23cf33e23c923a002073adf3fceb93cdc" Jun 03 16:19:48.213285 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:48.213270 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d10ad4283d97f9ab77109c3e6f19adf23cf33e23c923a002073adf3fceb93cdc"} err="failed to get container status \"d10ad4283d97f9ab77109c3e6f19adf23cf33e23c923a002073adf3fceb93cdc\": rpc error: code = NotFound desc = could not find container \"d10ad4283d97f9ab77109c3e6f19adf23cf33e23c923a002073adf3fceb93cdc\": container with ID starting with d10ad4283d97f9ab77109c3e6f19adf23cf33e23c923a002073adf3fceb93cdc not found: ID does not exist" Jun 03 16:19:48.213330 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:48.213286 2568 scope.go:117] "RemoveContainer" containerID="112ca06f598b29066261cc7cc7d7fa82fcb5421295d1686a55b04a3b930e4a0a" Jun 03 16:19:48.213493 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:48.213474 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"112ca06f598b29066261cc7cc7d7fa82fcb5421295d1686a55b04a3b930e4a0a"} err="failed to get container status \"112ca06f598b29066261cc7cc7d7fa82fcb5421295d1686a55b04a3b930e4a0a\": rpc error: code = NotFound desc = could not find container \"112ca06f598b29066261cc7cc7d7fa82fcb5421295d1686a55b04a3b930e4a0a\": container with ID starting with 112ca06f598b29066261cc7cc7d7fa82fcb5421295d1686a55b04a3b930e4a0a not found: ID does not exist" Jun 03 16:19:48.268793 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:48.268759 2568 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/41ddc394-b5da-48e1-ba7b-fc2e9fdfff0b-secret-alertmanager-main-tls\") pod \"41ddc394-b5da-48e1-ba7b-fc2e9fdfff0b\" (UID: \"41ddc394-b5da-48e1-ba7b-fc2e9fdfff0b\") " Jun 03 16:19:48.268938 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:48.268804 2568 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/41ddc394-b5da-48e1-ba7b-fc2e9fdfff0b-tls-assets\") pod \"41ddc394-b5da-48e1-ba7b-fc2e9fdfff0b\" (UID: \"41ddc394-b5da-48e1-ba7b-fc2e9fdfff0b\") " Jun 03 16:19:48.268938 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:48.268837 2568 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/41ddc394-b5da-48e1-ba7b-fc2e9fdfff0b-config-out\") pod \"41ddc394-b5da-48e1-ba7b-fc2e9fdfff0b\" (UID: \"41ddc394-b5da-48e1-ba7b-fc2e9fdfff0b\") " Jun 03 16:19:48.268938 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:48.268884 2568 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/41ddc394-b5da-48e1-ba7b-fc2e9fdfff0b-secret-alertmanager-kube-rbac-proxy-metric\") pod \"41ddc394-b5da-48e1-ba7b-fc2e9fdfff0b\" (UID: \"41ddc394-b5da-48e1-ba7b-fc2e9fdfff0b\") " Jun 03 16:19:48.268938 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:48.268915 2568 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/41ddc394-b5da-48e1-ba7b-fc2e9fdfff0b-secret-alertmanager-kube-rbac-proxy\") pod \"41ddc394-b5da-48e1-ba7b-fc2e9fdfff0b\" (UID: \"41ddc394-b5da-48e1-ba7b-fc2e9fdfff0b\") " Jun 03 16:19:48.269139 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:48.268948 2568 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-s2l7l\" (UniqueName: \"kubernetes.io/projected/41ddc394-b5da-48e1-ba7b-fc2e9fdfff0b-kube-api-access-s2l7l\") pod \"41ddc394-b5da-48e1-ba7b-fc2e9fdfff0b\" (UID: \"41ddc394-b5da-48e1-ba7b-fc2e9fdfff0b\") " Jun 03 16:19:48.269139 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:48.268973 2568 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/41ddc394-b5da-48e1-ba7b-fc2e9fdfff0b-web-config\") pod \"41ddc394-b5da-48e1-ba7b-fc2e9fdfff0b\" (UID: \"41ddc394-b5da-48e1-ba7b-fc2e9fdfff0b\") " Jun 03 16:19:48.269139 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:48.268999 2568 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/41ddc394-b5da-48e1-ba7b-fc2e9fdfff0b-config-volume\") pod \"41ddc394-b5da-48e1-ba7b-fc2e9fdfff0b\" (UID: \"41ddc394-b5da-48e1-ba7b-fc2e9fdfff0b\") " Jun 03 16:19:48.270323 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:48.269441 2568 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/41ddc394-b5da-48e1-ba7b-fc2e9fdfff0b-secret-alertmanager-kube-rbac-proxy-web\") pod \"41ddc394-b5da-48e1-ba7b-fc2e9fdfff0b\" (UID: \"41ddc394-b5da-48e1-ba7b-fc2e9fdfff0b\") " Jun 03 16:19:48.270323 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:48.269588 2568 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/41ddc394-b5da-48e1-ba7b-fc2e9fdfff0b-metrics-client-ca\") pod \"41ddc394-b5da-48e1-ba7b-fc2e9fdfff0b\" (UID: \"41ddc394-b5da-48e1-ba7b-fc2e9fdfff0b\") " Jun 03 16:19:48.270323 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:48.269619 2568 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/41ddc394-b5da-48e1-ba7b-fc2e9fdfff0b-alertmanager-main-db\") pod \"41ddc394-b5da-48e1-ba7b-fc2e9fdfff0b\" (UID: \"41ddc394-b5da-48e1-ba7b-fc2e9fdfff0b\") " Jun 03 16:19:48.270323 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:48.269647 2568 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/41ddc394-b5da-48e1-ba7b-fc2e9fdfff0b-cluster-tls-config\") pod \"41ddc394-b5da-48e1-ba7b-fc2e9fdfff0b\" (UID: \"41ddc394-b5da-48e1-ba7b-fc2e9fdfff0b\") " Jun 03 16:19:48.270323 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:48.269689 2568 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/41ddc394-b5da-48e1-ba7b-fc2e9fdfff0b-alertmanager-trusted-ca-bundle\") pod \"41ddc394-b5da-48e1-ba7b-fc2e9fdfff0b\" (UID: \"41ddc394-b5da-48e1-ba7b-fc2e9fdfff0b\") " Jun 03 16:19:48.270323 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:48.270113 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/41ddc394-b5da-48e1-ba7b-fc2e9fdfff0b-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "41ddc394-b5da-48e1-ba7b-fc2e9fdfff0b" (UID: "41ddc394-b5da-48e1-ba7b-fc2e9fdfff0b"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 03 16:19:48.270323 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:48.270228 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/41ddc394-b5da-48e1-ba7b-fc2e9fdfff0b-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "41ddc394-b5da-48e1-ba7b-fc2e9fdfff0b" (UID: "41ddc394-b5da-48e1-ba7b-fc2e9fdfff0b"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 03 16:19:48.270323 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:48.270265 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/41ddc394-b5da-48e1-ba7b-fc2e9fdfff0b-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "41ddc394-b5da-48e1-ba7b-fc2e9fdfff0b" (UID: "41ddc394-b5da-48e1-ba7b-fc2e9fdfff0b"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 03 16:19:48.271688 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:48.271652 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/41ddc394-b5da-48e1-ba7b-fc2e9fdfff0b-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "41ddc394-b5da-48e1-ba7b-fc2e9fdfff0b" (UID: "41ddc394-b5da-48e1-ba7b-fc2e9fdfff0b"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 03 16:19:48.272047 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:48.271959 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/41ddc394-b5da-48e1-ba7b-fc2e9fdfff0b-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "41ddc394-b5da-48e1-ba7b-fc2e9fdfff0b" (UID: "41ddc394-b5da-48e1-ba7b-fc2e9fdfff0b"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 03 16:19:48.272047 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:48.271973 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/41ddc394-b5da-48e1-ba7b-fc2e9fdfff0b-config-volume" (OuterVolumeSpecName: "config-volume") pod "41ddc394-b5da-48e1-ba7b-fc2e9fdfff0b" (UID: "41ddc394-b5da-48e1-ba7b-fc2e9fdfff0b"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 03 16:19:48.272391 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:48.272369 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/41ddc394-b5da-48e1-ba7b-fc2e9fdfff0b-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "41ddc394-b5da-48e1-ba7b-fc2e9fdfff0b" (UID: "41ddc394-b5da-48e1-ba7b-fc2e9fdfff0b"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 03 16:19:48.272474 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:48.272397 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/41ddc394-b5da-48e1-ba7b-fc2e9fdfff0b-kube-api-access-s2l7l" (OuterVolumeSpecName: "kube-api-access-s2l7l") pod "41ddc394-b5da-48e1-ba7b-fc2e9fdfff0b" (UID: "41ddc394-b5da-48e1-ba7b-fc2e9fdfff0b"). InnerVolumeSpecName "kube-api-access-s2l7l". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 03 16:19:48.272657 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:48.272625 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/41ddc394-b5da-48e1-ba7b-fc2e9fdfff0b-config-out" (OuterVolumeSpecName: "config-out") pod "41ddc394-b5da-48e1-ba7b-fc2e9fdfff0b" (UID: "41ddc394-b5da-48e1-ba7b-fc2e9fdfff0b"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 03 16:19:48.273304 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:48.273286 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/41ddc394-b5da-48e1-ba7b-fc2e9fdfff0b-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "41ddc394-b5da-48e1-ba7b-fc2e9fdfff0b" (UID: "41ddc394-b5da-48e1-ba7b-fc2e9fdfff0b"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 03 16:19:48.273842 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:48.273822 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/41ddc394-b5da-48e1-ba7b-fc2e9fdfff0b-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "41ddc394-b5da-48e1-ba7b-fc2e9fdfff0b" (UID: "41ddc394-b5da-48e1-ba7b-fc2e9fdfff0b"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 03 16:19:48.277239 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:48.277212 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/41ddc394-b5da-48e1-ba7b-fc2e9fdfff0b-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "41ddc394-b5da-48e1-ba7b-fc2e9fdfff0b" (UID: "41ddc394-b5da-48e1-ba7b-fc2e9fdfff0b"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 03 16:19:48.281880 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:48.281858 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/41ddc394-b5da-48e1-ba7b-fc2e9fdfff0b-web-config" (OuterVolumeSpecName: "web-config") pod "41ddc394-b5da-48e1-ba7b-fc2e9fdfff0b" (UID: "41ddc394-b5da-48e1-ba7b-fc2e9fdfff0b"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 03 16:19:48.370579 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:48.370519 2568 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/41ddc394-b5da-48e1-ba7b-fc2e9fdfff0b-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-128-88.ec2.internal\" DevicePath \"\"" Jun 03 16:19:48.370579 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:48.370576 2568 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/41ddc394-b5da-48e1-ba7b-fc2e9fdfff0b-secret-alertmanager-main-tls\") on node \"ip-10-0-128-88.ec2.internal\" DevicePath \"\"" Jun 03 16:19:48.370579 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:48.370587 2568 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/41ddc394-b5da-48e1-ba7b-fc2e9fdfff0b-tls-assets\") on node \"ip-10-0-128-88.ec2.internal\" DevicePath \"\"" Jun 03 16:19:48.370800 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:48.370596 2568 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/41ddc394-b5da-48e1-ba7b-fc2e9fdfff0b-config-out\") on node \"ip-10-0-128-88.ec2.internal\" DevicePath \"\"" Jun 03 16:19:48.370800 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:48.370606 2568 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/41ddc394-b5da-48e1-ba7b-fc2e9fdfff0b-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-128-88.ec2.internal\" DevicePath \"\"" Jun 03 16:19:48.370800 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:48.370616 2568 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/41ddc394-b5da-48e1-ba7b-fc2e9fdfff0b-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-128-88.ec2.internal\" DevicePath \"\"" Jun 03 16:19:48.370800 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:48.370625 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s2l7l\" (UniqueName: \"kubernetes.io/projected/41ddc394-b5da-48e1-ba7b-fc2e9fdfff0b-kube-api-access-s2l7l\") on node \"ip-10-0-128-88.ec2.internal\" DevicePath \"\"" Jun 03 16:19:48.370800 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:48.370635 2568 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/41ddc394-b5da-48e1-ba7b-fc2e9fdfff0b-web-config\") on node \"ip-10-0-128-88.ec2.internal\" DevicePath \"\"" Jun 03 16:19:48.370800 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:48.370643 2568 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/41ddc394-b5da-48e1-ba7b-fc2e9fdfff0b-config-volume\") on node \"ip-10-0-128-88.ec2.internal\" DevicePath \"\"" Jun 03 16:19:48.370800 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:48.370651 2568 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/41ddc394-b5da-48e1-ba7b-fc2e9fdfff0b-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-128-88.ec2.internal\" DevicePath \"\"" Jun 03 16:19:48.370800 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:48.370659 2568 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/41ddc394-b5da-48e1-ba7b-fc2e9fdfff0b-metrics-client-ca\") on node \"ip-10-0-128-88.ec2.internal\" DevicePath \"\"" Jun 03 16:19:48.370800 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:48.370669 2568 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/41ddc394-b5da-48e1-ba7b-fc2e9fdfff0b-alertmanager-main-db\") on node \"ip-10-0-128-88.ec2.internal\" DevicePath \"\"" Jun 03 16:19:48.370800 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:48.370678 2568 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/41ddc394-b5da-48e1-ba7b-fc2e9fdfff0b-cluster-tls-config\") on node \"ip-10-0-128-88.ec2.internal\" DevicePath \"\"" Jun 03 16:19:48.489974 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:48.489918 2568 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jun 03 16:19:48.494640 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:48.494617 2568 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jun 03 16:19:48.524519 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:48.524488 2568 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jun 03 16:19:48.524788 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:48.524774 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="41ddc394-b5da-48e1-ba7b-fc2e9fdfff0b" containerName="kube-rbac-proxy" Jun 03 16:19:48.524788 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:48.524789 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="41ddc394-b5da-48e1-ba7b-fc2e9fdfff0b" containerName="kube-rbac-proxy" Jun 03 16:19:48.524901 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:48.524801 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="41ddc394-b5da-48e1-ba7b-fc2e9fdfff0b" containerName="kube-rbac-proxy-web" Jun 03 16:19:48.524901 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:48.524807 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="41ddc394-b5da-48e1-ba7b-fc2e9fdfff0b" containerName="kube-rbac-proxy-web" Jun 03 16:19:48.524901 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:48.524817 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="41ddc394-b5da-48e1-ba7b-fc2e9fdfff0b" containerName="alertmanager" Jun 03 16:19:48.524901 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:48.524822 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="41ddc394-b5da-48e1-ba7b-fc2e9fdfff0b" containerName="alertmanager" Jun 03 16:19:48.524901 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:48.524829 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="41ddc394-b5da-48e1-ba7b-fc2e9fdfff0b" containerName="config-reloader" Jun 03 16:19:48.524901 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:48.524834 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="41ddc394-b5da-48e1-ba7b-fc2e9fdfff0b" containerName="config-reloader" Jun 03 16:19:48.524901 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:48.524841 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="41ddc394-b5da-48e1-ba7b-fc2e9fdfff0b" containerName="init-config-reloader" Jun 03 16:19:48.524901 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:48.524847 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="41ddc394-b5da-48e1-ba7b-fc2e9fdfff0b" containerName="init-config-reloader" Jun 03 16:19:48.524901 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:48.524853 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="99d39038-e9d1-45f4-916b-e8f846c1b36f" containerName="console" Jun 03 16:19:48.524901 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:48.524860 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="99d39038-e9d1-45f4-916b-e8f846c1b36f" containerName="console" Jun 03 16:19:48.524901 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:48.524876 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="36276610-0bf3-4055-87e8-7370c9e78c8a" containerName="console" Jun 03 16:19:48.524901 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:48.524881 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="36276610-0bf3-4055-87e8-7370c9e78c8a" containerName="console" Jun 03 16:19:48.524901 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:48.524888 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="41ddc394-b5da-48e1-ba7b-fc2e9fdfff0b" containerName="kube-rbac-proxy-metric" Jun 03 16:19:48.524901 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:48.524893 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="41ddc394-b5da-48e1-ba7b-fc2e9fdfff0b" containerName="kube-rbac-proxy-metric" Jun 03 16:19:48.524901 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:48.524900 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="41ddc394-b5da-48e1-ba7b-fc2e9fdfff0b" containerName="prom-label-proxy" Jun 03 16:19:48.524901 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:48.524904 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="41ddc394-b5da-48e1-ba7b-fc2e9fdfff0b" containerName="prom-label-proxy" Jun 03 16:19:48.525342 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:48.524970 2568 memory_manager.go:356] "RemoveStaleState removing state" podUID="41ddc394-b5da-48e1-ba7b-fc2e9fdfff0b" containerName="config-reloader" Jun 03 16:19:48.525342 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:48.524978 2568 memory_manager.go:356] "RemoveStaleState removing state" podUID="41ddc394-b5da-48e1-ba7b-fc2e9fdfff0b" containerName="alertmanager" Jun 03 16:19:48.525342 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:48.524984 2568 memory_manager.go:356] "RemoveStaleState removing state" podUID="41ddc394-b5da-48e1-ba7b-fc2e9fdfff0b" containerName="kube-rbac-proxy" Jun 03 16:19:48.525342 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:48.524991 2568 memory_manager.go:356] "RemoveStaleState removing state" podUID="41ddc394-b5da-48e1-ba7b-fc2e9fdfff0b" containerName="kube-rbac-proxy-metric" Jun 03 16:19:48.525342 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:48.524996 2568 memory_manager.go:356] "RemoveStaleState removing state" podUID="41ddc394-b5da-48e1-ba7b-fc2e9fdfff0b" containerName="prom-label-proxy" Jun 03 16:19:48.525342 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:48.525003 2568 memory_manager.go:356] "RemoveStaleState removing state" podUID="99d39038-e9d1-45f4-916b-e8f846c1b36f" containerName="console" Jun 03 16:19:48.525342 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:48.525009 2568 memory_manager.go:356] "RemoveStaleState removing state" podUID="36276610-0bf3-4055-87e8-7370c9e78c8a" containerName="console" Jun 03 16:19:48.525342 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:48.525015 2568 memory_manager.go:356] "RemoveStaleState removing state" podUID="41ddc394-b5da-48e1-ba7b-fc2e9fdfff0b" containerName="kube-rbac-proxy-web" Jun 03 16:19:48.528502 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:48.528486 2568 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jun 03 16:19:48.531306 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:48.531273 2568 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-g8nnm\"" Jun 03 16:19:48.531412 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:48.531364 2568 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jun 03 16:19:48.531693 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:48.531672 2568 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jun 03 16:19:48.532077 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:48.532061 2568 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jun 03 16:19:48.533003 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:48.532986 2568 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jun 03 16:19:48.533063 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:48.533011 2568 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jun 03 16:19:48.533122 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:48.533104 2568 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jun 03 16:19:48.533244 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:48.533230 2568 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jun 03 16:19:48.533506 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:48.533454 2568 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jun 03 16:19:48.538467 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:48.538449 2568 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jun 03 16:19:48.544320 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:48.544297 2568 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jun 03 16:19:48.671640 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:48.671595 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/d48d8d35-b7ce-4450-b96a-7070ed55406c-web-config\") pod \"alertmanager-main-0\" (UID: \"d48d8d35-b7ce-4450-b96a-7070ed55406c\") " pod="openshift-monitoring/alertmanager-main-0" Jun 03 16:19:48.671640 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:48.671651 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/d48d8d35-b7ce-4450-b96a-7070ed55406c-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"d48d8d35-b7ce-4450-b96a-7070ed55406c\") " pod="openshift-monitoring/alertmanager-main-0" Jun 03 16:19:48.671856 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:48.671672 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/d48d8d35-b7ce-4450-b96a-7070ed55406c-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"d48d8d35-b7ce-4450-b96a-7070ed55406c\") " pod="openshift-monitoring/alertmanager-main-0" Jun 03 16:19:48.671856 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:48.671692 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d48d8d35-b7ce-4450-b96a-7070ed55406c-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"d48d8d35-b7ce-4450-b96a-7070ed55406c\") " pod="openshift-monitoring/alertmanager-main-0" Jun 03 16:19:48.671856 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:48.671751 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/d48d8d35-b7ce-4450-b96a-7070ed55406c-tls-assets\") pod \"alertmanager-main-0\" (UID: \"d48d8d35-b7ce-4450-b96a-7070ed55406c\") " pod="openshift-monitoring/alertmanager-main-0" Jun 03 16:19:48.671856 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:48.671789 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/d48d8d35-b7ce-4450-b96a-7070ed55406c-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"d48d8d35-b7ce-4450-b96a-7070ed55406c\") " pod="openshift-monitoring/alertmanager-main-0" Jun 03 16:19:48.671856 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:48.671823 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d48d8d35-b7ce-4450-b96a-7070ed55406c-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"d48d8d35-b7ce-4450-b96a-7070ed55406c\") " pod="openshift-monitoring/alertmanager-main-0" Jun 03 16:19:48.671856 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:48.671845 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d48d8d35-b7ce-4450-b96a-7070ed55406c-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"d48d8d35-b7ce-4450-b96a-7070ed55406c\") " pod="openshift-monitoring/alertmanager-main-0" Jun 03 16:19:48.672073 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:48.671863 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/d48d8d35-b7ce-4450-b96a-7070ed55406c-config-volume\") pod \"alertmanager-main-0\" (UID: \"d48d8d35-b7ce-4450-b96a-7070ed55406c\") " pod="openshift-monitoring/alertmanager-main-0" Jun 03 16:19:48.672073 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:48.671884 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/d48d8d35-b7ce-4450-b96a-7070ed55406c-config-out\") pod \"alertmanager-main-0\" (UID: \"d48d8d35-b7ce-4450-b96a-7070ed55406c\") " pod="openshift-monitoring/alertmanager-main-0" Jun 03 16:19:48.672073 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:48.671928 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b6vc9\" (UniqueName: \"kubernetes.io/projected/d48d8d35-b7ce-4450-b96a-7070ed55406c-kube-api-access-b6vc9\") pod \"alertmanager-main-0\" (UID: \"d48d8d35-b7ce-4450-b96a-7070ed55406c\") " pod="openshift-monitoring/alertmanager-main-0" Jun 03 16:19:48.672073 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:48.671977 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d48d8d35-b7ce-4450-b96a-7070ed55406c-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"d48d8d35-b7ce-4450-b96a-7070ed55406c\") " pod="openshift-monitoring/alertmanager-main-0" Jun 03 16:19:48.672073 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:48.672032 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/d48d8d35-b7ce-4450-b96a-7070ed55406c-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"d48d8d35-b7ce-4450-b96a-7070ed55406c\") " pod="openshift-monitoring/alertmanager-main-0" Jun 03 16:19:48.773041 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:48.772927 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/d48d8d35-b7ce-4450-b96a-7070ed55406c-web-config\") pod \"alertmanager-main-0\" (UID: \"d48d8d35-b7ce-4450-b96a-7070ed55406c\") " pod="openshift-monitoring/alertmanager-main-0" Jun 03 16:19:48.773041 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:48.772997 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/d48d8d35-b7ce-4450-b96a-7070ed55406c-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"d48d8d35-b7ce-4450-b96a-7070ed55406c\") " pod="openshift-monitoring/alertmanager-main-0" Jun 03 16:19:48.773041 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:48.773025 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/d48d8d35-b7ce-4450-b96a-7070ed55406c-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"d48d8d35-b7ce-4450-b96a-7070ed55406c\") " pod="openshift-monitoring/alertmanager-main-0" Jun 03 16:19:48.773296 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:48.773051 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d48d8d35-b7ce-4450-b96a-7070ed55406c-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"d48d8d35-b7ce-4450-b96a-7070ed55406c\") " pod="openshift-monitoring/alertmanager-main-0" Jun 03 16:19:48.773296 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:48.773089 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/d48d8d35-b7ce-4450-b96a-7070ed55406c-tls-assets\") pod \"alertmanager-main-0\" (UID: \"d48d8d35-b7ce-4450-b96a-7070ed55406c\") " pod="openshift-monitoring/alertmanager-main-0" Jun 03 16:19:48.773296 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:48.773126 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/d48d8d35-b7ce-4450-b96a-7070ed55406c-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"d48d8d35-b7ce-4450-b96a-7070ed55406c\") " pod="openshift-monitoring/alertmanager-main-0" Jun 03 16:19:48.773296 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:48.773165 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d48d8d35-b7ce-4450-b96a-7070ed55406c-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"d48d8d35-b7ce-4450-b96a-7070ed55406c\") " pod="openshift-monitoring/alertmanager-main-0" Jun 03 16:19:48.773296 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:48.773193 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d48d8d35-b7ce-4450-b96a-7070ed55406c-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"d48d8d35-b7ce-4450-b96a-7070ed55406c\") " pod="openshift-monitoring/alertmanager-main-0" Jun 03 16:19:48.773568 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:48.773342 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/d48d8d35-b7ce-4450-b96a-7070ed55406c-config-volume\") pod \"alertmanager-main-0\" (UID: \"d48d8d35-b7ce-4450-b96a-7070ed55406c\") " pod="openshift-monitoring/alertmanager-main-0" Jun 03 16:19:48.773568 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:48.773375 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/d48d8d35-b7ce-4450-b96a-7070ed55406c-config-out\") pod \"alertmanager-main-0\" (UID: \"d48d8d35-b7ce-4450-b96a-7070ed55406c\") " pod="openshift-monitoring/alertmanager-main-0" Jun 03 16:19:48.773568 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:48.773404 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-b6vc9\" (UniqueName: \"kubernetes.io/projected/d48d8d35-b7ce-4450-b96a-7070ed55406c-kube-api-access-b6vc9\") pod \"alertmanager-main-0\" (UID: \"d48d8d35-b7ce-4450-b96a-7070ed55406c\") " pod="openshift-monitoring/alertmanager-main-0" Jun 03 16:19:48.773568 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:48.773438 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d48d8d35-b7ce-4450-b96a-7070ed55406c-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"d48d8d35-b7ce-4450-b96a-7070ed55406c\") " pod="openshift-monitoring/alertmanager-main-0" Jun 03 16:19:48.773568 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:48.773484 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/d48d8d35-b7ce-4450-b96a-7070ed55406c-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"d48d8d35-b7ce-4450-b96a-7070ed55406c\") " pod="openshift-monitoring/alertmanager-main-0" Jun 03 16:19:48.773985 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:48.773936 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/d48d8d35-b7ce-4450-b96a-7070ed55406c-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"d48d8d35-b7ce-4450-b96a-7070ed55406c\") " pod="openshift-monitoring/alertmanager-main-0" Jun 03 16:19:48.776321 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:48.776147 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/d48d8d35-b7ce-4450-b96a-7070ed55406c-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"d48d8d35-b7ce-4450-b96a-7070ed55406c\") " pod="openshift-monitoring/alertmanager-main-0" Jun 03 16:19:48.776321 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:48.776146 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/d48d8d35-b7ce-4450-b96a-7070ed55406c-web-config\") pod \"alertmanager-main-0\" (UID: \"d48d8d35-b7ce-4450-b96a-7070ed55406c\") " pod="openshift-monitoring/alertmanager-main-0" Jun 03 16:19:48.776321 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:48.776230 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/d48d8d35-b7ce-4450-b96a-7070ed55406c-tls-assets\") pod \"alertmanager-main-0\" (UID: \"d48d8d35-b7ce-4450-b96a-7070ed55406c\") " pod="openshift-monitoring/alertmanager-main-0" Jun 03 16:19:48.776321 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:48.776266 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d48d8d35-b7ce-4450-b96a-7070ed55406c-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"d48d8d35-b7ce-4450-b96a-7070ed55406c\") " pod="openshift-monitoring/alertmanager-main-0" Jun 03 16:19:48.776321 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:48.776272 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/d48d8d35-b7ce-4450-b96a-7070ed55406c-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"d48d8d35-b7ce-4450-b96a-7070ed55406c\") " pod="openshift-monitoring/alertmanager-main-0" Jun 03 16:19:48.776788 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:48.776326 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d48d8d35-b7ce-4450-b96a-7070ed55406c-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"d48d8d35-b7ce-4450-b96a-7070ed55406c\") " pod="openshift-monitoring/alertmanager-main-0" Jun 03 16:19:48.776788 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:48.776458 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/d48d8d35-b7ce-4450-b96a-7070ed55406c-config-volume\") pod \"alertmanager-main-0\" (UID: \"d48d8d35-b7ce-4450-b96a-7070ed55406c\") " pod="openshift-monitoring/alertmanager-main-0" Jun 03 16:19:48.776788 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:48.776699 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/d48d8d35-b7ce-4450-b96a-7070ed55406c-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"d48d8d35-b7ce-4450-b96a-7070ed55406c\") " pod="openshift-monitoring/alertmanager-main-0" Jun 03 16:19:48.776916 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:48.776900 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d48d8d35-b7ce-4450-b96a-7070ed55406c-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"d48d8d35-b7ce-4450-b96a-7070ed55406c\") " pod="openshift-monitoring/alertmanager-main-0" Jun 03 16:19:48.776957 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:48.776935 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d48d8d35-b7ce-4450-b96a-7070ed55406c-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"d48d8d35-b7ce-4450-b96a-7070ed55406c\") " pod="openshift-monitoring/alertmanager-main-0" Jun 03 16:19:48.778373 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:48.778353 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/d48d8d35-b7ce-4450-b96a-7070ed55406c-config-out\") pod \"alertmanager-main-0\" (UID: \"d48d8d35-b7ce-4450-b96a-7070ed55406c\") " pod="openshift-monitoring/alertmanager-main-0" Jun 03 16:19:48.784663 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:48.784646 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b6vc9\" (UniqueName: \"kubernetes.io/projected/d48d8d35-b7ce-4450-b96a-7070ed55406c-kube-api-access-b6vc9\") pod \"alertmanager-main-0\" (UID: \"d48d8d35-b7ce-4450-b96a-7070ed55406c\") " pod="openshift-monitoring/alertmanager-main-0" Jun 03 16:19:48.838519 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:48.838452 2568 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jun 03 16:19:48.971337 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:48.971309 2568 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jun 03 16:19:48.975101 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:19:48.975069 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd48d8d35_b7ce_4450_b96a_7070ed55406c.slice/crio-a812914c03b6a7b7beef584c2d6cd19fba3bc6cb17617a64e22e63117b7e28e2 WatchSource:0}: Error finding container a812914c03b6a7b7beef584c2d6cd19fba3bc6cb17617a64e22e63117b7e28e2: Status 404 returned error can't find the container with id a812914c03b6a7b7beef584c2d6cd19fba3bc6cb17617a64e22e63117b7e28e2 Jun 03 16:19:49.169664 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:49.169629 2568 generic.go:358] "Generic (PLEG): container finished" podID="d48d8d35-b7ce-4450-b96a-7070ed55406c" containerID="3ed409d635a723272a5931b1949edce9dbbed77c529608cd3a408f3324cbb555" exitCode=0 Jun 03 16:19:49.170059 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:49.169702 2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d48d8d35-b7ce-4450-b96a-7070ed55406c","Type":"ContainerDied","Data":"3ed409d635a723272a5931b1949edce9dbbed77c529608cd3a408f3324cbb555"} Jun 03 16:19:49.170059 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:49.169725 2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d48d8d35-b7ce-4450-b96a-7070ed55406c","Type":"ContainerStarted","Data":"a812914c03b6a7b7beef584c2d6cd19fba3bc6cb17617a64e22e63117b7e28e2"} Jun 03 16:19:49.567062 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:49.566992 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="41ddc394-b5da-48e1-ba7b-fc2e9fdfff0b" path="/var/lib/kubelet/pods/41ddc394-b5da-48e1-ba7b-fc2e9fdfff0b/volumes" Jun 03 16:19:50.176387 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:50.176351 2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d48d8d35-b7ce-4450-b96a-7070ed55406c","Type":"ContainerStarted","Data":"ed0c4ea62cf5ce48414c5d9ceaccddb60894aff5292b7ea587e169d2dbe9dc4e"} Jun 03 16:19:50.176772 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:50.176393 2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d48d8d35-b7ce-4450-b96a-7070ed55406c","Type":"ContainerStarted","Data":"6780638578c30fbcbd55f43f94016a41cb48ba99b51e928e406e7f3396ffffa5"} Jun 03 16:19:50.176772 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:50.176408 2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d48d8d35-b7ce-4450-b96a-7070ed55406c","Type":"ContainerStarted","Data":"99a7bace601418fcaac758df46329239511c92454a5cb9cded9d7b7a178ab684"} Jun 03 16:19:50.176772 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:50.176420 2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d48d8d35-b7ce-4450-b96a-7070ed55406c","Type":"ContainerStarted","Data":"a22ed9868b324689dd9f81ef1834fc0c1e01ca25634ffc455ba047dae7935db1"} Jun 03 16:19:50.176772 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:50.176432 2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d48d8d35-b7ce-4450-b96a-7070ed55406c","Type":"ContainerStarted","Data":"e56777eb5510f8a1b2e54961af98bbb79b87b141bb80872b6429b733f239fbba"} Jun 03 16:19:50.176772 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:50.176444 2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d48d8d35-b7ce-4450-b96a-7070ed55406c","Type":"ContainerStarted","Data":"6a1612f997568c167a40f0050f4b943cf33277c3eaba6c92e9224df30d57495b"} Jun 03 16:19:50.204490 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:50.204389 2568 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.204373819 podStartE2EDuration="2.204373819s" podCreationTimestamp="2026-06-03 16:19:48 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-06-03 16:19:50.202265312 +0000 UTC m=+161.290691901" watchObservedRunningTime="2026-06-03 16:19:50.204373819 +0000 UTC m=+161.292800408" Jun 03 16:19:50.893641 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:50.893603 2568 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-8586f67987-tdnvq"] Jun 03 16:19:50.896239 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:50.896220 2568 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-8586f67987-tdnvq" Jun 03 16:19:50.899241 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:50.898709 2568 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-ntdzk\"" Jun 03 16:19:50.899241 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:50.898954 2568 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\"" Jun 03 16:19:50.899407 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:50.899262 2568 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\"" Jun 03 16:19:50.899785 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:50.899658 2568 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\"" Jun 03 16:19:50.899785 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:50.899657 2568 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\"" Jun 03 16:19:50.899942 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:50.899811 2568 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\"" Jun 03 16:19:50.903757 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:50.903730 2568 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\"" Jun 03 16:19:50.908164 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:50.908135 2568 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-8586f67987-tdnvq"] Jun 03 16:19:50.989828 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:50.989793 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/32565e44-36b2-449c-983a-829acd788137-serving-certs-ca-bundle\") pod \"telemeter-client-8586f67987-tdnvq\" (UID: \"32565e44-36b2-449c-983a-829acd788137\") " pod="openshift-monitoring/telemeter-client-8586f67987-tdnvq" Jun 03 16:19:50.989828 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:50.989834 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/32565e44-36b2-449c-983a-829acd788137-federate-client-tls\") pod \"telemeter-client-8586f67987-tdnvq\" (UID: \"32565e44-36b2-449c-983a-829acd788137\") " pod="openshift-monitoring/telemeter-client-8586f67987-tdnvq" Jun 03 16:19:50.990034 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:50.989867 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/32565e44-36b2-449c-983a-829acd788137-telemeter-trusted-ca-bundle\") pod \"telemeter-client-8586f67987-tdnvq\" (UID: \"32565e44-36b2-449c-983a-829acd788137\") " pod="openshift-monitoring/telemeter-client-8586f67987-tdnvq" Jun 03 16:19:50.990034 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:50.989921 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/32565e44-36b2-449c-983a-829acd788137-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-8586f67987-tdnvq\" (UID: \"32565e44-36b2-449c-983a-829acd788137\") " pod="openshift-monitoring/telemeter-client-8586f67987-tdnvq" Jun 03 16:19:50.990034 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:50.989951 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jb4zs\" (UniqueName: \"kubernetes.io/projected/32565e44-36b2-449c-983a-829acd788137-kube-api-access-jb4zs\") pod \"telemeter-client-8586f67987-tdnvq\" (UID: \"32565e44-36b2-449c-983a-829acd788137\") " pod="openshift-monitoring/telemeter-client-8586f67987-tdnvq" Jun 03 16:19:50.990034 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:50.989968 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/32565e44-36b2-449c-983a-829acd788137-metrics-client-ca\") pod \"telemeter-client-8586f67987-tdnvq\" (UID: \"32565e44-36b2-449c-983a-829acd788137\") " pod="openshift-monitoring/telemeter-client-8586f67987-tdnvq" Jun 03 16:19:50.990034 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:50.989989 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/32565e44-36b2-449c-983a-829acd788137-secret-telemeter-client\") pod \"telemeter-client-8586f67987-tdnvq\" (UID: \"32565e44-36b2-449c-983a-829acd788137\") " pod="openshift-monitoring/telemeter-client-8586f67987-tdnvq" Jun 03 16:19:50.990231 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:50.990051 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/32565e44-36b2-449c-983a-829acd788137-telemeter-client-tls\") pod \"telemeter-client-8586f67987-tdnvq\" (UID: \"32565e44-36b2-449c-983a-829acd788137\") " pod="openshift-monitoring/telemeter-client-8586f67987-tdnvq" Jun 03 16:19:51.090872 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:51.090837 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/32565e44-36b2-449c-983a-829acd788137-metrics-client-ca\") pod \"telemeter-client-8586f67987-tdnvq\" (UID: \"32565e44-36b2-449c-983a-829acd788137\") " pod="openshift-monitoring/telemeter-client-8586f67987-tdnvq" Jun 03 16:19:51.090872 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:51.090875 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/32565e44-36b2-449c-983a-829acd788137-secret-telemeter-client\") pod \"telemeter-client-8586f67987-tdnvq\" (UID: \"32565e44-36b2-449c-983a-829acd788137\") " pod="openshift-monitoring/telemeter-client-8586f67987-tdnvq" Jun 03 16:19:51.091068 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:51.090907 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/32565e44-36b2-449c-983a-829acd788137-telemeter-client-tls\") pod \"telemeter-client-8586f67987-tdnvq\" (UID: \"32565e44-36b2-449c-983a-829acd788137\") " pod="openshift-monitoring/telemeter-client-8586f67987-tdnvq" Jun 03 16:19:51.091068 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:51.090940 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/32565e44-36b2-449c-983a-829acd788137-serving-certs-ca-bundle\") pod \"telemeter-client-8586f67987-tdnvq\" (UID: \"32565e44-36b2-449c-983a-829acd788137\") " pod="openshift-monitoring/telemeter-client-8586f67987-tdnvq" Jun 03 16:19:51.091068 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:51.090956 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/32565e44-36b2-449c-983a-829acd788137-federate-client-tls\") pod \"telemeter-client-8586f67987-tdnvq\" (UID: \"32565e44-36b2-449c-983a-829acd788137\") " pod="openshift-monitoring/telemeter-client-8586f67987-tdnvq" Jun 03 16:19:51.091068 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:51.090974 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/32565e44-36b2-449c-983a-829acd788137-telemeter-trusted-ca-bundle\") pod \"telemeter-client-8586f67987-tdnvq\" (UID: \"32565e44-36b2-449c-983a-829acd788137\") " pod="openshift-monitoring/telemeter-client-8586f67987-tdnvq" Jun 03 16:19:51.091068 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:51.091002 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/32565e44-36b2-449c-983a-829acd788137-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-8586f67987-tdnvq\" (UID: \"32565e44-36b2-449c-983a-829acd788137\") " pod="openshift-monitoring/telemeter-client-8586f67987-tdnvq" Jun 03 16:19:51.091068 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:51.091063 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-jb4zs\" (UniqueName: \"kubernetes.io/projected/32565e44-36b2-449c-983a-829acd788137-kube-api-access-jb4zs\") pod \"telemeter-client-8586f67987-tdnvq\" (UID: \"32565e44-36b2-449c-983a-829acd788137\") " pod="openshift-monitoring/telemeter-client-8586f67987-tdnvq" Jun 03 16:19:51.091748 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:51.091719 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/32565e44-36b2-449c-983a-829acd788137-serving-certs-ca-bundle\") pod \"telemeter-client-8586f67987-tdnvq\" (UID: \"32565e44-36b2-449c-983a-829acd788137\") " pod="openshift-monitoring/telemeter-client-8586f67987-tdnvq" Jun 03 16:19:51.091748 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:51.091732 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/32565e44-36b2-449c-983a-829acd788137-metrics-client-ca\") pod \"telemeter-client-8586f67987-tdnvq\" (UID: \"32565e44-36b2-449c-983a-829acd788137\") " pod="openshift-monitoring/telemeter-client-8586f67987-tdnvq" Jun 03 16:19:51.092074 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:51.092055 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/32565e44-36b2-449c-983a-829acd788137-telemeter-trusted-ca-bundle\") pod \"telemeter-client-8586f67987-tdnvq\" (UID: \"32565e44-36b2-449c-983a-829acd788137\") " pod="openshift-monitoring/telemeter-client-8586f67987-tdnvq" Jun 03 16:19:51.093459 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:51.093430 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/32565e44-36b2-449c-983a-829acd788137-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-8586f67987-tdnvq\" (UID: \"32565e44-36b2-449c-983a-829acd788137\") " pod="openshift-monitoring/telemeter-client-8586f67987-tdnvq" Jun 03 16:19:51.093571 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:51.093482 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/32565e44-36b2-449c-983a-829acd788137-telemeter-client-tls\") pod \"telemeter-client-8586f67987-tdnvq\" (UID: \"32565e44-36b2-449c-983a-829acd788137\") " pod="openshift-monitoring/telemeter-client-8586f67987-tdnvq" Jun 03 16:19:51.093804 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:51.093787 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/32565e44-36b2-449c-983a-829acd788137-federate-client-tls\") pod \"telemeter-client-8586f67987-tdnvq\" (UID: \"32565e44-36b2-449c-983a-829acd788137\") " pod="openshift-monitoring/telemeter-client-8586f67987-tdnvq" Jun 03 16:19:51.093880 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:51.093818 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/32565e44-36b2-449c-983a-829acd788137-secret-telemeter-client\") pod \"telemeter-client-8586f67987-tdnvq\" (UID: \"32565e44-36b2-449c-983a-829acd788137\") " pod="openshift-monitoring/telemeter-client-8586f67987-tdnvq" Jun 03 16:19:51.100034 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:51.100013 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jb4zs\" (UniqueName: \"kubernetes.io/projected/32565e44-36b2-449c-983a-829acd788137-kube-api-access-jb4zs\") pod \"telemeter-client-8586f67987-tdnvq\" (UID: \"32565e44-36b2-449c-983a-829acd788137\") " pod="openshift-monitoring/telemeter-client-8586f67987-tdnvq" Jun 03 16:19:51.209332 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:51.209261 2568 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-8586f67987-tdnvq" Jun 03 16:19:51.360210 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:51.360179 2568 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-8586f67987-tdnvq"] Jun 03 16:19:51.363493 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:19:51.363466 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod32565e44_36b2_449c_983a_829acd788137.slice/crio-8495afc5da9a99eb01e030f37b3b0289967decdc55ae1060d44b7622b3c485f2 WatchSource:0}: Error finding container 8495afc5da9a99eb01e030f37b3b0289967decdc55ae1060d44b7622b3c485f2: Status 404 returned error can't find the container with id 8495afc5da9a99eb01e030f37b3b0289967decdc55ae1060d44b7622b3c485f2 Jun 03 16:19:52.185571 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:52.185514 2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-8586f67987-tdnvq" event={"ID":"32565e44-36b2-449c-983a-829acd788137","Type":"ContainerStarted","Data":"8495afc5da9a99eb01e030f37b3b0289967decdc55ae1060d44b7622b3c485f2"} Jun 03 16:19:53.189601 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:53.189567 2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-8586f67987-tdnvq" event={"ID":"32565e44-36b2-449c-983a-829acd788137","Type":"ContainerStarted","Data":"7627e72e9cc94e64fed22d8dac5b0bd29b2ab77af562687ecae83df49d11edf3"} Jun 03 16:19:53.189884 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:53.189607 2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-8586f67987-tdnvq" event={"ID":"32565e44-36b2-449c-983a-829acd788137","Type":"ContainerStarted","Data":"1fd3d35c81ccb4f64b644414833d1d073fb2cf68d556da08f90ab80050014a1d"} Jun 03 16:19:54.194747 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:54.194710 2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-8586f67987-tdnvq" event={"ID":"32565e44-36b2-449c-983a-829acd788137","Type":"ContainerStarted","Data":"553bb44c23707c28ca1a9aa61458dc3e49565b75f9e096a76852889e267630be"} Jun 03 16:19:54.217052 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:54.216994 2568 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-8586f67987-tdnvq" podStartSLOduration=2.536435916 podStartE2EDuration="4.21697966s" podCreationTimestamp="2026-06-03 16:19:50 +0000 UTC" firstStartedPulling="2026-06-03 16:19:51.365419511 +0000 UTC m=+162.453846082" lastFinishedPulling="2026-06-03 16:19:53.045963255 +0000 UTC m=+164.134389826" observedRunningTime="2026-06-03 16:19:54.214721919 +0000 UTC m=+165.303148508" watchObservedRunningTime="2026-06-03 16:19:54.21697966 +0000 UTC m=+165.305406249" Jun 03 16:19:54.904682 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:54.904649 2568 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-console/console-69cf65c77-rbc5g"] Jun 03 16:19:54.908458 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:54.908435 2568 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-69cf65c77-rbc5g" Jun 03 16:19:54.919730 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:54.919705 2568 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-69cf65c77-rbc5g"] Jun 03 16:19:55.022826 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:55.022787 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3b738fe3-8e45-423d-b2e7-f489028311d9-console-serving-cert\") pod \"console-69cf65c77-rbc5g\" (UID: \"3b738fe3-8e45-423d-b2e7-f489028311d9\") " pod="openshift-console/console-69cf65c77-rbc5g" Jun 03 16:19:55.023000 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:55.022882 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3b738fe3-8e45-423d-b2e7-f489028311d9-service-ca\") pod \"console-69cf65c77-rbc5g\" (UID: \"3b738fe3-8e45-423d-b2e7-f489028311d9\") " pod="openshift-console/console-69cf65c77-rbc5g" Jun 03 16:19:55.023000 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:55.022927 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3b738fe3-8e45-423d-b2e7-f489028311d9-console-config\") pod \"console-69cf65c77-rbc5g\" (UID: \"3b738fe3-8e45-423d-b2e7-f489028311d9\") " pod="openshift-console/console-69cf65c77-rbc5g" Jun 03 16:19:55.023000 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:55.022951 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3b738fe3-8e45-423d-b2e7-f489028311d9-console-oauth-config\") pod \"console-69cf65c77-rbc5g\" (UID: \"3b738fe3-8e45-423d-b2e7-f489028311d9\") " pod="openshift-console/console-69cf65c77-rbc5g" Jun 03 16:19:55.023000 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:55.022978 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3b738fe3-8e45-423d-b2e7-f489028311d9-oauth-serving-cert\") pod \"console-69cf65c77-rbc5g\" (UID: \"3b738fe3-8e45-423d-b2e7-f489028311d9\") " pod="openshift-console/console-69cf65c77-rbc5g" Jun 03 16:19:55.023000 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:55.022999 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h5j2w\" (UniqueName: \"kubernetes.io/projected/3b738fe3-8e45-423d-b2e7-f489028311d9-kube-api-access-h5j2w\") pod \"console-69cf65c77-rbc5g\" (UID: \"3b738fe3-8e45-423d-b2e7-f489028311d9\") " pod="openshift-console/console-69cf65c77-rbc5g" Jun 03 16:19:55.023271 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:55.023058 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3b738fe3-8e45-423d-b2e7-f489028311d9-trusted-ca-bundle\") pod \"console-69cf65c77-rbc5g\" (UID: \"3b738fe3-8e45-423d-b2e7-f489028311d9\") " pod="openshift-console/console-69cf65c77-rbc5g" Jun 03 16:19:55.123857 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:55.123822 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3b738fe3-8e45-423d-b2e7-f489028311d9-trusted-ca-bundle\") pod \"console-69cf65c77-rbc5g\" (UID: \"3b738fe3-8e45-423d-b2e7-f489028311d9\") " pod="openshift-console/console-69cf65c77-rbc5g" Jun 03 16:19:55.124007 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:55.123871 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3b738fe3-8e45-423d-b2e7-f489028311d9-console-serving-cert\") pod \"console-69cf65c77-rbc5g\" (UID: \"3b738fe3-8e45-423d-b2e7-f489028311d9\") " pod="openshift-console/console-69cf65c77-rbc5g" Jun 03 16:19:55.124007 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:55.123901 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3b738fe3-8e45-423d-b2e7-f489028311d9-service-ca\") pod \"console-69cf65c77-rbc5g\" (UID: \"3b738fe3-8e45-423d-b2e7-f489028311d9\") " pod="openshift-console/console-69cf65c77-rbc5g" Jun 03 16:19:55.124007 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:55.123923 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3b738fe3-8e45-423d-b2e7-f489028311d9-console-config\") pod \"console-69cf65c77-rbc5g\" (UID: \"3b738fe3-8e45-423d-b2e7-f489028311d9\") " pod="openshift-console/console-69cf65c77-rbc5g" Jun 03 16:19:55.124007 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:55.123945 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3b738fe3-8e45-423d-b2e7-f489028311d9-console-oauth-config\") pod \"console-69cf65c77-rbc5g\" (UID: \"3b738fe3-8e45-423d-b2e7-f489028311d9\") " pod="openshift-console/console-69cf65c77-rbc5g" Jun 03 16:19:55.124007 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:55.123993 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3b738fe3-8e45-423d-b2e7-f489028311d9-oauth-serving-cert\") pod \"console-69cf65c77-rbc5g\" (UID: \"3b738fe3-8e45-423d-b2e7-f489028311d9\") " pod="openshift-console/console-69cf65c77-rbc5g" Jun 03 16:19:55.124262 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:55.124024 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-h5j2w\" (UniqueName: \"kubernetes.io/projected/3b738fe3-8e45-423d-b2e7-f489028311d9-kube-api-access-h5j2w\") pod \"console-69cf65c77-rbc5g\" (UID: \"3b738fe3-8e45-423d-b2e7-f489028311d9\") " pod="openshift-console/console-69cf65c77-rbc5g" Jun 03 16:19:55.124825 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:55.124750 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3b738fe3-8e45-423d-b2e7-f489028311d9-service-ca\") pod \"console-69cf65c77-rbc5g\" (UID: \"3b738fe3-8e45-423d-b2e7-f489028311d9\") " pod="openshift-console/console-69cf65c77-rbc5g" Jun 03 16:19:55.124825 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:55.124783 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3b738fe3-8e45-423d-b2e7-f489028311d9-trusted-ca-bundle\") pod \"console-69cf65c77-rbc5g\" (UID: \"3b738fe3-8e45-423d-b2e7-f489028311d9\") " pod="openshift-console/console-69cf65c77-rbc5g" Jun 03 16:19:55.124825 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:55.124764 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3b738fe3-8e45-423d-b2e7-f489028311d9-console-config\") pod \"console-69cf65c77-rbc5g\" (UID: \"3b738fe3-8e45-423d-b2e7-f489028311d9\") " pod="openshift-console/console-69cf65c77-rbc5g" Jun 03 16:19:55.125063 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:55.124919 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3b738fe3-8e45-423d-b2e7-f489028311d9-oauth-serving-cert\") pod \"console-69cf65c77-rbc5g\" (UID: \"3b738fe3-8e45-423d-b2e7-f489028311d9\") " pod="openshift-console/console-69cf65c77-rbc5g" Jun 03 16:19:55.127044 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:55.127017 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3b738fe3-8e45-423d-b2e7-f489028311d9-console-oauth-config\") pod \"console-69cf65c77-rbc5g\" (UID: \"3b738fe3-8e45-423d-b2e7-f489028311d9\") " pod="openshift-console/console-69cf65c77-rbc5g" Jun 03 16:19:55.127157 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:55.127120 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3b738fe3-8e45-423d-b2e7-f489028311d9-console-serving-cert\") pod \"console-69cf65c77-rbc5g\" (UID: \"3b738fe3-8e45-423d-b2e7-f489028311d9\") " pod="openshift-console/console-69cf65c77-rbc5g" Jun 03 16:19:55.136135 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:55.136116 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h5j2w\" (UniqueName: \"kubernetes.io/projected/3b738fe3-8e45-423d-b2e7-f489028311d9-kube-api-access-h5j2w\") pod \"console-69cf65c77-rbc5g\" (UID: \"3b738fe3-8e45-423d-b2e7-f489028311d9\") " pod="openshift-console/console-69cf65c77-rbc5g" Jun 03 16:19:55.219391 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:55.219304 2568 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-69cf65c77-rbc5g" Jun 03 16:19:55.347044 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:55.347013 2568 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-69cf65c77-rbc5g"] Jun 03 16:19:55.349790 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:19:55.349766 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3b738fe3_8e45_423d_b2e7_f489028311d9.slice/crio-4bbfec920012073c84e47e0a24bcab5c0e75486bcf5f3fd7063290c004c1c459 WatchSource:0}: Error finding container 4bbfec920012073c84e47e0a24bcab5c0e75486bcf5f3fd7063290c004c1c459: Status 404 returned error can't find the container with id 4bbfec920012073c84e47e0a24bcab5c0e75486bcf5f3fd7063290c004c1c459 Jun 03 16:19:56.201865 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:56.201827 2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-69cf65c77-rbc5g" event={"ID":"3b738fe3-8e45-423d-b2e7-f489028311d9","Type":"ContainerStarted","Data":"50e7afc4746adaaa064daa6e1b7e79b87cc9caddca24945a076e132e135acae3"} Jun 03 16:19:56.201865 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:56.201867 2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-69cf65c77-rbc5g" event={"ID":"3b738fe3-8e45-423d-b2e7-f489028311d9","Type":"ContainerStarted","Data":"4bbfec920012073c84e47e0a24bcab5c0e75486bcf5f3fd7063290c004c1c459"} Jun 03 16:19:56.222430 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:19:56.222381 2568 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-console/console-69cf65c77-rbc5g" podStartSLOduration=2.222368278 podStartE2EDuration="2.222368278s" podCreationTimestamp="2026-06-03 16:19:54 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-06-03 16:19:56.221672987 +0000 UTC m=+167.310099577" watchObservedRunningTime="2026-06-03 16:19:56.222368278 +0000 UTC m=+167.310794867" Jun 03 16:20:05.220167 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:20:05.220131 2568 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-69cf65c77-rbc5g" Jun 03 16:20:05.220522 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:20:05.220179 2568 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-69cf65c77-rbc5g" Jun 03 16:20:05.224942 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:20:05.224913 2568 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-69cf65c77-rbc5g" Jun 03 16:20:05.229215 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:20:05.229193 2568 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-69cf65c77-rbc5g" Jun 03 16:20:05.292956 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:20:05.292928 2568 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6c7fc6d78c-h7rmn"] Jun 03 16:20:30.312300 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:20:30.312246 2568 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-console/console-6c7fc6d78c-h7rmn" podUID="1f4b8d97-0852-40af-a85b-601aebc7c56f" containerName="console" containerID="cri-o://78305170c18904e9bb4903ddbd25e7b0c49142e926c7eecb1f5695cd90becad1" gracePeriod=15 Jun 03 16:20:30.547258 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:20:30.547238 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-6c7fc6d78c-h7rmn_1f4b8d97-0852-40af-a85b-601aebc7c56f/console/0.log" Jun 03 16:20:30.547362 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:20:30.547296 2568 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6c7fc6d78c-h7rmn" Jun 03 16:20:30.705051 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:20:30.705019 2568 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1f4b8d97-0852-40af-a85b-601aebc7c56f-console-oauth-config\") pod \"1f4b8d97-0852-40af-a85b-601aebc7c56f\" (UID: \"1f4b8d97-0852-40af-a85b-601aebc7c56f\") " Jun 03 16:20:30.705051 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:20:30.705054 2568 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1f4b8d97-0852-40af-a85b-601aebc7c56f-console-config\") pod \"1f4b8d97-0852-40af-a85b-601aebc7c56f\" (UID: \"1f4b8d97-0852-40af-a85b-601aebc7c56f\") " Jun 03 16:20:30.705235 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:20:30.705081 2568 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1f4b8d97-0852-40af-a85b-601aebc7c56f-trusted-ca-bundle\") pod \"1f4b8d97-0852-40af-a85b-601aebc7c56f\" (UID: \"1f4b8d97-0852-40af-a85b-601aebc7c56f\") " Jun 03 16:20:30.705235 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:20:30.705222 2568 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1f4b8d97-0852-40af-a85b-601aebc7c56f-service-ca\") pod \"1f4b8d97-0852-40af-a85b-601aebc7c56f\" (UID: \"1f4b8d97-0852-40af-a85b-601aebc7c56f\") " Jun 03 16:20:30.705336 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:20:30.705260 2568 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1f4b8d97-0852-40af-a85b-601aebc7c56f-console-serving-cert\") pod \"1f4b8d97-0852-40af-a85b-601aebc7c56f\" (UID: \"1f4b8d97-0852-40af-a85b-601aebc7c56f\") " Jun 03 16:20:30.705848 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:20:30.705417 2568 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1f4b8d97-0852-40af-a85b-601aebc7c56f-oauth-serving-cert\") pod \"1f4b8d97-0852-40af-a85b-601aebc7c56f\" (UID: \"1f4b8d97-0852-40af-a85b-601aebc7c56f\") " Jun 03 16:20:30.705848 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:20:30.705487 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1f4b8d97-0852-40af-a85b-601aebc7c56f-console-config" (OuterVolumeSpecName: "console-config") pod "1f4b8d97-0852-40af-a85b-601aebc7c56f" (UID: "1f4b8d97-0852-40af-a85b-601aebc7c56f"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 03 16:20:30.705848 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:20:30.705505 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1f4b8d97-0852-40af-a85b-601aebc7c56f-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "1f4b8d97-0852-40af-a85b-601aebc7c56f" (UID: "1f4b8d97-0852-40af-a85b-601aebc7c56f"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 03 16:20:30.705848 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:20:30.705530 2568 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-c77z6\" (UniqueName: \"kubernetes.io/projected/1f4b8d97-0852-40af-a85b-601aebc7c56f-kube-api-access-c77z6\") pod \"1f4b8d97-0852-40af-a85b-601aebc7c56f\" (UID: \"1f4b8d97-0852-40af-a85b-601aebc7c56f\") " Jun 03 16:20:30.705848 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:20:30.705589 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1f4b8d97-0852-40af-a85b-601aebc7c56f-service-ca" (OuterVolumeSpecName: "service-ca") pod "1f4b8d97-0852-40af-a85b-601aebc7c56f" (UID: "1f4b8d97-0852-40af-a85b-601aebc7c56f"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 03 16:20:30.705848 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:20:30.705789 2568 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1f4b8d97-0852-40af-a85b-601aebc7c56f-trusted-ca-bundle\") on node \"ip-10-0-128-88.ec2.internal\" DevicePath \"\"" Jun 03 16:20:30.705848 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:20:30.705810 2568 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1f4b8d97-0852-40af-a85b-601aebc7c56f-service-ca\") on node \"ip-10-0-128-88.ec2.internal\" DevicePath \"\"" Jun 03 16:20:30.705848 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:20:30.705833 2568 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1f4b8d97-0852-40af-a85b-601aebc7c56f-console-config\") on node \"ip-10-0-128-88.ec2.internal\" DevicePath \"\"" Jun 03 16:20:30.705848 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:20:30.705803 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1f4b8d97-0852-40af-a85b-601aebc7c56f-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "1f4b8d97-0852-40af-a85b-601aebc7c56f" (UID: "1f4b8d97-0852-40af-a85b-601aebc7c56f"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 03 16:20:30.707252 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:20:30.707230 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1f4b8d97-0852-40af-a85b-601aebc7c56f-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "1f4b8d97-0852-40af-a85b-601aebc7c56f" (UID: "1f4b8d97-0852-40af-a85b-601aebc7c56f"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 03 16:20:30.707356 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:20:30.707248 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1f4b8d97-0852-40af-a85b-601aebc7c56f-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "1f4b8d97-0852-40af-a85b-601aebc7c56f" (UID: "1f4b8d97-0852-40af-a85b-601aebc7c56f"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 03 16:20:30.707356 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:20:30.707336 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1f4b8d97-0852-40af-a85b-601aebc7c56f-kube-api-access-c77z6" (OuterVolumeSpecName: "kube-api-access-c77z6") pod "1f4b8d97-0852-40af-a85b-601aebc7c56f" (UID: "1f4b8d97-0852-40af-a85b-601aebc7c56f"). InnerVolumeSpecName "kube-api-access-c77z6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 03 16:20:30.806957 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:20:30.806931 2568 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1f4b8d97-0852-40af-a85b-601aebc7c56f-console-serving-cert\") on node \"ip-10-0-128-88.ec2.internal\" DevicePath \"\"" Jun 03 16:20:30.806957 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:20:30.806954 2568 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1f4b8d97-0852-40af-a85b-601aebc7c56f-oauth-serving-cert\") on node \"ip-10-0-128-88.ec2.internal\" DevicePath \"\"" Jun 03 16:20:30.807112 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:20:30.806964 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c77z6\" (UniqueName: \"kubernetes.io/projected/1f4b8d97-0852-40af-a85b-601aebc7c56f-kube-api-access-c77z6\") on node \"ip-10-0-128-88.ec2.internal\" DevicePath \"\"" Jun 03 16:20:30.807112 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:20:30.806973 2568 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1f4b8d97-0852-40af-a85b-601aebc7c56f-console-oauth-config\") on node \"ip-10-0-128-88.ec2.internal\" DevicePath \"\"" Jun 03 16:20:31.301996 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:20:31.301968 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-6c7fc6d78c-h7rmn_1f4b8d97-0852-40af-a85b-601aebc7c56f/console/0.log" Jun 03 16:20:31.302151 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:20:31.302006 2568 generic.go:358] "Generic (PLEG): container finished" podID="1f4b8d97-0852-40af-a85b-601aebc7c56f" containerID="78305170c18904e9bb4903ddbd25e7b0c49142e926c7eecb1f5695cd90becad1" exitCode=2 Jun 03 16:20:31.302151 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:20:31.302033 2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6c7fc6d78c-h7rmn" event={"ID":"1f4b8d97-0852-40af-a85b-601aebc7c56f","Type":"ContainerDied","Data":"78305170c18904e9bb4903ddbd25e7b0c49142e926c7eecb1f5695cd90becad1"} Jun 03 16:20:31.302151 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:20:31.302068 2568 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6c7fc6d78c-h7rmn" Jun 03 16:20:31.302151 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:20:31.302082 2568 scope.go:117] "RemoveContainer" containerID="78305170c18904e9bb4903ddbd25e7b0c49142e926c7eecb1f5695cd90becad1" Jun 03 16:20:31.302151 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:20:31.302070 2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6c7fc6d78c-h7rmn" event={"ID":"1f4b8d97-0852-40af-a85b-601aebc7c56f","Type":"ContainerDied","Data":"d1500fc0c486c8924261fce07e9b7a342fc51086c232095b7cd70b1d7d47c12a"} Jun 03 16:20:31.310020 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:20:31.309999 2568 scope.go:117] "RemoveContainer" containerID="78305170c18904e9bb4903ddbd25e7b0c49142e926c7eecb1f5695cd90becad1" Jun 03 16:20:31.310264 ip-10-0-128-88 kubenswrapper[2568]: E0603 16:20:31.310247 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"78305170c18904e9bb4903ddbd25e7b0c49142e926c7eecb1f5695cd90becad1\": container with ID starting with 78305170c18904e9bb4903ddbd25e7b0c49142e926c7eecb1f5695cd90becad1 not found: ID does not exist" containerID="78305170c18904e9bb4903ddbd25e7b0c49142e926c7eecb1f5695cd90becad1" Jun 03 16:20:31.310331 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:20:31.310274 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"78305170c18904e9bb4903ddbd25e7b0c49142e926c7eecb1f5695cd90becad1"} err="failed to get container status \"78305170c18904e9bb4903ddbd25e7b0c49142e926c7eecb1f5695cd90becad1\": rpc error: code = NotFound desc = could not find container \"78305170c18904e9bb4903ddbd25e7b0c49142e926c7eecb1f5695cd90becad1\": container with ID starting with 78305170c18904e9bb4903ddbd25e7b0c49142e926c7eecb1f5695cd90becad1 not found: ID does not exist" Jun 03 16:20:31.326036 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:20:31.326016 2568 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6c7fc6d78c-h7rmn"] Jun 03 16:20:31.332808 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:20:31.332789 2568 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-6c7fc6d78c-h7rmn"] Jun 03 16:20:31.564003 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:20:31.563928 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1f4b8d97-0852-40af-a85b-601aebc7c56f" path="/var/lib/kubelet/pods/1f4b8d97-0852-40af-a85b-601aebc7c56f/volumes" Jun 03 16:21:01.697238 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:21:01.697200 2568 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-console/console-df89b8f67-6rjh2"] Jun 03 16:21:01.697704 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:21:01.697501 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1f4b8d97-0852-40af-a85b-601aebc7c56f" containerName="console" Jun 03 16:21:01.697704 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:21:01.697513 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="1f4b8d97-0852-40af-a85b-601aebc7c56f" containerName="console" Jun 03 16:21:01.697704 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:21:01.697579 2568 memory_manager.go:356] "RemoveStaleState removing state" podUID="1f4b8d97-0852-40af-a85b-601aebc7c56f" containerName="console" Jun 03 16:21:01.700653 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:21:01.700638 2568 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-df89b8f67-6rjh2" Jun 03 16:21:01.711213 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:21:01.711189 2568 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-df89b8f67-6rjh2"] Jun 03 16:21:01.737951 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:21:01.737922 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/62ec1bd3-2f2f-4c73-b966-6e5f676ed79b-console-oauth-config\") pod \"console-df89b8f67-6rjh2\" (UID: \"62ec1bd3-2f2f-4c73-b966-6e5f676ed79b\") " pod="openshift-console/console-df89b8f67-6rjh2" Jun 03 16:21:01.737951 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:21:01.737956 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/62ec1bd3-2f2f-4c73-b966-6e5f676ed79b-service-ca\") pod \"console-df89b8f67-6rjh2\" (UID: \"62ec1bd3-2f2f-4c73-b966-6e5f676ed79b\") " pod="openshift-console/console-df89b8f67-6rjh2" Jun 03 16:21:01.738105 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:21:01.737977 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/62ec1bd3-2f2f-4c73-b966-6e5f676ed79b-oauth-serving-cert\") pod \"console-df89b8f67-6rjh2\" (UID: \"62ec1bd3-2f2f-4c73-b966-6e5f676ed79b\") " pod="openshift-console/console-df89b8f67-6rjh2" Jun 03 16:21:01.738105 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:21:01.737993 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/62ec1bd3-2f2f-4c73-b966-6e5f676ed79b-console-config\") pod \"console-df89b8f67-6rjh2\" (UID: \"62ec1bd3-2f2f-4c73-b966-6e5f676ed79b\") " pod="openshift-console/console-df89b8f67-6rjh2" Jun 03 16:21:01.738105 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:21:01.738033 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wxmrn\" (UniqueName: \"kubernetes.io/projected/62ec1bd3-2f2f-4c73-b966-6e5f676ed79b-kube-api-access-wxmrn\") pod \"console-df89b8f67-6rjh2\" (UID: \"62ec1bd3-2f2f-4c73-b966-6e5f676ed79b\") " pod="openshift-console/console-df89b8f67-6rjh2" Jun 03 16:21:01.738105 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:21:01.738078 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/62ec1bd3-2f2f-4c73-b966-6e5f676ed79b-console-serving-cert\") pod \"console-df89b8f67-6rjh2\" (UID: \"62ec1bd3-2f2f-4c73-b966-6e5f676ed79b\") " pod="openshift-console/console-df89b8f67-6rjh2" Jun 03 16:21:01.738105 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:21:01.738103 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/62ec1bd3-2f2f-4c73-b966-6e5f676ed79b-trusted-ca-bundle\") pod \"console-df89b8f67-6rjh2\" (UID: \"62ec1bd3-2f2f-4c73-b966-6e5f676ed79b\") " pod="openshift-console/console-df89b8f67-6rjh2" Jun 03 16:21:01.838633 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:21:01.838607 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/62ec1bd3-2f2f-4c73-b966-6e5f676ed79b-console-serving-cert\") pod \"console-df89b8f67-6rjh2\" (UID: \"62ec1bd3-2f2f-4c73-b966-6e5f676ed79b\") " pod="openshift-console/console-df89b8f67-6rjh2" Jun 03 16:21:01.838749 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:21:01.838643 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/62ec1bd3-2f2f-4c73-b966-6e5f676ed79b-trusted-ca-bundle\") pod \"console-df89b8f67-6rjh2\" (UID: \"62ec1bd3-2f2f-4c73-b966-6e5f676ed79b\") " pod="openshift-console/console-df89b8f67-6rjh2" Jun 03 16:21:01.838749 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:21:01.838686 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/62ec1bd3-2f2f-4c73-b966-6e5f676ed79b-console-oauth-config\") pod \"console-df89b8f67-6rjh2\" (UID: \"62ec1bd3-2f2f-4c73-b966-6e5f676ed79b\") " pod="openshift-console/console-df89b8f67-6rjh2" Jun 03 16:21:01.838749 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:21:01.838705 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/62ec1bd3-2f2f-4c73-b966-6e5f676ed79b-service-ca\") pod \"console-df89b8f67-6rjh2\" (UID: \"62ec1bd3-2f2f-4c73-b966-6e5f676ed79b\") " pod="openshift-console/console-df89b8f67-6rjh2" Jun 03 16:21:01.838749 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:21:01.838724 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/62ec1bd3-2f2f-4c73-b966-6e5f676ed79b-oauth-serving-cert\") pod \"console-df89b8f67-6rjh2\" (UID: \"62ec1bd3-2f2f-4c73-b966-6e5f676ed79b\") " pod="openshift-console/console-df89b8f67-6rjh2" Jun 03 16:21:01.838950 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:21:01.838750 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/62ec1bd3-2f2f-4c73-b966-6e5f676ed79b-console-config\") pod \"console-df89b8f67-6rjh2\" (UID: \"62ec1bd3-2f2f-4c73-b966-6e5f676ed79b\") " pod="openshift-console/console-df89b8f67-6rjh2" Jun 03 16:21:01.838950 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:21:01.838773 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-wxmrn\" (UniqueName: \"kubernetes.io/projected/62ec1bd3-2f2f-4c73-b966-6e5f676ed79b-kube-api-access-wxmrn\") pod \"console-df89b8f67-6rjh2\" (UID: \"62ec1bd3-2f2f-4c73-b966-6e5f676ed79b\") " pod="openshift-console/console-df89b8f67-6rjh2" Jun 03 16:21:01.839497 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:21:01.839469 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/62ec1bd3-2f2f-4c73-b966-6e5f676ed79b-service-ca\") pod \"console-df89b8f67-6rjh2\" (UID: \"62ec1bd3-2f2f-4c73-b966-6e5f676ed79b\") " pod="openshift-console/console-df89b8f67-6rjh2" Jun 03 16:21:01.839632 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:21:01.839518 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/62ec1bd3-2f2f-4c73-b966-6e5f676ed79b-oauth-serving-cert\") pod \"console-df89b8f67-6rjh2\" (UID: \"62ec1bd3-2f2f-4c73-b966-6e5f676ed79b\") " pod="openshift-console/console-df89b8f67-6rjh2" Jun 03 16:21:01.839691 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:21:01.839646 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/62ec1bd3-2f2f-4c73-b966-6e5f676ed79b-trusted-ca-bundle\") pod \"console-df89b8f67-6rjh2\" (UID: \"62ec1bd3-2f2f-4c73-b966-6e5f676ed79b\") " pod="openshift-console/console-df89b8f67-6rjh2" Jun 03 16:21:01.839736 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:21:01.839677 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/62ec1bd3-2f2f-4c73-b966-6e5f676ed79b-console-config\") pod \"console-df89b8f67-6rjh2\" (UID: \"62ec1bd3-2f2f-4c73-b966-6e5f676ed79b\") " pod="openshift-console/console-df89b8f67-6rjh2" Jun 03 16:21:01.841210 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:21:01.841187 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/62ec1bd3-2f2f-4c73-b966-6e5f676ed79b-console-oauth-config\") pod \"console-df89b8f67-6rjh2\" (UID: \"62ec1bd3-2f2f-4c73-b966-6e5f676ed79b\") " pod="openshift-console/console-df89b8f67-6rjh2" Jun 03 16:21:01.841292 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:21:01.841275 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/62ec1bd3-2f2f-4c73-b966-6e5f676ed79b-console-serving-cert\") pod \"console-df89b8f67-6rjh2\" (UID: \"62ec1bd3-2f2f-4c73-b966-6e5f676ed79b\") " pod="openshift-console/console-df89b8f67-6rjh2" Jun 03 16:21:01.846007 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:21:01.845985 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wxmrn\" (UniqueName: \"kubernetes.io/projected/62ec1bd3-2f2f-4c73-b966-6e5f676ed79b-kube-api-access-wxmrn\") pod \"console-df89b8f67-6rjh2\" (UID: \"62ec1bd3-2f2f-4c73-b966-6e5f676ed79b\") " pod="openshift-console/console-df89b8f67-6rjh2" Jun 03 16:21:02.010953 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:21:02.010867 2568 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-df89b8f67-6rjh2" Jun 03 16:21:02.136910 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:21:02.136880 2568 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-df89b8f67-6rjh2"] Jun 03 16:21:02.140318 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:21:02.140292 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod62ec1bd3_2f2f_4c73_b966_6e5f676ed79b.slice/crio-d9a2262827adf182289f27a9011955b04809a1c6e65dadabdef1f8bc39543bfe WatchSource:0}: Error finding container d9a2262827adf182289f27a9011955b04809a1c6e65dadabdef1f8bc39543bfe: Status 404 returned error can't find the container with id d9a2262827adf182289f27a9011955b04809a1c6e65dadabdef1f8bc39543bfe Jun 03 16:21:02.390902 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:21:02.390860 2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-df89b8f67-6rjh2" event={"ID":"62ec1bd3-2f2f-4c73-b966-6e5f676ed79b","Type":"ContainerStarted","Data":"fcd9e390262de7df2cd5f3981abb27f5f28cdc958abf07cdc732f04bf4e13277"} Jun 03 16:21:02.390902 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:21:02.390903 2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-df89b8f67-6rjh2" event={"ID":"62ec1bd3-2f2f-4c73-b966-6e5f676ed79b","Type":"ContainerStarted","Data":"d9a2262827adf182289f27a9011955b04809a1c6e65dadabdef1f8bc39543bfe"} Jun 03 16:21:02.409546 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:21:02.409499 2568 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-console/console-df89b8f67-6rjh2" podStartSLOduration=1.409484459 podStartE2EDuration="1.409484459s" podCreationTimestamp="2026-06-03 16:21:01 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-06-03 16:21:02.408127791 +0000 UTC m=+233.496554379" watchObservedRunningTime="2026-06-03 16:21:02.409484459 +0000 UTC m=+233.497911048" Jun 03 16:21:12.011093 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:21:12.011048 2568 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-df89b8f67-6rjh2" Jun 03 16:21:12.011093 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:21:12.011104 2568 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-df89b8f67-6rjh2" Jun 03 16:21:12.015595 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:21:12.015573 2568 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-df89b8f67-6rjh2" Jun 03 16:21:12.421335 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:21:12.421309 2568 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-df89b8f67-6rjh2" Jun 03 16:21:12.469911 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:21:12.469880 2568 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-console/console-69cf65c77-rbc5g"] Jun 03 16:21:37.491224 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:21:37.491138 2568 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-console/console-69cf65c77-rbc5g" podUID="3b738fe3-8e45-423d-b2e7-f489028311d9" containerName="console" containerID="cri-o://50e7afc4746adaaa064daa6e1b7e79b87cc9caddca24945a076e132e135acae3" gracePeriod=15 Jun 03 16:21:37.729721 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:21:37.729701 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-69cf65c77-rbc5g_3b738fe3-8e45-423d-b2e7-f489028311d9/console/0.log" Jun 03 16:21:37.729834 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:21:37.729760 2568 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-69cf65c77-rbc5g" Jun 03 16:21:37.816381 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:21:37.816312 2568 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3b738fe3-8e45-423d-b2e7-f489028311d9-oauth-serving-cert\") pod \"3b738fe3-8e45-423d-b2e7-f489028311d9\" (UID: \"3b738fe3-8e45-423d-b2e7-f489028311d9\") " Jun 03 16:21:37.816381 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:21:37.816360 2568 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3b738fe3-8e45-423d-b2e7-f489028311d9-console-oauth-config\") pod \"3b738fe3-8e45-423d-b2e7-f489028311d9\" (UID: \"3b738fe3-8e45-423d-b2e7-f489028311d9\") " Jun 03 16:21:37.816576 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:21:37.816383 2568 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h5j2w\" (UniqueName: \"kubernetes.io/projected/3b738fe3-8e45-423d-b2e7-f489028311d9-kube-api-access-h5j2w\") pod \"3b738fe3-8e45-423d-b2e7-f489028311d9\" (UID: \"3b738fe3-8e45-423d-b2e7-f489028311d9\") " Jun 03 16:21:37.816576 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:21:37.816404 2568 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3b738fe3-8e45-423d-b2e7-f489028311d9-console-config\") pod \"3b738fe3-8e45-423d-b2e7-f489028311d9\" (UID: \"3b738fe3-8e45-423d-b2e7-f489028311d9\") " Jun 03 16:21:37.816576 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:21:37.816426 2568 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3b738fe3-8e45-423d-b2e7-f489028311d9-service-ca\") pod \"3b738fe3-8e45-423d-b2e7-f489028311d9\" (UID: \"3b738fe3-8e45-423d-b2e7-f489028311d9\") " Jun 03 16:21:37.816576 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:21:37.816442 2568 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3b738fe3-8e45-423d-b2e7-f489028311d9-console-serving-cert\") pod \"3b738fe3-8e45-423d-b2e7-f489028311d9\" (UID: \"3b738fe3-8e45-423d-b2e7-f489028311d9\") " Jun 03 16:21:37.816576 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:21:37.816496 2568 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3b738fe3-8e45-423d-b2e7-f489028311d9-trusted-ca-bundle\") pod \"3b738fe3-8e45-423d-b2e7-f489028311d9\" (UID: \"3b738fe3-8e45-423d-b2e7-f489028311d9\") " Jun 03 16:21:37.816838 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:21:37.816811 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3b738fe3-8e45-423d-b2e7-f489028311d9-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "3b738fe3-8e45-423d-b2e7-f489028311d9" (UID: "3b738fe3-8e45-423d-b2e7-f489028311d9"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 03 16:21:37.816904 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:21:37.816870 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3b738fe3-8e45-423d-b2e7-f489028311d9-service-ca" (OuterVolumeSpecName: "service-ca") pod "3b738fe3-8e45-423d-b2e7-f489028311d9" (UID: "3b738fe3-8e45-423d-b2e7-f489028311d9"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 03 16:21:37.816975 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:21:37.816944 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3b738fe3-8e45-423d-b2e7-f489028311d9-console-config" (OuterVolumeSpecName: "console-config") pod "3b738fe3-8e45-423d-b2e7-f489028311d9" (UID: "3b738fe3-8e45-423d-b2e7-f489028311d9"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 03 16:21:37.817017 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:21:37.816975 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3b738fe3-8e45-423d-b2e7-f489028311d9-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "3b738fe3-8e45-423d-b2e7-f489028311d9" (UID: "3b738fe3-8e45-423d-b2e7-f489028311d9"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 03 16:21:37.818316 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:21:37.818288 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3b738fe3-8e45-423d-b2e7-f489028311d9-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "3b738fe3-8e45-423d-b2e7-f489028311d9" (UID: "3b738fe3-8e45-423d-b2e7-f489028311d9"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 03 16:21:37.818464 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:21:37.818445 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3b738fe3-8e45-423d-b2e7-f489028311d9-kube-api-access-h5j2w" (OuterVolumeSpecName: "kube-api-access-h5j2w") pod "3b738fe3-8e45-423d-b2e7-f489028311d9" (UID: "3b738fe3-8e45-423d-b2e7-f489028311d9"). InnerVolumeSpecName "kube-api-access-h5j2w". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 03 16:21:37.818591 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:21:37.818563 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3b738fe3-8e45-423d-b2e7-f489028311d9-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "3b738fe3-8e45-423d-b2e7-f489028311d9" (UID: "3b738fe3-8e45-423d-b2e7-f489028311d9"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 03 16:21:37.917578 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:21:37.917549 2568 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3b738fe3-8e45-423d-b2e7-f489028311d9-console-oauth-config\") on node \"ip-10-0-128-88.ec2.internal\" DevicePath \"\"" Jun 03 16:21:37.917700 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:21:37.917607 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h5j2w\" (UniqueName: \"kubernetes.io/projected/3b738fe3-8e45-423d-b2e7-f489028311d9-kube-api-access-h5j2w\") on node \"ip-10-0-128-88.ec2.internal\" DevicePath \"\"" Jun 03 16:21:37.917700 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:21:37.917626 2568 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3b738fe3-8e45-423d-b2e7-f489028311d9-console-config\") on node \"ip-10-0-128-88.ec2.internal\" DevicePath \"\"" Jun 03 16:21:37.917700 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:21:37.917639 2568 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3b738fe3-8e45-423d-b2e7-f489028311d9-service-ca\") on node \"ip-10-0-128-88.ec2.internal\" DevicePath \"\"" Jun 03 16:21:37.917700 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:21:37.917652 2568 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3b738fe3-8e45-423d-b2e7-f489028311d9-console-serving-cert\") on node \"ip-10-0-128-88.ec2.internal\" DevicePath \"\"" Jun 03 16:21:37.917700 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:21:37.917664 2568 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3b738fe3-8e45-423d-b2e7-f489028311d9-trusted-ca-bundle\") on node \"ip-10-0-128-88.ec2.internal\" DevicePath \"\"" Jun 03 16:21:37.917700 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:21:37.917679 2568 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3b738fe3-8e45-423d-b2e7-f489028311d9-oauth-serving-cert\") on node \"ip-10-0-128-88.ec2.internal\" DevicePath \"\"" Jun 03 16:21:38.492798 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:21:38.492770 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-69cf65c77-rbc5g_3b738fe3-8e45-423d-b2e7-f489028311d9/console/0.log" Jun 03 16:21:38.493245 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:21:38.492816 2568 generic.go:358] "Generic (PLEG): container finished" podID="3b738fe3-8e45-423d-b2e7-f489028311d9" containerID="50e7afc4746adaaa064daa6e1b7e79b87cc9caddca24945a076e132e135acae3" exitCode=2 Jun 03 16:21:38.493245 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:21:38.492849 2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-69cf65c77-rbc5g" event={"ID":"3b738fe3-8e45-423d-b2e7-f489028311d9","Type":"ContainerDied","Data":"50e7afc4746adaaa064daa6e1b7e79b87cc9caddca24945a076e132e135acae3"} Jun 03 16:21:38.493245 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:21:38.492876 2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-69cf65c77-rbc5g" event={"ID":"3b738fe3-8e45-423d-b2e7-f489028311d9","Type":"ContainerDied","Data":"4bbfec920012073c84e47e0a24bcab5c0e75486bcf5f3fd7063290c004c1c459"} Jun 03 16:21:38.493245 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:21:38.492891 2568 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-69cf65c77-rbc5g" Jun 03 16:21:38.493245 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:21:38.492894 2568 scope.go:117] "RemoveContainer" containerID="50e7afc4746adaaa064daa6e1b7e79b87cc9caddca24945a076e132e135acae3" Jun 03 16:21:38.502871 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:21:38.502835 2568 scope.go:117] "RemoveContainer" containerID="50e7afc4746adaaa064daa6e1b7e79b87cc9caddca24945a076e132e135acae3" Jun 03 16:21:38.503445 ip-10-0-128-88 kubenswrapper[2568]: E0603 16:21:38.503138 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"50e7afc4746adaaa064daa6e1b7e79b87cc9caddca24945a076e132e135acae3\": container with ID starting with 50e7afc4746adaaa064daa6e1b7e79b87cc9caddca24945a076e132e135acae3 not found: ID does not exist" containerID="50e7afc4746adaaa064daa6e1b7e79b87cc9caddca24945a076e132e135acae3" Jun 03 16:21:38.503445 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:21:38.503171 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"50e7afc4746adaaa064daa6e1b7e79b87cc9caddca24945a076e132e135acae3"} err="failed to get container status \"50e7afc4746adaaa064daa6e1b7e79b87cc9caddca24945a076e132e135acae3\": rpc error: code = NotFound desc = could not find container \"50e7afc4746adaaa064daa6e1b7e79b87cc9caddca24945a076e132e135acae3\": container with ID starting with 50e7afc4746adaaa064daa6e1b7e79b87cc9caddca24945a076e132e135acae3 not found: ID does not exist" Jun 03 16:21:38.517806 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:21:38.517780 2568 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-console/console-69cf65c77-rbc5g"] Jun 03 16:21:38.521852 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:21:38.521824 2568 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-69cf65c77-rbc5g"] Jun 03 16:21:39.563875 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:21:39.563843 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3b738fe3-8e45-423d-b2e7-f489028311d9" path="/var/lib/kubelet/pods/3b738fe3-8e45-423d-b2e7-f489028311d9/volumes" Jun 03 16:22:09.439209 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:22:09.439180 2568 kubelet.go:1628] "Image garbage collection succeeded" Jun 03 16:22:26.019711 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:22:26.019673 2568 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19ds8nk2"] Jun 03 16:22:26.021924 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:22:26.020088 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3b738fe3-8e45-423d-b2e7-f489028311d9" containerName="console" Jun 03 16:22:26.021924 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:22:26.020103 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="3b738fe3-8e45-423d-b2e7-f489028311d9" containerName="console" Jun 03 16:22:26.021924 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:22:26.020188 2568 memory_manager.go:356] "RemoveStaleState removing state" podUID="3b738fe3-8e45-423d-b2e7-f489028311d9" containerName="console" Jun 03 16:22:26.023403 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:22:26.023386 2568 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19ds8nk2" Jun 03 16:22:26.025730 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:22:26.025712 2568 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jun 03 16:22:26.025824 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:22:26.025759 2568 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jun 03 16:22:26.026227 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:22:26.026211 2568 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-vsjql\"" Jun 03 16:22:26.036149 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:22:26.036128 2568 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19ds8nk2"] Jun 03 16:22:26.079273 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:22:26.079249 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4af00017-1cb4-4348-ae89-0f83a34f0732-bundle\") pod \"2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19ds8nk2\" (UID: \"4af00017-1cb4-4348-ae89-0f83a34f0732\") " pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19ds8nk2" Jun 03 16:22:26.079382 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:22:26.079285 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4af00017-1cb4-4348-ae89-0f83a34f0732-util\") pod \"2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19ds8nk2\" (UID: \"4af00017-1cb4-4348-ae89-0f83a34f0732\") " pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19ds8nk2" Jun 03 16:22:26.079382 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:22:26.079307 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gpk5w\" (UniqueName: \"kubernetes.io/projected/4af00017-1cb4-4348-ae89-0f83a34f0732-kube-api-access-gpk5w\") pod \"2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19ds8nk2\" (UID: \"4af00017-1cb4-4348-ae89-0f83a34f0732\") " pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19ds8nk2" Jun 03 16:22:26.180352 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:22:26.180318 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4af00017-1cb4-4348-ae89-0f83a34f0732-util\") pod \"2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19ds8nk2\" (UID: \"4af00017-1cb4-4348-ae89-0f83a34f0732\") " pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19ds8nk2" Jun 03 16:22:26.180352 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:22:26.180356 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-gpk5w\" (UniqueName: \"kubernetes.io/projected/4af00017-1cb4-4348-ae89-0f83a34f0732-kube-api-access-gpk5w\") pod \"2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19ds8nk2\" (UID: \"4af00017-1cb4-4348-ae89-0f83a34f0732\") " pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19ds8nk2" Jun 03 16:22:26.180520 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:22:26.180438 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4af00017-1cb4-4348-ae89-0f83a34f0732-bundle\") pod \"2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19ds8nk2\" (UID: \"4af00017-1cb4-4348-ae89-0f83a34f0732\") " pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19ds8nk2" Jun 03 16:22:26.180807 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:22:26.180787 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4af00017-1cb4-4348-ae89-0f83a34f0732-util\") pod \"2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19ds8nk2\" (UID: \"4af00017-1cb4-4348-ae89-0f83a34f0732\") " pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19ds8nk2" Jun 03 16:22:26.180849 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:22:26.180798 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4af00017-1cb4-4348-ae89-0f83a34f0732-bundle\") pod \"2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19ds8nk2\" (UID: \"4af00017-1cb4-4348-ae89-0f83a34f0732\") " pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19ds8nk2" Jun 03 16:22:26.189419 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:22:26.189404 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gpk5w\" (UniqueName: \"kubernetes.io/projected/4af00017-1cb4-4348-ae89-0f83a34f0732-kube-api-access-gpk5w\") pod \"2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19ds8nk2\" (UID: \"4af00017-1cb4-4348-ae89-0f83a34f0732\") " pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19ds8nk2" Jun 03 16:22:26.332231 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:22:26.332157 2568 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19ds8nk2" Jun 03 16:22:26.463979 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:22:26.463947 2568 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19ds8nk2"] Jun 03 16:22:26.469221 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:22:26.469184 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4af00017_1cb4_4348_ae89_0f83a34f0732.slice/crio-0737eff47b9e918acc4b4b3eebf7608eed909cef013a683869073b637f2fb443 WatchSource:0}: Error finding container 0737eff47b9e918acc4b4b3eebf7608eed909cef013a683869073b637f2fb443: Status 404 returned error can't find the container with id 0737eff47b9e918acc4b4b3eebf7608eed909cef013a683869073b637f2fb443 Jun 03 16:22:26.471011 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:22:26.470993 2568 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jun 03 16:22:26.617843 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:22:26.617806 2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19ds8nk2" event={"ID":"4af00017-1cb4-4348-ae89-0f83a34f0732","Type":"ContainerStarted","Data":"0737eff47b9e918acc4b4b3eebf7608eed909cef013a683869073b637f2fb443"} Jun 03 16:22:32.636091 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:22:32.636054 2568 generic.go:358] "Generic (PLEG): container finished" podID="4af00017-1cb4-4348-ae89-0f83a34f0732" containerID="1b751f9fb25b97b1182409e99690887539fa4bb224a37a358c1604304b98f848" exitCode=0 Jun 03 16:22:32.636523 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:22:32.636140 2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19ds8nk2" event={"ID":"4af00017-1cb4-4348-ae89-0f83a34f0732","Type":"ContainerDied","Data":"1b751f9fb25b97b1182409e99690887539fa4bb224a37a358c1604304b98f848"} Jun 03 16:22:34.644075 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:22:34.644045 2568 generic.go:358] "Generic (PLEG): container finished" podID="4af00017-1cb4-4348-ae89-0f83a34f0732" containerID="d6dd99ca823fd46cee68d9362a9467c8ad2021487e06e59153f47383a0269296" exitCode=0 Jun 03 16:22:34.644514 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:22:34.644100 2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19ds8nk2" event={"ID":"4af00017-1cb4-4348-ae89-0f83a34f0732","Type":"ContainerDied","Data":"d6dd99ca823fd46cee68d9362a9467c8ad2021487e06e59153f47383a0269296"} Jun 03 16:22:44.673121 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:22:44.673082 2568 generic.go:358] "Generic (PLEG): container finished" podID="4af00017-1cb4-4348-ae89-0f83a34f0732" containerID="617762f381025ad8e653d97f15db0fe2b5157db215f4a7498d1a72ccdd598fe4" exitCode=0 Jun 03 16:22:44.673464 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:22:44.673123 2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19ds8nk2" event={"ID":"4af00017-1cb4-4348-ae89-0f83a34f0732","Type":"ContainerDied","Data":"617762f381025ad8e653d97f15db0fe2b5157db215f4a7498d1a72ccdd598fe4"} Jun 03 16:22:45.797613 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:22:45.797587 2568 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19ds8nk2" Jun 03 16:22:45.842766 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:22:45.842738 2568 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gpk5w\" (UniqueName: \"kubernetes.io/projected/4af00017-1cb4-4348-ae89-0f83a34f0732-kube-api-access-gpk5w\") pod \"4af00017-1cb4-4348-ae89-0f83a34f0732\" (UID: \"4af00017-1cb4-4348-ae89-0f83a34f0732\") " Jun 03 16:22:45.842919 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:22:45.842803 2568 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4af00017-1cb4-4348-ae89-0f83a34f0732-util\") pod \"4af00017-1cb4-4348-ae89-0f83a34f0732\" (UID: \"4af00017-1cb4-4348-ae89-0f83a34f0732\") " Jun 03 16:22:45.842919 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:22:45.842859 2568 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4af00017-1cb4-4348-ae89-0f83a34f0732-bundle\") pod \"4af00017-1cb4-4348-ae89-0f83a34f0732\" (UID: \"4af00017-1cb4-4348-ae89-0f83a34f0732\") " Jun 03 16:22:45.843494 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:22:45.843472 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4af00017-1cb4-4348-ae89-0f83a34f0732-bundle" (OuterVolumeSpecName: "bundle") pod "4af00017-1cb4-4348-ae89-0f83a34f0732" (UID: "4af00017-1cb4-4348-ae89-0f83a34f0732"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 03 16:22:45.844870 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:22:45.844850 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4af00017-1cb4-4348-ae89-0f83a34f0732-kube-api-access-gpk5w" (OuterVolumeSpecName: "kube-api-access-gpk5w") pod "4af00017-1cb4-4348-ae89-0f83a34f0732" (UID: "4af00017-1cb4-4348-ae89-0f83a34f0732"). InnerVolumeSpecName "kube-api-access-gpk5w". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 03 16:22:45.846995 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:22:45.846976 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4af00017-1cb4-4348-ae89-0f83a34f0732-util" (OuterVolumeSpecName: "util") pod "4af00017-1cb4-4348-ae89-0f83a34f0732" (UID: "4af00017-1cb4-4348-ae89-0f83a34f0732"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 03 16:22:45.944159 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:22:45.944090 2568 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4af00017-1cb4-4348-ae89-0f83a34f0732-bundle\") on node \"ip-10-0-128-88.ec2.internal\" DevicePath \"\"" Jun 03 16:22:45.944159 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:22:45.944116 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gpk5w\" (UniqueName: \"kubernetes.io/projected/4af00017-1cb4-4348-ae89-0f83a34f0732-kube-api-access-gpk5w\") on node \"ip-10-0-128-88.ec2.internal\" DevicePath \"\"" Jun 03 16:22:45.944159 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:22:45.944125 2568 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4af00017-1cb4-4348-ae89-0f83a34f0732-util\") on node \"ip-10-0-128-88.ec2.internal\" DevicePath \"\"" Jun 03 16:22:46.680138 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:22:46.680112 2568 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19ds8nk2" Jun 03 16:22:46.680138 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:22:46.680120 2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19ds8nk2" event={"ID":"4af00017-1cb4-4348-ae89-0f83a34f0732","Type":"ContainerDied","Data":"0737eff47b9e918acc4b4b3eebf7608eed909cef013a683869073b637f2fb443"} Jun 03 16:22:46.680334 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:22:46.680150 2568 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0737eff47b9e918acc4b4b3eebf7608eed909cef013a683869073b637f2fb443" Jun 03 16:22:54.272653 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:22:54.272570 2568 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-54b9655956-lrmlk"] Jun 03 16:22:54.273180 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:22:54.272886 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4af00017-1cb4-4348-ae89-0f83a34f0732" containerName="util" Jun 03 16:22:54.273180 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:22:54.272898 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="4af00017-1cb4-4348-ae89-0f83a34f0732" containerName="util" Jun 03 16:22:54.273180 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:22:54.272909 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4af00017-1cb4-4348-ae89-0f83a34f0732" containerName="pull" Jun 03 16:22:54.273180 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:22:54.272915 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="4af00017-1cb4-4348-ae89-0f83a34f0732" containerName="pull" Jun 03 16:22:54.273180 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:22:54.272933 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4af00017-1cb4-4348-ae89-0f83a34f0732" containerName="extract" Jun 03 16:22:54.273180 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:22:54.272943 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="4af00017-1cb4-4348-ae89-0f83a34f0732" containerName="extract" Jun 03 16:22:54.273180 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:22:54.272994 2568 memory_manager.go:356] "RemoveStaleState removing state" podUID="4af00017-1cb4-4348-ae89-0f83a34f0732" containerName="extract" Jun 03 16:22:54.274892 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:22:54.274876 2568 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-54b9655956-lrmlk" Jun 03 16:22:54.277684 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:22:54.277657 2568 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\"" Jun 03 16:22:54.277796 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:22:54.277743 2568 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-2tmls\"" Jun 03 16:22:54.279606 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:22:54.279592 2568 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\"" Jun 03 16:22:54.288320 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:22:54.288296 2568 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-54b9655956-lrmlk"] Jun 03 16:22:54.410843 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:22:54.410807 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a8445734-f84d-40d8-9e3f-d6fbe735a7d0-tmp\") pod \"cert-manager-operator-controller-manager-54b9655956-lrmlk\" (UID: \"a8445734-f84d-40d8-9e3f-d6fbe735a7d0\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-54b9655956-lrmlk" Jun 03 16:22:54.410963 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:22:54.410854 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q5lxl\" (UniqueName: \"kubernetes.io/projected/a8445734-f84d-40d8-9e3f-d6fbe735a7d0-kube-api-access-q5lxl\") pod \"cert-manager-operator-controller-manager-54b9655956-lrmlk\" (UID: \"a8445734-f84d-40d8-9e3f-d6fbe735a7d0\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-54b9655956-lrmlk" Jun 03 16:22:54.511783 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:22:54.511750 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a8445734-f84d-40d8-9e3f-d6fbe735a7d0-tmp\") pod \"cert-manager-operator-controller-manager-54b9655956-lrmlk\" (UID: \"a8445734-f84d-40d8-9e3f-d6fbe735a7d0\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-54b9655956-lrmlk" Jun 03 16:22:54.511864 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:22:54.511798 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-q5lxl\" (UniqueName: \"kubernetes.io/projected/a8445734-f84d-40d8-9e3f-d6fbe735a7d0-kube-api-access-q5lxl\") pod \"cert-manager-operator-controller-manager-54b9655956-lrmlk\" (UID: \"a8445734-f84d-40d8-9e3f-d6fbe735a7d0\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-54b9655956-lrmlk" Jun 03 16:22:54.512087 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:22:54.512072 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a8445734-f84d-40d8-9e3f-d6fbe735a7d0-tmp\") pod \"cert-manager-operator-controller-manager-54b9655956-lrmlk\" (UID: \"a8445734-f84d-40d8-9e3f-d6fbe735a7d0\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-54b9655956-lrmlk" Jun 03 16:22:54.519722 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:22:54.519698 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q5lxl\" (UniqueName: \"kubernetes.io/projected/a8445734-f84d-40d8-9e3f-d6fbe735a7d0-kube-api-access-q5lxl\") pod \"cert-manager-operator-controller-manager-54b9655956-lrmlk\" (UID: \"a8445734-f84d-40d8-9e3f-d6fbe735a7d0\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-54b9655956-lrmlk" Jun 03 16:22:54.584371 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:22:54.584306 2568 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-54b9655956-lrmlk" Jun 03 16:22:54.709364 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:22:54.709230 2568 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-54b9655956-lrmlk"] Jun 03 16:22:54.712158 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:22:54.712132 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda8445734_f84d_40d8_9e3f_d6fbe735a7d0.slice/crio-ad1b70cd0827b1995d1e7ed6a9c36dfd44c02242ebbc15c0895d2f98e179ccb5 WatchSource:0}: Error finding container ad1b70cd0827b1995d1e7ed6a9c36dfd44c02242ebbc15c0895d2f98e179ccb5: Status 404 returned error can't find the container with id ad1b70cd0827b1995d1e7ed6a9c36dfd44c02242ebbc15c0895d2f98e179ccb5 Jun 03 16:22:55.708332 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:22:55.708292 2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-54b9655956-lrmlk" event={"ID":"a8445734-f84d-40d8-9e3f-d6fbe735a7d0","Type":"ContainerStarted","Data":"ad1b70cd0827b1995d1e7ed6a9c36dfd44c02242ebbc15c0895d2f98e179ccb5"} Jun 03 16:22:57.716439 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:22:57.716401 2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-54b9655956-lrmlk" event={"ID":"a8445734-f84d-40d8-9e3f-d6fbe735a7d0","Type":"ContainerStarted","Data":"6a832ab5f4e54ce961d69f771749444fa985b4a9dba51f61f04a7202ac73a93e"} Jun 03 16:22:57.739564 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:22:57.739503 2568 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-54b9655956-lrmlk" podStartSLOduration=1.143872408 podStartE2EDuration="3.739487582s" podCreationTimestamp="2026-06-03 16:22:54 +0000 UTC" firstStartedPulling="2026-06-03 16:22:54.714597573 +0000 UTC m=+345.803024140" lastFinishedPulling="2026-06-03 16:22:57.310212736 +0000 UTC m=+348.398639314" observedRunningTime="2026-06-03 16:22:57.737686028 +0000 UTC m=+348.826112618" watchObservedRunningTime="2026-06-03 16:22:57.739487582 +0000 UTC m=+348.827914174" Jun 03 16:22:58.808065 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:22:58.808032 2568 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ft46zz"] Jun 03 16:22:58.810589 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:22:58.810573 2568 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ft46zz" Jun 03 16:22:58.812745 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:22:58.812720 2568 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jun 03 16:22:58.813374 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:22:58.813352 2568 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-vsjql\"" Jun 03 16:22:58.813437 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:22:58.813355 2568 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jun 03 16:22:58.823138 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:22:58.823088 2568 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ft46zz"] Jun 03 16:22:58.946190 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:22:58.946161 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4b23bb78-5da7-4856-a889-e66d49136c05-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ft46zz\" (UID: \"4b23bb78-5da7-4856-a889-e66d49136c05\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ft46zz" Jun 03 16:22:58.946190 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:22:58.946196 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4b23bb78-5da7-4856-a889-e66d49136c05-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ft46zz\" (UID: \"4b23bb78-5da7-4856-a889-e66d49136c05\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ft46zz" Jun 03 16:22:58.946365 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:22:58.946252 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xgdbf\" (UniqueName: \"kubernetes.io/projected/4b23bb78-5da7-4856-a889-e66d49136c05-kube-api-access-xgdbf\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ft46zz\" (UID: \"4b23bb78-5da7-4856-a889-e66d49136c05\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ft46zz" Jun 03 16:22:59.047085 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:22:59.047061 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-xgdbf\" (UniqueName: \"kubernetes.io/projected/4b23bb78-5da7-4856-a889-e66d49136c05-kube-api-access-xgdbf\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ft46zz\" (UID: \"4b23bb78-5da7-4856-a889-e66d49136c05\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ft46zz" Jun 03 16:22:59.047206 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:22:59.047110 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4b23bb78-5da7-4856-a889-e66d49136c05-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ft46zz\" (UID: \"4b23bb78-5da7-4856-a889-e66d49136c05\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ft46zz" Jun 03 16:22:59.047206 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:22:59.047133 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4b23bb78-5da7-4856-a889-e66d49136c05-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ft46zz\" (UID: \"4b23bb78-5da7-4856-a889-e66d49136c05\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ft46zz" Jun 03 16:22:59.047563 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:22:59.047519 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4b23bb78-5da7-4856-a889-e66d49136c05-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ft46zz\" (UID: \"4b23bb78-5da7-4856-a889-e66d49136c05\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ft46zz" Jun 03 16:22:59.047603 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:22:59.047555 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4b23bb78-5da7-4856-a889-e66d49136c05-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ft46zz\" (UID: \"4b23bb78-5da7-4856-a889-e66d49136c05\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ft46zz" Jun 03 16:22:59.055259 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:22:59.055231 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xgdbf\" (UniqueName: \"kubernetes.io/projected/4b23bb78-5da7-4856-a889-e66d49136c05-kube-api-access-xgdbf\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ft46zz\" (UID: \"4b23bb78-5da7-4856-a889-e66d49136c05\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ft46zz" Jun 03 16:22:59.125186 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:22:59.125169 2568 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ft46zz" Jun 03 16:22:59.250387 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:22:59.250364 2568 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ft46zz"] Jun 03 16:22:59.253069 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:22:59.253044 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4b23bb78_5da7_4856_a889_e66d49136c05.slice/crio-0e89133035dd6749ba96404f07dbccf1fb01ab81298ddf1004817d62e9f14339 WatchSource:0}: Error finding container 0e89133035dd6749ba96404f07dbccf1fb01ab81298ddf1004817d62e9f14339: Status 404 returned error can't find the container with id 0e89133035dd6749ba96404f07dbccf1fb01ab81298ddf1004817d62e9f14339 Jun 03 16:22:59.595229 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:22:59.595162 2568 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-587ccfb98-n4flj"] Jun 03 16:22:59.597297 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:22:59.597283 2568 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-587ccfb98-n4flj" Jun 03 16:22:59.599602 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:22:59.599575 2568 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Jun 03 16:22:59.599711 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:22:59.599615 2568 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-d8b4l\"" Jun 03 16:22:59.599711 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:22:59.599640 2568 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Jun 03 16:22:59.606271 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:22:59.606252 2568 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-587ccfb98-n4flj"] Jun 03 16:22:59.724330 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:22:59.724301 2568 generic.go:358] "Generic (PLEG): container finished" podID="4b23bb78-5da7-4856-a889-e66d49136c05" containerID="75c5abfe2f66ea0b4c03cfddebb9a4d2afa52dbe98c148a63c2db9bea89fc931" exitCode=0 Jun 03 16:22:59.724484 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:22:59.724375 2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ft46zz" event={"ID":"4b23bb78-5da7-4856-a889-e66d49136c05","Type":"ContainerDied","Data":"75c5abfe2f66ea0b4c03cfddebb9a4d2afa52dbe98c148a63c2db9bea89fc931"} Jun 03 16:22:59.724484 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:22:59.724410 2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ft46zz" event={"ID":"4b23bb78-5da7-4856-a889-e66d49136c05","Type":"ContainerStarted","Data":"0e89133035dd6749ba96404f07dbccf1fb01ab81298ddf1004817d62e9f14339"} Jun 03 16:22:59.752325 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:22:59.752294 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bnj97\" (UniqueName: \"kubernetes.io/projected/fc27c9a3-5876-49c6-9b50-784789477f9e-kube-api-access-bnj97\") pod \"cert-manager-webhook-587ccfb98-n4flj\" (UID: \"fc27c9a3-5876-49c6-9b50-784789477f9e\") " pod="cert-manager/cert-manager-webhook-587ccfb98-n4flj" Jun 03 16:22:59.752423 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:22:59.752350 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/fc27c9a3-5876-49c6-9b50-784789477f9e-bound-sa-token\") pod \"cert-manager-webhook-587ccfb98-n4flj\" (UID: \"fc27c9a3-5876-49c6-9b50-784789477f9e\") " pod="cert-manager/cert-manager-webhook-587ccfb98-n4flj" Jun 03 16:22:59.853217 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:22:59.853163 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/fc27c9a3-5876-49c6-9b50-784789477f9e-bound-sa-token\") pod \"cert-manager-webhook-587ccfb98-n4flj\" (UID: \"fc27c9a3-5876-49c6-9b50-784789477f9e\") " pod="cert-manager/cert-manager-webhook-587ccfb98-n4flj" Jun 03 16:22:59.853524 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:22:59.853249 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-bnj97\" (UniqueName: \"kubernetes.io/projected/fc27c9a3-5876-49c6-9b50-784789477f9e-kube-api-access-bnj97\") pod \"cert-manager-webhook-587ccfb98-n4flj\" (UID: \"fc27c9a3-5876-49c6-9b50-784789477f9e\") " pod="cert-manager/cert-manager-webhook-587ccfb98-n4flj" Jun 03 16:22:59.860642 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:22:59.860617 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/fc27c9a3-5876-49c6-9b50-784789477f9e-bound-sa-token\") pod \"cert-manager-webhook-587ccfb98-n4flj\" (UID: \"fc27c9a3-5876-49c6-9b50-784789477f9e\") " pod="cert-manager/cert-manager-webhook-587ccfb98-n4flj" Jun 03 16:22:59.860790 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:22:59.860775 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bnj97\" (UniqueName: \"kubernetes.io/projected/fc27c9a3-5876-49c6-9b50-784789477f9e-kube-api-access-bnj97\") pod \"cert-manager-webhook-587ccfb98-n4flj\" (UID: \"fc27c9a3-5876-49c6-9b50-784789477f9e\") " pod="cert-manager/cert-manager-webhook-587ccfb98-n4flj" Jun 03 16:22:59.922421 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:22:59.922400 2568 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-587ccfb98-n4flj" Jun 03 16:23:00.039823 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:23:00.039793 2568 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-587ccfb98-n4flj"] Jun 03 16:23:00.042598 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:23:00.042571 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podfc27c9a3_5876_49c6_9b50_784789477f9e.slice/crio-ae9bb91675bacca5e12ca489bdd2dca08fa69758f229efd851dc6d89c0e0af56 WatchSource:0}: Error finding container ae9bb91675bacca5e12ca489bdd2dca08fa69758f229efd851dc6d89c0e0af56: Status 404 returned error can't find the container with id ae9bb91675bacca5e12ca489bdd2dca08fa69758f229efd851dc6d89c0e0af56 Jun 03 16:23:00.728152 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:23:00.728115 2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-587ccfb98-n4flj" event={"ID":"fc27c9a3-5876-49c6-9b50-784789477f9e","Type":"ContainerStarted","Data":"ae9bb91675bacca5e12ca489bdd2dca08fa69758f229efd851dc6d89c0e0af56"} Jun 03 16:23:03.739838 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:23:03.739796 2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-587ccfb98-n4flj" event={"ID":"fc27c9a3-5876-49c6-9b50-784789477f9e","Type":"ContainerStarted","Data":"8b7d8c598004b38a44cc87373c2b87898a1043748672cb248e7c2e8865e91c44"} Jun 03 16:23:03.740247 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:23:03.739852 2568 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-587ccfb98-n4flj" Jun 03 16:23:03.741394 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:23:03.741370 2568 generic.go:358] "Generic (PLEG): container finished" podID="4b23bb78-5da7-4856-a889-e66d49136c05" containerID="bb294ff3c6c0b97bd0ef3537062052fadff3c63c1d0d2420f93ff4a1ad49e3ea" exitCode=0 Jun 03 16:23:03.741488 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:23:03.741407 2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ft46zz" event={"ID":"4b23bb78-5da7-4856-a889-e66d49136c05","Type":"ContainerDied","Data":"bb294ff3c6c0b97bd0ef3537062052fadff3c63c1d0d2420f93ff4a1ad49e3ea"} Jun 03 16:23:03.756614 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:23:03.756574 2568 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-587ccfb98-n4flj" podStartSLOduration=1.445958314 podStartE2EDuration="4.756562299s" podCreationTimestamp="2026-06-03 16:22:59 +0000 UTC" firstStartedPulling="2026-06-03 16:23:00.044488544 +0000 UTC m=+351.132915112" lastFinishedPulling="2026-06-03 16:23:03.355092517 +0000 UTC m=+354.443519097" observedRunningTime="2026-06-03 16:23:03.755907107 +0000 UTC m=+354.844333700" watchObservedRunningTime="2026-06-03 16:23:03.756562299 +0000 UTC m=+354.844988888" Jun 03 16:23:04.746911 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:23:04.746879 2568 generic.go:358] "Generic (PLEG): container finished" podID="4b23bb78-5da7-4856-a889-e66d49136c05" containerID="6b9a54b6f54a0d8bbfe7a1fb6f090e8ef434e9a2306679a29553a9b518f69119" exitCode=0 Jun 03 16:23:04.747291 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:23:04.746969 2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ft46zz" event={"ID":"4b23bb78-5da7-4856-a889-e66d49136c05","Type":"ContainerDied","Data":"6b9a54b6f54a0d8bbfe7a1fb6f090e8ef434e9a2306679a29553a9b518f69119"} Jun 03 16:23:05.876624 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:23:05.876603 2568 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ft46zz" Jun 03 16:23:06.002793 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:23:06.002733 2568 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xgdbf\" (UniqueName: \"kubernetes.io/projected/4b23bb78-5da7-4856-a889-e66d49136c05-kube-api-access-xgdbf\") pod \"4b23bb78-5da7-4856-a889-e66d49136c05\" (UID: \"4b23bb78-5da7-4856-a889-e66d49136c05\") " Jun 03 16:23:06.002793 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:23:06.002774 2568 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4b23bb78-5da7-4856-a889-e66d49136c05-bundle\") pod \"4b23bb78-5da7-4856-a889-e66d49136c05\" (UID: \"4b23bb78-5da7-4856-a889-e66d49136c05\") " Jun 03 16:23:06.002952 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:23:06.002824 2568 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4b23bb78-5da7-4856-a889-e66d49136c05-util\") pod \"4b23bb78-5da7-4856-a889-e66d49136c05\" (UID: \"4b23bb78-5da7-4856-a889-e66d49136c05\") " Jun 03 16:23:06.003161 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:23:06.003137 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4b23bb78-5da7-4856-a889-e66d49136c05-bundle" (OuterVolumeSpecName: "bundle") pod "4b23bb78-5da7-4856-a889-e66d49136c05" (UID: "4b23bb78-5da7-4856-a889-e66d49136c05"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 03 16:23:06.004724 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:23:06.004701 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4b23bb78-5da7-4856-a889-e66d49136c05-kube-api-access-xgdbf" (OuterVolumeSpecName: "kube-api-access-xgdbf") pod "4b23bb78-5da7-4856-a889-e66d49136c05" (UID: "4b23bb78-5da7-4856-a889-e66d49136c05"). InnerVolumeSpecName "kube-api-access-xgdbf". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 03 16:23:06.008782 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:23:06.008759 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4b23bb78-5da7-4856-a889-e66d49136c05-util" (OuterVolumeSpecName: "util") pod "4b23bb78-5da7-4856-a889-e66d49136c05" (UID: "4b23bb78-5da7-4856-a889-e66d49136c05"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 03 16:23:06.017814 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:23:06.017791 2568 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-68b757865b-pfh2r"] Jun 03 16:23:06.018107 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:23:06.018094 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4b23bb78-5da7-4856-a889-e66d49136c05" containerName="pull" Jun 03 16:23:06.018167 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:23:06.018109 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="4b23bb78-5da7-4856-a889-e66d49136c05" containerName="pull" Jun 03 16:23:06.018167 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:23:06.018122 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4b23bb78-5da7-4856-a889-e66d49136c05" containerName="util" Jun 03 16:23:06.018167 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:23:06.018128 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="4b23bb78-5da7-4856-a889-e66d49136c05" containerName="util" Jun 03 16:23:06.018167 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:23:06.018138 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4b23bb78-5da7-4856-a889-e66d49136c05" containerName="extract" Jun 03 16:23:06.018167 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:23:06.018144 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="4b23bb78-5da7-4856-a889-e66d49136c05" containerName="extract" Jun 03 16:23:06.018356 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:23:06.018203 2568 memory_manager.go:356] "RemoveStaleState removing state" podUID="4b23bb78-5da7-4856-a889-e66d49136c05" containerName="extract" Jun 03 16:23:06.021879 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:23:06.021862 2568 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-68b757865b-pfh2r"] Jun 03 16:23:06.021956 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:23:06.021949 2568 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-68b757865b-pfh2r" Jun 03 16:23:06.024691 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:23:06.024669 2568 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-h5j47\"" Jun 03 16:23:06.103958 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:23:06.103937 2568 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4b23bb78-5da7-4856-a889-e66d49136c05-util\") on node \"ip-10-0-128-88.ec2.internal\" DevicePath \"\"" Jun 03 16:23:06.103958 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:23:06.103956 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xgdbf\" (UniqueName: \"kubernetes.io/projected/4b23bb78-5da7-4856-a889-e66d49136c05-kube-api-access-xgdbf\") on node \"ip-10-0-128-88.ec2.internal\" DevicePath \"\"" Jun 03 16:23:06.104072 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:23:06.103969 2568 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4b23bb78-5da7-4856-a889-e66d49136c05-bundle\") on node \"ip-10-0-128-88.ec2.internal\" DevicePath \"\"" Jun 03 16:23:06.204352 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:23:06.204323 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dffdl\" (UniqueName: \"kubernetes.io/projected/38bcfa3b-3144-414a-bb2e-577134b615f6-kube-api-access-dffdl\") pod \"cert-manager-cainjector-68b757865b-pfh2r\" (UID: \"38bcfa3b-3144-414a-bb2e-577134b615f6\") " pod="cert-manager/cert-manager-cainjector-68b757865b-pfh2r" Jun 03 16:23:06.204434 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:23:06.204381 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/38bcfa3b-3144-414a-bb2e-577134b615f6-bound-sa-token\") pod \"cert-manager-cainjector-68b757865b-pfh2r\" (UID: \"38bcfa3b-3144-414a-bb2e-577134b615f6\") " pod="cert-manager/cert-manager-cainjector-68b757865b-pfh2r" Jun 03 16:23:06.305227 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:23:06.305172 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-dffdl\" (UniqueName: \"kubernetes.io/projected/38bcfa3b-3144-414a-bb2e-577134b615f6-kube-api-access-dffdl\") pod \"cert-manager-cainjector-68b757865b-pfh2r\" (UID: \"38bcfa3b-3144-414a-bb2e-577134b615f6\") " pod="cert-manager/cert-manager-cainjector-68b757865b-pfh2r" Jun 03 16:23:06.305227 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:23:06.305207 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/38bcfa3b-3144-414a-bb2e-577134b615f6-bound-sa-token\") pod \"cert-manager-cainjector-68b757865b-pfh2r\" (UID: \"38bcfa3b-3144-414a-bb2e-577134b615f6\") " pod="cert-manager/cert-manager-cainjector-68b757865b-pfh2r" Jun 03 16:23:06.315399 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:23:06.315372 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/38bcfa3b-3144-414a-bb2e-577134b615f6-bound-sa-token\") pod \"cert-manager-cainjector-68b757865b-pfh2r\" (UID: \"38bcfa3b-3144-414a-bb2e-577134b615f6\") " pod="cert-manager/cert-manager-cainjector-68b757865b-pfh2r" Jun 03 16:23:06.315671 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:23:06.315651 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dffdl\" (UniqueName: \"kubernetes.io/projected/38bcfa3b-3144-414a-bb2e-577134b615f6-kube-api-access-dffdl\") pod \"cert-manager-cainjector-68b757865b-pfh2r\" (UID: \"38bcfa3b-3144-414a-bb2e-577134b615f6\") " pod="cert-manager/cert-manager-cainjector-68b757865b-pfh2r" Jun 03 16:23:06.330521 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:23:06.330501 2568 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-68b757865b-pfh2r" Jun 03 16:23:06.450220 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:23:06.450197 2568 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-68b757865b-pfh2r"] Jun 03 16:23:06.452433 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:23:06.452404 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod38bcfa3b_3144_414a_bb2e_577134b615f6.slice/crio-ae4267fefd685e251139e69b4aa6377b942dd2c75fb94a716fe2699407c2f918 WatchSource:0}: Error finding container ae4267fefd685e251139e69b4aa6377b942dd2c75fb94a716fe2699407c2f918: Status 404 returned error can't find the container with id ae4267fefd685e251139e69b4aa6377b942dd2c75fb94a716fe2699407c2f918 Jun 03 16:23:06.754111 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:23:06.754079 2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-68b757865b-pfh2r" event={"ID":"38bcfa3b-3144-414a-bb2e-577134b615f6","Type":"ContainerStarted","Data":"0554def109cd5835c2350262f36bf668e75b750b1c8f084bc2c0f6762d246c74"} Jun 03 16:23:06.754111 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:23:06.754113 2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-68b757865b-pfh2r" event={"ID":"38bcfa3b-3144-414a-bb2e-577134b615f6","Type":"ContainerStarted","Data":"ae4267fefd685e251139e69b4aa6377b942dd2c75fb94a716fe2699407c2f918"} Jun 03 16:23:06.755734 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:23:06.755711 2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ft46zz" event={"ID":"4b23bb78-5da7-4856-a889-e66d49136c05","Type":"ContainerDied","Data":"0e89133035dd6749ba96404f07dbccf1fb01ab81298ddf1004817d62e9f14339"} Jun 03 16:23:06.755734 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:23:06.755728 2568 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ft46zz" Jun 03 16:23:06.755734 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:23:06.755737 2568 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0e89133035dd6749ba96404f07dbccf1fb01ab81298ddf1004817d62e9f14339" Jun 03 16:23:06.773463 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:23:06.773418 2568 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-68b757865b-pfh2r" podStartSLOduration=1.773401593 podStartE2EDuration="1.773401593s" podCreationTimestamp="2026-06-03 16:23:05 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-06-03 16:23:06.771924685 +0000 UTC m=+357.860351275" watchObservedRunningTime="2026-06-03 16:23:06.773401593 +0000 UTC m=+357.861828185" Jun 03 16:23:09.749052 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:23:09.749023 2568 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-587ccfb98-n4flj" Jun 03 16:23:13.223046 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:23:13.223009 2568 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-bfc7f696d-d7qj8"] Jun 03 16:23:13.241129 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:23:13.241087 2568 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-bfc7f696d-d7qj8"] Jun 03 16:23:13.241129 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:23:13.241117 2568 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-bfc7f696d-d7qj8" Jun 03 16:23:13.243114 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:23:13.243097 2568 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jun 03 16:23:13.243561 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:23:13.243526 2568 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jun 03 16:23:13.243561 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:23:13.243526 2568 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-7snk5\"" Jun 03 16:23:13.361608 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:23:13.361577 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/11ca5b7b-8c8d-41e6-9bf5-5025aafe9a96-tmp\") pod \"openshift-lws-operator-bfc7f696d-d7qj8\" (UID: \"11ca5b7b-8c8d-41e6-9bf5-5025aafe9a96\") " pod="openshift-lws-operator/openshift-lws-operator-bfc7f696d-d7qj8" Jun 03 16:23:13.361750 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:23:13.361630 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bq479\" (UniqueName: \"kubernetes.io/projected/11ca5b7b-8c8d-41e6-9bf5-5025aafe9a96-kube-api-access-bq479\") pod \"openshift-lws-operator-bfc7f696d-d7qj8\" (UID: \"11ca5b7b-8c8d-41e6-9bf5-5025aafe9a96\") " pod="openshift-lws-operator/openshift-lws-operator-bfc7f696d-d7qj8" Jun 03 16:23:13.462492 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:23:13.462458 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/11ca5b7b-8c8d-41e6-9bf5-5025aafe9a96-tmp\") pod \"openshift-lws-operator-bfc7f696d-d7qj8\" (UID: \"11ca5b7b-8c8d-41e6-9bf5-5025aafe9a96\") " pod="openshift-lws-operator/openshift-lws-operator-bfc7f696d-d7qj8" Jun 03 16:23:13.462650 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:23:13.462513 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-bq479\" (UniqueName: \"kubernetes.io/projected/11ca5b7b-8c8d-41e6-9bf5-5025aafe9a96-kube-api-access-bq479\") pod \"openshift-lws-operator-bfc7f696d-d7qj8\" (UID: \"11ca5b7b-8c8d-41e6-9bf5-5025aafe9a96\") " pod="openshift-lws-operator/openshift-lws-operator-bfc7f696d-d7qj8" Jun 03 16:23:13.462848 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:23:13.462829 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/11ca5b7b-8c8d-41e6-9bf5-5025aafe9a96-tmp\") pod \"openshift-lws-operator-bfc7f696d-d7qj8\" (UID: \"11ca5b7b-8c8d-41e6-9bf5-5025aafe9a96\") " pod="openshift-lws-operator/openshift-lws-operator-bfc7f696d-d7qj8" Jun 03 16:23:13.470790 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:23:13.470768 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bq479\" (UniqueName: \"kubernetes.io/projected/11ca5b7b-8c8d-41e6-9bf5-5025aafe9a96-kube-api-access-bq479\") pod \"openshift-lws-operator-bfc7f696d-d7qj8\" (UID: \"11ca5b7b-8c8d-41e6-9bf5-5025aafe9a96\") " pod="openshift-lws-operator/openshift-lws-operator-bfc7f696d-d7qj8" Jun 03 16:23:13.549935 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:23:13.549882 2568 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-bfc7f696d-d7qj8" Jun 03 16:23:13.676851 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:23:13.676811 2568 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-bfc7f696d-d7qj8"] Jun 03 16:23:13.679932 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:23:13.679886 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod11ca5b7b_8c8d_41e6_9bf5_5025aafe9a96.slice/crio-3ccc6f0a74a9f3c37e0605ed7782420d45ae39ff99e4252b00475e5b4cdcc075 WatchSource:0}: Error finding container 3ccc6f0a74a9f3c37e0605ed7782420d45ae39ff99e4252b00475e5b4cdcc075: Status 404 returned error can't find the container with id 3ccc6f0a74a9f3c37e0605ed7782420d45ae39ff99e4252b00475e5b4cdcc075 Jun 03 16:23:13.777322 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:23:13.777281 2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-bfc7f696d-d7qj8" event={"ID":"11ca5b7b-8c8d-41e6-9bf5-5025aafe9a96","Type":"ContainerStarted","Data":"3ccc6f0a74a9f3c37e0605ed7782420d45ae39ff99e4252b00475e5b4cdcc075"} Jun 03 16:23:16.792792 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:23:16.792717 2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-bfc7f696d-d7qj8" event={"ID":"11ca5b7b-8c8d-41e6-9bf5-5025aafe9a96","Type":"ContainerStarted","Data":"525f1279018e016bb75265f19815811cf0573f719ddc475af9151ec9f65c20e6"} Jun 03 16:23:16.809563 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:23:16.809508 2568 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-bfc7f696d-d7qj8" podStartSLOduration=1.076969238 podStartE2EDuration="3.809495147s" podCreationTimestamp="2026-06-03 16:23:13 +0000 UTC" firstStartedPulling="2026-06-03 16:23:13.681298816 +0000 UTC m=+364.769725385" lastFinishedPulling="2026-06-03 16:23:16.413824719 +0000 UTC m=+367.502251294" observedRunningTime="2026-06-03 16:23:16.808324509 +0000 UTC m=+367.896751115" watchObservedRunningTime="2026-06-03 16:23:16.809495147 +0000 UTC m=+367.897921737" Jun 03 16:23:18.980426 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:23:18.980395 2568 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-79c8d999ff-bj2lm"] Jun 03 16:23:18.982639 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:23:18.982623 2568 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-79c8d999ff-bj2lm" Jun 03 16:23:18.984672 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:23:18.984655 2568 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-pbcgs\"" Jun 03 16:23:18.992247 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:23:18.992222 2568 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-79c8d999ff-bj2lm"] Jun 03 16:23:19.001618 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:23:19.001597 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lzft4\" (UniqueName: \"kubernetes.io/projected/4f48bb40-ece5-44b2-b796-378922f2aa69-kube-api-access-lzft4\") pod \"cert-manager-79c8d999ff-bj2lm\" (UID: \"4f48bb40-ece5-44b2-b796-378922f2aa69\") " pod="cert-manager/cert-manager-79c8d999ff-bj2lm" Jun 03 16:23:19.001719 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:23:19.001699 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4f48bb40-ece5-44b2-b796-378922f2aa69-bound-sa-token\") pod \"cert-manager-79c8d999ff-bj2lm\" (UID: \"4f48bb40-ece5-44b2-b796-378922f2aa69\") " pod="cert-manager/cert-manager-79c8d999ff-bj2lm" Jun 03 16:23:19.102107 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:23:19.102078 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4f48bb40-ece5-44b2-b796-378922f2aa69-bound-sa-token\") pod \"cert-manager-79c8d999ff-bj2lm\" (UID: \"4f48bb40-ece5-44b2-b796-378922f2aa69\") " pod="cert-manager/cert-manager-79c8d999ff-bj2lm" Jun 03 16:23:19.102242 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:23:19.102122 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-lzft4\" (UniqueName: \"kubernetes.io/projected/4f48bb40-ece5-44b2-b796-378922f2aa69-kube-api-access-lzft4\") pod \"cert-manager-79c8d999ff-bj2lm\" (UID: \"4f48bb40-ece5-44b2-b796-378922f2aa69\") " pod="cert-manager/cert-manager-79c8d999ff-bj2lm" Jun 03 16:23:19.109136 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:23:19.109109 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4f48bb40-ece5-44b2-b796-378922f2aa69-bound-sa-token\") pod \"cert-manager-79c8d999ff-bj2lm\" (UID: \"4f48bb40-ece5-44b2-b796-378922f2aa69\") " pod="cert-manager/cert-manager-79c8d999ff-bj2lm" Jun 03 16:23:19.109406 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:23:19.109385 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lzft4\" (UniqueName: \"kubernetes.io/projected/4f48bb40-ece5-44b2-b796-378922f2aa69-kube-api-access-lzft4\") pod \"cert-manager-79c8d999ff-bj2lm\" (UID: \"4f48bb40-ece5-44b2-b796-378922f2aa69\") " pod="cert-manager/cert-manager-79c8d999ff-bj2lm" Jun 03 16:23:19.118631 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:23:19.118608 2568 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-marketplace/3081035efbc3f76b8ae1b663abbd2a6bc5b4896fe94818011b247755c556bmw"] Jun 03 16:23:19.121023 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:23:19.121009 2568 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/3081035efbc3f76b8ae1b663abbd2a6bc5b4896fe94818011b247755c556bmw" Jun 03 16:23:19.123137 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:23:19.123115 2568 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jun 03 16:23:19.123238 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:23:19.123166 2568 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-vsjql\"" Jun 03 16:23:19.123238 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:23:19.123190 2568 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jun 03 16:23:19.129352 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:23:19.129332 2568 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/3081035efbc3f76b8ae1b663abbd2a6bc5b4896fe94818011b247755c556bmw"] Jun 03 16:23:19.202583 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:23:19.202552 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4p6n7\" (UniqueName: \"kubernetes.io/projected/c84016cb-5bd2-4ec8-9078-1ca3d986d03e-kube-api-access-4p6n7\") pod \"3081035efbc3f76b8ae1b663abbd2a6bc5b4896fe94818011b247755c556bmw\" (UID: \"c84016cb-5bd2-4ec8-9078-1ca3d986d03e\") " pod="openshift-marketplace/3081035efbc3f76b8ae1b663abbd2a6bc5b4896fe94818011b247755c556bmw" Jun 03 16:23:19.202735 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:23:19.202618 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c84016cb-5bd2-4ec8-9078-1ca3d986d03e-bundle\") pod \"3081035efbc3f76b8ae1b663abbd2a6bc5b4896fe94818011b247755c556bmw\" (UID: \"c84016cb-5bd2-4ec8-9078-1ca3d986d03e\") " pod="openshift-marketplace/3081035efbc3f76b8ae1b663abbd2a6bc5b4896fe94818011b247755c556bmw" Jun 03 16:23:19.202735 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:23:19.202700 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c84016cb-5bd2-4ec8-9078-1ca3d986d03e-util\") pod \"3081035efbc3f76b8ae1b663abbd2a6bc5b4896fe94818011b247755c556bmw\" (UID: \"c84016cb-5bd2-4ec8-9078-1ca3d986d03e\") " pod="openshift-marketplace/3081035efbc3f76b8ae1b663abbd2a6bc5b4896fe94818011b247755c556bmw" Jun 03 16:23:19.292025 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:23:19.291967 2568 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-79c8d999ff-bj2lm" Jun 03 16:23:19.303064 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:23:19.303040 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c84016cb-5bd2-4ec8-9078-1ca3d986d03e-bundle\") pod \"3081035efbc3f76b8ae1b663abbd2a6bc5b4896fe94818011b247755c556bmw\" (UID: \"c84016cb-5bd2-4ec8-9078-1ca3d986d03e\") " pod="openshift-marketplace/3081035efbc3f76b8ae1b663abbd2a6bc5b4896fe94818011b247755c556bmw" Jun 03 16:23:19.303186 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:23:19.303093 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c84016cb-5bd2-4ec8-9078-1ca3d986d03e-util\") pod \"3081035efbc3f76b8ae1b663abbd2a6bc5b4896fe94818011b247755c556bmw\" (UID: \"c84016cb-5bd2-4ec8-9078-1ca3d986d03e\") " pod="openshift-marketplace/3081035efbc3f76b8ae1b663abbd2a6bc5b4896fe94818011b247755c556bmw" Jun 03 16:23:19.303186 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:23:19.303139 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-4p6n7\" (UniqueName: \"kubernetes.io/projected/c84016cb-5bd2-4ec8-9078-1ca3d986d03e-kube-api-access-4p6n7\") pod \"3081035efbc3f76b8ae1b663abbd2a6bc5b4896fe94818011b247755c556bmw\" (UID: \"c84016cb-5bd2-4ec8-9078-1ca3d986d03e\") " pod="openshift-marketplace/3081035efbc3f76b8ae1b663abbd2a6bc5b4896fe94818011b247755c556bmw" Jun 03 16:23:19.303436 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:23:19.303416 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c84016cb-5bd2-4ec8-9078-1ca3d986d03e-bundle\") pod \"3081035efbc3f76b8ae1b663abbd2a6bc5b4896fe94818011b247755c556bmw\" (UID: \"c84016cb-5bd2-4ec8-9078-1ca3d986d03e\") " pod="openshift-marketplace/3081035efbc3f76b8ae1b663abbd2a6bc5b4896fe94818011b247755c556bmw" Jun 03 16:23:19.303595 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:23:19.303521 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c84016cb-5bd2-4ec8-9078-1ca3d986d03e-util\") pod \"3081035efbc3f76b8ae1b663abbd2a6bc5b4896fe94818011b247755c556bmw\" (UID: \"c84016cb-5bd2-4ec8-9078-1ca3d986d03e\") " pod="openshift-marketplace/3081035efbc3f76b8ae1b663abbd2a6bc5b4896fe94818011b247755c556bmw" Jun 03 16:23:19.311582 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:23:19.311555 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4p6n7\" (UniqueName: \"kubernetes.io/projected/c84016cb-5bd2-4ec8-9078-1ca3d986d03e-kube-api-access-4p6n7\") pod \"3081035efbc3f76b8ae1b663abbd2a6bc5b4896fe94818011b247755c556bmw\" (UID: \"c84016cb-5bd2-4ec8-9078-1ca3d986d03e\") " pod="openshift-marketplace/3081035efbc3f76b8ae1b663abbd2a6bc5b4896fe94818011b247755c556bmw" Jun 03 16:23:19.416805 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:23:19.416772 2568 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-79c8d999ff-bj2lm"] Jun 03 16:23:19.419808 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:23:19.419783 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4f48bb40_ece5_44b2_b796_378922f2aa69.slice/crio-6c1e59b0f488f53ac5e8c93c1dd4602ef6874a88af676d5b91a1d9afc7716cb7 WatchSource:0}: Error finding container 6c1e59b0f488f53ac5e8c93c1dd4602ef6874a88af676d5b91a1d9afc7716cb7: Status 404 returned error can't find the container with id 6c1e59b0f488f53ac5e8c93c1dd4602ef6874a88af676d5b91a1d9afc7716cb7 Jun 03 16:23:19.430608 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:23:19.430590 2568 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/3081035efbc3f76b8ae1b663abbd2a6bc5b4896fe94818011b247755c556bmw" Jun 03 16:23:19.560366 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:23:19.560289 2568 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/3081035efbc3f76b8ae1b663abbd2a6bc5b4896fe94818011b247755c556bmw"] Jun 03 16:23:19.563471 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:23:19.563447 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc84016cb_5bd2_4ec8_9078_1ca3d986d03e.slice/crio-d50c5966235c29ddb3435551e2bfac4f34de46b5481c7972c17ea1a309ca143f WatchSource:0}: Error finding container d50c5966235c29ddb3435551e2bfac4f34de46b5481c7972c17ea1a309ca143f: Status 404 returned error can't find the container with id d50c5966235c29ddb3435551e2bfac4f34de46b5481c7972c17ea1a309ca143f Jun 03 16:23:19.802203 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:23:19.802168 2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-79c8d999ff-bj2lm" event={"ID":"4f48bb40-ece5-44b2-b796-378922f2aa69","Type":"ContainerStarted","Data":"838954dd975d157567a690a3a8cbcff672cf00e0e164cdb228bb66ca634dd11c"} Jun 03 16:23:19.802383 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:23:19.802209 2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-79c8d999ff-bj2lm" event={"ID":"4f48bb40-ece5-44b2-b796-378922f2aa69","Type":"ContainerStarted","Data":"6c1e59b0f488f53ac5e8c93c1dd4602ef6874a88af676d5b91a1d9afc7716cb7"} Jun 03 16:23:19.803355 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:23:19.803330 2568 generic.go:358] "Generic (PLEG): container finished" podID="c84016cb-5bd2-4ec8-9078-1ca3d986d03e" containerID="de800a46e146711c0aae34e5ffa89cbe43e8e0d3df680ceca0500984d76b7d48" exitCode=0 Jun 03 16:23:19.803454 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:23:19.803383 2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/3081035efbc3f76b8ae1b663abbd2a6bc5b4896fe94818011b247755c556bmw" event={"ID":"c84016cb-5bd2-4ec8-9078-1ca3d986d03e","Type":"ContainerDied","Data":"de800a46e146711c0aae34e5ffa89cbe43e8e0d3df680ceca0500984d76b7d48"} Jun 03 16:23:19.803454 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:23:19.803402 2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/3081035efbc3f76b8ae1b663abbd2a6bc5b4896fe94818011b247755c556bmw" event={"ID":"c84016cb-5bd2-4ec8-9078-1ca3d986d03e","Type":"ContainerStarted","Data":"d50c5966235c29ddb3435551e2bfac4f34de46b5481c7972c17ea1a309ca143f"} Jun 03 16:23:19.819006 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:23:19.818941 2568 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="cert-manager/cert-manager-79c8d999ff-bj2lm" podStartSLOduration=1.818927111 podStartE2EDuration="1.818927111s" podCreationTimestamp="2026-06-03 16:23:18 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-06-03 16:23:19.817078037 +0000 UTC m=+370.905504627" watchObservedRunningTime="2026-06-03 16:23:19.818927111 +0000 UTC m=+370.907353702" Jun 03 16:23:20.808279 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:23:20.808186 2568 generic.go:358] "Generic (PLEG): container finished" podID="c84016cb-5bd2-4ec8-9078-1ca3d986d03e" containerID="789ebc4a92944f3aed1e52cbdb40793cbfc16ce3914b5aab86a6dbba461bf40d" exitCode=0 Jun 03 16:23:20.808641 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:23:20.808280 2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/3081035efbc3f76b8ae1b663abbd2a6bc5b4896fe94818011b247755c556bmw" event={"ID":"c84016cb-5bd2-4ec8-9078-1ca3d986d03e","Type":"ContainerDied","Data":"789ebc4a92944f3aed1e52cbdb40793cbfc16ce3914b5aab86a6dbba461bf40d"} Jun 03 16:23:21.813206 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:23:21.813168 2568 generic.go:358] "Generic (PLEG): container finished" podID="c84016cb-5bd2-4ec8-9078-1ca3d986d03e" containerID="e3633f149bed0750557a83291056bab78f7974dfa489e2a3766252fbf224e507" exitCode=0 Jun 03 16:23:21.813684 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:23:21.813234 2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/3081035efbc3f76b8ae1b663abbd2a6bc5b4896fe94818011b247755c556bmw" event={"ID":"c84016cb-5bd2-4ec8-9078-1ca3d986d03e","Type":"ContainerDied","Data":"e3633f149bed0750557a83291056bab78f7974dfa489e2a3766252fbf224e507"} Jun 03 16:23:22.937463 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:23:22.937438 2568 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/3081035efbc3f76b8ae1b663abbd2a6bc5b4896fe94818011b247755c556bmw" Jun 03 16:23:23.030188 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:23:23.030160 2568 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4p6n7\" (UniqueName: \"kubernetes.io/projected/c84016cb-5bd2-4ec8-9078-1ca3d986d03e-kube-api-access-4p6n7\") pod \"c84016cb-5bd2-4ec8-9078-1ca3d986d03e\" (UID: \"c84016cb-5bd2-4ec8-9078-1ca3d986d03e\") " Jun 03 16:23:23.030353 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:23:23.030228 2568 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c84016cb-5bd2-4ec8-9078-1ca3d986d03e-bundle\") pod \"c84016cb-5bd2-4ec8-9078-1ca3d986d03e\" (UID: \"c84016cb-5bd2-4ec8-9078-1ca3d986d03e\") " Jun 03 16:23:23.030353 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:23:23.030255 2568 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c84016cb-5bd2-4ec8-9078-1ca3d986d03e-util\") pod \"c84016cb-5bd2-4ec8-9078-1ca3d986d03e\" (UID: \"c84016cb-5bd2-4ec8-9078-1ca3d986d03e\") " Jun 03 16:23:23.030911 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:23:23.030884 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c84016cb-5bd2-4ec8-9078-1ca3d986d03e-bundle" (OuterVolumeSpecName: "bundle") pod "c84016cb-5bd2-4ec8-9078-1ca3d986d03e" (UID: "c84016cb-5bd2-4ec8-9078-1ca3d986d03e"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 03 16:23:23.032271 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:23:23.032238 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c84016cb-5bd2-4ec8-9078-1ca3d986d03e-kube-api-access-4p6n7" (OuterVolumeSpecName: "kube-api-access-4p6n7") pod "c84016cb-5bd2-4ec8-9078-1ca3d986d03e" (UID: "c84016cb-5bd2-4ec8-9078-1ca3d986d03e"). InnerVolumeSpecName "kube-api-access-4p6n7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 03 16:23:23.035723 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:23:23.035699 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c84016cb-5bd2-4ec8-9078-1ca3d986d03e-util" (OuterVolumeSpecName: "util") pod "c84016cb-5bd2-4ec8-9078-1ca3d986d03e" (UID: "c84016cb-5bd2-4ec8-9078-1ca3d986d03e"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 03 16:23:23.131164 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:23:23.131140 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4p6n7\" (UniqueName: \"kubernetes.io/projected/c84016cb-5bd2-4ec8-9078-1ca3d986d03e-kube-api-access-4p6n7\") on node \"ip-10-0-128-88.ec2.internal\" DevicePath \"\"" Jun 03 16:23:23.131249 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:23:23.131164 2568 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c84016cb-5bd2-4ec8-9078-1ca3d986d03e-bundle\") on node \"ip-10-0-128-88.ec2.internal\" DevicePath \"\"" Jun 03 16:23:23.131249 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:23:23.131175 2568 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c84016cb-5bd2-4ec8-9078-1ca3d986d03e-util\") on node \"ip-10-0-128-88.ec2.internal\" DevicePath \"\"" Jun 03 16:23:23.821654 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:23:23.821628 2568 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/3081035efbc3f76b8ae1b663abbd2a6bc5b4896fe94818011b247755c556bmw" Jun 03 16:23:23.821812 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:23:23.821619 2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/3081035efbc3f76b8ae1b663abbd2a6bc5b4896fe94818011b247755c556bmw" event={"ID":"c84016cb-5bd2-4ec8-9078-1ca3d986d03e","Type":"ContainerDied","Data":"d50c5966235c29ddb3435551e2bfac4f34de46b5481c7972c17ea1a309ca143f"} Jun 03 16:23:23.821812 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:23:23.821735 2568 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d50c5966235c29ddb3435551e2bfac4f34de46b5481c7972c17ea1a309ca143f" Jun 03 16:23:33.373318 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:23:33.373283 2568 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-marketplace/f77c1a82ade775f7608969957d57ee0b8db93eeada9825bd6f7f7156c97np7n"] Jun 03 16:23:33.373702 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:23:33.373601 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c84016cb-5bd2-4ec8-9078-1ca3d986d03e" containerName="util" Jun 03 16:23:33.373702 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:23:33.373612 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="c84016cb-5bd2-4ec8-9078-1ca3d986d03e" containerName="util" Jun 03 16:23:33.373702 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:23:33.373620 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c84016cb-5bd2-4ec8-9078-1ca3d986d03e" containerName="extract" Jun 03 16:23:33.373702 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:23:33.373625 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="c84016cb-5bd2-4ec8-9078-1ca3d986d03e" containerName="extract" Jun 03 16:23:33.373702 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:23:33.373633 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c84016cb-5bd2-4ec8-9078-1ca3d986d03e" containerName="pull" Jun 03 16:23:33.373702 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:23:33.373638 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="c84016cb-5bd2-4ec8-9078-1ca3d986d03e" containerName="pull" Jun 03 16:23:33.373702 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:23:33.373686 2568 memory_manager.go:356] "RemoveStaleState removing state" podUID="c84016cb-5bd2-4ec8-9078-1ca3d986d03e" containerName="extract" Jun 03 16:23:33.376320 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:23:33.376304 2568 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/f77c1a82ade775f7608969957d57ee0b8db93eeada9825bd6f7f7156c97np7n" Jun 03 16:23:33.378373 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:23:33.378347 2568 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-vsjql\"" Jun 03 16:23:33.378488 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:23:33.378456 2568 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jun 03 16:23:33.378827 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:23:33.378809 2568 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jun 03 16:23:33.385049 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:23:33.385029 2568 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/f77c1a82ade775f7608969957d57ee0b8db93eeada9825bd6f7f7156c97np7n"] Jun 03 16:23:33.405201 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:23:33.405168 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5645a5d8-9658-4c73-8c4d-ef0f7c48d722-util\") pod \"f77c1a82ade775f7608969957d57ee0b8db93eeada9825bd6f7f7156c97np7n\" (UID: \"5645a5d8-9658-4c73-8c4d-ef0f7c48d722\") " pod="openshift-marketplace/f77c1a82ade775f7608969957d57ee0b8db93eeada9825bd6f7f7156c97np7n" Jun 03 16:23:33.405322 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:23:33.405254 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5645a5d8-9658-4c73-8c4d-ef0f7c48d722-bundle\") pod \"f77c1a82ade775f7608969957d57ee0b8db93eeada9825bd6f7f7156c97np7n\" (UID: \"5645a5d8-9658-4c73-8c4d-ef0f7c48d722\") " pod="openshift-marketplace/f77c1a82ade775f7608969957d57ee0b8db93eeada9825bd6f7f7156c97np7n" Jun 03 16:23:33.405322 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:23:33.405289 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lmbnd\" (UniqueName: \"kubernetes.io/projected/5645a5d8-9658-4c73-8c4d-ef0f7c48d722-kube-api-access-lmbnd\") pod \"f77c1a82ade775f7608969957d57ee0b8db93eeada9825bd6f7f7156c97np7n\" (UID: \"5645a5d8-9658-4c73-8c4d-ef0f7c48d722\") " pod="openshift-marketplace/f77c1a82ade775f7608969957d57ee0b8db93eeada9825bd6f7f7156c97np7n" Jun 03 16:23:33.506375 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:23:33.506336 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5645a5d8-9658-4c73-8c4d-ef0f7c48d722-util\") pod \"f77c1a82ade775f7608969957d57ee0b8db93eeada9825bd6f7f7156c97np7n\" (UID: \"5645a5d8-9658-4c73-8c4d-ef0f7c48d722\") " pod="openshift-marketplace/f77c1a82ade775f7608969957d57ee0b8db93eeada9825bd6f7f7156c97np7n" Jun 03 16:23:33.506500 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:23:33.506395 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5645a5d8-9658-4c73-8c4d-ef0f7c48d722-bundle\") pod \"f77c1a82ade775f7608969957d57ee0b8db93eeada9825bd6f7f7156c97np7n\" (UID: \"5645a5d8-9658-4c73-8c4d-ef0f7c48d722\") " pod="openshift-marketplace/f77c1a82ade775f7608969957d57ee0b8db93eeada9825bd6f7f7156c97np7n" Jun 03 16:23:33.506500 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:23:33.506415 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-lmbnd\" (UniqueName: \"kubernetes.io/projected/5645a5d8-9658-4c73-8c4d-ef0f7c48d722-kube-api-access-lmbnd\") pod \"f77c1a82ade775f7608969957d57ee0b8db93eeada9825bd6f7f7156c97np7n\" (UID: \"5645a5d8-9658-4c73-8c4d-ef0f7c48d722\") " pod="openshift-marketplace/f77c1a82ade775f7608969957d57ee0b8db93eeada9825bd6f7f7156c97np7n" Jun 03 16:23:33.506808 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:23:33.506786 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5645a5d8-9658-4c73-8c4d-ef0f7c48d722-util\") pod \"f77c1a82ade775f7608969957d57ee0b8db93eeada9825bd6f7f7156c97np7n\" (UID: \"5645a5d8-9658-4c73-8c4d-ef0f7c48d722\") " pod="openshift-marketplace/f77c1a82ade775f7608969957d57ee0b8db93eeada9825bd6f7f7156c97np7n" Jun 03 16:23:33.506847 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:23:33.506797 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5645a5d8-9658-4c73-8c4d-ef0f7c48d722-bundle\") pod \"f77c1a82ade775f7608969957d57ee0b8db93eeada9825bd6f7f7156c97np7n\" (UID: \"5645a5d8-9658-4c73-8c4d-ef0f7c48d722\") " pod="openshift-marketplace/f77c1a82ade775f7608969957d57ee0b8db93eeada9825bd6f7f7156c97np7n" Jun 03 16:23:33.521460 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:23:33.521431 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lmbnd\" (UniqueName: \"kubernetes.io/projected/5645a5d8-9658-4c73-8c4d-ef0f7c48d722-kube-api-access-lmbnd\") pod \"f77c1a82ade775f7608969957d57ee0b8db93eeada9825bd6f7f7156c97np7n\" (UID: \"5645a5d8-9658-4c73-8c4d-ef0f7c48d722\") " pod="openshift-marketplace/f77c1a82ade775f7608969957d57ee0b8db93eeada9825bd6f7f7156c97np7n" Jun 03 16:23:33.685753 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:23:33.685693 2568 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/f77c1a82ade775f7608969957d57ee0b8db93eeada9825bd6f7f7156c97np7n" Jun 03 16:23:33.815441 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:23:33.815415 2568 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/f77c1a82ade775f7608969957d57ee0b8db93eeada9825bd6f7f7156c97np7n"] Jun 03 16:23:33.817464 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:23:33.817438 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5645a5d8_9658_4c73_8c4d_ef0f7c48d722.slice/crio-e91797063cdd6ecf42d324374d79d3d36724a9b10df4b0f0554fb52bed4b7e8a WatchSource:0}: Error finding container e91797063cdd6ecf42d324374d79d3d36724a9b10df4b0f0554fb52bed4b7e8a: Status 404 returned error can't find the container with id e91797063cdd6ecf42d324374d79d3d36724a9b10df4b0f0554fb52bed4b7e8a Jun 03 16:23:33.855415 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:23:33.855388 2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/f77c1a82ade775f7608969957d57ee0b8db93eeada9825bd6f7f7156c97np7n" event={"ID":"5645a5d8-9658-4c73-8c4d-ef0f7c48d722","Type":"ContainerStarted","Data":"e91797063cdd6ecf42d324374d79d3d36724a9b10df4b0f0554fb52bed4b7e8a"} Jun 03 16:23:34.146765 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:23:34.146732 2568 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6c656db94d-j7rhw"] Jun 03 16:23:34.149293 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:23:34.149277 2568 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-6c656db94d-j7rhw" Jun 03 16:23:34.151679 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:23:34.151657 2568 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jun 03 16:23:34.151933 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:23:34.151778 2568 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-lbsp8\"" Jun 03 16:23:34.151933 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:23:34.151900 2568 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jun 03 16:23:34.152068 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:23:34.152054 2568 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jun 03 16:23:34.155746 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:23:34.155590 2568 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jun 03 16:23:34.173257 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:23:34.173233 2568 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6c656db94d-j7rhw"] Jun 03 16:23:34.213324 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:23:34.213301 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5b560183-6f5f-4f17-823e-7131b664fde3-webhook-cert\") pod \"opendatahub-operator-controller-manager-6c656db94d-j7rhw\" (UID: \"5b560183-6f5f-4f17-823e-7131b664fde3\") " pod="opendatahub/opendatahub-operator-controller-manager-6c656db94d-j7rhw" Jun 03 16:23:34.213440 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:23:34.213361 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/5b560183-6f5f-4f17-823e-7131b664fde3-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6c656db94d-j7rhw\" (UID: \"5b560183-6f5f-4f17-823e-7131b664fde3\") " pod="opendatahub/opendatahub-operator-controller-manager-6c656db94d-j7rhw" Jun 03 16:23:34.213440 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:23:34.213413 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2sp2r\" (UniqueName: \"kubernetes.io/projected/5b560183-6f5f-4f17-823e-7131b664fde3-kube-api-access-2sp2r\") pod \"opendatahub-operator-controller-manager-6c656db94d-j7rhw\" (UID: \"5b560183-6f5f-4f17-823e-7131b664fde3\") " pod="opendatahub/opendatahub-operator-controller-manager-6c656db94d-j7rhw" Jun 03 16:23:34.314810 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:23:34.314774 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/5b560183-6f5f-4f17-823e-7131b664fde3-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6c656db94d-j7rhw\" (UID: \"5b560183-6f5f-4f17-823e-7131b664fde3\") " pod="opendatahub/opendatahub-operator-controller-manager-6c656db94d-j7rhw" Jun 03 16:23:34.314810 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:23:34.314821 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-2sp2r\" (UniqueName: \"kubernetes.io/projected/5b560183-6f5f-4f17-823e-7131b664fde3-kube-api-access-2sp2r\") pod \"opendatahub-operator-controller-manager-6c656db94d-j7rhw\" (UID: \"5b560183-6f5f-4f17-823e-7131b664fde3\") " pod="opendatahub/opendatahub-operator-controller-manager-6c656db94d-j7rhw" Jun 03 16:23:34.315040 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:23:34.314946 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5b560183-6f5f-4f17-823e-7131b664fde3-webhook-cert\") pod \"opendatahub-operator-controller-manager-6c656db94d-j7rhw\" (UID: \"5b560183-6f5f-4f17-823e-7131b664fde3\") " pod="opendatahub/opendatahub-operator-controller-manager-6c656db94d-j7rhw" Jun 03 16:23:34.317195 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:23:34.317174 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5b560183-6f5f-4f17-823e-7131b664fde3-webhook-cert\") pod \"opendatahub-operator-controller-manager-6c656db94d-j7rhw\" (UID: \"5b560183-6f5f-4f17-823e-7131b664fde3\") " pod="opendatahub/opendatahub-operator-controller-manager-6c656db94d-j7rhw" Jun 03 16:23:34.317287 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:23:34.317215 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/5b560183-6f5f-4f17-823e-7131b664fde3-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6c656db94d-j7rhw\" (UID: \"5b560183-6f5f-4f17-823e-7131b664fde3\") " pod="opendatahub/opendatahub-operator-controller-manager-6c656db94d-j7rhw" Jun 03 16:23:34.323763 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:23:34.323746 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2sp2r\" (UniqueName: \"kubernetes.io/projected/5b560183-6f5f-4f17-823e-7131b664fde3-kube-api-access-2sp2r\") pod \"opendatahub-operator-controller-manager-6c656db94d-j7rhw\" (UID: \"5b560183-6f5f-4f17-823e-7131b664fde3\") " pod="opendatahub/opendatahub-operator-controller-manager-6c656db94d-j7rhw" Jun 03 16:23:34.461278 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:23:34.461188 2568 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-6c656db94d-j7rhw" Jun 03 16:23:34.591989 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:23:34.591962 2568 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6c656db94d-j7rhw"] Jun 03 16:23:34.593760 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:23:34.593736 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5b560183_6f5f_4f17_823e_7131b664fde3.slice/crio-39e5708aead9b4948b47f34f4ee65e2a5c6d8c610aca8592df54e27172831723 WatchSource:0}: Error finding container 39e5708aead9b4948b47f34f4ee65e2a5c6d8c610aca8592df54e27172831723: Status 404 returned error can't find the container with id 39e5708aead9b4948b47f34f4ee65e2a5c6d8c610aca8592df54e27172831723 Jun 03 16:23:34.862229 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:23:34.862197 2568 generic.go:358] "Generic (PLEG): container finished" podID="5645a5d8-9658-4c73-8c4d-ef0f7c48d722" containerID="e011ae2a0cf758fc0f1bfb20ccbf27d6b745fa44756773d7a12a0ced972c6a53" exitCode=0 Jun 03 16:23:34.862422 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:23:34.862286 2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/f77c1a82ade775f7608969957d57ee0b8db93eeada9825bd6f7f7156c97np7n" event={"ID":"5645a5d8-9658-4c73-8c4d-ef0f7c48d722","Type":"ContainerDied","Data":"e011ae2a0cf758fc0f1bfb20ccbf27d6b745fa44756773d7a12a0ced972c6a53"} Jun 03 16:23:34.863734 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:23:34.863713 2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6c656db94d-j7rhw" event={"ID":"5b560183-6f5f-4f17-823e-7131b664fde3","Type":"ContainerStarted","Data":"39e5708aead9b4948b47f34f4ee65e2a5c6d8c610aca8592df54e27172831723"} Jun 03 16:23:35.869244 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:23:35.869210 2568 generic.go:358] "Generic (PLEG): container finished" podID="5645a5d8-9658-4c73-8c4d-ef0f7c48d722" containerID="248f77e2991165cd1754d29fac4ef88454c365cf9ba5ce6f806d815e426e1367" exitCode=0 Jun 03 16:23:35.869244 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:23:35.869247 2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/f77c1a82ade775f7608969957d57ee0b8db93eeada9825bd6f7f7156c97np7n" event={"ID":"5645a5d8-9658-4c73-8c4d-ef0f7c48d722","Type":"ContainerDied","Data":"248f77e2991165cd1754d29fac4ef88454c365cf9ba5ce6f806d815e426e1367"} Jun 03 16:23:36.876121 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:23:36.876074 2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/f77c1a82ade775f7608969957d57ee0b8db93eeada9825bd6f7f7156c97np7n" event={"ID":"5645a5d8-9658-4c73-8c4d-ef0f7c48d722","Type":"ContainerStarted","Data":"3c8a76af2b7c86a782b7ae5e72a5863005452babab0c894779724b34ff0f1675"} Jun 03 16:23:36.901850 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:23:36.901802 2568 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-marketplace/f77c1a82ade775f7608969957d57ee0b8db93eeada9825bd6f7f7156c97np7n" podStartSLOduration=3.2979569140000002 podStartE2EDuration="3.901788815s" podCreationTimestamp="2026-06-03 16:23:33 +0000 UTC" firstStartedPulling="2026-06-03 16:23:34.862885806 +0000 UTC m=+385.951312377" lastFinishedPulling="2026-06-03 16:23:35.466717706 +0000 UTC m=+386.555144278" observedRunningTime="2026-06-03 16:23:36.900266714 +0000 UTC m=+387.988693309" watchObservedRunningTime="2026-06-03 16:23:36.901788815 +0000 UTC m=+387.990215405" Jun 03 16:23:37.881553 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:23:37.881496 2568 generic.go:358] "Generic (PLEG): container finished" podID="5645a5d8-9658-4c73-8c4d-ef0f7c48d722" containerID="3c8a76af2b7c86a782b7ae5e72a5863005452babab0c894779724b34ff0f1675" exitCode=0 Jun 03 16:23:37.882017 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:23:37.881572 2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/f77c1a82ade775f7608969957d57ee0b8db93eeada9825bd6f7f7156c97np7n" event={"ID":"5645a5d8-9658-4c73-8c4d-ef0f7c48d722","Type":"ContainerDied","Data":"3c8a76af2b7c86a782b7ae5e72a5863005452babab0c894779724b34ff0f1675"} Jun 03 16:23:37.883063 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:23:37.883041 2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6c656db94d-j7rhw" event={"ID":"5b560183-6f5f-4f17-823e-7131b664fde3","Type":"ContainerStarted","Data":"f879977ec97b18174a31e87750302fe4f1efe812ea6eec500cc0b622b4b6fc80"} Jun 03 16:23:37.883172 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:23:37.883147 2568 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-6c656db94d-j7rhw" Jun 03 16:23:37.919605 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:23:37.919560 2568 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-6c656db94d-j7rhw" podStartSLOduration=1.309900231 podStartE2EDuration="3.919527587s" podCreationTimestamp="2026-06-03 16:23:34 +0000 UTC" firstStartedPulling="2026-06-03 16:23:34.596229941 +0000 UTC m=+385.684656510" lastFinishedPulling="2026-06-03 16:23:37.205857296 +0000 UTC m=+388.294283866" observedRunningTime="2026-06-03 16:23:37.91891491 +0000 UTC m=+389.007341500" watchObservedRunningTime="2026-06-03 16:23:37.919527587 +0000 UTC m=+389.007954176" Jun 03 16:23:39.007253 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:23:39.007235 2568 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/f77c1a82ade775f7608969957d57ee0b8db93eeada9825bd6f7f7156c97np7n" Jun 03 16:23:39.061044 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:23:39.061019 2568 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lmbnd\" (UniqueName: \"kubernetes.io/projected/5645a5d8-9658-4c73-8c4d-ef0f7c48d722-kube-api-access-lmbnd\") pod \"5645a5d8-9658-4c73-8c4d-ef0f7c48d722\" (UID: \"5645a5d8-9658-4c73-8c4d-ef0f7c48d722\") " Jun 03 16:23:39.061181 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:23:39.061126 2568 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5645a5d8-9658-4c73-8c4d-ef0f7c48d722-util\") pod \"5645a5d8-9658-4c73-8c4d-ef0f7c48d722\" (UID: \"5645a5d8-9658-4c73-8c4d-ef0f7c48d722\") " Jun 03 16:23:39.061181 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:23:39.061174 2568 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5645a5d8-9658-4c73-8c4d-ef0f7c48d722-bundle\") pod \"5645a5d8-9658-4c73-8c4d-ef0f7c48d722\" (UID: \"5645a5d8-9658-4c73-8c4d-ef0f7c48d722\") " Jun 03 16:23:39.061990 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:23:39.061960 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5645a5d8-9658-4c73-8c4d-ef0f7c48d722-bundle" (OuterVolumeSpecName: "bundle") pod "5645a5d8-9658-4c73-8c4d-ef0f7c48d722" (UID: "5645a5d8-9658-4c73-8c4d-ef0f7c48d722"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 03 16:23:39.063015 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:23:39.062996 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5645a5d8-9658-4c73-8c4d-ef0f7c48d722-kube-api-access-lmbnd" (OuterVolumeSpecName: "kube-api-access-lmbnd") pod "5645a5d8-9658-4c73-8c4d-ef0f7c48d722" (UID: "5645a5d8-9658-4c73-8c4d-ef0f7c48d722"). InnerVolumeSpecName "kube-api-access-lmbnd". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 03 16:23:39.069878 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:23:39.069858 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5645a5d8-9658-4c73-8c4d-ef0f7c48d722-util" (OuterVolumeSpecName: "util") pod "5645a5d8-9658-4c73-8c4d-ef0f7c48d722" (UID: "5645a5d8-9658-4c73-8c4d-ef0f7c48d722"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 03 16:23:39.161912 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:23:39.161855 2568 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5645a5d8-9658-4c73-8c4d-ef0f7c48d722-bundle\") on node \"ip-10-0-128-88.ec2.internal\" DevicePath \"\"" Jun 03 16:23:39.161912 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:23:39.161878 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lmbnd\" (UniqueName: \"kubernetes.io/projected/5645a5d8-9658-4c73-8c4d-ef0f7c48d722-kube-api-access-lmbnd\") on node \"ip-10-0-128-88.ec2.internal\" DevicePath \"\"" Jun 03 16:23:39.161912 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:23:39.161888 2568 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5645a5d8-9658-4c73-8c4d-ef0f7c48d722-util\") on node \"ip-10-0-128-88.ec2.internal\" DevicePath \"\"" Jun 03 16:23:39.891787 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:23:39.891755 2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/f77c1a82ade775f7608969957d57ee0b8db93eeada9825bd6f7f7156c97np7n" event={"ID":"5645a5d8-9658-4c73-8c4d-ef0f7c48d722","Type":"ContainerDied","Data":"e91797063cdd6ecf42d324374d79d3d36724a9b10df4b0f0554fb52bed4b7e8a"} Jun 03 16:23:39.891787 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:23:39.891787 2568 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e91797063cdd6ecf42d324374d79d3d36724a9b10df4b0f0554fb52bed4b7e8a" Jun 03 16:23:39.891977 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:23:39.891804 2568 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/f77c1a82ade775f7608969957d57ee0b8db93eeada9825bd6f7f7156c97np7n" Jun 03 16:23:40.002884 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:23:40.002853 2568 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-bd65fcf99-phvvj"] Jun 03 16:23:40.003151 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:23:40.003139 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5645a5d8-9658-4c73-8c4d-ef0f7c48d722" containerName="pull" Jun 03 16:23:40.003187 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:23:40.003152 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="5645a5d8-9658-4c73-8c4d-ef0f7c48d722" containerName="pull" Jun 03 16:23:40.003187 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:23:40.003183 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5645a5d8-9658-4c73-8c4d-ef0f7c48d722" containerName="util" Jun 03 16:23:40.003258 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:23:40.003189 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="5645a5d8-9658-4c73-8c4d-ef0f7c48d722" containerName="util" Jun 03 16:23:40.003258 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:23:40.003199 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5645a5d8-9658-4c73-8c4d-ef0f7c48d722" containerName="extract" Jun 03 16:23:40.003258 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:23:40.003205 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="5645a5d8-9658-4c73-8c4d-ef0f7c48d722" containerName="extract" Jun 03 16:23:40.003345 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:23:40.003260 2568 memory_manager.go:356] "RemoveStaleState removing state" podUID="5645a5d8-9658-4c73-8c4d-ef0f7c48d722" containerName="extract" Jun 03 16:23:40.005831 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:23:40.005816 2568 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-bd65fcf99-phvvj" Jun 03 16:23:40.012407 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:23:40.012386 2568 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-tm59q\"" Jun 03 16:23:40.012792 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:23:40.012493 2568 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Jun 03 16:23:40.012792 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:23:40.012587 2568 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Jun 03 16:23:40.012792 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:23:40.012587 2568 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Jun 03 16:23:40.019226 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:23:40.019208 2568 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-bd65fcf99-phvvj"] Jun 03 16:23:40.069529 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:23:40.069498 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/8a83f4a5-c55d-4aae-b972-3fdf6daea302-manager-config\") pod \"lws-controller-manager-bd65fcf99-phvvj\" (UID: \"8a83f4a5-c55d-4aae-b972-3fdf6daea302\") " pod="openshift-lws-operator/lws-controller-manager-bd65fcf99-phvvj" Jun 03 16:23:40.069661 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:23:40.069561 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-np6mr\" (UniqueName: \"kubernetes.io/projected/8a83f4a5-c55d-4aae-b972-3fdf6daea302-kube-api-access-np6mr\") pod \"lws-controller-manager-bd65fcf99-phvvj\" (UID: \"8a83f4a5-c55d-4aae-b972-3fdf6daea302\") " pod="openshift-lws-operator/lws-controller-manager-bd65fcf99-phvvj" Jun 03 16:23:40.069661 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:23:40.069623 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/8a83f4a5-c55d-4aae-b972-3fdf6daea302-metrics-cert\") pod \"lws-controller-manager-bd65fcf99-phvvj\" (UID: \"8a83f4a5-c55d-4aae-b972-3fdf6daea302\") " pod="openshift-lws-operator/lws-controller-manager-bd65fcf99-phvvj" Jun 03 16:23:40.069661 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:23:40.069639 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8a83f4a5-c55d-4aae-b972-3fdf6daea302-cert\") pod \"lws-controller-manager-bd65fcf99-phvvj\" (UID: \"8a83f4a5-c55d-4aae-b972-3fdf6daea302\") " pod="openshift-lws-operator/lws-controller-manager-bd65fcf99-phvvj" Jun 03 16:23:40.171044 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:23:40.170959 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/8a83f4a5-c55d-4aae-b972-3fdf6daea302-metrics-cert\") pod \"lws-controller-manager-bd65fcf99-phvvj\" (UID: \"8a83f4a5-c55d-4aae-b972-3fdf6daea302\") " pod="openshift-lws-operator/lws-controller-manager-bd65fcf99-phvvj" Jun 03 16:23:40.171044 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:23:40.170992 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8a83f4a5-c55d-4aae-b972-3fdf6daea302-cert\") pod \"lws-controller-manager-bd65fcf99-phvvj\" (UID: \"8a83f4a5-c55d-4aae-b972-3fdf6daea302\") " pod="openshift-lws-operator/lws-controller-manager-bd65fcf99-phvvj" Jun 03 16:23:40.171044 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:23:40.171027 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/8a83f4a5-c55d-4aae-b972-3fdf6daea302-manager-config\") pod \"lws-controller-manager-bd65fcf99-phvvj\" (UID: \"8a83f4a5-c55d-4aae-b972-3fdf6daea302\") " pod="openshift-lws-operator/lws-controller-manager-bd65fcf99-phvvj" Jun 03 16:23:40.171295 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:23:40.171110 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-np6mr\" (UniqueName: \"kubernetes.io/projected/8a83f4a5-c55d-4aae-b972-3fdf6daea302-kube-api-access-np6mr\") pod \"lws-controller-manager-bd65fcf99-phvvj\" (UID: \"8a83f4a5-c55d-4aae-b972-3fdf6daea302\") " pod="openshift-lws-operator/lws-controller-manager-bd65fcf99-phvvj" Jun 03 16:23:40.171643 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:23:40.171624 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/8a83f4a5-c55d-4aae-b972-3fdf6daea302-manager-config\") pod \"lws-controller-manager-bd65fcf99-phvvj\" (UID: \"8a83f4a5-c55d-4aae-b972-3fdf6daea302\") " pod="openshift-lws-operator/lws-controller-manager-bd65fcf99-phvvj" Jun 03 16:23:40.173470 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:23:40.173448 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8a83f4a5-c55d-4aae-b972-3fdf6daea302-cert\") pod \"lws-controller-manager-bd65fcf99-phvvj\" (UID: \"8a83f4a5-c55d-4aae-b972-3fdf6daea302\") " pod="openshift-lws-operator/lws-controller-manager-bd65fcf99-phvvj" Jun 03 16:23:40.173576 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:23:40.173492 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/8a83f4a5-c55d-4aae-b972-3fdf6daea302-metrics-cert\") pod \"lws-controller-manager-bd65fcf99-phvvj\" (UID: \"8a83f4a5-c55d-4aae-b972-3fdf6daea302\") " pod="openshift-lws-operator/lws-controller-manager-bd65fcf99-phvvj" Jun 03 16:23:40.182288 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:23:40.182267 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-np6mr\" (UniqueName: \"kubernetes.io/projected/8a83f4a5-c55d-4aae-b972-3fdf6daea302-kube-api-access-np6mr\") pod \"lws-controller-manager-bd65fcf99-phvvj\" (UID: \"8a83f4a5-c55d-4aae-b972-3fdf6daea302\") " pod="openshift-lws-operator/lws-controller-manager-bd65fcf99-phvvj" Jun 03 16:23:40.314224 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:23:40.314200 2568 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-bd65fcf99-phvvj" Jun 03 16:23:40.475035 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:23:40.475010 2568 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-bd65fcf99-phvvj"] Jun 03 16:23:40.476311 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:23:40.476287 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8a83f4a5_c55d_4aae_b972_3fdf6daea302.slice/crio-5ce37c529079ae05b06aed329321177d4bc827040075e98abee2b0b89e56b68c WatchSource:0}: Error finding container 5ce37c529079ae05b06aed329321177d4bc827040075e98abee2b0b89e56b68c: Status 404 returned error can't find the container with id 5ce37c529079ae05b06aed329321177d4bc827040075e98abee2b0b89e56b68c Jun 03 16:23:40.896185 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:23:40.896156 2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-bd65fcf99-phvvj" event={"ID":"8a83f4a5-c55d-4aae-b972-3fdf6daea302","Type":"ContainerStarted","Data":"5ce37c529079ae05b06aed329321177d4bc827040075e98abee2b0b89e56b68c"} Jun 03 16:23:43.907624 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:23:43.907591 2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-bd65fcf99-phvvj" event={"ID":"8a83f4a5-c55d-4aae-b972-3fdf6daea302","Type":"ContainerStarted","Data":"50e52e2c1f4a2a72db675bf8cdfe98bafddc1260dac1d13a9a32d3654f546a5e"} Jun 03 16:23:43.908008 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:23:43.907672 2568 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-bd65fcf99-phvvj" Jun 03 16:23:43.926859 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:23:43.926819 2568 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-bd65fcf99-phvvj" podStartSLOduration=2.48119975 podStartE2EDuration="4.926807081s" podCreationTimestamp="2026-06-03 16:23:39 +0000 UTC" firstStartedPulling="2026-06-03 16:23:40.477971788 +0000 UTC m=+391.566398356" lastFinishedPulling="2026-06-03 16:23:42.923579117 +0000 UTC m=+394.012005687" observedRunningTime="2026-06-03 16:23:43.925014797 +0000 UTC m=+395.013441387" watchObservedRunningTime="2026-06-03 16:23:43.926807081 +0000 UTC m=+395.015233672" Jun 03 16:23:48.888794 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:23:48.888764 2568 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-6c656db94d-j7rhw" Jun 03 16:23:54.913835 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:23:54.913805 2568 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-bd65fcf99-phvvj" Jun 03 16:24:03.144247 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:24:03.144216 2568 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-marketplace/4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835n5jc9"] Jun 03 16:24:03.147529 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:24:03.147508 2568 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835n5jc9" Jun 03 16:24:03.149681 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:24:03.149663 2568 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-vsjql\"" Jun 03 16:24:03.150063 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:24:03.150044 2568 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jun 03 16:24:03.150171 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:24:03.150155 2568 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jun 03 16:24:03.164812 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:24:03.164792 2568 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835n5jc9"] Jun 03 16:24:03.270902 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:24:03.270866 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cv6pb\" (UniqueName: \"kubernetes.io/projected/80eb26e7-7995-400b-943a-28301eae7c39-kube-api-access-cv6pb\") pod \"4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835n5jc9\" (UID: \"80eb26e7-7995-400b-943a-28301eae7c39\") " pod="openshift-marketplace/4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835n5jc9" Jun 03 16:24:03.271059 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:24:03.270968 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/80eb26e7-7995-400b-943a-28301eae7c39-util\") pod \"4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835n5jc9\" (UID: \"80eb26e7-7995-400b-943a-28301eae7c39\") " pod="openshift-marketplace/4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835n5jc9" Jun 03 16:24:03.271059 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:24:03.271019 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/80eb26e7-7995-400b-943a-28301eae7c39-bundle\") pod \"4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835n5jc9\" (UID: \"80eb26e7-7995-400b-943a-28301eae7c39\") " pod="openshift-marketplace/4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835n5jc9" Jun 03 16:24:03.372168 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:24:03.372138 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-cv6pb\" (UniqueName: \"kubernetes.io/projected/80eb26e7-7995-400b-943a-28301eae7c39-kube-api-access-cv6pb\") pod \"4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835n5jc9\" (UID: \"80eb26e7-7995-400b-943a-28301eae7c39\") " pod="openshift-marketplace/4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835n5jc9" Jun 03 16:24:03.372317 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:24:03.372192 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/80eb26e7-7995-400b-943a-28301eae7c39-util\") pod \"4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835n5jc9\" (UID: \"80eb26e7-7995-400b-943a-28301eae7c39\") " pod="openshift-marketplace/4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835n5jc9" Jun 03 16:24:03.372317 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:24:03.372217 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/80eb26e7-7995-400b-943a-28301eae7c39-bundle\") pod \"4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835n5jc9\" (UID: \"80eb26e7-7995-400b-943a-28301eae7c39\") " pod="openshift-marketplace/4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835n5jc9" Jun 03 16:24:03.372577 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:24:03.372560 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/80eb26e7-7995-400b-943a-28301eae7c39-bundle\") pod \"4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835n5jc9\" (UID: \"80eb26e7-7995-400b-943a-28301eae7c39\") " pod="openshift-marketplace/4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835n5jc9" Jun 03 16:24:03.372629 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:24:03.372611 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/80eb26e7-7995-400b-943a-28301eae7c39-util\") pod \"4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835n5jc9\" (UID: \"80eb26e7-7995-400b-943a-28301eae7c39\") " pod="openshift-marketplace/4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835n5jc9" Jun 03 16:24:03.380504 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:24:03.380481 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cv6pb\" (UniqueName: \"kubernetes.io/projected/80eb26e7-7995-400b-943a-28301eae7c39-kube-api-access-cv6pb\") pod \"4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835n5jc9\" (UID: \"80eb26e7-7995-400b-943a-28301eae7c39\") " pod="openshift-marketplace/4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835n5jc9" Jun 03 16:24:03.456883 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:24:03.456828 2568 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835n5jc9" Jun 03 16:24:03.589319 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:24:03.589296 2568 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835n5jc9"] Jun 03 16:24:03.590415 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:24:03.590386 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod80eb26e7_7995_400b_943a_28301eae7c39.slice/crio-10fb69f7eb02bd64a1c5b7aa509c6d4fd980856b6ac8779a5132ca00a0d2d504 WatchSource:0}: Error finding container 10fb69f7eb02bd64a1c5b7aa509c6d4fd980856b6ac8779a5132ca00a0d2d504: Status 404 returned error can't find the container with id 10fb69f7eb02bd64a1c5b7aa509c6d4fd980856b6ac8779a5132ca00a0d2d504 Jun 03 16:24:03.978479 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:24:03.978449 2568 generic.go:358] "Generic (PLEG): container finished" podID="80eb26e7-7995-400b-943a-28301eae7c39" containerID="ead4928ceca3c8434e3dff9c2c1d6457f8e54b15364874b311ef9a0d821c81e0" exitCode=0 Jun 03 16:24:03.978679 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:24:03.978560 2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835n5jc9" event={"ID":"80eb26e7-7995-400b-943a-28301eae7c39","Type":"ContainerDied","Data":"ead4928ceca3c8434e3dff9c2c1d6457f8e54b15364874b311ef9a0d821c81e0"} Jun 03 16:24:03.978679 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:24:03.978598 2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835n5jc9" event={"ID":"80eb26e7-7995-400b-943a-28301eae7c39","Type":"ContainerStarted","Data":"10fb69f7eb02bd64a1c5b7aa509c6d4fd980856b6ac8779a5132ca00a0d2d504"} Jun 03 16:24:04.984310 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:24:04.984219 2568 generic.go:358] "Generic (PLEG): container finished" podID="80eb26e7-7995-400b-943a-28301eae7c39" containerID="1c4589d1ed2d550a9be5909365a34b58c0abba55ab4339e0a66b86879bbdb354" exitCode=0 Jun 03 16:24:04.984676 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:24:04.984308 2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835n5jc9" event={"ID":"80eb26e7-7995-400b-943a-28301eae7c39","Type":"ContainerDied","Data":"1c4589d1ed2d550a9be5909365a34b58c0abba55ab4339e0a66b86879bbdb354"} Jun 03 16:24:05.990042 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:24:05.990009 2568 generic.go:358] "Generic (PLEG): container finished" podID="80eb26e7-7995-400b-943a-28301eae7c39" containerID="1a82a8cfbaeb06318488e869082321960c712ce0017a514d6e6b03e126190cfb" exitCode=0 Jun 03 16:24:05.990409 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:24:05.990059 2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835n5jc9" event={"ID":"80eb26e7-7995-400b-943a-28301eae7c39","Type":"ContainerDied","Data":"1a82a8cfbaeb06318488e869082321960c712ce0017a514d6e6b03e126190cfb"} Jun 03 16:24:07.118778 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:24:07.118752 2568 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835n5jc9" Jun 03 16:24:07.307524 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:24:07.307451 2568 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/80eb26e7-7995-400b-943a-28301eae7c39-bundle\") pod \"80eb26e7-7995-400b-943a-28301eae7c39\" (UID: \"80eb26e7-7995-400b-943a-28301eae7c39\") " Jun 03 16:24:07.307524 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:24:07.307503 2568 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/80eb26e7-7995-400b-943a-28301eae7c39-util\") pod \"80eb26e7-7995-400b-943a-28301eae7c39\" (UID: \"80eb26e7-7995-400b-943a-28301eae7c39\") " Jun 03 16:24:07.307764 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:24:07.307562 2568 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cv6pb\" (UniqueName: \"kubernetes.io/projected/80eb26e7-7995-400b-943a-28301eae7c39-kube-api-access-cv6pb\") pod \"80eb26e7-7995-400b-943a-28301eae7c39\" (UID: \"80eb26e7-7995-400b-943a-28301eae7c39\") " Jun 03 16:24:07.308280 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:24:07.308258 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/80eb26e7-7995-400b-943a-28301eae7c39-bundle" (OuterVolumeSpecName: "bundle") pod "80eb26e7-7995-400b-943a-28301eae7c39" (UID: "80eb26e7-7995-400b-943a-28301eae7c39"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 03 16:24:07.309604 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:24:07.309572 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/80eb26e7-7995-400b-943a-28301eae7c39-kube-api-access-cv6pb" (OuterVolumeSpecName: "kube-api-access-cv6pb") pod "80eb26e7-7995-400b-943a-28301eae7c39" (UID: "80eb26e7-7995-400b-943a-28301eae7c39"). InnerVolumeSpecName "kube-api-access-cv6pb". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 03 16:24:07.315466 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:24:07.315439 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/80eb26e7-7995-400b-943a-28301eae7c39-util" (OuterVolumeSpecName: "util") pod "80eb26e7-7995-400b-943a-28301eae7c39" (UID: "80eb26e7-7995-400b-943a-28301eae7c39"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 03 16:24:07.408802 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:24:07.408775 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cv6pb\" (UniqueName: \"kubernetes.io/projected/80eb26e7-7995-400b-943a-28301eae7c39-kube-api-access-cv6pb\") on node \"ip-10-0-128-88.ec2.internal\" DevicePath \"\"" Jun 03 16:24:07.408802 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:24:07.408800 2568 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/80eb26e7-7995-400b-943a-28301eae7c39-bundle\") on node \"ip-10-0-128-88.ec2.internal\" DevicePath \"\"" Jun 03 16:24:07.408963 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:24:07.408810 2568 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/80eb26e7-7995-400b-943a-28301eae7c39-util\") on node \"ip-10-0-128-88.ec2.internal\" DevicePath \"\"" Jun 03 16:24:07.999639 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:24:07.999604 2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835n5jc9" event={"ID":"80eb26e7-7995-400b-943a-28301eae7c39","Type":"ContainerDied","Data":"10fb69f7eb02bd64a1c5b7aa509c6d4fd980856b6ac8779a5132ca00a0d2d504"} Jun 03 16:24:07.999639 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:24:07.999633 2568 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835n5jc9" Jun 03 16:24:07.999639 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:24:07.999642 2568 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="10fb69f7eb02bd64a1c5b7aa509c6d4fd980856b6ac8779a5132ca00a0d2d504" Jun 03 16:24:17.558266 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:24:17.558231 2568 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762avzpc6"] Jun 03 16:24:17.558688 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:24:17.558570 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="80eb26e7-7995-400b-943a-28301eae7c39" containerName="util" Jun 03 16:24:17.558688 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:24:17.558581 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="80eb26e7-7995-400b-943a-28301eae7c39" containerName="util" Jun 03 16:24:17.558688 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:24:17.558598 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="80eb26e7-7995-400b-943a-28301eae7c39" containerName="extract" Jun 03 16:24:17.558688 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:24:17.558604 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="80eb26e7-7995-400b-943a-28301eae7c39" containerName="extract" Jun 03 16:24:17.558688 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:24:17.558621 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="80eb26e7-7995-400b-943a-28301eae7c39" containerName="pull" Jun 03 16:24:17.558688 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:24:17.558627 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="80eb26e7-7995-400b-943a-28301eae7c39" containerName="pull" Jun 03 16:24:17.558688 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:24:17.558675 2568 memory_manager.go:356] "RemoveStaleState removing state" podUID="80eb26e7-7995-400b-943a-28301eae7c39" containerName="extract" Jun 03 16:24:17.568816 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:24:17.568793 2568 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762avzpc6" Jun 03 16:24:17.575092 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:24:17.575040 2568 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jun 03 16:24:17.575960 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:24:17.575941 2568 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-vsjql\"" Jun 03 16:24:17.576513 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:24:17.576496 2568 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jun 03 16:24:17.580888 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:24:17.580866 2568 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762avzpc6"] Jun 03 16:24:17.582779 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:24:17.582758 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/fec6583a-7552-42de-9e98-6b6a550d276e-util\") pod \"c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762avzpc6\" (UID: \"fec6583a-7552-42de-9e98-6b6a550d276e\") " pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762avzpc6" Jun 03 16:24:17.582950 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:24:17.582931 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7lrvj\" (UniqueName: \"kubernetes.io/projected/fec6583a-7552-42de-9e98-6b6a550d276e-kube-api-access-7lrvj\") pod \"c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762avzpc6\" (UID: \"fec6583a-7552-42de-9e98-6b6a550d276e\") " pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762avzpc6" Jun 03 16:24:17.583099 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:24:17.583077 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/fec6583a-7552-42de-9e98-6b6a550d276e-bundle\") pod \"c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762avzpc6\" (UID: \"fec6583a-7552-42de-9e98-6b6a550d276e\") " pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762avzpc6" Jun 03 16:24:17.684464 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:24:17.684435 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/fec6583a-7552-42de-9e98-6b6a550d276e-util\") pod \"c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762avzpc6\" (UID: \"fec6583a-7552-42de-9e98-6b6a550d276e\") " pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762avzpc6" Jun 03 16:24:17.684620 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:24:17.684476 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-7lrvj\" (UniqueName: \"kubernetes.io/projected/fec6583a-7552-42de-9e98-6b6a550d276e-kube-api-access-7lrvj\") pod \"c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762avzpc6\" (UID: \"fec6583a-7552-42de-9e98-6b6a550d276e\") " pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762avzpc6" Jun 03 16:24:17.684620 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:24:17.684503 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/fec6583a-7552-42de-9e98-6b6a550d276e-bundle\") pod \"c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762avzpc6\" (UID: \"fec6583a-7552-42de-9e98-6b6a550d276e\") " pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762avzpc6" Jun 03 16:24:17.684897 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:24:17.684874 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/fec6583a-7552-42de-9e98-6b6a550d276e-util\") pod \"c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762avzpc6\" (UID: \"fec6583a-7552-42de-9e98-6b6a550d276e\") " pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762avzpc6" Jun 03 16:24:17.684950 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:24:17.684886 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/fec6583a-7552-42de-9e98-6b6a550d276e-bundle\") pod \"c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762avzpc6\" (UID: \"fec6583a-7552-42de-9e98-6b6a550d276e\") " pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762avzpc6" Jun 03 16:24:17.696524 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:24:17.696490 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7lrvj\" (UniqueName: \"kubernetes.io/projected/fec6583a-7552-42de-9e98-6b6a550d276e-kube-api-access-7lrvj\") pod \"c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762avzpc6\" (UID: \"fec6583a-7552-42de-9e98-6b6a550d276e\") " pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762avzpc6" Jun 03 16:24:17.887112 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:24:17.887080 2568 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762avzpc6" Jun 03 16:24:18.027643 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:24:18.027616 2568 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762avzpc6"] Jun 03 16:24:18.029323 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:24:18.029295 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfec6583a_7552_42de_9e98_6b6a550d276e.slice/crio-93851ed69b9c04485adc1daa9e914b686cd62a376a0622756d9a87fdcb04912c WatchSource:0}: Error finding container 93851ed69b9c04485adc1daa9e914b686cd62a376a0622756d9a87fdcb04912c: Status 404 returned error can't find the container with id 93851ed69b9c04485adc1daa9e914b686cd62a376a0622756d9a87fdcb04912c Jun 03 16:24:18.035970 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:24:18.035943 2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762avzpc6" event={"ID":"fec6583a-7552-42de-9e98-6b6a550d276e","Type":"ContainerStarted","Data":"93851ed69b9c04485adc1daa9e914b686cd62a376a0622756d9a87fdcb04912c"} Jun 03 16:24:19.041593 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:24:19.041466 2568 generic.go:358] "Generic (PLEG): container finished" podID="fec6583a-7552-42de-9e98-6b6a550d276e" containerID="1fbc0aef24dc5d5d53bb6f919e91df7f35c765bcd6df7279e94fe085044eb29e" exitCode=0 Jun 03 16:24:19.041593 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:24:19.041577 2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762avzpc6" event={"ID":"fec6583a-7552-42de-9e98-6b6a550d276e","Type":"ContainerDied","Data":"1fbc0aef24dc5d5d53bb6f919e91df7f35c765bcd6df7279e94fe085044eb29e"} Jun 03 16:24:20.046654 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:24:20.046566 2568 generic.go:358] "Generic (PLEG): container finished" podID="fec6583a-7552-42de-9e98-6b6a550d276e" containerID="20e8340b160fed1fe524802cd08498b200bca0543351f294244fc7d0222b9fe9" exitCode=0 Jun 03 16:24:20.046654 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:24:20.046641 2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762avzpc6" event={"ID":"fec6583a-7552-42de-9e98-6b6a550d276e","Type":"ContainerDied","Data":"20e8340b160fed1fe524802cd08498b200bca0543351f294244fc7d0222b9fe9"} Jun 03 16:24:21.052953 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:24:21.052915 2568 generic.go:358] "Generic (PLEG): container finished" podID="fec6583a-7552-42de-9e98-6b6a550d276e" containerID="5a4a4981e57a98be62cb81a3a401e99e4476e4beb112e703b053a8ccf29a8e7b" exitCode=0 Jun 03 16:24:21.053330 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:24:21.052974 2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762avzpc6" event={"ID":"fec6583a-7552-42de-9e98-6b6a550d276e","Type":"ContainerDied","Data":"5a4a4981e57a98be62cb81a3a401e99e4476e4beb112e703b053a8ccf29a8e7b"} Jun 03 16:24:22.181186 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:24:22.181164 2568 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762avzpc6" Jun 03 16:24:22.222373 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:24:22.222349 2568 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/fec6583a-7552-42de-9e98-6b6a550d276e-util\") pod \"fec6583a-7552-42de-9e98-6b6a550d276e\" (UID: \"fec6583a-7552-42de-9e98-6b6a550d276e\") " Jun 03 16:24:22.222502 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:24:22.222412 2568 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/fec6583a-7552-42de-9e98-6b6a550d276e-bundle\") pod \"fec6583a-7552-42de-9e98-6b6a550d276e\" (UID: \"fec6583a-7552-42de-9e98-6b6a550d276e\") " Jun 03 16:24:22.222502 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:24:22.222434 2568 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7lrvj\" (UniqueName: \"kubernetes.io/projected/fec6583a-7552-42de-9e98-6b6a550d276e-kube-api-access-7lrvj\") pod \"fec6583a-7552-42de-9e98-6b6a550d276e\" (UID: \"fec6583a-7552-42de-9e98-6b6a550d276e\") " Jun 03 16:24:22.223250 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:24:22.223225 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fec6583a-7552-42de-9e98-6b6a550d276e-bundle" (OuterVolumeSpecName: "bundle") pod "fec6583a-7552-42de-9e98-6b6a550d276e" (UID: "fec6583a-7552-42de-9e98-6b6a550d276e"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 03 16:24:22.224411 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:24:22.224390 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fec6583a-7552-42de-9e98-6b6a550d276e-kube-api-access-7lrvj" (OuterVolumeSpecName: "kube-api-access-7lrvj") pod "fec6583a-7552-42de-9e98-6b6a550d276e" (UID: "fec6583a-7552-42de-9e98-6b6a550d276e"). InnerVolumeSpecName "kube-api-access-7lrvj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 03 16:24:22.227835 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:24:22.227809 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fec6583a-7552-42de-9e98-6b6a550d276e-util" (OuterVolumeSpecName: "util") pod "fec6583a-7552-42de-9e98-6b6a550d276e" (UID: "fec6583a-7552-42de-9e98-6b6a550d276e"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 03 16:24:22.323744 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:24:22.323673 2568 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/fec6583a-7552-42de-9e98-6b6a550d276e-bundle\") on node \"ip-10-0-128-88.ec2.internal\" DevicePath \"\"" Jun 03 16:24:22.323744 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:24:22.323702 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7lrvj\" (UniqueName: \"kubernetes.io/projected/fec6583a-7552-42de-9e98-6b6a550d276e-kube-api-access-7lrvj\") on node \"ip-10-0-128-88.ec2.internal\" DevicePath \"\"" Jun 03 16:24:22.323744 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:24:22.323713 2568 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/fec6583a-7552-42de-9e98-6b6a550d276e-util\") on node \"ip-10-0-128-88.ec2.internal\" DevicePath \"\"" Jun 03 16:24:23.063933 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:24:23.063899 2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762avzpc6" event={"ID":"fec6583a-7552-42de-9e98-6b6a550d276e","Type":"ContainerDied","Data":"93851ed69b9c04485adc1daa9e914b686cd62a376a0622756d9a87fdcb04912c"} Jun 03 16:24:23.063933 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:24:23.063922 2568 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762avzpc6" Jun 03 16:24:23.063933 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:24:23.063932 2568 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="93851ed69b9c04485adc1daa9e914b686cd62a376a0622756d9a87fdcb04912c" Jun 03 16:24:39.289521 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:24:39.289490 2568 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4bsjmtj"] Jun 03 16:24:39.289993 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:24:39.289836 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fec6583a-7552-42de-9e98-6b6a550d276e" containerName="extract" Jun 03 16:24:39.289993 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:24:39.289851 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="fec6583a-7552-42de-9e98-6b6a550d276e" containerName="extract" Jun 03 16:24:39.289993 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:24:39.289865 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fec6583a-7552-42de-9e98-6b6a550d276e" containerName="util" Jun 03 16:24:39.289993 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:24:39.289870 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="fec6583a-7552-42de-9e98-6b6a550d276e" containerName="util" Jun 03 16:24:39.289993 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:24:39.289884 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fec6583a-7552-42de-9e98-6b6a550d276e" containerName="pull" Jun 03 16:24:39.289993 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:24:39.289889 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="fec6583a-7552-42de-9e98-6b6a550d276e" containerName="pull" Jun 03 16:24:39.289993 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:24:39.289939 2568 memory_manager.go:356] "RemoveStaleState removing state" podUID="fec6583a-7552-42de-9e98-6b6a550d276e" containerName="extract" Jun 03 16:24:39.294639 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:24:39.294611 2568 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4bsjmtj" Jun 03 16:24:39.296818 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:24:39.296796 2568 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jun 03 16:24:39.297085 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:24:39.297062 2568 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jun 03 16:24:39.297217 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:24:39.297119 2568 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jun 03 16:24:39.297217 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:24:39.297136 2568 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-hwnbk\"" Jun 03 16:24:39.304742 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:24:39.304719 2568 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4bsjmtj"] Jun 03 16:24:39.362643 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:24:39.362617 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/44aff344-e167-440b-b710-a464694c38af-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-686dffbf4bsjmtj\" (UID: \"44aff344-e167-440b-b710-a464694c38af\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4bsjmtj" Jun 03 16:24:39.362773 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:24:39.362656 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/44aff344-e167-440b-b710-a464694c38af-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-686dffbf4bsjmtj\" (UID: \"44aff344-e167-440b-b710-a464694c38af\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4bsjmtj" Jun 03 16:24:39.362773 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:24:39.362687 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/44aff344-e167-440b-b710-a464694c38af-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-686dffbf4bsjmtj\" (UID: \"44aff344-e167-440b-b710-a464694c38af\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4bsjmtj" Jun 03 16:24:39.362773 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:24:39.362735 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/44aff344-e167-440b-b710-a464694c38af-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-686dffbf4bsjmtj\" (UID: \"44aff344-e167-440b-b710-a464694c38af\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4bsjmtj" Jun 03 16:24:39.362773 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:24:39.362766 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/44aff344-e167-440b-b710-a464694c38af-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-686dffbf4bsjmtj\" (UID: \"44aff344-e167-440b-b710-a464694c38af\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4bsjmtj" Jun 03 16:24:39.362903 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:24:39.362804 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/44aff344-e167-440b-b710-a464694c38af-istio-token\") pod \"data-science-gateway-data-science-gateway-class-686dffbf4bsjmtj\" (UID: \"44aff344-e167-440b-b710-a464694c38af\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4bsjmtj" Jun 03 16:24:39.362903 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:24:39.362824 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/44aff344-e167-440b-b710-a464694c38af-istio-data\") pod \"data-science-gateway-data-science-gateway-class-686dffbf4bsjmtj\" (UID: \"44aff344-e167-440b-b710-a464694c38af\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4bsjmtj" Jun 03 16:24:39.362903 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:24:39.362880 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w2pvv\" (UniqueName: \"kubernetes.io/projected/44aff344-e167-440b-b710-a464694c38af-kube-api-access-w2pvv\") pod \"data-science-gateway-data-science-gateway-class-686dffbf4bsjmtj\" (UID: \"44aff344-e167-440b-b710-a464694c38af\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4bsjmtj" Jun 03 16:24:39.362998 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:24:39.362924 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/44aff344-e167-440b-b710-a464694c38af-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-686dffbf4bsjmtj\" (UID: \"44aff344-e167-440b-b710-a464694c38af\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4bsjmtj" Jun 03 16:24:39.463768 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:24:39.463743 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/44aff344-e167-440b-b710-a464694c38af-istio-data\") pod \"data-science-gateway-data-science-gateway-class-686dffbf4bsjmtj\" (UID: \"44aff344-e167-440b-b710-a464694c38af\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4bsjmtj" Jun 03 16:24:39.463912 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:24:39.463796 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-w2pvv\" (UniqueName: \"kubernetes.io/projected/44aff344-e167-440b-b710-a464694c38af-kube-api-access-w2pvv\") pod \"data-science-gateway-data-science-gateway-class-686dffbf4bsjmtj\" (UID: \"44aff344-e167-440b-b710-a464694c38af\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4bsjmtj" Jun 03 16:24:39.463912 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:24:39.463834 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/44aff344-e167-440b-b710-a464694c38af-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-686dffbf4bsjmtj\" (UID: \"44aff344-e167-440b-b710-a464694c38af\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4bsjmtj" Jun 03 16:24:39.463912 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:24:39.463860 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/44aff344-e167-440b-b710-a464694c38af-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-686dffbf4bsjmtj\" (UID: \"44aff344-e167-440b-b710-a464694c38af\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4bsjmtj" Jun 03 16:24:39.463912 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:24:39.463886 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/44aff344-e167-440b-b710-a464694c38af-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-686dffbf4bsjmtj\" (UID: \"44aff344-e167-440b-b710-a464694c38af\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4bsjmtj" Jun 03 16:24:39.464126 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:24:39.464007 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/44aff344-e167-440b-b710-a464694c38af-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-686dffbf4bsjmtj\" (UID: \"44aff344-e167-440b-b710-a464694c38af\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4bsjmtj" Jun 03 16:24:39.464126 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:24:39.464054 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/44aff344-e167-440b-b710-a464694c38af-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-686dffbf4bsjmtj\" (UID: \"44aff344-e167-440b-b710-a464694c38af\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4bsjmtj" Jun 03 16:24:39.464126 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:24:39.464087 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/44aff344-e167-440b-b710-a464694c38af-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-686dffbf4bsjmtj\" (UID: \"44aff344-e167-440b-b710-a464694c38af\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4bsjmtj" Jun 03 16:24:39.464266 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:24:39.464125 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/44aff344-e167-440b-b710-a464694c38af-istio-token\") pod \"data-science-gateway-data-science-gateway-class-686dffbf4bsjmtj\" (UID: \"44aff344-e167-440b-b710-a464694c38af\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4bsjmtj" Jun 03 16:24:39.464266 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:24:39.464165 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/44aff344-e167-440b-b710-a464694c38af-istio-data\") pod \"data-science-gateway-data-science-gateway-class-686dffbf4bsjmtj\" (UID: \"44aff344-e167-440b-b710-a464694c38af\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4bsjmtj" Jun 03 16:24:39.464266 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:24:39.464212 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/44aff344-e167-440b-b710-a464694c38af-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-686dffbf4bsjmtj\" (UID: \"44aff344-e167-440b-b710-a464694c38af\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4bsjmtj" Jun 03 16:24:39.464398 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:24:39.464308 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/44aff344-e167-440b-b710-a464694c38af-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-686dffbf4bsjmtj\" (UID: \"44aff344-e167-440b-b710-a464694c38af\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4bsjmtj" Jun 03 16:24:39.464398 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:24:39.464342 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/44aff344-e167-440b-b710-a464694c38af-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-686dffbf4bsjmtj\" (UID: \"44aff344-e167-440b-b710-a464694c38af\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4bsjmtj" Jun 03 16:24:39.464765 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:24:39.464740 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/44aff344-e167-440b-b710-a464694c38af-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-686dffbf4bsjmtj\" (UID: \"44aff344-e167-440b-b710-a464694c38af\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4bsjmtj" Jun 03 16:24:39.466397 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:24:39.466377 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/44aff344-e167-440b-b710-a464694c38af-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-686dffbf4bsjmtj\" (UID: \"44aff344-e167-440b-b710-a464694c38af\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4bsjmtj" Jun 03 16:24:39.466627 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:24:39.466610 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/44aff344-e167-440b-b710-a464694c38af-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-686dffbf4bsjmtj\" (UID: \"44aff344-e167-440b-b710-a464694c38af\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4bsjmtj" Jun 03 16:24:39.472443 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:24:39.472417 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w2pvv\" (UniqueName: \"kubernetes.io/projected/44aff344-e167-440b-b710-a464694c38af-kube-api-access-w2pvv\") pod \"data-science-gateway-data-science-gateway-class-686dffbf4bsjmtj\" (UID: \"44aff344-e167-440b-b710-a464694c38af\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4bsjmtj" Jun 03 16:24:39.475589 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:24:39.475568 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/44aff344-e167-440b-b710-a464694c38af-istio-token\") pod \"data-science-gateway-data-science-gateway-class-686dffbf4bsjmtj\" (UID: \"44aff344-e167-440b-b710-a464694c38af\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4bsjmtj" Jun 03 16:24:39.606683 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:24:39.606657 2568 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4bsjmtj" Jun 03 16:24:39.741297 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:24:39.741271 2568 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4bsjmtj"] Jun 03 16:24:39.743731 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:24:39.743705 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod44aff344_e167_440b_b710_a464694c38af.slice/crio-ad65b1c599feb8734d11aaa94df434a817ce83fe8e6c04cc48591abd2d97b4d4 WatchSource:0}: Error finding container ad65b1c599feb8734d11aaa94df434a817ce83fe8e6c04cc48591abd2d97b4d4: Status 404 returned error can't find the container with id ad65b1c599feb8734d11aaa94df434a817ce83fe8e6c04cc48591abd2d97b4d4 Jun 03 16:24:40.131023 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:24:40.130989 2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4bsjmtj" event={"ID":"44aff344-e167-440b-b710-a464694c38af","Type":"ContainerStarted","Data":"ad65b1c599feb8734d11aaa94df434a817ce83fe8e6c04cc48591abd2d97b4d4"} Jun 03 16:24:42.362249 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:24:42.362217 2568 kubelet_resources.go:45] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jun 03 16:24:42.362484 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:24:42.362287 2568 kubelet_resources.go:45] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jun 03 16:24:42.362484 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:24:42.362337 2568 kubelet_resources.go:45] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jun 03 16:24:43.144028 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:24:43.143992 2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4bsjmtj" event={"ID":"44aff344-e167-440b-b710-a464694c38af","Type":"ContainerStarted","Data":"bde4398f35e12306feb53d06951a41caa12d29ca41153e11ade7aab77505ba10"} Jun 03 16:24:43.164525 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:24:43.164476 2568 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4bsjmtj" podStartSLOduration=1.5480618019999999 podStartE2EDuration="4.164462496s" podCreationTimestamp="2026-06-03 16:24:39 +0000 UTC" firstStartedPulling="2026-06-03 16:24:39.745562973 +0000 UTC m=+450.833989541" lastFinishedPulling="2026-06-03 16:24:42.361963664 +0000 UTC m=+453.450390235" observedRunningTime="2026-06-03 16:24:43.162982719 +0000 UTC m=+454.251409309" watchObservedRunningTime="2026-06-03 16:24:43.164462496 +0000 UTC m=+454.252889086" Jun 03 16:24:43.607152 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:24:43.607126 2568 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4bsjmtj" Jun 03 16:24:43.611511 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:24:43.611489 2568 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4bsjmtj" Jun 03 16:24:44.147868 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:24:44.147834 2568 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4bsjmtj" Jun 03 16:24:44.148834 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:24:44.148814 2568 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-686dffbf4bsjmtj" Jun 03 16:24:52.780688 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:24:52.780657 2568 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-8tndb"] Jun 03 16:24:52.784645 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:24:52.784625 2568 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-8tndb" Jun 03 16:24:52.788759 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:24:52.788742 2568 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jun 03 16:24:52.788866 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:24:52.788743 2568 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jun 03 16:24:52.788866 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:24:52.788743 2568 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-catalog-dockercfg-cqrvb\"" Jun 03 16:24:52.801235 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:24:52.801214 2568 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-8tndb"] Jun 03 16:24:52.875830 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:24:52.875804 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fhdbq\" (UniqueName: \"kubernetes.io/projected/96d5ee00-4eef-46b9-9360-36ddaa7b2800-kube-api-access-fhdbq\") pod \"kuadrant-operator-catalog-8tndb\" (UID: \"96d5ee00-4eef-46b9-9360-36ddaa7b2800\") " pod="kuadrant-system/kuadrant-operator-catalog-8tndb" Jun 03 16:24:52.976843 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:24:52.976814 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-fhdbq\" (UniqueName: \"kubernetes.io/projected/96d5ee00-4eef-46b9-9360-36ddaa7b2800-kube-api-access-fhdbq\") pod \"kuadrant-operator-catalog-8tndb\" (UID: \"96d5ee00-4eef-46b9-9360-36ddaa7b2800\") " pod="kuadrant-system/kuadrant-operator-catalog-8tndb" Jun 03 16:24:52.986007 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:24:52.985975 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fhdbq\" (UniqueName: \"kubernetes.io/projected/96d5ee00-4eef-46b9-9360-36ddaa7b2800-kube-api-access-fhdbq\") pod \"kuadrant-operator-catalog-8tndb\" (UID: \"96d5ee00-4eef-46b9-9360-36ddaa7b2800\") " pod="kuadrant-system/kuadrant-operator-catalog-8tndb" Jun 03 16:24:53.093558 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:24:53.093478 2568 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-8tndb" Jun 03 16:24:53.120457 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:24:53.120433 2568 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-8tndb"] Jun 03 16:24:53.222000 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:24:53.221968 2568 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-8tndb"] Jun 03 16:24:53.224494 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:24:53.224468 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod96d5ee00_4eef_46b9_9360_36ddaa7b2800.slice/crio-805e868064b98405d47a2ec773efea01b61464820cf72f05ac68cddaf9e6b6c5 WatchSource:0}: Error finding container 805e868064b98405d47a2ec773efea01b61464820cf72f05ac68cddaf9e6b6c5: Status 404 returned error can't find the container with id 805e868064b98405d47a2ec773efea01b61464820cf72f05ac68cddaf9e6b6c5 Jun 03 16:24:53.332859 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:24:53.332834 2568 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-pjwz8"] Jun 03 16:24:53.337879 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:24:53.337864 2568 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-pjwz8" Jun 03 16:24:53.343301 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:24:53.343277 2568 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-pjwz8"] Jun 03 16:24:53.481243 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:24:53.481211 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dsw7l\" (UniqueName: \"kubernetes.io/projected/6b856dd2-5f84-428d-8cf3-5cb0a80302f1-kube-api-access-dsw7l\") pod \"kuadrant-operator-catalog-pjwz8\" (UID: \"6b856dd2-5f84-428d-8cf3-5cb0a80302f1\") " pod="kuadrant-system/kuadrant-operator-catalog-pjwz8" Jun 03 16:24:53.581807 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:24:53.581783 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-dsw7l\" (UniqueName: \"kubernetes.io/projected/6b856dd2-5f84-428d-8cf3-5cb0a80302f1-kube-api-access-dsw7l\") pod \"kuadrant-operator-catalog-pjwz8\" (UID: \"6b856dd2-5f84-428d-8cf3-5cb0a80302f1\") " pod="kuadrant-system/kuadrant-operator-catalog-pjwz8" Jun 03 16:24:53.590675 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:24:53.590655 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dsw7l\" (UniqueName: \"kubernetes.io/projected/6b856dd2-5f84-428d-8cf3-5cb0a80302f1-kube-api-access-dsw7l\") pod \"kuadrant-operator-catalog-pjwz8\" (UID: \"6b856dd2-5f84-428d-8cf3-5cb0a80302f1\") " pod="kuadrant-system/kuadrant-operator-catalog-pjwz8" Jun 03 16:24:53.649406 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:24:53.649382 2568 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-pjwz8" Jun 03 16:24:53.785220 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:24:53.785196 2568 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-pjwz8"] Jun 03 16:24:53.805505 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:24:53.805476 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6b856dd2_5f84_428d_8cf3_5cb0a80302f1.slice/crio-04b8987b745771979d8789e4e565c791520949d1e0c4b82c00db1c539bebf086 WatchSource:0}: Error finding container 04b8987b745771979d8789e4e565c791520949d1e0c4b82c00db1c539bebf086: Status 404 returned error can't find the container with id 04b8987b745771979d8789e4e565c791520949d1e0c4b82c00db1c539bebf086 Jun 03 16:24:54.188397 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:24:54.188367 2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-8tndb" event={"ID":"96d5ee00-4eef-46b9-9360-36ddaa7b2800","Type":"ContainerStarted","Data":"805e868064b98405d47a2ec773efea01b61464820cf72f05ac68cddaf9e6b6c5"} Jun 03 16:24:54.189395 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:24:54.189370 2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-pjwz8" event={"ID":"6b856dd2-5f84-428d-8cf3-5cb0a80302f1","Type":"ContainerStarted","Data":"04b8987b745771979d8789e4e565c791520949d1e0c4b82c00db1c539bebf086"} Jun 03 16:24:56.198085 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:24:56.198043 2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-pjwz8" event={"ID":"6b856dd2-5f84-428d-8cf3-5cb0a80302f1","Type":"ContainerStarted","Data":"252c753e0654b90189832e17e6545350b459544ba5cb924058ded533a757b382"} Jun 03 16:24:56.199272 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:24:56.199248 2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-8tndb" event={"ID":"96d5ee00-4eef-46b9-9360-36ddaa7b2800","Type":"ContainerStarted","Data":"dbfe425302298bd07e29a51a08cd635a3a368e067cce11dd85085b8afcb80042"} Jun 03 16:24:56.199380 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:24:56.199327 2568 kuberuntime_container.go:864] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-catalog-8tndb" podUID="96d5ee00-4eef-46b9-9360-36ddaa7b2800" containerName="registry-server" containerID="cri-o://dbfe425302298bd07e29a51a08cd635a3a368e067cce11dd85085b8afcb80042" gracePeriod=2 Jun 03 16:24:56.214678 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:24:56.214628 2568 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-pjwz8" podStartSLOduration=1.5444079099999999 podStartE2EDuration="3.214611298s" podCreationTimestamp="2026-06-03 16:24:53 +0000 UTC" firstStartedPulling="2026-06-03 16:24:53.806913104 +0000 UTC m=+464.895339672" lastFinishedPulling="2026-06-03 16:24:55.477116481 +0000 UTC m=+466.565543060" observedRunningTime="2026-06-03 16:24:56.213163166 +0000 UTC m=+467.301589757" watchObservedRunningTime="2026-06-03 16:24:56.214611298 +0000 UTC m=+467.303037889" Jun 03 16:24:56.228746 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:24:56.228703 2568 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-8tndb" podStartSLOduration=1.979381394 podStartE2EDuration="4.228691014s" podCreationTimestamp="2026-06-03 16:24:52 +0000 UTC" firstStartedPulling="2026-06-03 16:24:53.22572403 +0000 UTC m=+464.314150597" lastFinishedPulling="2026-06-03 16:24:55.475033645 +0000 UTC m=+466.563460217" observedRunningTime="2026-06-03 16:24:56.226448655 +0000 UTC m=+467.314875244" watchObservedRunningTime="2026-06-03 16:24:56.228691014 +0000 UTC m=+467.317117624" Jun 03 16:24:56.436493 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:24:56.436474 2568 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-8tndb" Jun 03 16:24:56.504469 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:24:56.504401 2568 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fhdbq\" (UniqueName: \"kubernetes.io/projected/96d5ee00-4eef-46b9-9360-36ddaa7b2800-kube-api-access-fhdbq\") pod \"96d5ee00-4eef-46b9-9360-36ddaa7b2800\" (UID: \"96d5ee00-4eef-46b9-9360-36ddaa7b2800\") " Jun 03 16:24:56.506448 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:24:56.506418 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/96d5ee00-4eef-46b9-9360-36ddaa7b2800-kube-api-access-fhdbq" (OuterVolumeSpecName: "kube-api-access-fhdbq") pod "96d5ee00-4eef-46b9-9360-36ddaa7b2800" (UID: "96d5ee00-4eef-46b9-9360-36ddaa7b2800"). InnerVolumeSpecName "kube-api-access-fhdbq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 03 16:24:56.605867 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:24:56.605841 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fhdbq\" (UniqueName: \"kubernetes.io/projected/96d5ee00-4eef-46b9-9360-36ddaa7b2800-kube-api-access-fhdbq\") on node \"ip-10-0-128-88.ec2.internal\" DevicePath \"\"" Jun 03 16:24:57.204230 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:24:57.204193 2568 generic.go:358] "Generic (PLEG): container finished" podID="96d5ee00-4eef-46b9-9360-36ddaa7b2800" containerID="dbfe425302298bd07e29a51a08cd635a3a368e067cce11dd85085b8afcb80042" exitCode=0 Jun 03 16:24:57.204678 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:24:57.204250 2568 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-8tndb" Jun 03 16:24:57.204678 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:24:57.204274 2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-8tndb" event={"ID":"96d5ee00-4eef-46b9-9360-36ddaa7b2800","Type":"ContainerDied","Data":"dbfe425302298bd07e29a51a08cd635a3a368e067cce11dd85085b8afcb80042"} Jun 03 16:24:57.204678 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:24:57.204306 2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-8tndb" event={"ID":"96d5ee00-4eef-46b9-9360-36ddaa7b2800","Type":"ContainerDied","Data":"805e868064b98405d47a2ec773efea01b61464820cf72f05ac68cddaf9e6b6c5"} Jun 03 16:24:57.204678 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:24:57.204320 2568 scope.go:117] "RemoveContainer" containerID="dbfe425302298bd07e29a51a08cd635a3a368e067cce11dd85085b8afcb80042" Jun 03 16:24:57.215963 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:24:57.215932 2568 scope.go:117] "RemoveContainer" containerID="dbfe425302298bd07e29a51a08cd635a3a368e067cce11dd85085b8afcb80042" Jun 03 16:24:57.216550 ip-10-0-128-88 kubenswrapper[2568]: E0603 16:24:57.216506 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dbfe425302298bd07e29a51a08cd635a3a368e067cce11dd85085b8afcb80042\": container with ID starting with dbfe425302298bd07e29a51a08cd635a3a368e067cce11dd85085b8afcb80042 not found: ID does not exist" containerID="dbfe425302298bd07e29a51a08cd635a3a368e067cce11dd85085b8afcb80042" Jun 03 16:24:57.216649 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:24:57.216561 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dbfe425302298bd07e29a51a08cd635a3a368e067cce11dd85085b8afcb80042"} err="failed to get container status \"dbfe425302298bd07e29a51a08cd635a3a368e067cce11dd85085b8afcb80042\": rpc error: code = NotFound desc = could not find container \"dbfe425302298bd07e29a51a08cd635a3a368e067cce11dd85085b8afcb80042\": container with ID starting with dbfe425302298bd07e29a51a08cd635a3a368e067cce11dd85085b8afcb80042 not found: ID does not exist" Jun 03 16:24:57.230369 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:24:57.230345 2568 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-8tndb"] Jun 03 16:24:57.234811 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:24:57.234791 2568 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-8tndb"] Jun 03 16:24:57.564280 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:24:57.564205 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="96d5ee00-4eef-46b9-9360-36ddaa7b2800" path="/var/lib/kubelet/pods/96d5ee00-4eef-46b9-9360-36ddaa7b2800/volumes" Jun 03 16:25:00.860151 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:25:00.860118 2568 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-console/console-6c84757667-7mnxn"] Jun 03 16:25:00.860609 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:25:00.860592 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="96d5ee00-4eef-46b9-9360-36ddaa7b2800" containerName="registry-server" Jun 03 16:25:00.860609 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:25:00.860610 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="96d5ee00-4eef-46b9-9360-36ddaa7b2800" containerName="registry-server" Jun 03 16:25:00.860747 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:25:00.860732 2568 memory_manager.go:356] "RemoveStaleState removing state" podUID="96d5ee00-4eef-46b9-9360-36ddaa7b2800" containerName="registry-server" Jun 03 16:25:00.864854 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:25:00.864832 2568 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6c84757667-7mnxn" Jun 03 16:25:00.874985 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:25:00.874961 2568 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6c84757667-7mnxn"] Jun 03 16:25:00.941180 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:25:00.941142 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/fd36c718-aaaa-4d15-a586-70d48dc664a8-console-serving-cert\") pod \"console-6c84757667-7mnxn\" (UID: \"fd36c718-aaaa-4d15-a586-70d48dc664a8\") " pod="openshift-console/console-6c84757667-7mnxn" Jun 03 16:25:00.941180 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:25:00.941184 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/fd36c718-aaaa-4d15-a586-70d48dc664a8-console-config\") pod \"console-6c84757667-7mnxn\" (UID: \"fd36c718-aaaa-4d15-a586-70d48dc664a8\") " pod="openshift-console/console-6c84757667-7mnxn" Jun 03 16:25:00.941382 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:25:00.941289 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/fd36c718-aaaa-4d15-a586-70d48dc664a8-oauth-serving-cert\") pod \"console-6c84757667-7mnxn\" (UID: \"fd36c718-aaaa-4d15-a586-70d48dc664a8\") " pod="openshift-console/console-6c84757667-7mnxn" Jun 03 16:25:00.941382 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:25:00.941317 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/fd36c718-aaaa-4d15-a586-70d48dc664a8-console-oauth-config\") pod \"console-6c84757667-7mnxn\" (UID: \"fd36c718-aaaa-4d15-a586-70d48dc664a8\") " pod="openshift-console/console-6c84757667-7mnxn" Jun 03 16:25:00.941382 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:25:00.941371 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rjb6m\" (UniqueName: \"kubernetes.io/projected/fd36c718-aaaa-4d15-a586-70d48dc664a8-kube-api-access-rjb6m\") pod \"console-6c84757667-7mnxn\" (UID: \"fd36c718-aaaa-4d15-a586-70d48dc664a8\") " pod="openshift-console/console-6c84757667-7mnxn" Jun 03 16:25:00.941494 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:25:00.941395 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fd36c718-aaaa-4d15-a586-70d48dc664a8-trusted-ca-bundle\") pod \"console-6c84757667-7mnxn\" (UID: \"fd36c718-aaaa-4d15-a586-70d48dc664a8\") " pod="openshift-console/console-6c84757667-7mnxn" Jun 03 16:25:00.941494 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:25:00.941418 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/fd36c718-aaaa-4d15-a586-70d48dc664a8-service-ca\") pod \"console-6c84757667-7mnxn\" (UID: \"fd36c718-aaaa-4d15-a586-70d48dc664a8\") " pod="openshift-console/console-6c84757667-7mnxn" Jun 03 16:25:01.042120 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:25:01.042090 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/fd36c718-aaaa-4d15-a586-70d48dc664a8-oauth-serving-cert\") pod \"console-6c84757667-7mnxn\" (UID: \"fd36c718-aaaa-4d15-a586-70d48dc664a8\") " pod="openshift-console/console-6c84757667-7mnxn" Jun 03 16:25:01.042120 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:25:01.042123 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/fd36c718-aaaa-4d15-a586-70d48dc664a8-console-oauth-config\") pod \"console-6c84757667-7mnxn\" (UID: \"fd36c718-aaaa-4d15-a586-70d48dc664a8\") " pod="openshift-console/console-6c84757667-7mnxn" Jun 03 16:25:01.042321 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:25:01.042149 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-rjb6m\" (UniqueName: \"kubernetes.io/projected/fd36c718-aaaa-4d15-a586-70d48dc664a8-kube-api-access-rjb6m\") pod \"console-6c84757667-7mnxn\" (UID: \"fd36c718-aaaa-4d15-a586-70d48dc664a8\") " pod="openshift-console/console-6c84757667-7mnxn" Jun 03 16:25:01.042321 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:25:01.042203 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fd36c718-aaaa-4d15-a586-70d48dc664a8-trusted-ca-bundle\") pod \"console-6c84757667-7mnxn\" (UID: \"fd36c718-aaaa-4d15-a586-70d48dc664a8\") " pod="openshift-console/console-6c84757667-7mnxn" Jun 03 16:25:01.042321 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:25:01.042232 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/fd36c718-aaaa-4d15-a586-70d48dc664a8-service-ca\") pod \"console-6c84757667-7mnxn\" (UID: \"fd36c718-aaaa-4d15-a586-70d48dc664a8\") " pod="openshift-console/console-6c84757667-7mnxn" Jun 03 16:25:01.042321 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:25:01.042275 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/fd36c718-aaaa-4d15-a586-70d48dc664a8-console-serving-cert\") pod \"console-6c84757667-7mnxn\" (UID: \"fd36c718-aaaa-4d15-a586-70d48dc664a8\") " pod="openshift-console/console-6c84757667-7mnxn" Jun 03 16:25:01.042321 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:25:01.042294 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/fd36c718-aaaa-4d15-a586-70d48dc664a8-console-config\") pod \"console-6c84757667-7mnxn\" (UID: \"fd36c718-aaaa-4d15-a586-70d48dc664a8\") " pod="openshift-console/console-6c84757667-7mnxn" Jun 03 16:25:01.043011 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:25:01.042990 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/fd36c718-aaaa-4d15-a586-70d48dc664a8-service-ca\") pod \"console-6c84757667-7mnxn\" (UID: \"fd36c718-aaaa-4d15-a586-70d48dc664a8\") " pod="openshift-console/console-6c84757667-7mnxn" Jun 03 16:25:01.043159 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:25:01.043102 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/fd36c718-aaaa-4d15-a586-70d48dc664a8-console-config\") pod \"console-6c84757667-7mnxn\" (UID: \"fd36c718-aaaa-4d15-a586-70d48dc664a8\") " pod="openshift-console/console-6c84757667-7mnxn" Jun 03 16:25:01.043159 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:25:01.043105 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/fd36c718-aaaa-4d15-a586-70d48dc664a8-oauth-serving-cert\") pod \"console-6c84757667-7mnxn\" (UID: \"fd36c718-aaaa-4d15-a586-70d48dc664a8\") " pod="openshift-console/console-6c84757667-7mnxn" Jun 03 16:25:01.043282 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:25:01.043166 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fd36c718-aaaa-4d15-a586-70d48dc664a8-trusted-ca-bundle\") pod \"console-6c84757667-7mnxn\" (UID: \"fd36c718-aaaa-4d15-a586-70d48dc664a8\") " pod="openshift-console/console-6c84757667-7mnxn" Jun 03 16:25:01.044633 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:25:01.044608 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/fd36c718-aaaa-4d15-a586-70d48dc664a8-console-serving-cert\") pod \"console-6c84757667-7mnxn\" (UID: \"fd36c718-aaaa-4d15-a586-70d48dc664a8\") " pod="openshift-console/console-6c84757667-7mnxn" Jun 03 16:25:01.044713 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:25:01.044663 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/fd36c718-aaaa-4d15-a586-70d48dc664a8-console-oauth-config\") pod \"console-6c84757667-7mnxn\" (UID: \"fd36c718-aaaa-4d15-a586-70d48dc664a8\") " pod="openshift-console/console-6c84757667-7mnxn" Jun 03 16:25:01.050731 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:25:01.050712 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rjb6m\" (UniqueName: \"kubernetes.io/projected/fd36c718-aaaa-4d15-a586-70d48dc664a8-kube-api-access-rjb6m\") pod \"console-6c84757667-7mnxn\" (UID: \"fd36c718-aaaa-4d15-a586-70d48dc664a8\") " pod="openshift-console/console-6c84757667-7mnxn" Jun 03 16:25:01.175476 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:25:01.175395 2568 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6c84757667-7mnxn" Jun 03 16:25:01.303195 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:25:01.303167 2568 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6c84757667-7mnxn"] Jun 03 16:25:01.305410 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:25:01.305378 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfd36c718_aaaa_4d15_a586_70d48dc664a8.slice/crio-2f60e8669c6fb7371cafbef877645b3b00f95914468bdcb61fcaa07a54a05a25 WatchSource:0}: Error finding container 2f60e8669c6fb7371cafbef877645b3b00f95914468bdcb61fcaa07a54a05a25: Status 404 returned error can't find the container with id 2f60e8669c6fb7371cafbef877645b3b00f95914468bdcb61fcaa07a54a05a25 Jun 03 16:25:02.227936 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:25:02.227900 2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6c84757667-7mnxn" event={"ID":"fd36c718-aaaa-4d15-a586-70d48dc664a8","Type":"ContainerStarted","Data":"c7e260a08941193ce429511aac393c682df287dad1282d221942bc37622dedce"} Jun 03 16:25:02.227936 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:25:02.227939 2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6c84757667-7mnxn" event={"ID":"fd36c718-aaaa-4d15-a586-70d48dc664a8","Type":"ContainerStarted","Data":"2f60e8669c6fb7371cafbef877645b3b00f95914468bdcb61fcaa07a54a05a25"} Jun 03 16:25:02.247067 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:25:02.247021 2568 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-console/console-6c84757667-7mnxn" podStartSLOduration=2.247010832 podStartE2EDuration="2.247010832s" podCreationTimestamp="2026-06-03 16:25:00 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-06-03 16:25:02.244646525 +0000 UTC m=+473.333073116" watchObservedRunningTime="2026-06-03 16:25:02.247010832 +0000 UTC m=+473.335437422" Jun 03 16:25:03.649761 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:25:03.649718 2568 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kuadrant-system/kuadrant-operator-catalog-pjwz8" Jun 03 16:25:03.649761 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:25:03.649762 2568 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-catalog-pjwz8" Jun 03 16:25:03.672195 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:25:03.672170 2568 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="kuadrant-system/kuadrant-operator-catalog-pjwz8" Jun 03 16:25:04.256017 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:25:04.255990 2568 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-catalog-pjwz8" Jun 03 16:25:08.322736 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:25:08.322698 2568 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759tg6xr"] Jun 03 16:25:08.326377 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:25:08.326360 2568 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759tg6xr" Jun 03 16:25:08.328573 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:25:08.328556 2568 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-5t7wf\"" Jun 03 16:25:08.333816 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:25:08.333793 2568 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759tg6xr"] Jun 03 16:25:08.502478 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:25:08.502445 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/094f549f-494e-486a-aca5-303c7492dacd-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759tg6xr\" (UID: \"094f549f-494e-486a-aca5-303c7492dacd\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759tg6xr" Jun 03 16:25:08.502635 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:25:08.502498 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mq462\" (UniqueName: \"kubernetes.io/projected/094f549f-494e-486a-aca5-303c7492dacd-kube-api-access-mq462\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759tg6xr\" (UID: \"094f549f-494e-486a-aca5-303c7492dacd\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759tg6xr" Jun 03 16:25:08.502635 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:25:08.502624 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/094f549f-494e-486a-aca5-303c7492dacd-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759tg6xr\" (UID: \"094f549f-494e-486a-aca5-303c7492dacd\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759tg6xr" Jun 03 16:25:08.603051 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:25:08.602985 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/094f549f-494e-486a-aca5-303c7492dacd-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759tg6xr\" (UID: \"094f549f-494e-486a-aca5-303c7492dacd\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759tg6xr" Jun 03 16:25:08.603051 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:25:08.603027 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/094f549f-494e-486a-aca5-303c7492dacd-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759tg6xr\" (UID: \"094f549f-494e-486a-aca5-303c7492dacd\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759tg6xr" Jun 03 16:25:08.603211 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:25:08.603062 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-mq462\" (UniqueName: \"kubernetes.io/projected/094f549f-494e-486a-aca5-303c7492dacd-kube-api-access-mq462\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759tg6xr\" (UID: \"094f549f-494e-486a-aca5-303c7492dacd\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759tg6xr" Jun 03 16:25:08.603335 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:25:08.603318 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/094f549f-494e-486a-aca5-303c7492dacd-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759tg6xr\" (UID: \"094f549f-494e-486a-aca5-303c7492dacd\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759tg6xr" Jun 03 16:25:08.603402 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:25:08.603381 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/094f549f-494e-486a-aca5-303c7492dacd-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759tg6xr\" (UID: \"094f549f-494e-486a-aca5-303c7492dacd\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759tg6xr" Jun 03 16:25:08.611146 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:25:08.611128 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mq462\" (UniqueName: \"kubernetes.io/projected/094f549f-494e-486a-aca5-303c7492dacd-kube-api-access-mq462\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759tg6xr\" (UID: \"094f549f-494e-486a-aca5-303c7492dacd\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759tg6xr" Jun 03 16:25:08.637102 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:25:08.637081 2568 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759tg6xr" Jun 03 16:25:08.760760 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:25:08.760691 2568 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759tg6xr"] Jun 03 16:25:08.763002 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:25:08.762972 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod094f549f_494e_486a_aca5_303c7492dacd.slice/crio-5e3ad9075108c094315b70eb03e165fee7ceb76d88cbb2f800a73e4a20c0a1d0 WatchSource:0}: Error finding container 5e3ad9075108c094315b70eb03e165fee7ceb76d88cbb2f800a73e4a20c0a1d0: Status 404 returned error can't find the container with id 5e3ad9075108c094315b70eb03e165fee7ceb76d88cbb2f800a73e4a20c0a1d0 Jun 03 16:25:09.122465 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:25:09.122390 2568 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73tzwb2"] Jun 03 16:25:09.125871 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:25:09.125851 2568 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73tzwb2" Jun 03 16:25:09.135709 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:25:09.135689 2568 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73tzwb2"] Jun 03 16:25:09.207286 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:25:09.207255 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e12d559c-bbf9-4f80-b155-8a9fa65fa0c1-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73tzwb2\" (UID: \"e12d559c-bbf9-4f80-b155-8a9fa65fa0c1\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73tzwb2" Jun 03 16:25:09.207430 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:25:09.207295 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9fsln\" (UniqueName: \"kubernetes.io/projected/e12d559c-bbf9-4f80-b155-8a9fa65fa0c1-kube-api-access-9fsln\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73tzwb2\" (UID: \"e12d559c-bbf9-4f80-b155-8a9fa65fa0c1\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73tzwb2" Jun 03 16:25:09.207430 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:25:09.207395 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e12d559c-bbf9-4f80-b155-8a9fa65fa0c1-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73tzwb2\" (UID: \"e12d559c-bbf9-4f80-b155-8a9fa65fa0c1\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73tzwb2" Jun 03 16:25:09.253929 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:25:09.253900 2568 generic.go:358] "Generic (PLEG): container finished" podID="094f549f-494e-486a-aca5-303c7492dacd" containerID="767d4a4b9ce3fcb34f66884ca26a27c438187586f8f8d7b87b228edca4e776b3" exitCode=0 Jun 03 16:25:09.254053 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:25:09.253981 2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759tg6xr" event={"ID":"094f549f-494e-486a-aca5-303c7492dacd","Type":"ContainerDied","Data":"767d4a4b9ce3fcb34f66884ca26a27c438187586f8f8d7b87b228edca4e776b3"} Jun 03 16:25:09.254053 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:25:09.254014 2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759tg6xr" event={"ID":"094f549f-494e-486a-aca5-303c7492dacd","Type":"ContainerStarted","Data":"5e3ad9075108c094315b70eb03e165fee7ceb76d88cbb2f800a73e4a20c0a1d0"} Jun 03 16:25:09.308318 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:25:09.308293 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e12d559c-bbf9-4f80-b155-8a9fa65fa0c1-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73tzwb2\" (UID: \"e12d559c-bbf9-4f80-b155-8a9fa65fa0c1\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73tzwb2" Jun 03 16:25:09.308432 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:25:09.308341 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e12d559c-bbf9-4f80-b155-8a9fa65fa0c1-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73tzwb2\" (UID: \"e12d559c-bbf9-4f80-b155-8a9fa65fa0c1\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73tzwb2" Jun 03 16:25:09.308432 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:25:09.308372 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-9fsln\" (UniqueName: \"kubernetes.io/projected/e12d559c-bbf9-4f80-b155-8a9fa65fa0c1-kube-api-access-9fsln\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73tzwb2\" (UID: \"e12d559c-bbf9-4f80-b155-8a9fa65fa0c1\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73tzwb2" Jun 03 16:25:09.308654 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:25:09.308637 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e12d559c-bbf9-4f80-b155-8a9fa65fa0c1-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73tzwb2\" (UID: \"e12d559c-bbf9-4f80-b155-8a9fa65fa0c1\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73tzwb2" Jun 03 16:25:09.308764 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:25:09.308748 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e12d559c-bbf9-4f80-b155-8a9fa65fa0c1-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73tzwb2\" (UID: \"e12d559c-bbf9-4f80-b155-8a9fa65fa0c1\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73tzwb2" Jun 03 16:25:09.316234 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:25:09.316208 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9fsln\" (UniqueName: \"kubernetes.io/projected/e12d559c-bbf9-4f80-b155-8a9fa65fa0c1-kube-api-access-9fsln\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73tzwb2\" (UID: \"e12d559c-bbf9-4f80-b155-8a9fa65fa0c1\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73tzwb2" Jun 03 16:25:09.434940 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:25:09.434878 2568 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73tzwb2" Jun 03 16:25:09.565849 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:25:09.565804 2568 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73tzwb2"] Jun 03 16:25:09.567355 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:25:09.567330 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode12d559c_bbf9_4f80_b155_8a9fa65fa0c1.slice/crio-f11f4b73a24763b9e0d1b80482aae1f8731739d45edc15f54e43d47ccbc02de5 WatchSource:0}: Error finding container f11f4b73a24763b9e0d1b80482aae1f8731739d45edc15f54e43d47ccbc02de5: Status 404 returned error can't find the container with id f11f4b73a24763b9e0d1b80482aae1f8731739d45edc15f54e43d47ccbc02de5 Jun 03 16:25:09.724605 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:25:09.724526 2568 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1knbcb"] Jun 03 16:25:09.728269 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:25:09.728250 2568 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1knbcb" Jun 03 16:25:09.737059 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:25:09.737036 2568 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1knbcb"] Jun 03 16:25:09.812913 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:25:09.812889 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a5bdab96-bea9-4fbc-8bc1-e987d74623c8-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1knbcb\" (UID: \"a5bdab96-bea9-4fbc-8bc1-e987d74623c8\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1knbcb" Jun 03 16:25:09.813085 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:25:09.812975 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a5bdab96-bea9-4fbc-8bc1-e987d74623c8-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1knbcb\" (UID: \"a5bdab96-bea9-4fbc-8bc1-e987d74623c8\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1knbcb" Jun 03 16:25:09.813085 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:25:09.813027 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r5mjv\" (UniqueName: \"kubernetes.io/projected/a5bdab96-bea9-4fbc-8bc1-e987d74623c8-kube-api-access-r5mjv\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1knbcb\" (UID: \"a5bdab96-bea9-4fbc-8bc1-e987d74623c8\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1knbcb" Jun 03 16:25:09.914155 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:25:09.913765 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a5bdab96-bea9-4fbc-8bc1-e987d74623c8-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1knbcb\" (UID: \"a5bdab96-bea9-4fbc-8bc1-e987d74623c8\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1knbcb" Jun 03 16:25:09.914155 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:25:09.913861 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a5bdab96-bea9-4fbc-8bc1-e987d74623c8-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1knbcb\" (UID: \"a5bdab96-bea9-4fbc-8bc1-e987d74623c8\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1knbcb" Jun 03 16:25:09.914155 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:25:09.913932 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-r5mjv\" (UniqueName: \"kubernetes.io/projected/a5bdab96-bea9-4fbc-8bc1-e987d74623c8-kube-api-access-r5mjv\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1knbcb\" (UID: \"a5bdab96-bea9-4fbc-8bc1-e987d74623c8\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1knbcb" Jun 03 16:25:09.914155 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:25:09.914148 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a5bdab96-bea9-4fbc-8bc1-e987d74623c8-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1knbcb\" (UID: \"a5bdab96-bea9-4fbc-8bc1-e987d74623c8\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1knbcb" Jun 03 16:25:09.914453 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:25:09.914433 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a5bdab96-bea9-4fbc-8bc1-e987d74623c8-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1knbcb\" (UID: \"a5bdab96-bea9-4fbc-8bc1-e987d74623c8\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1knbcb" Jun 03 16:25:09.921898 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:25:09.921870 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r5mjv\" (UniqueName: \"kubernetes.io/projected/a5bdab96-bea9-4fbc-8bc1-e987d74623c8-kube-api-access-r5mjv\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1knbcb\" (UID: \"a5bdab96-bea9-4fbc-8bc1-e987d74623c8\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1knbcb" Jun 03 16:25:10.037720 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:25:10.037676 2568 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1knbcb" Jun 03 16:25:10.128105 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:25:10.128078 2568 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0fqfpf"] Jun 03 16:25:10.133517 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:25:10.133499 2568 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0fqfpf" Jun 03 16:25:10.139370 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:25:10.139342 2568 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0fqfpf"] Jun 03 16:25:10.167492 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:25:10.167469 2568 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1knbcb"] Jun 03 16:25:10.169106 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:25:10.169083 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda5bdab96_bea9_4fbc_8bc1_e987d74623c8.slice/crio-1630b3887bf6f3814a56c9a57829cf26de9a6800b33087d3cd8b1561d788696a WatchSource:0}: Error finding container 1630b3887bf6f3814a56c9a57829cf26de9a6800b33087d3cd8b1561d788696a: Status 404 returned error can't find the container with id 1630b3887bf6f3814a56c9a57829cf26de9a6800b33087d3cd8b1561d788696a Jun 03 16:25:10.216300 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:25:10.216274 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f73bd334-e4cc-4916-8ad4-07b5b88e7b8e-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0fqfpf\" (UID: \"f73bd334-e4cc-4916-8ad4-07b5b88e7b8e\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0fqfpf" Jun 03 16:25:10.216395 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:25:10.216313 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f73bd334-e4cc-4916-8ad4-07b5b88e7b8e-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0fqfpf\" (UID: \"f73bd334-e4cc-4916-8ad4-07b5b88e7b8e\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0fqfpf" Jun 03 16:25:10.216395 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:25:10.216385 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c452b\" (UniqueName: \"kubernetes.io/projected/f73bd334-e4cc-4916-8ad4-07b5b88e7b8e-kube-api-access-c452b\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0fqfpf\" (UID: \"f73bd334-e4cc-4916-8ad4-07b5b88e7b8e\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0fqfpf" Jun 03 16:25:10.259198 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:25:10.259122 2568 generic.go:358] "Generic (PLEG): container finished" podID="e12d559c-bbf9-4f80-b155-8a9fa65fa0c1" containerID="f8999da48ea1c4cc5bcf8dd079d7cac99f627d6386859a3269461cbd4a8de34f" exitCode=0 Jun 03 16:25:10.259378 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:25:10.259204 2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73tzwb2" event={"ID":"e12d559c-bbf9-4f80-b155-8a9fa65fa0c1","Type":"ContainerDied","Data":"f8999da48ea1c4cc5bcf8dd079d7cac99f627d6386859a3269461cbd4a8de34f"} Jun 03 16:25:10.259378 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:25:10.259234 2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73tzwb2" event={"ID":"e12d559c-bbf9-4f80-b155-8a9fa65fa0c1","Type":"ContainerStarted","Data":"f11f4b73a24763b9e0d1b80482aae1f8731739d45edc15f54e43d47ccbc02de5"} Jun 03 16:25:10.261265 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:25:10.261138 2568 generic.go:358] "Generic (PLEG): container finished" podID="094f549f-494e-486a-aca5-303c7492dacd" containerID="67290ceb7778baa777d185a88a3066e4315190265b1354a43825ef7cbdecf67a" exitCode=0 Jun 03 16:25:10.261265 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:25:10.261200 2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759tg6xr" event={"ID":"094f549f-494e-486a-aca5-303c7492dacd","Type":"ContainerDied","Data":"67290ceb7778baa777d185a88a3066e4315190265b1354a43825ef7cbdecf67a"} Jun 03 16:25:10.262835 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:25:10.262815 2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1knbcb" event={"ID":"a5bdab96-bea9-4fbc-8bc1-e987d74623c8","Type":"ContainerStarted","Data":"fd3773669cc28f67660e834d0a4e93e73f9a4e33e39a58b480faca37a34de5ed"} Jun 03 16:25:10.262835 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:25:10.262842 2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1knbcb" event={"ID":"a5bdab96-bea9-4fbc-8bc1-e987d74623c8","Type":"ContainerStarted","Data":"1630b3887bf6f3814a56c9a57829cf26de9a6800b33087d3cd8b1561d788696a"} Jun 03 16:25:10.317634 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:25:10.317597 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f73bd334-e4cc-4916-8ad4-07b5b88e7b8e-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0fqfpf\" (UID: \"f73bd334-e4cc-4916-8ad4-07b5b88e7b8e\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0fqfpf" Jun 03 16:25:10.317772 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:25:10.317639 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f73bd334-e4cc-4916-8ad4-07b5b88e7b8e-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0fqfpf\" (UID: \"f73bd334-e4cc-4916-8ad4-07b5b88e7b8e\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0fqfpf" Jun 03 16:25:10.317772 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:25:10.317686 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-c452b\" (UniqueName: \"kubernetes.io/projected/f73bd334-e4cc-4916-8ad4-07b5b88e7b8e-kube-api-access-c452b\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0fqfpf\" (UID: \"f73bd334-e4cc-4916-8ad4-07b5b88e7b8e\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0fqfpf" Jun 03 16:25:10.317971 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:25:10.317950 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f73bd334-e4cc-4916-8ad4-07b5b88e7b8e-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0fqfpf\" (UID: \"f73bd334-e4cc-4916-8ad4-07b5b88e7b8e\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0fqfpf" Jun 03 16:25:10.318034 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:25:10.318002 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f73bd334-e4cc-4916-8ad4-07b5b88e7b8e-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0fqfpf\" (UID: \"f73bd334-e4cc-4916-8ad4-07b5b88e7b8e\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0fqfpf" Jun 03 16:25:10.331283 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:25:10.331253 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c452b\" (UniqueName: \"kubernetes.io/projected/f73bd334-e4cc-4916-8ad4-07b5b88e7b8e-kube-api-access-c452b\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0fqfpf\" (UID: \"f73bd334-e4cc-4916-8ad4-07b5b88e7b8e\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0fqfpf" Jun 03 16:25:10.454330 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:25:10.454303 2568 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0fqfpf" Jun 03 16:25:10.578199 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:25:10.578175 2568 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0fqfpf"] Jun 03 16:25:10.579221 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:25:10.579197 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf73bd334_e4cc_4916_8ad4_07b5b88e7b8e.slice/crio-acd26eb77ff210bc1bf7fbffaac6cd7a97b34c88b839883eecef19af070a2e58 WatchSource:0}: Error finding container acd26eb77ff210bc1bf7fbffaac6cd7a97b34c88b839883eecef19af070a2e58: Status 404 returned error can't find the container with id acd26eb77ff210bc1bf7fbffaac6cd7a97b34c88b839883eecef19af070a2e58 Jun 03 16:25:11.176493 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:25:11.176469 2568 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6c84757667-7mnxn" Jun 03 16:25:11.176618 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:25:11.176507 2568 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-6c84757667-7mnxn" Jun 03 16:25:11.182820 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:25:11.182801 2568 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-6c84757667-7mnxn" Jun 03 16:25:11.268732 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:25:11.268699 2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73tzwb2" event={"ID":"e12d559c-bbf9-4f80-b155-8a9fa65fa0c1","Type":"ContainerStarted","Data":"029356e53b39b0e737d8798fa987e0dd1ddd03183a24436ca8d76b6a2edd9cb7"} Jun 03 16:25:11.270615 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:25:11.270582 2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759tg6xr" event={"ID":"094f549f-494e-486a-aca5-303c7492dacd","Type":"ContainerDied","Data":"5a56fdd52e5221eec0522a6d523f286e4e1060a21e1a49e046c98e86ccf58054"} Jun 03 16:25:11.270738 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:25:11.270528 2568 generic.go:358] "Generic (PLEG): container finished" podID="094f549f-494e-486a-aca5-303c7492dacd" containerID="5a56fdd52e5221eec0522a6d523f286e4e1060a21e1a49e046c98e86ccf58054" exitCode=0 Jun 03 16:25:11.271828 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:25:11.271806 2568 generic.go:358] "Generic (PLEG): container finished" podID="a5bdab96-bea9-4fbc-8bc1-e987d74623c8" containerID="fd3773669cc28f67660e834d0a4e93e73f9a4e33e39a58b480faca37a34de5ed" exitCode=0 Jun 03 16:25:11.271900 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:25:11.271884 2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1knbcb" event={"ID":"a5bdab96-bea9-4fbc-8bc1-e987d74623c8","Type":"ContainerDied","Data":"fd3773669cc28f67660e834d0a4e93e73f9a4e33e39a58b480faca37a34de5ed"} Jun 03 16:25:11.273237 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:25:11.273210 2568 generic.go:358] "Generic (PLEG): container finished" podID="f73bd334-e4cc-4916-8ad4-07b5b88e7b8e" containerID="b545573e564712af48644f19ad7573365847ab47832a94c695c59a36290e7aed" exitCode=0 Jun 03 16:25:11.273343 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:25:11.273281 2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0fqfpf" event={"ID":"f73bd334-e4cc-4916-8ad4-07b5b88e7b8e","Type":"ContainerDied","Data":"b545573e564712af48644f19ad7573365847ab47832a94c695c59a36290e7aed"} Jun 03 16:25:11.273343 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:25:11.273306 2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0fqfpf" event={"ID":"f73bd334-e4cc-4916-8ad4-07b5b88e7b8e","Type":"ContainerStarted","Data":"acd26eb77ff210bc1bf7fbffaac6cd7a97b34c88b839883eecef19af070a2e58"} Jun 03 16:25:11.277642 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:25:11.277622 2568 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-6c84757667-7mnxn" Jun 03 16:25:11.348682 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:25:11.348656 2568 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-console/console-df89b8f67-6rjh2"] Jun 03 16:25:12.278343 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:25:12.278319 2568 generic.go:358] "Generic (PLEG): container finished" podID="a5bdab96-bea9-4fbc-8bc1-e987d74623c8" containerID="38698195f35bc0be2db27be0c449725dbb0270fa60752ed8dff5c69fae191bc3" exitCode=0 Jun 03 16:25:12.278705 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:25:12.278411 2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1knbcb" event={"ID":"a5bdab96-bea9-4fbc-8bc1-e987d74623c8","Type":"ContainerDied","Data":"38698195f35bc0be2db27be0c449725dbb0270fa60752ed8dff5c69fae191bc3"} Jun 03 16:25:12.279991 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:25:12.279978 2568 generic.go:358] "Generic (PLEG): container finished" podID="f73bd334-e4cc-4916-8ad4-07b5b88e7b8e" containerID="10a1cf5a96a2d3bdb7a6d2cc3adb2e58699c780495f26a513d5fda0955382e06" exitCode=0 Jun 03 16:25:12.280060 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:25:12.280048 2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0fqfpf" event={"ID":"f73bd334-e4cc-4916-8ad4-07b5b88e7b8e","Type":"ContainerDied","Data":"10a1cf5a96a2d3bdb7a6d2cc3adb2e58699c780495f26a513d5fda0955382e06"} Jun 03 16:25:12.281991 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:25:12.281952 2568 generic.go:358] "Generic (PLEG): container finished" podID="e12d559c-bbf9-4f80-b155-8a9fa65fa0c1" containerID="029356e53b39b0e737d8798fa987e0dd1ddd03183a24436ca8d76b6a2edd9cb7" exitCode=0 Jun 03 16:25:12.281991 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:25:12.281970 2568 generic.go:358] "Generic (PLEG): container finished" podID="e12d559c-bbf9-4f80-b155-8a9fa65fa0c1" containerID="ae14121d816e63f1db94abf8dd7c00e574166ad11c33f717aa84d24d54bd4c6d" exitCode=0 Jun 03 16:25:12.282108 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:25:12.282021 2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73tzwb2" event={"ID":"e12d559c-bbf9-4f80-b155-8a9fa65fa0c1","Type":"ContainerDied","Data":"029356e53b39b0e737d8798fa987e0dd1ddd03183a24436ca8d76b6a2edd9cb7"} Jun 03 16:25:12.282108 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:25:12.282050 2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73tzwb2" event={"ID":"e12d559c-bbf9-4f80-b155-8a9fa65fa0c1","Type":"ContainerDied","Data":"ae14121d816e63f1db94abf8dd7c00e574166ad11c33f717aa84d24d54bd4c6d"} Jun 03 16:25:12.408719 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:25:12.408688 2568 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759tg6xr" Jun 03 16:25:12.537068 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:25:12.537039 2568 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/094f549f-494e-486a-aca5-303c7492dacd-util\") pod \"094f549f-494e-486a-aca5-303c7492dacd\" (UID: \"094f549f-494e-486a-aca5-303c7492dacd\") " Jun 03 16:25:12.537213 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:25:12.537119 2568 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/094f549f-494e-486a-aca5-303c7492dacd-bundle\") pod \"094f549f-494e-486a-aca5-303c7492dacd\" (UID: \"094f549f-494e-486a-aca5-303c7492dacd\") " Jun 03 16:25:12.537213 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:25:12.537195 2568 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mq462\" (UniqueName: \"kubernetes.io/projected/094f549f-494e-486a-aca5-303c7492dacd-kube-api-access-mq462\") pod \"094f549f-494e-486a-aca5-303c7492dacd\" (UID: \"094f549f-494e-486a-aca5-303c7492dacd\") " Jun 03 16:25:12.537634 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:25:12.537612 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/094f549f-494e-486a-aca5-303c7492dacd-bundle" (OuterVolumeSpecName: "bundle") pod "094f549f-494e-486a-aca5-303c7492dacd" (UID: "094f549f-494e-486a-aca5-303c7492dacd"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 03 16:25:12.539299 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:25:12.539277 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/094f549f-494e-486a-aca5-303c7492dacd-kube-api-access-mq462" (OuterVolumeSpecName: "kube-api-access-mq462") pod "094f549f-494e-486a-aca5-303c7492dacd" (UID: "094f549f-494e-486a-aca5-303c7492dacd"). InnerVolumeSpecName "kube-api-access-mq462". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 03 16:25:12.542559 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:25:12.542513 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/094f549f-494e-486a-aca5-303c7492dacd-util" (OuterVolumeSpecName: "util") pod "094f549f-494e-486a-aca5-303c7492dacd" (UID: "094f549f-494e-486a-aca5-303c7492dacd"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 03 16:25:12.638745 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:25:12.638720 2568 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/094f549f-494e-486a-aca5-303c7492dacd-util\") on node \"ip-10-0-128-88.ec2.internal\" DevicePath \"\"" Jun 03 16:25:12.638745 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:25:12.638744 2568 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/094f549f-494e-486a-aca5-303c7492dacd-bundle\") on node \"ip-10-0-128-88.ec2.internal\" DevicePath \"\"" Jun 03 16:25:12.638873 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:25:12.638753 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mq462\" (UniqueName: \"kubernetes.io/projected/094f549f-494e-486a-aca5-303c7492dacd-kube-api-access-mq462\") on node \"ip-10-0-128-88.ec2.internal\" DevicePath \"\"" Jun 03 16:25:13.288120 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:25:13.288082 2568 generic.go:358] "Generic (PLEG): container finished" podID="a5bdab96-bea9-4fbc-8bc1-e987d74623c8" containerID="50138f69513f5cbc75b01fa00b3961df6762aede604f4edec366cfc10ac67942" exitCode=0 Jun 03 16:25:13.288586 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:25:13.288167 2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1knbcb" event={"ID":"a5bdab96-bea9-4fbc-8bc1-e987d74623c8","Type":"ContainerDied","Data":"50138f69513f5cbc75b01fa00b3961df6762aede604f4edec366cfc10ac67942"} Jun 03 16:25:13.290002 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:25:13.289978 2568 generic.go:358] "Generic (PLEG): container finished" podID="f73bd334-e4cc-4916-8ad4-07b5b88e7b8e" containerID="4a52da55133ee984885050fcd2fd0764c26087531d1f64bdf2846549ad72da19" exitCode=0 Jun 03 16:25:13.290122 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:25:13.290054 2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0fqfpf" event={"ID":"f73bd334-e4cc-4916-8ad4-07b5b88e7b8e","Type":"ContainerDied","Data":"4a52da55133ee984885050fcd2fd0764c26087531d1f64bdf2846549ad72da19"} Jun 03 16:25:13.291548 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:25:13.291518 2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759tg6xr" event={"ID":"094f549f-494e-486a-aca5-303c7492dacd","Type":"ContainerDied","Data":"5e3ad9075108c094315b70eb03e165fee7ceb76d88cbb2f800a73e4a20c0a1d0"} Jun 03 16:25:13.291619 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:25:13.291556 2568 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="5e3ad9075108c094315b70eb03e165fee7ceb76d88cbb2f800a73e4a20c0a1d0" Jun 03 16:25:13.291653 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:25:13.291618 2568 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759tg6xr" Jun 03 16:25:13.416764 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:25:13.416742 2568 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73tzwb2" Jun 03 16:25:13.546465 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:25:13.546396 2568 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e12d559c-bbf9-4f80-b155-8a9fa65fa0c1-bundle\") pod \"e12d559c-bbf9-4f80-b155-8a9fa65fa0c1\" (UID: \"e12d559c-bbf9-4f80-b155-8a9fa65fa0c1\") " Jun 03 16:25:13.546465 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:25:13.546438 2568 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e12d559c-bbf9-4f80-b155-8a9fa65fa0c1-util\") pod \"e12d559c-bbf9-4f80-b155-8a9fa65fa0c1\" (UID: \"e12d559c-bbf9-4f80-b155-8a9fa65fa0c1\") " Jun 03 16:25:13.546811 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:25:13.546469 2568 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9fsln\" (UniqueName: \"kubernetes.io/projected/e12d559c-bbf9-4f80-b155-8a9fa65fa0c1-kube-api-access-9fsln\") pod \"e12d559c-bbf9-4f80-b155-8a9fa65fa0c1\" (UID: \"e12d559c-bbf9-4f80-b155-8a9fa65fa0c1\") " Jun 03 16:25:13.547062 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:25:13.547036 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e12d559c-bbf9-4f80-b155-8a9fa65fa0c1-bundle" (OuterVolumeSpecName: "bundle") pod "e12d559c-bbf9-4f80-b155-8a9fa65fa0c1" (UID: "e12d559c-bbf9-4f80-b155-8a9fa65fa0c1"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 03 16:25:13.548508 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:25:13.548475 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e12d559c-bbf9-4f80-b155-8a9fa65fa0c1-kube-api-access-9fsln" (OuterVolumeSpecName: "kube-api-access-9fsln") pod "e12d559c-bbf9-4f80-b155-8a9fa65fa0c1" (UID: "e12d559c-bbf9-4f80-b155-8a9fa65fa0c1"). InnerVolumeSpecName "kube-api-access-9fsln". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 03 16:25:13.551220 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:25:13.551186 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e12d559c-bbf9-4f80-b155-8a9fa65fa0c1-util" (OuterVolumeSpecName: "util") pod "e12d559c-bbf9-4f80-b155-8a9fa65fa0c1" (UID: "e12d559c-bbf9-4f80-b155-8a9fa65fa0c1"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 03 16:25:13.647905 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:25:13.647880 2568 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e12d559c-bbf9-4f80-b155-8a9fa65fa0c1-bundle\") on node \"ip-10-0-128-88.ec2.internal\" DevicePath \"\"" Jun 03 16:25:13.647905 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:25:13.647903 2568 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e12d559c-bbf9-4f80-b155-8a9fa65fa0c1-util\") on node \"ip-10-0-128-88.ec2.internal\" DevicePath \"\"" Jun 03 16:25:13.648040 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:25:13.647912 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9fsln\" (UniqueName: \"kubernetes.io/projected/e12d559c-bbf9-4f80-b155-8a9fa65fa0c1-kube-api-access-9fsln\") on node \"ip-10-0-128-88.ec2.internal\" DevicePath \"\"" Jun 03 16:25:14.297034 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:25:14.297004 2568 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73tzwb2" Jun 03 16:25:14.297034 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:25:14.296995 2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73tzwb2" event={"ID":"e12d559c-bbf9-4f80-b155-8a9fa65fa0c1","Type":"ContainerDied","Data":"f11f4b73a24763b9e0d1b80482aae1f8731739d45edc15f54e43d47ccbc02de5"} Jun 03 16:25:14.297459 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:25:14.297052 2568 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f11f4b73a24763b9e0d1b80482aae1f8731739d45edc15f54e43d47ccbc02de5" Jun 03 16:25:14.438584 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:25:14.438563 2568 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0fqfpf" Jun 03 16:25:14.441949 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:25:14.441925 2568 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1knbcb" Jun 03 16:25:14.556054 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:25:14.555979 2568 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r5mjv\" (UniqueName: \"kubernetes.io/projected/a5bdab96-bea9-4fbc-8bc1-e987d74623c8-kube-api-access-r5mjv\") pod \"a5bdab96-bea9-4fbc-8bc1-e987d74623c8\" (UID: \"a5bdab96-bea9-4fbc-8bc1-e987d74623c8\") " Jun 03 16:25:14.556054 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:25:14.556017 2568 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f73bd334-e4cc-4916-8ad4-07b5b88e7b8e-bundle\") pod \"f73bd334-e4cc-4916-8ad4-07b5b88e7b8e\" (UID: \"f73bd334-e4cc-4916-8ad4-07b5b88e7b8e\") " Jun 03 16:25:14.556054 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:25:14.556045 2568 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f73bd334-e4cc-4916-8ad4-07b5b88e7b8e-util\") pod \"f73bd334-e4cc-4916-8ad4-07b5b88e7b8e\" (UID: \"f73bd334-e4cc-4916-8ad4-07b5b88e7b8e\") " Jun 03 16:25:14.556323 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:25:14.556103 2568 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-c452b\" (UniqueName: \"kubernetes.io/projected/f73bd334-e4cc-4916-8ad4-07b5b88e7b8e-kube-api-access-c452b\") pod \"f73bd334-e4cc-4916-8ad4-07b5b88e7b8e\" (UID: \"f73bd334-e4cc-4916-8ad4-07b5b88e7b8e\") " Jun 03 16:25:14.556323 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:25:14.556129 2568 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a5bdab96-bea9-4fbc-8bc1-e987d74623c8-util\") pod \"a5bdab96-bea9-4fbc-8bc1-e987d74623c8\" (UID: \"a5bdab96-bea9-4fbc-8bc1-e987d74623c8\") " Jun 03 16:25:14.556323 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:25:14.556149 2568 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a5bdab96-bea9-4fbc-8bc1-e987d74623c8-bundle\") pod \"a5bdab96-bea9-4fbc-8bc1-e987d74623c8\" (UID: \"a5bdab96-bea9-4fbc-8bc1-e987d74623c8\") " Jun 03 16:25:14.556788 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:25:14.556597 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f73bd334-e4cc-4916-8ad4-07b5b88e7b8e-bundle" (OuterVolumeSpecName: "bundle") pod "f73bd334-e4cc-4916-8ad4-07b5b88e7b8e" (UID: "f73bd334-e4cc-4916-8ad4-07b5b88e7b8e"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 03 16:25:14.556902 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:25:14.556807 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a5bdab96-bea9-4fbc-8bc1-e987d74623c8-bundle" (OuterVolumeSpecName: "bundle") pod "a5bdab96-bea9-4fbc-8bc1-e987d74623c8" (UID: "a5bdab96-bea9-4fbc-8bc1-e987d74623c8"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 03 16:25:14.558214 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:25:14.558179 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f73bd334-e4cc-4916-8ad4-07b5b88e7b8e-kube-api-access-c452b" (OuterVolumeSpecName: "kube-api-access-c452b") pod "f73bd334-e4cc-4916-8ad4-07b5b88e7b8e" (UID: "f73bd334-e4cc-4916-8ad4-07b5b88e7b8e"). InnerVolumeSpecName "kube-api-access-c452b". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 03 16:25:14.558311 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:25:14.558282 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a5bdab96-bea9-4fbc-8bc1-e987d74623c8-kube-api-access-r5mjv" (OuterVolumeSpecName: "kube-api-access-r5mjv") pod "a5bdab96-bea9-4fbc-8bc1-e987d74623c8" (UID: "a5bdab96-bea9-4fbc-8bc1-e987d74623c8"). InnerVolumeSpecName "kube-api-access-r5mjv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 03 16:25:14.562017 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:25:14.561992 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a5bdab96-bea9-4fbc-8bc1-e987d74623c8-util" (OuterVolumeSpecName: "util") pod "a5bdab96-bea9-4fbc-8bc1-e987d74623c8" (UID: "a5bdab96-bea9-4fbc-8bc1-e987d74623c8"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 03 16:25:14.562129 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:25:14.562115 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f73bd334-e4cc-4916-8ad4-07b5b88e7b8e-util" (OuterVolumeSpecName: "util") pod "f73bd334-e4cc-4916-8ad4-07b5b88e7b8e" (UID: "f73bd334-e4cc-4916-8ad4-07b5b88e7b8e"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 03 16:25:14.657108 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:25:14.657081 2568 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a5bdab96-bea9-4fbc-8bc1-e987d74623c8-util\") on node \"ip-10-0-128-88.ec2.internal\" DevicePath \"\"" Jun 03 16:25:14.657108 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:25:14.657103 2568 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a5bdab96-bea9-4fbc-8bc1-e987d74623c8-bundle\") on node \"ip-10-0-128-88.ec2.internal\" DevicePath \"\"" Jun 03 16:25:14.657245 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:25:14.657112 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r5mjv\" (UniqueName: \"kubernetes.io/projected/a5bdab96-bea9-4fbc-8bc1-e987d74623c8-kube-api-access-r5mjv\") on node \"ip-10-0-128-88.ec2.internal\" DevicePath \"\"" Jun 03 16:25:14.657245 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:25:14.657122 2568 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f73bd334-e4cc-4916-8ad4-07b5b88e7b8e-bundle\") on node \"ip-10-0-128-88.ec2.internal\" DevicePath \"\"" Jun 03 16:25:14.657245 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:25:14.657132 2568 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f73bd334-e4cc-4916-8ad4-07b5b88e7b8e-util\") on node \"ip-10-0-128-88.ec2.internal\" DevicePath \"\"" Jun 03 16:25:14.657245 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:25:14.657140 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c452b\" (UniqueName: \"kubernetes.io/projected/f73bd334-e4cc-4916-8ad4-07b5b88e7b8e-kube-api-access-c452b\") on node \"ip-10-0-128-88.ec2.internal\" DevicePath \"\"" Jun 03 16:25:15.302159 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:25:15.302125 2568 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1knbcb" Jun 03 16:25:15.302616 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:25:15.302132 2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1knbcb" event={"ID":"a5bdab96-bea9-4fbc-8bc1-e987d74623c8","Type":"ContainerDied","Data":"1630b3887bf6f3814a56c9a57829cf26de9a6800b33087d3cd8b1561d788696a"} Jun 03 16:25:15.302616 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:25:15.302234 2568 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1630b3887bf6f3814a56c9a57829cf26de9a6800b33087d3cd8b1561d788696a" Jun 03 16:25:15.303809 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:25:15.303788 2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0fqfpf" event={"ID":"f73bd334-e4cc-4916-8ad4-07b5b88e7b8e","Type":"ContainerDied","Data":"acd26eb77ff210bc1bf7fbffaac6cd7a97b34c88b839883eecef19af070a2e58"} Jun 03 16:25:15.303809 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:25:15.303811 2568 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="acd26eb77ff210bc1bf7fbffaac6cd7a97b34c88b839883eecef19af070a2e58" Jun 03 16:25:15.303943 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:25:15.303821 2568 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0fqfpf" Jun 03 16:25:24.470571 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:25:24.470513 2568 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-648d5c98bc-qb8dv"] Jun 03 16:25:24.470991 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:25:24.470887 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f73bd334-e4cc-4916-8ad4-07b5b88e7b8e" containerName="pull" Jun 03 16:25:24.470991 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:25:24.470904 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="f73bd334-e4cc-4916-8ad4-07b5b88e7b8e" containerName="pull" Jun 03 16:25:24.470991 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:25:24.470915 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a5bdab96-bea9-4fbc-8bc1-e987d74623c8" containerName="extract" Jun 03 16:25:24.470991 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:25:24.470923 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="a5bdab96-bea9-4fbc-8bc1-e987d74623c8" containerName="extract" Jun 03 16:25:24.470991 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:25:24.470933 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f73bd334-e4cc-4916-8ad4-07b5b88e7b8e" containerName="extract" Jun 03 16:25:24.470991 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:25:24.470942 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="f73bd334-e4cc-4916-8ad4-07b5b88e7b8e" containerName="extract" Jun 03 16:25:24.470991 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:25:24.470955 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e12d559c-bbf9-4f80-b155-8a9fa65fa0c1" containerName="extract" Jun 03 16:25:24.470991 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:25:24.470964 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="e12d559c-bbf9-4f80-b155-8a9fa65fa0c1" containerName="extract" Jun 03 16:25:24.470991 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:25:24.470976 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e12d559c-bbf9-4f80-b155-8a9fa65fa0c1" containerName="util" Jun 03 16:25:24.470991 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:25:24.470984 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="e12d559c-bbf9-4f80-b155-8a9fa65fa0c1" containerName="util" Jun 03 16:25:24.470991 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:25:24.470992 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a5bdab96-bea9-4fbc-8bc1-e987d74623c8" containerName="pull" Jun 03 16:25:24.471349 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:25:24.471001 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="a5bdab96-bea9-4fbc-8bc1-e987d74623c8" containerName="pull" Jun 03 16:25:24.471349 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:25:24.471013 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="094f549f-494e-486a-aca5-303c7492dacd" containerName="util" Jun 03 16:25:24.471349 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:25:24.471021 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="094f549f-494e-486a-aca5-303c7492dacd" containerName="util" Jun 03 16:25:24.471349 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:25:24.471036 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e12d559c-bbf9-4f80-b155-8a9fa65fa0c1" containerName="pull" Jun 03 16:25:24.471349 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:25:24.471044 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="e12d559c-bbf9-4f80-b155-8a9fa65fa0c1" containerName="pull" Jun 03 16:25:24.471349 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:25:24.471065 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="094f549f-494e-486a-aca5-303c7492dacd" containerName="pull" Jun 03 16:25:24.471349 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:25:24.471073 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="094f549f-494e-486a-aca5-303c7492dacd" containerName="pull" Jun 03 16:25:24.471349 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:25:24.471082 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a5bdab96-bea9-4fbc-8bc1-e987d74623c8" containerName="util" Jun 03 16:25:24.471349 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:25:24.471090 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="a5bdab96-bea9-4fbc-8bc1-e987d74623c8" containerName="util" Jun 03 16:25:24.471349 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:25:24.471100 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f73bd334-e4cc-4916-8ad4-07b5b88e7b8e" containerName="util" Jun 03 16:25:24.471349 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:25:24.471105 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="f73bd334-e4cc-4916-8ad4-07b5b88e7b8e" containerName="util" Jun 03 16:25:24.471349 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:25:24.471114 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="094f549f-494e-486a-aca5-303c7492dacd" containerName="extract" Jun 03 16:25:24.471349 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:25:24.471119 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="094f549f-494e-486a-aca5-303c7492dacd" containerName="extract" Jun 03 16:25:24.471349 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:25:24.471198 2568 memory_manager.go:356] "RemoveStaleState removing state" podUID="a5bdab96-bea9-4fbc-8bc1-e987d74623c8" containerName="extract" Jun 03 16:25:24.471349 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:25:24.471207 2568 memory_manager.go:356] "RemoveStaleState removing state" podUID="f73bd334-e4cc-4916-8ad4-07b5b88e7b8e" containerName="extract" Jun 03 16:25:24.471349 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:25:24.471213 2568 memory_manager.go:356] "RemoveStaleState removing state" podUID="094f549f-494e-486a-aca5-303c7492dacd" containerName="extract" Jun 03 16:25:24.471349 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:25:24.471220 2568 memory_manager.go:356] "RemoveStaleState removing state" podUID="e12d559c-bbf9-4f80-b155-8a9fa65fa0c1" containerName="extract" Jun 03 16:25:24.473282 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:25:24.473266 2568 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-648d5c98bc-qb8dv" Jun 03 16:25:24.477138 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:25:24.477113 2568 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\"" Jun 03 16:25:24.477138 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:25:24.477129 2568 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-b6b5x\"" Jun 03 16:25:24.487247 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:25:24.487227 2568 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-648d5c98bc-qb8dv"] Jun 03 16:25:24.531106 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:25:24.531080 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-swcfk\" (UniqueName: \"kubernetes.io/projected/a183631b-3c99-4de5-a235-e1bbf27efcb5-kube-api-access-swcfk\") pod \"dns-operator-controller-manager-648d5c98bc-qb8dv\" (UID: \"a183631b-3c99-4de5-a235-e1bbf27efcb5\") " pod="kuadrant-system/dns-operator-controller-manager-648d5c98bc-qb8dv" Jun 03 16:25:24.632190 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:25:24.632164 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-swcfk\" (UniqueName: \"kubernetes.io/projected/a183631b-3c99-4de5-a235-e1bbf27efcb5-kube-api-access-swcfk\") pod \"dns-operator-controller-manager-648d5c98bc-qb8dv\" (UID: \"a183631b-3c99-4de5-a235-e1bbf27efcb5\") " pod="kuadrant-system/dns-operator-controller-manager-648d5c98bc-qb8dv" Jun 03 16:25:24.642620 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:25:24.642596 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-swcfk\" (UniqueName: \"kubernetes.io/projected/a183631b-3c99-4de5-a235-e1bbf27efcb5-kube-api-access-swcfk\") pod \"dns-operator-controller-manager-648d5c98bc-qb8dv\" (UID: \"a183631b-3c99-4de5-a235-e1bbf27efcb5\") " pod="kuadrant-system/dns-operator-controller-manager-648d5c98bc-qb8dv" Jun 03 16:25:24.783031 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:25:24.782967 2568 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-648d5c98bc-qb8dv" Jun 03 16:25:24.908684 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:25:24.908655 2568 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-648d5c98bc-qb8dv"] Jun 03 16:25:24.910596 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:25:24.910570 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda183631b_3c99_4de5_a235_e1bbf27efcb5.slice/crio-7f8f052915d516ef15b9b86c74ce6a2ae93d2e138c08925ac8b52729a8e28879 WatchSource:0}: Error finding container 7f8f052915d516ef15b9b86c74ce6a2ae93d2e138c08925ac8b52729a8e28879: Status 404 returned error can't find the container with id 7f8f052915d516ef15b9b86c74ce6a2ae93d2e138c08925ac8b52729a8e28879 Jun 03 16:25:25.341357 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:25:25.341320 2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-648d5c98bc-qb8dv" event={"ID":"a183631b-3c99-4de5-a235-e1bbf27efcb5","Type":"ContainerStarted","Data":"7f8f052915d516ef15b9b86c74ce6a2ae93d2e138c08925ac8b52729a8e28879"} Jun 03 16:25:28.357285 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:25:28.357251 2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-648d5c98bc-qb8dv" event={"ID":"a183631b-3c99-4de5-a235-e1bbf27efcb5","Type":"ContainerStarted","Data":"a7f9bad905a5a42d15110e95284914edd47402ee2e784194ff521b0120e5487d"} Jun 03 16:25:28.357668 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:25:28.357366 2568 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-648d5c98bc-qb8dv" Jun 03 16:25:28.374820 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:25:28.374778 2568 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-648d5c98bc-qb8dv" podStartSLOduration=1.955064097 podStartE2EDuration="4.374759295s" podCreationTimestamp="2026-06-03 16:25:24 +0000 UTC" firstStartedPulling="2026-06-03 16:25:24.91247928 +0000 UTC m=+496.000905849" lastFinishedPulling="2026-06-03 16:25:27.33217447 +0000 UTC m=+498.420601047" observedRunningTime="2026-06-03 16:25:28.372449167 +0000 UTC m=+499.460875757" watchObservedRunningTime="2026-06-03 16:25:28.374759295 +0000 UTC m=+499.463185889" Jun 03 16:25:28.745233 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:25:28.745145 2568 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-657f44b778-22cg6"] Jun 03 16:25:28.765525 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:25:28.765497 2568 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-657f44b778-22cg6" Jun 03 16:25:28.768890 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:25:28.768605 2568 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-657f44b778-22cg6"] Jun 03 16:25:28.769088 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:25:28.769034 2568 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-qnfwx\"" Jun 03 16:25:28.870421 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:25:28.870390 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fmnc5\" (UniqueName: \"kubernetes.io/projected/3a592c0c-0b15-4908-8bf7-787646a82d01-kube-api-access-fmnc5\") pod \"authorino-operator-657f44b778-22cg6\" (UID: \"3a592c0c-0b15-4908-8bf7-787646a82d01\") " pod="kuadrant-system/authorino-operator-657f44b778-22cg6" Jun 03 16:25:28.971452 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:25:28.971422 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-fmnc5\" (UniqueName: \"kubernetes.io/projected/3a592c0c-0b15-4908-8bf7-787646a82d01-kube-api-access-fmnc5\") pod \"authorino-operator-657f44b778-22cg6\" (UID: \"3a592c0c-0b15-4908-8bf7-787646a82d01\") " pod="kuadrant-system/authorino-operator-657f44b778-22cg6" Jun 03 16:25:28.982135 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:25:28.982102 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fmnc5\" (UniqueName: \"kubernetes.io/projected/3a592c0c-0b15-4908-8bf7-787646a82d01-kube-api-access-fmnc5\") pod \"authorino-operator-657f44b778-22cg6\" (UID: \"3a592c0c-0b15-4908-8bf7-787646a82d01\") " pod="kuadrant-system/authorino-operator-657f44b778-22cg6" Jun 03 16:25:29.081413 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:25:29.081342 2568 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-657f44b778-22cg6" Jun 03 16:25:29.211068 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:25:29.211045 2568 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-657f44b778-22cg6"] Jun 03 16:25:29.212630 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:25:29.212598 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3a592c0c_0b15_4908_8bf7_787646a82d01.slice/crio-2255ac5a33ea7dc60d72ee7ff931119f7d8d37891863030f5c00ce68871598b6 WatchSource:0}: Error finding container 2255ac5a33ea7dc60d72ee7ff931119f7d8d37891863030f5c00ce68871598b6: Status 404 returned error can't find the container with id 2255ac5a33ea7dc60d72ee7ff931119f7d8d37891863030f5c00ce68871598b6 Jun 03 16:25:29.361823 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:25:29.361789 2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-657f44b778-22cg6" event={"ID":"3a592c0c-0b15-4908-8bf7-787646a82d01","Type":"ContainerStarted","Data":"2255ac5a33ea7dc60d72ee7ff931119f7d8d37891863030f5c00ce68871598b6"} Jun 03 16:25:32.375610 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:25:32.375572 2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-657f44b778-22cg6" event={"ID":"3a592c0c-0b15-4908-8bf7-787646a82d01","Type":"ContainerStarted","Data":"fc32655795377d201fd7232728869e754d929ffb6b4e60e9f60b6cadc92fc933"} Jun 03 16:25:32.376036 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:25:32.375665 2568 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-657f44b778-22cg6" Jun 03 16:25:32.397895 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:25:32.397851 2568 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-657f44b778-22cg6" podStartSLOduration=2.038632476 podStartE2EDuration="4.397838347s" podCreationTimestamp="2026-06-03 16:25:28 +0000 UTC" firstStartedPulling="2026-06-03 16:25:29.214573028 +0000 UTC m=+500.302999596" lastFinishedPulling="2026-06-03 16:25:31.573778885 +0000 UTC m=+502.662205467" observedRunningTime="2026-06-03 16:25:32.394728446 +0000 UTC m=+503.483155036" watchObservedRunningTime="2026-06-03 16:25:32.397838347 +0000 UTC m=+503.486264938" Jun 03 16:25:36.372461 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:25:36.372394 2568 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-console/console-df89b8f67-6rjh2" podUID="62ec1bd3-2f2f-4c73-b966-6e5f676ed79b" containerName="console" containerID="cri-o://fcd9e390262de7df2cd5f3981abb27f5f28cdc958abf07cdc732f04bf4e13277" gracePeriod=15 Jun 03 16:25:36.613862 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:25:36.613841 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-df89b8f67-6rjh2_62ec1bd3-2f2f-4c73-b966-6e5f676ed79b/console/0.log" Jun 03 16:25:36.613971 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:25:36.613897 2568 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-df89b8f67-6rjh2" Jun 03 16:25:36.737805 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:25:36.737732 2568 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/62ec1bd3-2f2f-4c73-b966-6e5f676ed79b-service-ca\") pod \"62ec1bd3-2f2f-4c73-b966-6e5f676ed79b\" (UID: \"62ec1bd3-2f2f-4c73-b966-6e5f676ed79b\") " Jun 03 16:25:36.737805 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:25:36.737770 2568 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/62ec1bd3-2f2f-4c73-b966-6e5f676ed79b-trusted-ca-bundle\") pod \"62ec1bd3-2f2f-4c73-b966-6e5f676ed79b\" (UID: \"62ec1bd3-2f2f-4c73-b966-6e5f676ed79b\") " Jun 03 16:25:36.737966 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:25:36.737811 2568 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/62ec1bd3-2f2f-4c73-b966-6e5f676ed79b-oauth-serving-cert\") pod \"62ec1bd3-2f2f-4c73-b966-6e5f676ed79b\" (UID: \"62ec1bd3-2f2f-4c73-b966-6e5f676ed79b\") " Jun 03 16:25:36.737966 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:25:36.737915 2568 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/62ec1bd3-2f2f-4c73-b966-6e5f676ed79b-console-oauth-config\") pod \"62ec1bd3-2f2f-4c73-b966-6e5f676ed79b\" (UID: \"62ec1bd3-2f2f-4c73-b966-6e5f676ed79b\") " Jun 03 16:25:36.737966 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:25:36.737960 2568 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/62ec1bd3-2f2f-4c73-b966-6e5f676ed79b-console-config\") pod \"62ec1bd3-2f2f-4c73-b966-6e5f676ed79b\" (UID: \"62ec1bd3-2f2f-4c73-b966-6e5f676ed79b\") " Jun 03 16:25:36.738105 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:25:36.737992 2568 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/62ec1bd3-2f2f-4c73-b966-6e5f676ed79b-console-serving-cert\") pod \"62ec1bd3-2f2f-4c73-b966-6e5f676ed79b\" (UID: \"62ec1bd3-2f2f-4c73-b966-6e5f676ed79b\") " Jun 03 16:25:36.738105 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:25:36.738041 2568 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wxmrn\" (UniqueName: \"kubernetes.io/projected/62ec1bd3-2f2f-4c73-b966-6e5f676ed79b-kube-api-access-wxmrn\") pod \"62ec1bd3-2f2f-4c73-b966-6e5f676ed79b\" (UID: \"62ec1bd3-2f2f-4c73-b966-6e5f676ed79b\") " Jun 03 16:25:36.738324 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:25:36.738246 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/62ec1bd3-2f2f-4c73-b966-6e5f676ed79b-service-ca" (OuterVolumeSpecName: "service-ca") pod "62ec1bd3-2f2f-4c73-b966-6e5f676ed79b" (UID: "62ec1bd3-2f2f-4c73-b966-6e5f676ed79b"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 03 16:25:36.738324 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:25:36.738258 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/62ec1bd3-2f2f-4c73-b966-6e5f676ed79b-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "62ec1bd3-2f2f-4c73-b966-6e5f676ed79b" (UID: "62ec1bd3-2f2f-4c73-b966-6e5f676ed79b"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 03 16:25:36.738488 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:25:36.738371 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/62ec1bd3-2f2f-4c73-b966-6e5f676ed79b-console-config" (OuterVolumeSpecName: "console-config") pod "62ec1bd3-2f2f-4c73-b966-6e5f676ed79b" (UID: "62ec1bd3-2f2f-4c73-b966-6e5f676ed79b"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 03 16:25:36.738488 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:25:36.738412 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/62ec1bd3-2f2f-4c73-b966-6e5f676ed79b-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "62ec1bd3-2f2f-4c73-b966-6e5f676ed79b" (UID: "62ec1bd3-2f2f-4c73-b966-6e5f676ed79b"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 03 16:25:36.740098 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:25:36.740076 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/62ec1bd3-2f2f-4c73-b966-6e5f676ed79b-kube-api-access-wxmrn" (OuterVolumeSpecName: "kube-api-access-wxmrn") pod "62ec1bd3-2f2f-4c73-b966-6e5f676ed79b" (UID: "62ec1bd3-2f2f-4c73-b966-6e5f676ed79b"). InnerVolumeSpecName "kube-api-access-wxmrn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 03 16:25:36.740505 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:25:36.740483 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/62ec1bd3-2f2f-4c73-b966-6e5f676ed79b-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "62ec1bd3-2f2f-4c73-b966-6e5f676ed79b" (UID: "62ec1bd3-2f2f-4c73-b966-6e5f676ed79b"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 03 16:25:36.740579 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:25:36.740493 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/62ec1bd3-2f2f-4c73-b966-6e5f676ed79b-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "62ec1bd3-2f2f-4c73-b966-6e5f676ed79b" (UID: "62ec1bd3-2f2f-4c73-b966-6e5f676ed79b"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 03 16:25:36.839235 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:25:36.839195 2568 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/62ec1bd3-2f2f-4c73-b966-6e5f676ed79b-oauth-serving-cert\") on node \"ip-10-0-128-88.ec2.internal\" DevicePath \"\"" Jun 03 16:25:36.839235 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:25:36.839231 2568 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/62ec1bd3-2f2f-4c73-b966-6e5f676ed79b-console-oauth-config\") on node \"ip-10-0-128-88.ec2.internal\" DevicePath \"\"" Jun 03 16:25:36.839235 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:25:36.839245 2568 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/62ec1bd3-2f2f-4c73-b966-6e5f676ed79b-console-config\") on node \"ip-10-0-128-88.ec2.internal\" DevicePath \"\"" Jun 03 16:25:36.839435 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:25:36.839258 2568 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/62ec1bd3-2f2f-4c73-b966-6e5f676ed79b-console-serving-cert\") on node \"ip-10-0-128-88.ec2.internal\" DevicePath \"\"" Jun 03 16:25:36.839435 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:25:36.839270 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wxmrn\" (UniqueName: \"kubernetes.io/projected/62ec1bd3-2f2f-4c73-b966-6e5f676ed79b-kube-api-access-wxmrn\") on node \"ip-10-0-128-88.ec2.internal\" DevicePath \"\"" Jun 03 16:25:36.839435 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:25:36.839282 2568 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/62ec1bd3-2f2f-4c73-b966-6e5f676ed79b-service-ca\") on node \"ip-10-0-128-88.ec2.internal\" DevicePath \"\"" Jun 03 16:25:36.839435 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:25:36.839292 2568 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/62ec1bd3-2f2f-4c73-b966-6e5f676ed79b-trusted-ca-bundle\") on node \"ip-10-0-128-88.ec2.internal\" DevicePath \"\"" Jun 03 16:25:37.394845 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:25:37.394820 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-df89b8f67-6rjh2_62ec1bd3-2f2f-4c73-b966-6e5f676ed79b/console/0.log" Jun 03 16:25:37.395257 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:25:37.394859 2568 generic.go:358] "Generic (PLEG): container finished" podID="62ec1bd3-2f2f-4c73-b966-6e5f676ed79b" containerID="fcd9e390262de7df2cd5f3981abb27f5f28cdc958abf07cdc732f04bf4e13277" exitCode=2 Jun 03 16:25:37.395257 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:25:37.394923 2568 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-df89b8f67-6rjh2" Jun 03 16:25:37.395257 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:25:37.394956 2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-df89b8f67-6rjh2" event={"ID":"62ec1bd3-2f2f-4c73-b966-6e5f676ed79b","Type":"ContainerDied","Data":"fcd9e390262de7df2cd5f3981abb27f5f28cdc958abf07cdc732f04bf4e13277"} Jun 03 16:25:37.395257 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:25:37.394992 2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-df89b8f67-6rjh2" event={"ID":"62ec1bd3-2f2f-4c73-b966-6e5f676ed79b","Type":"ContainerDied","Data":"d9a2262827adf182289f27a9011955b04809a1c6e65dadabdef1f8bc39543bfe"} Jun 03 16:25:37.395257 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:25:37.395007 2568 scope.go:117] "RemoveContainer" containerID="fcd9e390262de7df2cd5f3981abb27f5f28cdc958abf07cdc732f04bf4e13277" Jun 03 16:25:37.404179 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:25:37.404166 2568 scope.go:117] "RemoveContainer" containerID="fcd9e390262de7df2cd5f3981abb27f5f28cdc958abf07cdc732f04bf4e13277" Jun 03 16:25:37.404423 ip-10-0-128-88 kubenswrapper[2568]: E0603 16:25:37.404408 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fcd9e390262de7df2cd5f3981abb27f5f28cdc958abf07cdc732f04bf4e13277\": container with ID starting with fcd9e390262de7df2cd5f3981abb27f5f28cdc958abf07cdc732f04bf4e13277 not found: ID does not exist" containerID="fcd9e390262de7df2cd5f3981abb27f5f28cdc958abf07cdc732f04bf4e13277" Jun 03 16:25:37.404462 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:25:37.404432 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fcd9e390262de7df2cd5f3981abb27f5f28cdc958abf07cdc732f04bf4e13277"} err="failed to get container status \"fcd9e390262de7df2cd5f3981abb27f5f28cdc958abf07cdc732f04bf4e13277\": rpc error: code = NotFound desc = could not find container \"fcd9e390262de7df2cd5f3981abb27f5f28cdc958abf07cdc732f04bf4e13277\": container with ID starting with fcd9e390262de7df2cd5f3981abb27f5f28cdc958abf07cdc732f04bf4e13277 not found: ID does not exist" Jun 03 16:25:37.420190 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:25:37.420166 2568 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-console/console-df89b8f67-6rjh2"] Jun 03 16:25:37.425848 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:25:37.425828 2568 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-df89b8f67-6rjh2"] Jun 03 16:25:37.570119 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:25:37.570083 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="62ec1bd3-2f2f-4c73-b966-6e5f676ed79b" path="/var/lib/kubelet/pods/62ec1bd3-2f2f-4c73-b966-6e5f676ed79b/volumes" Jun 03 16:25:39.364789 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:25:39.364760 2568 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-648d5c98bc-qb8dv" Jun 03 16:25:43.381262 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:25:43.381235 2568 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-657f44b778-22cg6" Jun 03 16:25:46.098880 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:25:46.098842 2568 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-6cb54b5c86-5kzgr"] Jun 03 16:25:46.099308 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:25:46.099180 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="62ec1bd3-2f2f-4c73-b966-6e5f676ed79b" containerName="console" Jun 03 16:25:46.099308 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:25:46.099190 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="62ec1bd3-2f2f-4c73-b966-6e5f676ed79b" containerName="console" Jun 03 16:25:46.099308 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:25:46.099262 2568 memory_manager.go:356] "RemoveStaleState removing state" podUID="62ec1bd3-2f2f-4c73-b966-6e5f676ed79b" containerName="console" Jun 03 16:25:46.105279 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:25:46.105253 2568 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-6cb54b5c86-5kzgr" Jun 03 16:25:46.107705 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:25:46.107680 2568 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\"" Jun 03 16:25:46.108036 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:25:46.107998 2568 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\"" Jun 03 16:25:46.108207 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:25:46.108191 2568 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-5t7wf\"" Jun 03 16:25:46.109816 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:25:46.109791 2568 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-6cb54b5c86-5kzgr"] Jun 03 16:25:46.217637 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:25:46.217602 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/c9e9db85-9f68-4fc7-8e11-82433d9a6977-plugin-serving-cert\") pod \"kuadrant-console-plugin-6cb54b5c86-5kzgr\" (UID: \"c9e9db85-9f68-4fc7-8e11-82433d9a6977\") " pod="kuadrant-system/kuadrant-console-plugin-6cb54b5c86-5kzgr" Jun 03 16:25:46.217816 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:25:46.217645 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qs8n5\" (UniqueName: \"kubernetes.io/projected/c9e9db85-9f68-4fc7-8e11-82433d9a6977-kube-api-access-qs8n5\") pod \"kuadrant-console-plugin-6cb54b5c86-5kzgr\" (UID: \"c9e9db85-9f68-4fc7-8e11-82433d9a6977\") " pod="kuadrant-system/kuadrant-console-plugin-6cb54b5c86-5kzgr" Jun 03 16:25:46.217816 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:25:46.217713 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/c9e9db85-9f68-4fc7-8e11-82433d9a6977-nginx-conf\") pod \"kuadrant-console-plugin-6cb54b5c86-5kzgr\" (UID: \"c9e9db85-9f68-4fc7-8e11-82433d9a6977\") " pod="kuadrant-system/kuadrant-console-plugin-6cb54b5c86-5kzgr" Jun 03 16:25:46.318419 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:25:46.318385 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/c9e9db85-9f68-4fc7-8e11-82433d9a6977-plugin-serving-cert\") pod \"kuadrant-console-plugin-6cb54b5c86-5kzgr\" (UID: \"c9e9db85-9f68-4fc7-8e11-82433d9a6977\") " pod="kuadrant-system/kuadrant-console-plugin-6cb54b5c86-5kzgr" Jun 03 16:25:46.318587 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:25:46.318427 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-qs8n5\" (UniqueName: \"kubernetes.io/projected/c9e9db85-9f68-4fc7-8e11-82433d9a6977-kube-api-access-qs8n5\") pod \"kuadrant-console-plugin-6cb54b5c86-5kzgr\" (UID: \"c9e9db85-9f68-4fc7-8e11-82433d9a6977\") " pod="kuadrant-system/kuadrant-console-plugin-6cb54b5c86-5kzgr" Jun 03 16:25:46.318587 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:25:46.318469 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/c9e9db85-9f68-4fc7-8e11-82433d9a6977-nginx-conf\") pod \"kuadrant-console-plugin-6cb54b5c86-5kzgr\" (UID: \"c9e9db85-9f68-4fc7-8e11-82433d9a6977\") " pod="kuadrant-system/kuadrant-console-plugin-6cb54b5c86-5kzgr" Jun 03 16:25:46.318587 ip-10-0-128-88 kubenswrapper[2568]: E0603 16:25:46.318554 2568 secret.go:189] Couldn't get secret kuadrant-system/plugin-serving-cert: secret "plugin-serving-cert" not found Jun 03 16:25:46.318703 ip-10-0-128-88 kubenswrapper[2568]: E0603 16:25:46.318630 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c9e9db85-9f68-4fc7-8e11-82433d9a6977-plugin-serving-cert podName:c9e9db85-9f68-4fc7-8e11-82433d9a6977 nodeName:}" failed. No retries permitted until 2026-06-03 16:25:46.818609155 +0000 UTC m=+517.907035742 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "plugin-serving-cert" (UniqueName: "kubernetes.io/secret/c9e9db85-9f68-4fc7-8e11-82433d9a6977-plugin-serving-cert") pod "kuadrant-console-plugin-6cb54b5c86-5kzgr" (UID: "c9e9db85-9f68-4fc7-8e11-82433d9a6977") : secret "plugin-serving-cert" not found Jun 03 16:25:46.319123 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:25:46.319105 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/c9e9db85-9f68-4fc7-8e11-82433d9a6977-nginx-conf\") pod \"kuadrant-console-plugin-6cb54b5c86-5kzgr\" (UID: \"c9e9db85-9f68-4fc7-8e11-82433d9a6977\") " pod="kuadrant-system/kuadrant-console-plugin-6cb54b5c86-5kzgr" Jun 03 16:25:46.328938 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:25:46.328911 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qs8n5\" (UniqueName: \"kubernetes.io/projected/c9e9db85-9f68-4fc7-8e11-82433d9a6977-kube-api-access-qs8n5\") pod \"kuadrant-console-plugin-6cb54b5c86-5kzgr\" (UID: \"c9e9db85-9f68-4fc7-8e11-82433d9a6977\") " pod="kuadrant-system/kuadrant-console-plugin-6cb54b5c86-5kzgr" Jun 03 16:25:46.823012 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:25:46.822981 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/c9e9db85-9f68-4fc7-8e11-82433d9a6977-plugin-serving-cert\") pod \"kuadrant-console-plugin-6cb54b5c86-5kzgr\" (UID: \"c9e9db85-9f68-4fc7-8e11-82433d9a6977\") " pod="kuadrant-system/kuadrant-console-plugin-6cb54b5c86-5kzgr" Jun 03 16:25:46.825385 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:25:46.825362 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/c9e9db85-9f68-4fc7-8e11-82433d9a6977-plugin-serving-cert\") pod \"kuadrant-console-plugin-6cb54b5c86-5kzgr\" (UID: \"c9e9db85-9f68-4fc7-8e11-82433d9a6977\") " pod="kuadrant-system/kuadrant-console-plugin-6cb54b5c86-5kzgr" Jun 03 16:25:47.015612 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:25:47.015577 2568 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-6cb54b5c86-5kzgr" Jun 03 16:25:47.143115 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:25:47.143087 2568 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-6cb54b5c86-5kzgr"] Jun 03 16:25:47.144370 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:25:47.144345 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc9e9db85_9f68_4fc7_8e11_82433d9a6977.slice/crio-44e1fa7ae240c5d1014611716430f8bd7941551f0981d504bb47390d053fca46 WatchSource:0}: Error finding container 44e1fa7ae240c5d1014611716430f8bd7941551f0981d504bb47390d053fca46: Status 404 returned error can't find the container with id 44e1fa7ae240c5d1014611716430f8bd7941551f0981d504bb47390d053fca46 Jun 03 16:25:47.435632 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:25:47.435600 2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-6cb54b5c86-5kzgr" event={"ID":"c9e9db85-9f68-4fc7-8e11-82433d9a6977","Type":"ContainerStarted","Data":"44e1fa7ae240c5d1014611716430f8bd7941551f0981d504bb47390d053fca46"} Jun 03 16:25:56.758940 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:25:56.758740 2568 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-85c4996f8c-zwvrp"] Jun 03 16:25:56.764870 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:25:56.764841 2568 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-zwvrp" Jun 03 16:25:56.769006 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:25:56.768980 2568 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-2j2lq\"" Jun 03 16:25:56.774681 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:25:56.774233 2568 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-85c4996f8c-zwvrp"] Jun 03 16:25:56.913359 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:25:56.913314 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zmc2c\" (UniqueName: \"kubernetes.io/projected/4143b490-3201-49e7-99fa-bc16a828ecb0-kube-api-access-zmc2c\") pod \"limitador-operator-controller-manager-85c4996f8c-zwvrp\" (UID: \"4143b490-3201-49e7-99fa-bc16a828ecb0\") " pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-zwvrp" Jun 03 16:25:57.014826 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:25:57.014745 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-zmc2c\" (UniqueName: \"kubernetes.io/projected/4143b490-3201-49e7-99fa-bc16a828ecb0-kube-api-access-zmc2c\") pod \"limitador-operator-controller-manager-85c4996f8c-zwvrp\" (UID: \"4143b490-3201-49e7-99fa-bc16a828ecb0\") " pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-zwvrp" Jun 03 16:25:57.028842 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:25:57.028812 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zmc2c\" (UniqueName: \"kubernetes.io/projected/4143b490-3201-49e7-99fa-bc16a828ecb0-kube-api-access-zmc2c\") pod \"limitador-operator-controller-manager-85c4996f8c-zwvrp\" (UID: \"4143b490-3201-49e7-99fa-bc16a828ecb0\") " pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-zwvrp" Jun 03 16:25:57.079063 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:25:57.079019 2568 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-zwvrp" Jun 03 16:25:57.215119 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:25:57.215093 2568 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-85c4996f8c-zwvrp"] Jun 03 16:25:57.216777 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:25:57.216747 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4143b490_3201_49e7_99fa_bc16a828ecb0.slice/crio-1d3a68c149a2ab2b906e5c1b196f0406f1d5f2f2c9aecb84008ed6d8086fc885 WatchSource:0}: Error finding container 1d3a68c149a2ab2b906e5c1b196f0406f1d5f2f2c9aecb84008ed6d8086fc885: Status 404 returned error can't find the container with id 1d3a68c149a2ab2b906e5c1b196f0406f1d5f2f2c9aecb84008ed6d8086fc885 Jun 03 16:25:57.481593 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:25:57.481556 2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-zwvrp" event={"ID":"4143b490-3201-49e7-99fa-bc16a828ecb0","Type":"ContainerStarted","Data":"1d3a68c149a2ab2b906e5c1b196f0406f1d5f2f2c9aecb84008ed6d8086fc885"} Jun 03 16:26:13.564716 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:26:13.564688 2568 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-zwvrp" Jun 03 16:26:13.564716 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:26:13.564718 2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-zwvrp" event={"ID":"4143b490-3201-49e7-99fa-bc16a828ecb0","Type":"ContainerStarted","Data":"6a59824eb5b0961b6db8d56108fea3752668f12852e0b36aa067420f33978842"} Jun 03 16:26:13.565157 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:26:13.564733 2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-6cb54b5c86-5kzgr" event={"ID":"c9e9db85-9f68-4fc7-8e11-82433d9a6977","Type":"ContainerStarted","Data":"929f1ab25ac1dd96afc2b2c3e08c1eb57dc1ec7bdd695e6e992cce070f02daab"} Jun 03 16:26:13.582069 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:26:13.582028 2568 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-zwvrp" podStartSLOduration=2.241562998 podStartE2EDuration="17.582015739s" podCreationTimestamp="2026-06-03 16:25:56 +0000 UTC" firstStartedPulling="2026-06-03 16:25:57.21862064 +0000 UTC m=+528.307047207" lastFinishedPulling="2026-06-03 16:26:12.559073374 +0000 UTC m=+543.647499948" observedRunningTime="2026-06-03 16:26:13.579403308 +0000 UTC m=+544.667829897" watchObservedRunningTime="2026-06-03 16:26:13.582015739 +0000 UTC m=+544.670442328" Jun 03 16:26:13.596705 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:26:13.596664 2568 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-6cb54b5c86-5kzgr" podStartSLOduration=2.150584377 podStartE2EDuration="27.596652328s" podCreationTimestamp="2026-06-03 16:25:46 +0000 UTC" firstStartedPulling="2026-06-03 16:25:47.14559331 +0000 UTC m=+518.234019879" lastFinishedPulling="2026-06-03 16:26:12.591661262 +0000 UTC m=+543.680087830" observedRunningTime="2026-06-03 16:26:13.596500821 +0000 UTC m=+544.684927410" watchObservedRunningTime="2026-06-03 16:26:13.596652328 +0000 UTC m=+544.685078917" Jun 03 16:26:24.568955 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:26:24.568916 2568 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-85c4996f8c-zwvrp" Jun 03 16:26:39.220228 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:26:39.220188 2568 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-f99f4b5cd-sgfbt"] Jun 03 16:26:39.321423 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:26:39.321393 2568 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-f99f4b5cd-sgfbt"] Jun 03 16:26:39.321610 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:26:39.321505 2568 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-f99f4b5cd-sgfbt" Jun 03 16:26:39.324082 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:26:39.324060 2568 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-f2t4m\"" Jun 03 16:26:39.398056 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:26:39.398020 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2kk8x\" (UniqueName: \"kubernetes.io/projected/ee8a8189-2f35-4ad5-8459-07ab2e6b58e8-kube-api-access-2kk8x\") pod \"authorino-f99f4b5cd-sgfbt\" (UID: \"ee8a8189-2f35-4ad5-8459-07ab2e6b58e8\") " pod="kuadrant-system/authorino-f99f4b5cd-sgfbt" Jun 03 16:26:39.499478 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:26:39.499401 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-2kk8x\" (UniqueName: \"kubernetes.io/projected/ee8a8189-2f35-4ad5-8459-07ab2e6b58e8-kube-api-access-2kk8x\") pod \"authorino-f99f4b5cd-sgfbt\" (UID: \"ee8a8189-2f35-4ad5-8459-07ab2e6b58e8\") " pod="kuadrant-system/authorino-f99f4b5cd-sgfbt" Jun 03 16:26:39.507058 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:26:39.507037 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2kk8x\" (UniqueName: \"kubernetes.io/projected/ee8a8189-2f35-4ad5-8459-07ab2e6b58e8-kube-api-access-2kk8x\") pod \"authorino-f99f4b5cd-sgfbt\" (UID: \"ee8a8189-2f35-4ad5-8459-07ab2e6b58e8\") " pod="kuadrant-system/authorino-f99f4b5cd-sgfbt" Jun 03 16:26:39.631306 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:26:39.631268 2568 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-f99f4b5cd-sgfbt" Jun 03 16:26:39.768113 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:26:39.767965 2568 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-f99f4b5cd-sgfbt"] Jun 03 16:26:39.771006 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:26:39.770972 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podee8a8189_2f35_4ad5_8459_07ab2e6b58e8.slice/crio-855cc08b5ca1aaa82c2ca467414a7e919b4f5a656a82ca822148baffa4c33e7c WatchSource:0}: Error finding container 855cc08b5ca1aaa82c2ca467414a7e919b4f5a656a82ca822148baffa4c33e7c: Status 404 returned error can't find the container with id 855cc08b5ca1aaa82c2ca467414a7e919b4f5a656a82ca822148baffa4c33e7c Jun 03 16:26:40.664638 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:26:40.664600 2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-f99f4b5cd-sgfbt" event={"ID":"ee8a8189-2f35-4ad5-8459-07ab2e6b58e8","Type":"ContainerStarted","Data":"855cc08b5ca1aaa82c2ca467414a7e919b4f5a656a82ca822148baffa4c33e7c"} Jun 03 16:26:44.121168 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:26:44.121133 2568 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-f99f4b5cd-sgfbt"] Jun 03 16:26:44.683377 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:26:44.683337 2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-f99f4b5cd-sgfbt" event={"ID":"ee8a8189-2f35-4ad5-8459-07ab2e6b58e8","Type":"ContainerStarted","Data":"e63967455a1458a5965bb9f2e7ebd0d8b4482bb89afdbdabbe5573fb2cce5f47"} Jun 03 16:26:44.698893 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:26:44.698847 2568 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/authorino-f99f4b5cd-sgfbt" podStartSLOduration=1.570600478 podStartE2EDuration="5.698832687s" podCreationTimestamp="2026-06-03 16:26:39 +0000 UTC" firstStartedPulling="2026-06-03 16:26:39.772164858 +0000 UTC m=+570.860591427" lastFinishedPulling="2026-06-03 16:26:43.900397069 +0000 UTC m=+574.988823636" observedRunningTime="2026-06-03 16:26:44.697372871 +0000 UTC m=+575.785799461" watchObservedRunningTime="2026-06-03 16:26:44.698832687 +0000 UTC m=+575.787259276" Jun 03 16:26:45.687126 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:26:45.687088 2568 kuberuntime_container.go:864] "Killing container with a grace period" pod="kuadrant-system/authorino-f99f4b5cd-sgfbt" podUID="ee8a8189-2f35-4ad5-8459-07ab2e6b58e8" containerName="authorino" containerID="cri-o://e63967455a1458a5965bb9f2e7ebd0d8b4482bb89afdbdabbe5573fb2cce5f47" gracePeriod=30 Jun 03 16:26:45.953664 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:26:45.953644 2568 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-f99f4b5cd-sgfbt" Jun 03 16:26:46.063443 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:26:46.063410 2568 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2kk8x\" (UniqueName: \"kubernetes.io/projected/ee8a8189-2f35-4ad5-8459-07ab2e6b58e8-kube-api-access-2kk8x\") pod \"ee8a8189-2f35-4ad5-8459-07ab2e6b58e8\" (UID: \"ee8a8189-2f35-4ad5-8459-07ab2e6b58e8\") " Jun 03 16:26:46.065469 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:26:46.065440 2568 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ee8a8189-2f35-4ad5-8459-07ab2e6b58e8-kube-api-access-2kk8x" (OuterVolumeSpecName: "kube-api-access-2kk8x") pod "ee8a8189-2f35-4ad5-8459-07ab2e6b58e8" (UID: "ee8a8189-2f35-4ad5-8459-07ab2e6b58e8"). InnerVolumeSpecName "kube-api-access-2kk8x". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 03 16:26:46.164993 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:26:46.164958 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2kk8x\" (UniqueName: \"kubernetes.io/projected/ee8a8189-2f35-4ad5-8459-07ab2e6b58e8-kube-api-access-2kk8x\") on node \"ip-10-0-128-88.ec2.internal\" DevicePath \"\"" Jun 03 16:26:46.691952 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:26:46.691916 2568 generic.go:358] "Generic (PLEG): container finished" podID="ee8a8189-2f35-4ad5-8459-07ab2e6b58e8" containerID="e63967455a1458a5965bb9f2e7ebd0d8b4482bb89afdbdabbe5573fb2cce5f47" exitCode=0 Jun 03 16:26:46.692412 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:26:46.691970 2568 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-f99f4b5cd-sgfbt" Jun 03 16:26:46.692412 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:26:46.691995 2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-f99f4b5cd-sgfbt" event={"ID":"ee8a8189-2f35-4ad5-8459-07ab2e6b58e8","Type":"ContainerDied","Data":"e63967455a1458a5965bb9f2e7ebd0d8b4482bb89afdbdabbe5573fb2cce5f47"} Jun 03 16:26:46.692412 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:26:46.692037 2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-f99f4b5cd-sgfbt" event={"ID":"ee8a8189-2f35-4ad5-8459-07ab2e6b58e8","Type":"ContainerDied","Data":"855cc08b5ca1aaa82c2ca467414a7e919b4f5a656a82ca822148baffa4c33e7c"} Jun 03 16:26:46.692412 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:26:46.692077 2568 scope.go:117] "RemoveContainer" containerID="e63967455a1458a5965bb9f2e7ebd0d8b4482bb89afdbdabbe5573fb2cce5f47" Jun 03 16:26:46.700728 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:26:46.700714 2568 scope.go:117] "RemoveContainer" containerID="e63967455a1458a5965bb9f2e7ebd0d8b4482bb89afdbdabbe5573fb2cce5f47" Jun 03 16:26:46.700971 ip-10-0-128-88 kubenswrapper[2568]: E0603 16:26:46.700952 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e63967455a1458a5965bb9f2e7ebd0d8b4482bb89afdbdabbe5573fb2cce5f47\": container with ID starting with e63967455a1458a5965bb9f2e7ebd0d8b4482bb89afdbdabbe5573fb2cce5f47 not found: ID does not exist" containerID="e63967455a1458a5965bb9f2e7ebd0d8b4482bb89afdbdabbe5573fb2cce5f47" Jun 03 16:26:46.701023 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:26:46.700980 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e63967455a1458a5965bb9f2e7ebd0d8b4482bb89afdbdabbe5573fb2cce5f47"} err="failed to get container status \"e63967455a1458a5965bb9f2e7ebd0d8b4482bb89afdbdabbe5573fb2cce5f47\": rpc error: code = NotFound desc = could not find container \"e63967455a1458a5965bb9f2e7ebd0d8b4482bb89afdbdabbe5573fb2cce5f47\": container with ID starting with e63967455a1458a5965bb9f2e7ebd0d8b4482bb89afdbdabbe5573fb2cce5f47 not found: ID does not exist" Jun 03 16:26:46.716194 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:26:46.716165 2568 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-f99f4b5cd-sgfbt"] Jun 03 16:26:46.719928 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:26:46.719909 2568 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-f99f4b5cd-sgfbt"] Jun 03 16:26:47.565916 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:26:47.565822 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ee8a8189-2f35-4ad5-8459-07ab2e6b58e8" path="/var/lib/kubelet/pods/ee8a8189-2f35-4ad5-8459-07ab2e6b58e8/volumes" Jun 03 16:27:58.979589 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:27:58.979559 2568 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-nmf8k"] Jun 03 16:27:58.980031 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:27:58.979925 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ee8a8189-2f35-4ad5-8459-07ab2e6b58e8" containerName="authorino" Jun 03 16:27:58.980031 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:27:58.979936 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="ee8a8189-2f35-4ad5-8459-07ab2e6b58e8" containerName="authorino" Jun 03 16:27:58.980031 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:27:58.980002 2568 memory_manager.go:356] "RemoveStaleState removing state" podUID="ee8a8189-2f35-4ad5-8459-07ab2e6b58e8" containerName="authorino" Jun 03 16:27:58.982210 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:27:58.982190 2568 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-nmf8k" Jun 03 16:27:58.984948 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:27:58.984919 2568 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jun 03 16:27:58.985070 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:27:58.984923 2568 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-vxrmx\"" Jun 03 16:27:58.985070 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:27:58.984962 2568 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\"" Jun 03 16:27:58.985070 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:27:58.984955 2568 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jun 03 16:27:58.992682 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:27:58.992660 2568 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-nmf8k"] Jun 03 16:27:59.076104 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:27:59.076077 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ec14e607-775c-4c96-bb3f-e5b4b8f04b6d-model-cache\") pod \"facebook-opt-125m-simulated-kserve-8f8dc67b7-nmf8k\" (UID: \"ec14e607-775c-4c96-bb3f-e5b4b8f04b6d\") " pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-nmf8k" Jun 03 16:27:59.076261 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:27:59.076116 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ec14e607-775c-4c96-bb3f-e5b4b8f04b6d-dshm\") pod \"facebook-opt-125m-simulated-kserve-8f8dc67b7-nmf8k\" (UID: \"ec14e607-775c-4c96-bb3f-e5b4b8f04b6d\") " pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-nmf8k" Jun 03 16:27:59.076261 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:27:59.076208 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2jftf\" (UniqueName: \"kubernetes.io/projected/ec14e607-775c-4c96-bb3f-e5b4b8f04b6d-kube-api-access-2jftf\") pod \"facebook-opt-125m-simulated-kserve-8f8dc67b7-nmf8k\" (UID: \"ec14e607-775c-4c96-bb3f-e5b4b8f04b6d\") " pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-nmf8k" Jun 03 16:27:59.076261 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:27:59.076240 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ec14e607-775c-4c96-bb3f-e5b4b8f04b6d-home\") pod \"facebook-opt-125m-simulated-kserve-8f8dc67b7-nmf8k\" (UID: \"ec14e607-775c-4c96-bb3f-e5b4b8f04b6d\") " pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-nmf8k" Jun 03 16:27:59.076369 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:27:59.076268 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ec14e607-775c-4c96-bb3f-e5b4b8f04b6d-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-8f8dc67b7-nmf8k\" (UID: \"ec14e607-775c-4c96-bb3f-e5b4b8f04b6d\") " pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-nmf8k" Jun 03 16:27:59.076369 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:27:59.076322 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/ec14e607-775c-4c96-bb3f-e5b4b8f04b6d-kserve-provision-location\") pod \"facebook-opt-125m-simulated-kserve-8f8dc67b7-nmf8k\" (UID: \"ec14e607-775c-4c96-bb3f-e5b4b8f04b6d\") " pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-nmf8k" Jun 03 16:27:59.176900 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:27:59.176869 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-2jftf\" (UniqueName: \"kubernetes.io/projected/ec14e607-775c-4c96-bb3f-e5b4b8f04b6d-kube-api-access-2jftf\") pod \"facebook-opt-125m-simulated-kserve-8f8dc67b7-nmf8k\" (UID: \"ec14e607-775c-4c96-bb3f-e5b4b8f04b6d\") " pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-nmf8k" Jun 03 16:27:59.177035 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:27:59.176906 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ec14e607-775c-4c96-bb3f-e5b4b8f04b6d-home\") pod \"facebook-opt-125m-simulated-kserve-8f8dc67b7-nmf8k\" (UID: \"ec14e607-775c-4c96-bb3f-e5b4b8f04b6d\") " pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-nmf8k" Jun 03 16:27:59.177035 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:27:59.176950 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ec14e607-775c-4c96-bb3f-e5b4b8f04b6d-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-8f8dc67b7-nmf8k\" (UID: \"ec14e607-775c-4c96-bb3f-e5b4b8f04b6d\") " pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-nmf8k" Jun 03 16:27:59.177035 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:27:59.176983 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/ec14e607-775c-4c96-bb3f-e5b4b8f04b6d-kserve-provision-location\") pod \"facebook-opt-125m-simulated-kserve-8f8dc67b7-nmf8k\" (UID: \"ec14e607-775c-4c96-bb3f-e5b4b8f04b6d\") " pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-nmf8k" Jun 03 16:27:59.177035 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:27:59.177026 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ec14e607-775c-4c96-bb3f-e5b4b8f04b6d-model-cache\") pod \"facebook-opt-125m-simulated-kserve-8f8dc67b7-nmf8k\" (UID: \"ec14e607-775c-4c96-bb3f-e5b4b8f04b6d\") " pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-nmf8k" Jun 03 16:27:59.177204 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:27:59.177082 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ec14e607-775c-4c96-bb3f-e5b4b8f04b6d-dshm\") pod \"facebook-opt-125m-simulated-kserve-8f8dc67b7-nmf8k\" (UID: \"ec14e607-775c-4c96-bb3f-e5b4b8f04b6d\") " pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-nmf8k" Jun 03 16:27:59.177274 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:27:59.177252 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ec14e607-775c-4c96-bb3f-e5b4b8f04b6d-home\") pod \"facebook-opt-125m-simulated-kserve-8f8dc67b7-nmf8k\" (UID: \"ec14e607-775c-4c96-bb3f-e5b4b8f04b6d\") " pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-nmf8k" Jun 03 16:27:59.177371 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:27:59.177354 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/ec14e607-775c-4c96-bb3f-e5b4b8f04b6d-kserve-provision-location\") pod \"facebook-opt-125m-simulated-kserve-8f8dc67b7-nmf8k\" (UID: \"ec14e607-775c-4c96-bb3f-e5b4b8f04b6d\") " pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-nmf8k" Jun 03 16:27:59.177496 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:27:59.177476 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ec14e607-775c-4c96-bb3f-e5b4b8f04b6d-model-cache\") pod \"facebook-opt-125m-simulated-kserve-8f8dc67b7-nmf8k\" (UID: \"ec14e607-775c-4c96-bb3f-e5b4b8f04b6d\") " pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-nmf8k" Jun 03 16:27:59.179304 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:27:59.179287 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ec14e607-775c-4c96-bb3f-e5b4b8f04b6d-dshm\") pod \"facebook-opt-125m-simulated-kserve-8f8dc67b7-nmf8k\" (UID: \"ec14e607-775c-4c96-bb3f-e5b4b8f04b6d\") " pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-nmf8k" Jun 03 16:27:59.179392 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:27:59.179376 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ec14e607-775c-4c96-bb3f-e5b4b8f04b6d-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-8f8dc67b7-nmf8k\" (UID: \"ec14e607-775c-4c96-bb3f-e5b4b8f04b6d\") " pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-nmf8k" Jun 03 16:27:59.184951 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:27:59.184921 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2jftf\" (UniqueName: \"kubernetes.io/projected/ec14e607-775c-4c96-bb3f-e5b4b8f04b6d-kube-api-access-2jftf\") pod \"facebook-opt-125m-simulated-kserve-8f8dc67b7-nmf8k\" (UID: \"ec14e607-775c-4c96-bb3f-e5b4b8f04b6d\") " pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-nmf8k" Jun 03 16:27:59.293873 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:27:59.293819 2568 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-nmf8k" Jun 03 16:27:59.424994 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:27:59.424957 2568 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-nmf8k"] Jun 03 16:27:59.427972 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:27:59.427945 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podec14e607_775c_4c96_bb3f_e5b4b8f04b6d.slice/crio-1fdb5c9f80672380572376ca488ba1556852db325d43e93150fcd901d9cb7b7a WatchSource:0}: Error finding container 1fdb5c9f80672380572376ca488ba1556852db325d43e93150fcd901d9cb7b7a: Status 404 returned error can't find the container with id 1fdb5c9f80672380572376ca488ba1556852db325d43e93150fcd901d9cb7b7a Jun 03 16:27:59.429599 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:27:59.429584 2568 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jun 03 16:27:59.968742 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:27:59.968709 2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-nmf8k" event={"ID":"ec14e607-775c-4c96-bb3f-e5b4b8f04b6d","Type":"ContainerStarted","Data":"1fdb5c9f80672380572376ca488ba1556852db325d43e93150fcd901d9cb7b7a"} Jun 03 16:28:10.011636 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:28:10.011594 2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-nmf8k" event={"ID":"ec14e607-775c-4c96-bb3f-e5b4b8f04b6d","Type":"ContainerStarted","Data":"75ce1a81552fca98b9620cadc078783d85840f7fb4bcfea19302180bb0d7b718"} Jun 03 16:28:15.575282 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:28:15.575251 2568 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-84db68679b-fq625"] Jun 03 16:28:15.578378 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:28:15.578362 2568 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-84db68679b-fq625" Jun 03 16:28:15.580414 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:28:15.580394 2568 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\"" Jun 03 16:28:15.586157 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:28:15.586138 2568 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-84db68679b-fq625"] Jun 03 16:28:15.733951 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:28:15.733912 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/4f802b10-22aa-4f74-9922-0643a27e15af-kserve-provision-location\") pod \"e2e-trlp-test-simulated-kserve-84db68679b-fq625\" (UID: \"4f802b10-22aa-4f74-9922-0643a27e15af\") " pod="llm/e2e-trlp-test-simulated-kserve-84db68679b-fq625" Jun 03 16:28:15.734135 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:28:15.733965 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4f802b10-22aa-4f74-9922-0643a27e15af-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-84db68679b-fq625\" (UID: \"4f802b10-22aa-4f74-9922-0643a27e15af\") " pod="llm/e2e-trlp-test-simulated-kserve-84db68679b-fq625" Jun 03 16:28:15.734135 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:28:15.734002 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4f802b10-22aa-4f74-9922-0643a27e15af-dshm\") pod \"e2e-trlp-test-simulated-kserve-84db68679b-fq625\" (UID: \"4f802b10-22aa-4f74-9922-0643a27e15af\") " pod="llm/e2e-trlp-test-simulated-kserve-84db68679b-fq625" Jun 03 16:28:15.734135 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:28:15.734029 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mfrmf\" (UniqueName: \"kubernetes.io/projected/4f802b10-22aa-4f74-9922-0643a27e15af-kube-api-access-mfrmf\") pod \"e2e-trlp-test-simulated-kserve-84db68679b-fq625\" (UID: \"4f802b10-22aa-4f74-9922-0643a27e15af\") " pod="llm/e2e-trlp-test-simulated-kserve-84db68679b-fq625" Jun 03 16:28:15.734135 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:28:15.734061 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4f802b10-22aa-4f74-9922-0643a27e15af-home\") pod \"e2e-trlp-test-simulated-kserve-84db68679b-fq625\" (UID: \"4f802b10-22aa-4f74-9922-0643a27e15af\") " pod="llm/e2e-trlp-test-simulated-kserve-84db68679b-fq625" Jun 03 16:28:15.734135 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:28:15.734122 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4f802b10-22aa-4f74-9922-0643a27e15af-model-cache\") pod \"e2e-trlp-test-simulated-kserve-84db68679b-fq625\" (UID: \"4f802b10-22aa-4f74-9922-0643a27e15af\") " pod="llm/e2e-trlp-test-simulated-kserve-84db68679b-fq625" Jun 03 16:28:15.834805 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:28:15.834728 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4f802b10-22aa-4f74-9922-0643a27e15af-home\") pod \"e2e-trlp-test-simulated-kserve-84db68679b-fq625\" (UID: \"4f802b10-22aa-4f74-9922-0643a27e15af\") " pod="llm/e2e-trlp-test-simulated-kserve-84db68679b-fq625" Jun 03 16:28:15.834805 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:28:15.834779 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4f802b10-22aa-4f74-9922-0643a27e15af-model-cache\") pod \"e2e-trlp-test-simulated-kserve-84db68679b-fq625\" (UID: \"4f802b10-22aa-4f74-9922-0643a27e15af\") " pod="llm/e2e-trlp-test-simulated-kserve-84db68679b-fq625" Jun 03 16:28:15.834993 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:28:15.834814 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/4f802b10-22aa-4f74-9922-0643a27e15af-kserve-provision-location\") pod \"e2e-trlp-test-simulated-kserve-84db68679b-fq625\" (UID: \"4f802b10-22aa-4f74-9922-0643a27e15af\") " pod="llm/e2e-trlp-test-simulated-kserve-84db68679b-fq625" Jun 03 16:28:15.834993 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:28:15.834841 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4f802b10-22aa-4f74-9922-0643a27e15af-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-84db68679b-fq625\" (UID: \"4f802b10-22aa-4f74-9922-0643a27e15af\") " pod="llm/e2e-trlp-test-simulated-kserve-84db68679b-fq625" Jun 03 16:28:15.834993 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:28:15.834862 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4f802b10-22aa-4f74-9922-0643a27e15af-dshm\") pod \"e2e-trlp-test-simulated-kserve-84db68679b-fq625\" (UID: \"4f802b10-22aa-4f74-9922-0643a27e15af\") " pod="llm/e2e-trlp-test-simulated-kserve-84db68679b-fq625" Jun 03 16:28:15.834993 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:28:15.834886 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-mfrmf\" (UniqueName: \"kubernetes.io/projected/4f802b10-22aa-4f74-9922-0643a27e15af-kube-api-access-mfrmf\") pod \"e2e-trlp-test-simulated-kserve-84db68679b-fq625\" (UID: \"4f802b10-22aa-4f74-9922-0643a27e15af\") " pod="llm/e2e-trlp-test-simulated-kserve-84db68679b-fq625" Jun 03 16:28:15.835276 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:28:15.835168 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4f802b10-22aa-4f74-9922-0643a27e15af-home\") pod \"e2e-trlp-test-simulated-kserve-84db68679b-fq625\" (UID: \"4f802b10-22aa-4f74-9922-0643a27e15af\") " pod="llm/e2e-trlp-test-simulated-kserve-84db68679b-fq625" Jun 03 16:28:15.835276 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:28:15.835197 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4f802b10-22aa-4f74-9922-0643a27e15af-model-cache\") pod \"e2e-trlp-test-simulated-kserve-84db68679b-fq625\" (UID: \"4f802b10-22aa-4f74-9922-0643a27e15af\") " pod="llm/e2e-trlp-test-simulated-kserve-84db68679b-fq625" Jun 03 16:28:15.835378 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:28:15.835320 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/4f802b10-22aa-4f74-9922-0643a27e15af-kserve-provision-location\") pod \"e2e-trlp-test-simulated-kserve-84db68679b-fq625\" (UID: \"4f802b10-22aa-4f74-9922-0643a27e15af\") " pod="llm/e2e-trlp-test-simulated-kserve-84db68679b-fq625" Jun 03 16:28:15.836906 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:28:15.836880 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4f802b10-22aa-4f74-9922-0643a27e15af-dshm\") pod \"e2e-trlp-test-simulated-kserve-84db68679b-fq625\" (UID: \"4f802b10-22aa-4f74-9922-0643a27e15af\") " pod="llm/e2e-trlp-test-simulated-kserve-84db68679b-fq625" Jun 03 16:28:15.837193 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:28:15.837178 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4f802b10-22aa-4f74-9922-0643a27e15af-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-84db68679b-fq625\" (UID: \"4f802b10-22aa-4f74-9922-0643a27e15af\") " pod="llm/e2e-trlp-test-simulated-kserve-84db68679b-fq625" Jun 03 16:28:15.842645 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:28:15.842626 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mfrmf\" (UniqueName: \"kubernetes.io/projected/4f802b10-22aa-4f74-9922-0643a27e15af-kube-api-access-mfrmf\") pod \"e2e-trlp-test-simulated-kserve-84db68679b-fq625\" (UID: \"4f802b10-22aa-4f74-9922-0643a27e15af\") " pod="llm/e2e-trlp-test-simulated-kserve-84db68679b-fq625" Jun 03 16:28:15.889349 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:28:15.889328 2568 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-84db68679b-fq625" Jun 03 16:28:16.020244 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:28:16.020213 2568 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-84db68679b-fq625"] Jun 03 16:28:16.023483 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:28:16.023453 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4f802b10_22aa_4f74_9922_0643a27e15af.slice/crio-759a4ad1961eba591b857e1ff76c987fc04ef69e5e78e79a8fc78a6e157e810f WatchSource:0}: Error finding container 759a4ad1961eba591b857e1ff76c987fc04ef69e5e78e79a8fc78a6e157e810f: Status 404 returned error can't find the container with id 759a4ad1961eba591b857e1ff76c987fc04ef69e5e78e79a8fc78a6e157e810f Jun 03 16:28:16.036520 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:28:16.036488 2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-84db68679b-fq625" event={"ID":"4f802b10-22aa-4f74-9922-0643a27e15af","Type":"ContainerStarted","Data":"759a4ad1961eba591b857e1ff76c987fc04ef69e5e78e79a8fc78a6e157e810f"} Jun 03 16:28:16.575335 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:28:16.575301 2568 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccmrgx8"] Jun 03 16:28:16.577818 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:28:16.577800 2568 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccmrgx8" Jun 03 16:28:16.580084 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:28:16.580059 2568 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\"" Jun 03 16:28:16.589765 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:28:16.589743 2568 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccmrgx8"] Jun 03 16:28:16.742655 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:28:16.742622 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/373d4597-ee86-4395-b945-ee8ecc277c42-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccmrgx8\" (UID: \"373d4597-ee86-4395-b945-ee8ecc277c42\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccmrgx8" Jun 03 16:28:16.742812 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:28:16.742670 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/373d4597-ee86-4395-b945-ee8ecc277c42-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccmrgx8\" (UID: \"373d4597-ee86-4395-b945-ee8ecc277c42\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccmrgx8" Jun 03 16:28:16.742812 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:28:16.742708 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/373d4597-ee86-4395-b945-ee8ecc277c42-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccmrgx8\" (UID: \"373d4597-ee86-4395-b945-ee8ecc277c42\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccmrgx8" Jun 03 16:28:16.742812 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:28:16.742762 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qjv5h\" (UniqueName: \"kubernetes.io/projected/373d4597-ee86-4395-b945-ee8ecc277c42-kube-api-access-qjv5h\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccmrgx8\" (UID: \"373d4597-ee86-4395-b945-ee8ecc277c42\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccmrgx8" Jun 03 16:28:16.742812 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:28:16.742790 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/373d4597-ee86-4395-b945-ee8ecc277c42-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccmrgx8\" (UID: \"373d4597-ee86-4395-b945-ee8ecc277c42\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccmrgx8" Jun 03 16:28:16.742946 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:28:16.742838 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/373d4597-ee86-4395-b945-ee8ecc277c42-kserve-provision-location\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccmrgx8\" (UID: \"373d4597-ee86-4395-b945-ee8ecc277c42\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccmrgx8" Jun 03 16:28:16.843384 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:28:16.843295 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-qjv5h\" (UniqueName: \"kubernetes.io/projected/373d4597-ee86-4395-b945-ee8ecc277c42-kube-api-access-qjv5h\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccmrgx8\" (UID: \"373d4597-ee86-4395-b945-ee8ecc277c42\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccmrgx8" Jun 03 16:28:16.843384 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:28:16.843351 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/373d4597-ee86-4395-b945-ee8ecc277c42-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccmrgx8\" (UID: \"373d4597-ee86-4395-b945-ee8ecc277c42\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccmrgx8" Jun 03 16:28:16.843672 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:28:16.843424 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/373d4597-ee86-4395-b945-ee8ecc277c42-kserve-provision-location\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccmrgx8\" (UID: \"373d4597-ee86-4395-b945-ee8ecc277c42\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccmrgx8" Jun 03 16:28:16.843672 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:28:16.843476 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/373d4597-ee86-4395-b945-ee8ecc277c42-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccmrgx8\" (UID: \"373d4597-ee86-4395-b945-ee8ecc277c42\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccmrgx8" Jun 03 16:28:16.843672 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:28:16.843528 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/373d4597-ee86-4395-b945-ee8ecc277c42-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccmrgx8\" (UID: \"373d4597-ee86-4395-b945-ee8ecc277c42\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccmrgx8" Jun 03 16:28:16.843672 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:28:16.843573 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/373d4597-ee86-4395-b945-ee8ecc277c42-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccmrgx8\" (UID: \"373d4597-ee86-4395-b945-ee8ecc277c42\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccmrgx8" Jun 03 16:28:16.843905 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:28:16.843771 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/373d4597-ee86-4395-b945-ee8ecc277c42-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccmrgx8\" (UID: \"373d4597-ee86-4395-b945-ee8ecc277c42\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccmrgx8" Jun 03 16:28:16.844005 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:28:16.843979 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/373d4597-ee86-4395-b945-ee8ecc277c42-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccmrgx8\" (UID: \"373d4597-ee86-4395-b945-ee8ecc277c42\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccmrgx8" Jun 03 16:28:16.844290 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:28:16.844262 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/373d4597-ee86-4395-b945-ee8ecc277c42-kserve-provision-location\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccmrgx8\" (UID: \"373d4597-ee86-4395-b945-ee8ecc277c42\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccmrgx8" Jun 03 16:28:16.845941 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:28:16.845924 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/373d4597-ee86-4395-b945-ee8ecc277c42-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccmrgx8\" (UID: \"373d4597-ee86-4395-b945-ee8ecc277c42\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccmrgx8" Jun 03 16:28:16.846411 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:28:16.846395 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/373d4597-ee86-4395-b945-ee8ecc277c42-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccmrgx8\" (UID: \"373d4597-ee86-4395-b945-ee8ecc277c42\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccmrgx8" Jun 03 16:28:16.850796 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:28:16.850778 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qjv5h\" (UniqueName: \"kubernetes.io/projected/373d4597-ee86-4395-b945-ee8ecc277c42-kube-api-access-qjv5h\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccmrgx8\" (UID: \"373d4597-ee86-4395-b945-ee8ecc277c42\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccmrgx8" Jun 03 16:28:16.888721 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:28:16.888698 2568 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccmrgx8" Jun 03 16:28:17.025802 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:28:17.025768 2568 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccmrgx8"] Jun 03 16:28:17.027463 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:28:17.027434 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod373d4597_ee86_4395_b945_ee8ecc277c42.slice/crio-36a3948a88c5d0f9455e3c8573bbc41763b7c82c45dee190dc8b6f222d0412e9 WatchSource:0}: Error finding container 36a3948a88c5d0f9455e3c8573bbc41763b7c82c45dee190dc8b6f222d0412e9: Status 404 returned error can't find the container with id 36a3948a88c5d0f9455e3c8573bbc41763b7c82c45dee190dc8b6f222d0412e9 Jun 03 16:28:17.042291 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:28:17.042257 2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-84db68679b-fq625" event={"ID":"4f802b10-22aa-4f74-9922-0643a27e15af","Type":"ContainerStarted","Data":"6684be4dc266be5d385624dc7a20fe4228913bec46ed845705ce7cb1699203b7"} Jun 03 16:28:17.043432 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:28:17.043408 2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccmrgx8" event={"ID":"373d4597-ee86-4395-b945-ee8ecc277c42","Type":"ContainerStarted","Data":"36a3948a88c5d0f9455e3c8573bbc41763b7c82c45dee190dc8b6f222d0412e9"} Jun 03 16:28:18.048944 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:28:18.048903 2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccmrgx8" event={"ID":"373d4597-ee86-4395-b945-ee8ecc277c42","Type":"ContainerStarted","Data":"9b7b448afac76387d4a3cc5e1115b1e2bb23a5bf80f2281818fdbe6122ef35c0"} Jun 03 16:28:20.060754 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:28:20.060713 2568 generic.go:358] "Generic (PLEG): container finished" podID="ec14e607-775c-4c96-bb3f-e5b4b8f04b6d" containerID="75ce1a81552fca98b9620cadc078783d85840f7fb4bcfea19302180bb0d7b718" exitCode=0 Jun 03 16:28:20.061136 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:28:20.060797 2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-nmf8k" event={"ID":"ec14e607-775c-4c96-bb3f-e5b4b8f04b6d","Type":"ContainerDied","Data":"75ce1a81552fca98b9620cadc078783d85840f7fb4bcfea19302180bb0d7b718"} Jun 03 16:28:22.071038 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:28:22.070995 2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-nmf8k" event={"ID":"ec14e607-775c-4c96-bb3f-e5b4b8f04b6d","Type":"ContainerStarted","Data":"28107f088027068d8772b5f55669c4486ac197ef3690d78caad015b4fb696cac"} Jun 03 16:28:22.071423 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:28:22.071218 2568 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-nmf8k" Jun 03 16:28:22.090446 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:28:22.090401 2568 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-nmf8k" podStartSLOduration=2.085311506 podStartE2EDuration="24.090387838s" podCreationTimestamp="2026-06-03 16:27:58 +0000 UTC" firstStartedPulling="2026-06-03 16:27:59.429713081 +0000 UTC m=+650.518139649" lastFinishedPulling="2026-06-03 16:28:21.43478941 +0000 UTC m=+672.523215981" observedRunningTime="2026-06-03 16:28:22.087785396 +0000 UTC m=+673.176211985" watchObservedRunningTime="2026-06-03 16:28:22.090387838 +0000 UTC m=+673.178814427" Jun 03 16:28:23.672979 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:28:23.672949 2568 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-7f849f6b56-l6584"] Jun 03 16:28:23.681892 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:28:23.681867 2568 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-l6584" Jun 03 16:28:23.684129 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:28:23.684102 2568 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\"" Jun 03 16:28:23.686689 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:28:23.686667 2568 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-7f849f6b56-l6584"] Jun 03 16:28:23.809916 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:28:23.809878 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d91a37d9-3ee4-4fc6-bbe4-39eadea22ee5-home\") pod \"e2e-distinct-2-simulated-kserve-7f849f6b56-l6584\" (UID: \"d91a37d9-3ee4-4fc6-bbe4-39eadea22ee5\") " pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-l6584" Jun 03 16:28:23.809916 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:28:23.809918 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/d91a37d9-3ee4-4fc6-bbe4-39eadea22ee5-kserve-provision-location\") pod \"e2e-distinct-2-simulated-kserve-7f849f6b56-l6584\" (UID: \"d91a37d9-3ee4-4fc6-bbe4-39eadea22ee5\") " pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-l6584" Jun 03 16:28:23.810101 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:28:23.809941 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-59s4d\" (UniqueName: \"kubernetes.io/projected/d91a37d9-3ee4-4fc6-bbe4-39eadea22ee5-kube-api-access-59s4d\") pod \"e2e-distinct-2-simulated-kserve-7f849f6b56-l6584\" (UID: \"d91a37d9-3ee4-4fc6-bbe4-39eadea22ee5\") " pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-l6584" Jun 03 16:28:23.810101 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:28:23.810008 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d91a37d9-3ee4-4fc6-bbe4-39eadea22ee5-dshm\") pod \"e2e-distinct-2-simulated-kserve-7f849f6b56-l6584\" (UID: \"d91a37d9-3ee4-4fc6-bbe4-39eadea22ee5\") " pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-l6584" Jun 03 16:28:23.810101 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:28:23.810053 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d91a37d9-3ee4-4fc6-bbe4-39eadea22ee5-model-cache\") pod \"e2e-distinct-2-simulated-kserve-7f849f6b56-l6584\" (UID: \"d91a37d9-3ee4-4fc6-bbe4-39eadea22ee5\") " pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-l6584" Jun 03 16:28:23.810203 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:28:23.810110 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d91a37d9-3ee4-4fc6-bbe4-39eadea22ee5-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-7f849f6b56-l6584\" (UID: \"d91a37d9-3ee4-4fc6-bbe4-39eadea22ee5\") " pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-l6584" Jun 03 16:28:23.911260 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:28:23.911230 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/d91a37d9-3ee4-4fc6-bbe4-39eadea22ee5-kserve-provision-location\") pod \"e2e-distinct-2-simulated-kserve-7f849f6b56-l6584\" (UID: \"d91a37d9-3ee4-4fc6-bbe4-39eadea22ee5\") " pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-l6584" Jun 03 16:28:23.911435 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:28:23.911265 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-59s4d\" (UniqueName: \"kubernetes.io/projected/d91a37d9-3ee4-4fc6-bbe4-39eadea22ee5-kube-api-access-59s4d\") pod \"e2e-distinct-2-simulated-kserve-7f849f6b56-l6584\" (UID: \"d91a37d9-3ee4-4fc6-bbe4-39eadea22ee5\") " pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-l6584" Jun 03 16:28:23.911435 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:28:23.911294 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d91a37d9-3ee4-4fc6-bbe4-39eadea22ee5-dshm\") pod \"e2e-distinct-2-simulated-kserve-7f849f6b56-l6584\" (UID: \"d91a37d9-3ee4-4fc6-bbe4-39eadea22ee5\") " pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-l6584" Jun 03 16:28:23.911435 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:28:23.911341 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d91a37d9-3ee4-4fc6-bbe4-39eadea22ee5-model-cache\") pod \"e2e-distinct-2-simulated-kserve-7f849f6b56-l6584\" (UID: \"d91a37d9-3ee4-4fc6-bbe4-39eadea22ee5\") " pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-l6584" Jun 03 16:28:23.911435 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:28:23.911399 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d91a37d9-3ee4-4fc6-bbe4-39eadea22ee5-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-7f849f6b56-l6584\" (UID: \"d91a37d9-3ee4-4fc6-bbe4-39eadea22ee5\") " pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-l6584" Jun 03 16:28:23.911699 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:28:23.911447 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d91a37d9-3ee4-4fc6-bbe4-39eadea22ee5-home\") pod \"e2e-distinct-2-simulated-kserve-7f849f6b56-l6584\" (UID: \"d91a37d9-3ee4-4fc6-bbe4-39eadea22ee5\") " pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-l6584" Jun 03 16:28:23.911759 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:28:23.911736 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d91a37d9-3ee4-4fc6-bbe4-39eadea22ee5-model-cache\") pod \"e2e-distinct-2-simulated-kserve-7f849f6b56-l6584\" (UID: \"d91a37d9-3ee4-4fc6-bbe4-39eadea22ee5\") " pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-l6584" Jun 03 16:28:23.911799 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:28:23.911735 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/d91a37d9-3ee4-4fc6-bbe4-39eadea22ee5-kserve-provision-location\") pod \"e2e-distinct-2-simulated-kserve-7f849f6b56-l6584\" (UID: \"d91a37d9-3ee4-4fc6-bbe4-39eadea22ee5\") " pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-l6584" Jun 03 16:28:23.911897 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:28:23.911871 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d91a37d9-3ee4-4fc6-bbe4-39eadea22ee5-home\") pod \"e2e-distinct-2-simulated-kserve-7f849f6b56-l6584\" (UID: \"d91a37d9-3ee4-4fc6-bbe4-39eadea22ee5\") " pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-l6584" Jun 03 16:28:23.913628 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:28:23.913608 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d91a37d9-3ee4-4fc6-bbe4-39eadea22ee5-dshm\") pod \"e2e-distinct-2-simulated-kserve-7f849f6b56-l6584\" (UID: \"d91a37d9-3ee4-4fc6-bbe4-39eadea22ee5\") " pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-l6584" Jun 03 16:28:23.913920 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:28:23.913900 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d91a37d9-3ee4-4fc6-bbe4-39eadea22ee5-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-7f849f6b56-l6584\" (UID: \"d91a37d9-3ee4-4fc6-bbe4-39eadea22ee5\") " pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-l6584" Jun 03 16:28:23.919685 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:28:23.919666 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-59s4d\" (UniqueName: \"kubernetes.io/projected/d91a37d9-3ee4-4fc6-bbe4-39eadea22ee5-kube-api-access-59s4d\") pod \"e2e-distinct-2-simulated-kserve-7f849f6b56-l6584\" (UID: \"d91a37d9-3ee4-4fc6-bbe4-39eadea22ee5\") " pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-l6584" Jun 03 16:28:23.994887 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:28:23.994811 2568 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-l6584" Jun 03 16:28:24.128088 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:28:24.128061 2568 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-7f849f6b56-l6584"] Jun 03 16:28:24.130152 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:28:24.130124 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd91a37d9_3ee4_4fc6_bbe4_39eadea22ee5.slice/crio-98569e78245df3d63b142737e5719b8f8b4cb03691db3f944544a6c9ac683b39 WatchSource:0}: Error finding container 98569e78245df3d63b142737e5719b8f8b4cb03691db3f944544a6c9ac683b39: Status 404 returned error can't find the container with id 98569e78245df3d63b142737e5719b8f8b4cb03691db3f944544a6c9ac683b39 Jun 03 16:28:25.085143 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:28:25.085103 2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-l6584" event={"ID":"d91a37d9-3ee4-4fc6-bbe4-39eadea22ee5","Type":"ContainerStarted","Data":"5ec404a0e2673abfaa1383c298a936b9ae24a4feb70bc36e8f595352c5c7f85a"} Jun 03 16:28:25.085143 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:28:25.085148 2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-l6584" event={"ID":"d91a37d9-3ee4-4fc6-bbe4-39eadea22ee5","Type":"ContainerStarted","Data":"98569e78245df3d63b142737e5719b8f8b4cb03691db3f944544a6c9ac683b39"} Jun 03 16:28:28.099825 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:28:28.099790 2568 generic.go:358] "Generic (PLEG): container finished" podID="4f802b10-22aa-4f74-9922-0643a27e15af" containerID="6684be4dc266be5d385624dc7a20fe4228913bec46ed845705ce7cb1699203b7" exitCode=0 Jun 03 16:28:28.100295 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:28:28.099858 2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-84db68679b-fq625" event={"ID":"4f802b10-22aa-4f74-9922-0643a27e15af","Type":"ContainerDied","Data":"6684be4dc266be5d385624dc7a20fe4228913bec46ed845705ce7cb1699203b7"} Jun 03 16:28:29.104918 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:28:29.104890 2568 generic.go:358] "Generic (PLEG): container finished" podID="373d4597-ee86-4395-b945-ee8ecc277c42" containerID="9b7b448afac76387d4a3cc5e1115b1e2bb23a5bf80f2281818fdbe6122ef35c0" exitCode=0 Jun 03 16:28:29.105315 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:28:29.104968 2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccmrgx8" event={"ID":"373d4597-ee86-4395-b945-ee8ecc277c42","Type":"ContainerDied","Data":"9b7b448afac76387d4a3cc5e1115b1e2bb23a5bf80f2281818fdbe6122ef35c0"} Jun 03 16:28:29.106786 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:28:29.106762 2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-84db68679b-fq625" event={"ID":"4f802b10-22aa-4f74-9922-0643a27e15af","Type":"ContainerStarted","Data":"766e60dec37a7c86af8f8ce00aa07cfe15c0eb79b7854af3c73a99f77bc3f961"} Jun 03 16:28:29.106998 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:28:29.106964 2568 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-84db68679b-fq625" Jun 03 16:28:29.144258 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:28:29.144205 2568 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-84db68679b-fq625" podStartSLOduration=13.828190993 podStartE2EDuration="14.144190882s" podCreationTimestamp="2026-06-03 16:28:15 +0000 UTC" firstStartedPulling="2026-06-03 16:28:28.100617929 +0000 UTC m=+679.189044497" lastFinishedPulling="2026-06-03 16:28:28.4166178 +0000 UTC m=+679.505044386" observedRunningTime="2026-06-03 16:28:29.143693703 +0000 UTC m=+680.232120294" watchObservedRunningTime="2026-06-03 16:28:29.144190882 +0000 UTC m=+680.232617472" Jun 03 16:28:30.111899 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:28:30.111857 2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccmrgx8" event={"ID":"373d4597-ee86-4395-b945-ee8ecc277c42","Type":"ContainerStarted","Data":"3ffc645668968aabb0f0e0fe017aacf6f24603a6c740ce87b6e2a1c62d27bca5"} Jun 03 16:28:30.132043 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:28:30.131999 2568 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccmrgx8" podStartSLOduration=13.728720766 podStartE2EDuration="14.131983648s" podCreationTimestamp="2026-06-03 16:28:16 +0000 UTC" firstStartedPulling="2026-06-03 16:28:29.105670866 +0000 UTC m=+680.194097438" lastFinishedPulling="2026-06-03 16:28:29.508933735 +0000 UTC m=+680.597360320" observedRunningTime="2026-06-03 16:28:30.130400661 +0000 UTC m=+681.218827275" watchObservedRunningTime="2026-06-03 16:28:30.131983648 +0000 UTC m=+681.220410238" Jun 03 16:28:33.088456 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:28:33.088427 2568 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-8f8dc67b7-nmf8k" Jun 03 16:28:36.137417 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:28:36.137383 2568 generic.go:358] "Generic (PLEG): container finished" podID="d91a37d9-3ee4-4fc6-bbe4-39eadea22ee5" containerID="5ec404a0e2673abfaa1383c298a936b9ae24a4feb70bc36e8f595352c5c7f85a" exitCode=0 Jun 03 16:28:36.137822 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:28:36.137455 2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-l6584" event={"ID":"d91a37d9-3ee4-4fc6-bbe4-39eadea22ee5","Type":"ContainerDied","Data":"5ec404a0e2673abfaa1383c298a936b9ae24a4feb70bc36e8f595352c5c7f85a"} Jun 03 16:28:36.986797 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:28:36.986718 2568 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-hzn4t"] Jun 03 16:28:37.026663 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:28:37.026633 2568 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-hzn4t"] Jun 03 16:28:37.026808 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:28:37.026743 2568 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-hzn4t" Jun 03 16:28:37.033948 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:28:37.033925 2568 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\"" Jun 03 16:28:37.125206 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:28:37.125178 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f45c240d-6f62-44f7-9e26-f57ca7ad6124-model-cache\") pod \"e2e-distinct-simulated-kserve-7bb4cdb4d7-hzn4t\" (UID: \"f45c240d-6f62-44f7-9e26-f57ca7ad6124\") " pod="llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-hzn4t" Jun 03 16:28:37.125328 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:28:37.125211 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/f45c240d-6f62-44f7-9e26-f57ca7ad6124-kserve-provision-location\") pod \"e2e-distinct-simulated-kserve-7bb4cdb4d7-hzn4t\" (UID: \"f45c240d-6f62-44f7-9e26-f57ca7ad6124\") " pod="llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-hzn4t" Jun 03 16:28:37.125328 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:28:37.125243 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f45c240d-6f62-44f7-9e26-f57ca7ad6124-tls-certs\") pod \"e2e-distinct-simulated-kserve-7bb4cdb4d7-hzn4t\" (UID: \"f45c240d-6f62-44f7-9e26-f57ca7ad6124\") " pod="llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-hzn4t" Jun 03 16:28:37.125328 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:28:37.125285 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zl7h5\" (UniqueName: \"kubernetes.io/projected/f45c240d-6f62-44f7-9e26-f57ca7ad6124-kube-api-access-zl7h5\") pod \"e2e-distinct-simulated-kserve-7bb4cdb4d7-hzn4t\" (UID: \"f45c240d-6f62-44f7-9e26-f57ca7ad6124\") " pod="llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-hzn4t" Jun 03 16:28:37.125433 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:28:37.125328 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f45c240d-6f62-44f7-9e26-f57ca7ad6124-dshm\") pod \"e2e-distinct-simulated-kserve-7bb4cdb4d7-hzn4t\" (UID: \"f45c240d-6f62-44f7-9e26-f57ca7ad6124\") " pod="llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-hzn4t" Jun 03 16:28:37.125433 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:28:37.125406 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f45c240d-6f62-44f7-9e26-f57ca7ad6124-home\") pod \"e2e-distinct-simulated-kserve-7bb4cdb4d7-hzn4t\" (UID: \"f45c240d-6f62-44f7-9e26-f57ca7ad6124\") " pod="llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-hzn4t" Jun 03 16:28:37.143512 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:28:37.143481 2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-l6584" event={"ID":"d91a37d9-3ee4-4fc6-bbe4-39eadea22ee5","Type":"ContainerStarted","Data":"6dfcf5ed666e3673cde30508e05572274c8de6d816358fa840fd27d12fe215e9"} Jun 03 16:28:37.143840 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:28:37.143698 2568 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-l6584" Jun 03 16:28:37.164915 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:28:37.164813 2568 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-l6584" podStartSLOduration=13.692451132 podStartE2EDuration="14.164793835s" podCreationTimestamp="2026-06-03 16:28:23 +0000 UTC" firstStartedPulling="2026-06-03 16:28:36.138078034 +0000 UTC m=+687.226504601" lastFinishedPulling="2026-06-03 16:28:36.610420733 +0000 UTC m=+687.698847304" observedRunningTime="2026-06-03 16:28:37.163995343 +0000 UTC m=+688.252421932" watchObservedRunningTime="2026-06-03 16:28:37.164793835 +0000 UTC m=+688.253220425" Jun 03 16:28:37.225979 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:28:37.225949 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f45c240d-6f62-44f7-9e26-f57ca7ad6124-model-cache\") pod \"e2e-distinct-simulated-kserve-7bb4cdb4d7-hzn4t\" (UID: \"f45c240d-6f62-44f7-9e26-f57ca7ad6124\") " pod="llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-hzn4t" Jun 03 16:28:37.226130 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:28:37.225993 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/f45c240d-6f62-44f7-9e26-f57ca7ad6124-kserve-provision-location\") pod \"e2e-distinct-simulated-kserve-7bb4cdb4d7-hzn4t\" (UID: \"f45c240d-6f62-44f7-9e26-f57ca7ad6124\") " pod="llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-hzn4t" Jun 03 16:28:37.226130 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:28:37.226119 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f45c240d-6f62-44f7-9e26-f57ca7ad6124-tls-certs\") pod \"e2e-distinct-simulated-kserve-7bb4cdb4d7-hzn4t\" (UID: \"f45c240d-6f62-44f7-9e26-f57ca7ad6124\") " pod="llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-hzn4t" Jun 03 16:28:37.226270 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:28:37.226155 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-zl7h5\" (UniqueName: \"kubernetes.io/projected/f45c240d-6f62-44f7-9e26-f57ca7ad6124-kube-api-access-zl7h5\") pod \"e2e-distinct-simulated-kserve-7bb4cdb4d7-hzn4t\" (UID: \"f45c240d-6f62-44f7-9e26-f57ca7ad6124\") " pod="llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-hzn4t" Jun 03 16:28:37.226270 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:28:37.226245 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f45c240d-6f62-44f7-9e26-f57ca7ad6124-dshm\") pod \"e2e-distinct-simulated-kserve-7bb4cdb4d7-hzn4t\" (UID: \"f45c240d-6f62-44f7-9e26-f57ca7ad6124\") " pod="llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-hzn4t" Jun 03 16:28:37.226377 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:28:37.226353 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f45c240d-6f62-44f7-9e26-f57ca7ad6124-home\") pod \"e2e-distinct-simulated-kserve-7bb4cdb4d7-hzn4t\" (UID: \"f45c240d-6f62-44f7-9e26-f57ca7ad6124\") " pod="llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-hzn4t" Jun 03 16:28:37.226429 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:28:37.226410 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f45c240d-6f62-44f7-9e26-f57ca7ad6124-model-cache\") pod \"e2e-distinct-simulated-kserve-7bb4cdb4d7-hzn4t\" (UID: \"f45c240d-6f62-44f7-9e26-f57ca7ad6124\") " pod="llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-hzn4t" Jun 03 16:28:37.226493 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:28:37.226472 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/f45c240d-6f62-44f7-9e26-f57ca7ad6124-kserve-provision-location\") pod \"e2e-distinct-simulated-kserve-7bb4cdb4d7-hzn4t\" (UID: \"f45c240d-6f62-44f7-9e26-f57ca7ad6124\") " pod="llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-hzn4t" Jun 03 16:28:37.226703 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:28:37.226687 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f45c240d-6f62-44f7-9e26-f57ca7ad6124-home\") pod \"e2e-distinct-simulated-kserve-7bb4cdb4d7-hzn4t\" (UID: \"f45c240d-6f62-44f7-9e26-f57ca7ad6124\") " pod="llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-hzn4t" Jun 03 16:28:37.228386 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:28:37.228364 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f45c240d-6f62-44f7-9e26-f57ca7ad6124-dshm\") pod \"e2e-distinct-simulated-kserve-7bb4cdb4d7-hzn4t\" (UID: \"f45c240d-6f62-44f7-9e26-f57ca7ad6124\") " pod="llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-hzn4t" Jun 03 16:28:37.228730 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:28:37.228715 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f45c240d-6f62-44f7-9e26-f57ca7ad6124-tls-certs\") pod \"e2e-distinct-simulated-kserve-7bb4cdb4d7-hzn4t\" (UID: \"f45c240d-6f62-44f7-9e26-f57ca7ad6124\") " pod="llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-hzn4t" Jun 03 16:28:37.233852 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:28:37.233833 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zl7h5\" (UniqueName: \"kubernetes.io/projected/f45c240d-6f62-44f7-9e26-f57ca7ad6124-kube-api-access-zl7h5\") pod \"e2e-distinct-simulated-kserve-7bb4cdb4d7-hzn4t\" (UID: \"f45c240d-6f62-44f7-9e26-f57ca7ad6124\") " pod="llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-hzn4t" Jun 03 16:28:37.336993 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:28:37.336913 2568 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-hzn4t" Jun 03 16:28:37.476031 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:28:37.475942 2568 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-hzn4t"] Jun 03 16:28:37.478856 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:28:37.478830 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf45c240d_6f62_44f7_9e26_f57ca7ad6124.slice/crio-f35a3a186098110fa3aa3cde3f50526f16bd48aa056385541d950eae9fb83acd WatchSource:0}: Error finding container f35a3a186098110fa3aa3cde3f50526f16bd48aa056385541d950eae9fb83acd: Status 404 returned error can't find the container with id f35a3a186098110fa3aa3cde3f50526f16bd48aa056385541d950eae9fb83acd Jun 03 16:28:38.149673 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:28:38.149638 2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-hzn4t" event={"ID":"f45c240d-6f62-44f7-9e26-f57ca7ad6124","Type":"ContainerStarted","Data":"9d0325d66bd6dce6453c045888a3e66c6d0676e8defddd904337c1fea820f672"} Jun 03 16:28:38.150132 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:28:38.149682 2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-hzn4t" event={"ID":"f45c240d-6f62-44f7-9e26-f57ca7ad6124","Type":"ContainerStarted","Data":"f35a3a186098110fa3aa3cde3f50526f16bd48aa056385541d950eae9fb83acd"} Jun 03 16:28:40.112146 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:28:40.112110 2568 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccmrgx8" Jun 03 16:28:40.124450 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:28:40.124424 2568 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-84db68679b-fq625" Jun 03 16:28:40.130104 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:28:40.130081 2568 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccmrgx8" Jun 03 16:28:42.276021 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:28:42.275981 2568 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-6b97b89985-ddvlq"] Jun 03 16:28:42.279661 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:28:42.279637 2568 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-6b97b89985-ddvlq" Jun 03 16:28:42.281795 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:28:42.281774 2568 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\"" Jun 03 16:28:42.290326 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:28:42.290304 2568 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-6b97b89985-ddvlq"] Jun 03 16:28:42.375267 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:28:42.375235 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f043a50e-6e27-4dad-92de-a2a6137d126e-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-6b97b89985-ddvlq\" (UID: \"f043a50e-6e27-4dad-92de-a2a6137d126e\") " pod="llm/premium-simulated-simulated-premium-kserve-6b97b89985-ddvlq" Jun 03 16:28:42.375423 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:28:42.375282 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f043a50e-6e27-4dad-92de-a2a6137d126e-dshm\") pod \"premium-simulated-simulated-premium-kserve-6b97b89985-ddvlq\" (UID: \"f043a50e-6e27-4dad-92de-a2a6137d126e\") " pod="llm/premium-simulated-simulated-premium-kserve-6b97b89985-ddvlq" Jun 03 16:28:42.375423 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:28:42.375302 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f043a50e-6e27-4dad-92de-a2a6137d126e-home\") pod \"premium-simulated-simulated-premium-kserve-6b97b89985-ddvlq\" (UID: \"f043a50e-6e27-4dad-92de-a2a6137d126e\") " pod="llm/premium-simulated-simulated-premium-kserve-6b97b89985-ddvlq" Jun 03 16:28:42.375500 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:28:42.375434 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f043a50e-6e27-4dad-92de-a2a6137d126e-model-cache\") pod \"premium-simulated-simulated-premium-kserve-6b97b89985-ddvlq\" (UID: \"f043a50e-6e27-4dad-92de-a2a6137d126e\") " pod="llm/premium-simulated-simulated-premium-kserve-6b97b89985-ddvlq" Jun 03 16:28:42.375500 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:28:42.375464 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/f043a50e-6e27-4dad-92de-a2a6137d126e-kserve-provision-location\") pod \"premium-simulated-simulated-premium-kserve-6b97b89985-ddvlq\" (UID: \"f043a50e-6e27-4dad-92de-a2a6137d126e\") " pod="llm/premium-simulated-simulated-premium-kserve-6b97b89985-ddvlq" Jun 03 16:28:42.375500 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:28:42.375485 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-df94h\" (UniqueName: \"kubernetes.io/projected/f043a50e-6e27-4dad-92de-a2a6137d126e-kube-api-access-df94h\") pod \"premium-simulated-simulated-premium-kserve-6b97b89985-ddvlq\" (UID: \"f043a50e-6e27-4dad-92de-a2a6137d126e\") " pod="llm/premium-simulated-simulated-premium-kserve-6b97b89985-ddvlq" Jun 03 16:28:42.476795 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:28:42.476762 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f043a50e-6e27-4dad-92de-a2a6137d126e-model-cache\") pod \"premium-simulated-simulated-premium-kserve-6b97b89985-ddvlq\" (UID: \"f043a50e-6e27-4dad-92de-a2a6137d126e\") " pod="llm/premium-simulated-simulated-premium-kserve-6b97b89985-ddvlq" Jun 03 16:28:42.476795 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:28:42.476795 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/f043a50e-6e27-4dad-92de-a2a6137d126e-kserve-provision-location\") pod \"premium-simulated-simulated-premium-kserve-6b97b89985-ddvlq\" (UID: \"f043a50e-6e27-4dad-92de-a2a6137d126e\") " pod="llm/premium-simulated-simulated-premium-kserve-6b97b89985-ddvlq" Jun 03 16:28:42.477033 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:28:42.476814 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-df94h\" (UniqueName: \"kubernetes.io/projected/f043a50e-6e27-4dad-92de-a2a6137d126e-kube-api-access-df94h\") pod \"premium-simulated-simulated-premium-kserve-6b97b89985-ddvlq\" (UID: \"f043a50e-6e27-4dad-92de-a2a6137d126e\") " pod="llm/premium-simulated-simulated-premium-kserve-6b97b89985-ddvlq" Jun 03 16:28:42.477033 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:28:42.476845 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f043a50e-6e27-4dad-92de-a2a6137d126e-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-6b97b89985-ddvlq\" (UID: \"f043a50e-6e27-4dad-92de-a2a6137d126e\") " pod="llm/premium-simulated-simulated-premium-kserve-6b97b89985-ddvlq" Jun 03 16:28:42.477033 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:28:42.476893 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f043a50e-6e27-4dad-92de-a2a6137d126e-dshm\") pod \"premium-simulated-simulated-premium-kserve-6b97b89985-ddvlq\" (UID: \"f043a50e-6e27-4dad-92de-a2a6137d126e\") " pod="llm/premium-simulated-simulated-premium-kserve-6b97b89985-ddvlq" Jun 03 16:28:42.477033 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:28:42.476923 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f043a50e-6e27-4dad-92de-a2a6137d126e-home\") pod \"premium-simulated-simulated-premium-kserve-6b97b89985-ddvlq\" (UID: \"f043a50e-6e27-4dad-92de-a2a6137d126e\") " pod="llm/premium-simulated-simulated-premium-kserve-6b97b89985-ddvlq" Jun 03 16:28:42.477243 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:28:42.477156 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f043a50e-6e27-4dad-92de-a2a6137d126e-model-cache\") pod \"premium-simulated-simulated-premium-kserve-6b97b89985-ddvlq\" (UID: \"f043a50e-6e27-4dad-92de-a2a6137d126e\") " pod="llm/premium-simulated-simulated-premium-kserve-6b97b89985-ddvlq" Jun 03 16:28:42.477243 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:28:42.477215 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/f043a50e-6e27-4dad-92de-a2a6137d126e-kserve-provision-location\") pod \"premium-simulated-simulated-premium-kserve-6b97b89985-ddvlq\" (UID: \"f043a50e-6e27-4dad-92de-a2a6137d126e\") " pod="llm/premium-simulated-simulated-premium-kserve-6b97b89985-ddvlq" Jun 03 16:28:42.477358 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:28:42.477266 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f043a50e-6e27-4dad-92de-a2a6137d126e-home\") pod \"premium-simulated-simulated-premium-kserve-6b97b89985-ddvlq\" (UID: \"f043a50e-6e27-4dad-92de-a2a6137d126e\") " pod="llm/premium-simulated-simulated-premium-kserve-6b97b89985-ddvlq" Jun 03 16:28:42.479179 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:28:42.479148 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f043a50e-6e27-4dad-92de-a2a6137d126e-dshm\") pod \"premium-simulated-simulated-premium-kserve-6b97b89985-ddvlq\" (UID: \"f043a50e-6e27-4dad-92de-a2a6137d126e\") " pod="llm/premium-simulated-simulated-premium-kserve-6b97b89985-ddvlq" Jun 03 16:28:42.479389 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:28:42.479372 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f043a50e-6e27-4dad-92de-a2a6137d126e-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-6b97b89985-ddvlq\" (UID: \"f043a50e-6e27-4dad-92de-a2a6137d126e\") " pod="llm/premium-simulated-simulated-premium-kserve-6b97b89985-ddvlq" Jun 03 16:28:42.483711 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:28:42.483688 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-df94h\" (UniqueName: \"kubernetes.io/projected/f043a50e-6e27-4dad-92de-a2a6137d126e-kube-api-access-df94h\") pod \"premium-simulated-simulated-premium-kserve-6b97b89985-ddvlq\" (UID: \"f043a50e-6e27-4dad-92de-a2a6137d126e\") " pod="llm/premium-simulated-simulated-premium-kserve-6b97b89985-ddvlq" Jun 03 16:28:42.590408 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:28:42.590336 2568 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-6b97b89985-ddvlq" Jun 03 16:28:42.717907 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:28:42.717880 2568 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-6b97b89985-ddvlq"] Jun 03 16:28:42.719281 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:28:42.719247 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf043a50e_6e27_4dad_92de_a2a6137d126e.slice/crio-1a7db06a406a1f2fdbbf9f8b6da44751d4749cc9c2cad3d6b8e581dbaf1603c4 WatchSource:0}: Error finding container 1a7db06a406a1f2fdbbf9f8b6da44751d4749cc9c2cad3d6b8e581dbaf1603c4: Status 404 returned error can't find the container with id 1a7db06a406a1f2fdbbf9f8b6da44751d4749cc9c2cad3d6b8e581dbaf1603c4 Jun 03 16:28:43.170989 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:28:43.170939 2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-6b97b89985-ddvlq" event={"ID":"f043a50e-6e27-4dad-92de-a2a6137d126e","Type":"ContainerStarted","Data":"7fd9de3e349a824693a5673711dd9c11abc3d122d2ca0ce5e44fe7fff42ffa2b"} Jun 03 16:28:43.170989 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:28:43.170982 2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-6b97b89985-ddvlq" event={"ID":"f043a50e-6e27-4dad-92de-a2a6137d126e","Type":"ContainerStarted","Data":"1a7db06a406a1f2fdbbf9f8b6da44751d4749cc9c2cad3d6b8e581dbaf1603c4"} Jun 03 16:28:48.163413 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:28:48.163325 2568 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-l6584" Jun 03 16:28:48.192032 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:28:48.192001 2568 generic.go:358] "Generic (PLEG): container finished" podID="f45c240d-6f62-44f7-9e26-f57ca7ad6124" containerID="9d0325d66bd6dce6453c045888a3e66c6d0676e8defddd904337c1fea820f672" exitCode=0 Jun 03 16:28:48.192203 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:28:48.192040 2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-hzn4t" event={"ID":"f45c240d-6f62-44f7-9e26-f57ca7ad6124","Type":"ContainerDied","Data":"9d0325d66bd6dce6453c045888a3e66c6d0676e8defddd904337c1fea820f672"} Jun 03 16:28:49.198548 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:28:49.198500 2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-hzn4t" event={"ID":"f45c240d-6f62-44f7-9e26-f57ca7ad6124","Type":"ContainerStarted","Data":"6b7c4e6ca4ef3c6a3a255d002492d5b158e1e8d3d3f0247e69fcaf02ea3dbdbe"} Jun 03 16:28:49.198948 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:28:49.198763 2568 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-hzn4t" Jun 03 16:28:49.216963 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:28:49.216900 2568 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-hzn4t" podStartSLOduration=12.781121085 podStartE2EDuration="13.216881228s" podCreationTimestamp="2026-06-03 16:28:36 +0000 UTC" firstStartedPulling="2026-06-03 16:28:48.192828643 +0000 UTC m=+699.281255211" lastFinishedPulling="2026-06-03 16:28:48.628588784 +0000 UTC m=+699.717015354" observedRunningTime="2026-06-03 16:28:49.215702926 +0000 UTC m=+700.304129516" watchObservedRunningTime="2026-06-03 16:28:49.216881228 +0000 UTC m=+700.305307819" Jun 03 16:29:00.216993 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:29:00.216956 2568 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-7bb4cdb4d7-hzn4t" Jun 03 16:33:42.190017 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:33:42.189943 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-6b97b89985-ddvlq_f043a50e-6e27-4dad-92de-a2a6137d126e/storage-initializer/0.log" Jun 03 16:33:48.223234 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:33:48.223197 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6c656db94d-j7rhw_5b560183-6f5f-4f17-823e-7131b664fde3/manager/0.log" Jun 03 16:33:49.388780 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:33:49.388750 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759tg6xr_094f549f-494e-486a-aca5-303c7492dacd/util/0.log" Jun 03 16:33:49.395499 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:33:49.395473 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759tg6xr_094f549f-494e-486a-aca5-303c7492dacd/pull/0.log" Jun 03 16:33:49.401872 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:33:49.401839 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759tg6xr_094f549f-494e-486a-aca5-303c7492dacd/extract/0.log" Jun 03 16:33:49.511003 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:33:49.510972 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0fqfpf_f73bd334-e4cc-4916-8ad4-07b5b88e7b8e/pull/0.log" Jun 03 16:33:49.517595 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:33:49.517562 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0fqfpf_f73bd334-e4cc-4916-8ad4-07b5b88e7b8e/extract/0.log" Jun 03 16:33:49.523570 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:33:49.523530 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0fqfpf_f73bd334-e4cc-4916-8ad4-07b5b88e7b8e/util/0.log" Jun 03 16:33:49.635733 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:33:49.635699 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73tzwb2_e12d559c-bbf9-4f80-b155-8a9fa65fa0c1/extract/0.log" Jun 03 16:33:49.642678 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:33:49.642601 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73tzwb2_e12d559c-bbf9-4f80-b155-8a9fa65fa0c1/util/0.log" Jun 03 16:33:49.650075 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:33:49.650041 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73tzwb2_e12d559c-bbf9-4f80-b155-8a9fa65fa0c1/pull/0.log" Jun 03 16:33:49.766852 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:33:49.766820 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1knbcb_a5bdab96-bea9-4fbc-8bc1-e987d74623c8/util/0.log" Jun 03 16:33:49.774657 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:33:49.774630 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1knbcb_a5bdab96-bea9-4fbc-8bc1-e987d74623c8/pull/0.log" Jun 03 16:33:49.782293 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:33:49.782275 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1knbcb_a5bdab96-bea9-4fbc-8bc1-e987d74623c8/extract/0.log" Jun 03 16:33:50.016643 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:33:50.016565 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-657f44b778-22cg6_3a592c0c-0b15-4908-8bf7-787646a82d01/manager/0.log" Jun 03 16:33:50.124837 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:33:50.124805 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-648d5c98bc-qb8dv_a183631b-3c99-4de5-a235-e1bbf27efcb5/manager/0.log" Jun 03 16:33:50.236548 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:33:50.236512 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-6cb54b5c86-5kzgr_c9e9db85-9f68-4fc7-8e11-82433d9a6977/kuadrant-console-plugin/0.log" Jun 03 16:33:50.357592 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:33:50.357563 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-pjwz8_6b856dd2-5f84-428d-8cf3-5cb0a80302f1/registry-server/0.log" Jun 03 16:33:50.730956 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:33:50.730863 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-85c4996f8c-zwvrp_4143b490-3201-49e7-99fa-bc16a828ecb0/manager/0.log" Jun 03 16:33:51.121043 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:33:51.121011 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-686dffbf4bsjmtj_44aff344-e167-440b-b710-a464694c38af/istio-proxy/0.log" Jun 03 16:33:52.192519 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:33:52.192488 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-7f849f6b56-l6584_d91a37d9-3ee4-4fc6-bbe4-39eadea22ee5/storage-initializer/0.log" Jun 03 16:33:52.200697 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:33:52.200675 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-7f849f6b56-l6584_d91a37d9-3ee4-4fc6-bbe4-39eadea22ee5/main/0.log" Jun 03 16:33:52.316415 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:33:52.316371 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-7bb4cdb4d7-hzn4t_f45c240d-6f62-44f7-9e26-f57ca7ad6124/storage-initializer/0.log" Jun 03 16:33:52.328740 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:33:52.328704 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-7bb4cdb4d7-hzn4t_f45c240d-6f62-44f7-9e26-f57ca7ad6124/main/0.log" Jun 03 16:33:52.432081 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:33:52.432052 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-84db68679b-fq625_4f802b10-22aa-4f74-9922-0643a27e15af/storage-initializer/0.log" Jun 03 16:33:52.439928 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:33:52.439908 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-84db68679b-fq625_4f802b10-22aa-4f74-9922-0643a27e15af/main/0.log" Jun 03 16:33:52.551496 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:33:52.551401 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccmrgx8_373d4597-ee86-4395-b945-ee8ecc277c42/storage-initializer/0.log" Jun 03 16:33:52.559248 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:33:52.559223 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-75cdccmrgx8_373d4597-ee86-4395-b945-ee8ecc277c42/main/0.log" Jun 03 16:33:52.679371 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:33:52.679331 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-8f8dc67b7-nmf8k_ec14e607-775c-4c96-bb3f-e5b4b8f04b6d/storage-initializer/0.log" Jun 03 16:33:52.687005 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:33:52.686987 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-8f8dc67b7-nmf8k_ec14e607-775c-4c96-bb3f-e5b4b8f04b6d/main/0.log" Jun 03 16:33:52.801737 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:33:52.801660 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-6b97b89985-ddvlq_f043a50e-6e27-4dad-92de-a2a6137d126e/storage-initializer/0.log" Jun 03 16:34:00.193639 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:34:00.193592 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-6g497_778470c3-9be3-440f-b177-7b846a0c7d32/global-pull-secret-syncer/0.log" Jun 03 16:34:00.329001 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:34:00.328971 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-5hlfw_130349f2-fa26-4e51-bf29-6819611104db/konnectivity-agent/0.log" Jun 03 16:34:00.409800 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:34:00.409766 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-128-88.ec2.internal_e4c4dbf6c50859cd8172ab3fc4042545/haproxy/0.log" Jun 03 16:34:04.431764 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:34:04.431732 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759tg6xr_094f549f-494e-486a-aca5-303c7492dacd/extract/0.log" Jun 03 16:34:04.463575 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:34:04.463519 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759tg6xr_094f549f-494e-486a-aca5-303c7492dacd/util/0.log" Jun 03 16:34:04.547036 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:34:04.547006 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759tg6xr_094f549f-494e-486a-aca5-303c7492dacd/pull/0.log" Jun 03 16:34:04.643511 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:34:04.643482 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0fqfpf_f73bd334-e4cc-4916-8ad4-07b5b88e7b8e/extract/0.log" Jun 03 16:34:04.671006 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:34:04.670979 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0fqfpf_f73bd334-e4cc-4916-8ad4-07b5b88e7b8e/util/0.log" Jun 03 16:34:04.704818 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:34:04.704747 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0fqfpf_f73bd334-e4cc-4916-8ad4-07b5b88e7b8e/pull/0.log" Jun 03 16:34:04.746161 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:34:04.746125 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73tzwb2_e12d559c-bbf9-4f80-b155-8a9fa65fa0c1/extract/0.log" Jun 03 16:34:04.774114 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:34:04.774086 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73tzwb2_e12d559c-bbf9-4f80-b155-8a9fa65fa0c1/util/0.log" Jun 03 16:34:04.799423 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:34:04.799394 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73tzwb2_e12d559c-bbf9-4f80-b155-8a9fa65fa0c1/pull/0.log" Jun 03 16:34:04.838516 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:34:04.838490 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1knbcb_a5bdab96-bea9-4fbc-8bc1-e987d74623c8/extract/0.log" Jun 03 16:34:04.873805 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:34:04.873776 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1knbcb_a5bdab96-bea9-4fbc-8bc1-e987d74623c8/util/0.log" Jun 03 16:34:04.904549 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:34:04.904510 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1knbcb_a5bdab96-bea9-4fbc-8bc1-e987d74623c8/pull/0.log" Jun 03 16:34:04.982265 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:34:04.982173 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-657f44b778-22cg6_3a592c0c-0b15-4908-8bf7-787646a82d01/manager/0.log" Jun 03 16:34:05.015827 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:34:05.015802 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-648d5c98bc-qb8dv_a183631b-3c99-4de5-a235-e1bbf27efcb5/manager/0.log" Jun 03 16:34:05.046916 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:34:05.046893 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-6cb54b5c86-5kzgr_c9e9db85-9f68-4fc7-8e11-82433d9a6977/kuadrant-console-plugin/0.log" Jun 03 16:34:05.093584 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:34:05.093553 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-pjwz8_6b856dd2-5f84-428d-8cf3-5cb0a80302f1/registry-server/0.log" Jun 03 16:34:05.223664 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:34:05.223631 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-85c4996f8c-zwvrp_4143b490-3201-49e7-99fa-bc16a828ecb0/manager/0.log" Jun 03 16:34:06.683143 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:34:06.683119 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_d48d8d35-b7ce-4450-b96a-7070ed55406c/alertmanager/0.log" Jun 03 16:34:06.713599 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:34:06.713563 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_d48d8d35-b7ce-4450-b96a-7070ed55406c/config-reloader/0.log" Jun 03 16:34:06.744392 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:34:06.744363 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_d48d8d35-b7ce-4450-b96a-7070ed55406c/kube-rbac-proxy-web/0.log" Jun 03 16:34:06.771504 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:34:06.771482 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_d48d8d35-b7ce-4450-b96a-7070ed55406c/kube-rbac-proxy/0.log" Jun 03 16:34:06.795472 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:34:06.795451 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_d48d8d35-b7ce-4450-b96a-7070ed55406c/kube-rbac-proxy-metric/0.log" Jun 03 16:34:06.819632 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:34:06.819613 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_d48d8d35-b7ce-4450-b96a-7070ed55406c/prom-label-proxy/0.log" Jun 03 16:34:06.846254 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:34:06.846231 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_d48d8d35-b7ce-4450-b96a-7070ed55406c/init-config-reloader/0.log" Jun 03 16:34:07.044039 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:34:07.043960 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-5768fcc9dd-vhfbl_2d77eba0-7912-4d36-9647-f1b651f01299/monitoring-plugin/0.log" Jun 03 16:34:07.096708 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:34:07.096680 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-6fhtj_f1e80c68-33ac-44f8-88d2-b8abf7ecd943/node-exporter/0.log" Jun 03 16:34:07.121220 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:34:07.121193 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-6fhtj_f1e80c68-33ac-44f8-88d2-b8abf7ecd943/kube-rbac-proxy/0.log" Jun 03 16:34:07.145976 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:34:07.145958 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-6fhtj_f1e80c68-33ac-44f8-88d2-b8abf7ecd943/init-textfile/0.log" Jun 03 16:34:07.686754 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:34:07.686651 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-658bcd8bdf-cfg8g_eed996ff-81a2-4df1-b751-be50acf655e0/prometheus-operator/0.log" Jun 03 16:34:07.724789 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:34:07.724760 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-658bcd8bdf-cfg8g_eed996ff-81a2-4df1-b751-be50acf655e0/kube-rbac-proxy/0.log" Jun 03 16:34:07.751653 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:34:07.751627 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-59b7f8756b-jpmb7_7b9b2d8d-9463-4a44-9477-faa61cc8c0d1/prometheus-operator-admission-webhook/0.log" Jun 03 16:34:07.784605 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:34:07.784578 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-8586f67987-tdnvq_32565e44-36b2-449c-983a-829acd788137/telemeter-client/0.log" Jun 03 16:34:07.811179 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:34:07.811149 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-8586f67987-tdnvq_32565e44-36b2-449c-983a-829acd788137/reload/0.log" Jun 03 16:34:07.844974 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:34:07.844953 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-8586f67987-tdnvq_32565e44-36b2-449c-983a-829acd788137/kube-rbac-proxy/0.log" Jun 03 16:34:07.881257 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:34:07.881232 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-755bbc489c-bptwf_1885968b-3bc3-437f-bfb9-6777652fb84c/thanos-query/0.log" Jun 03 16:34:07.907644 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:34:07.907626 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-755bbc489c-bptwf_1885968b-3bc3-437f-bfb9-6777652fb84c/kube-rbac-proxy-web/0.log" Jun 03 16:34:07.936407 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:34:07.936375 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-755bbc489c-bptwf_1885968b-3bc3-437f-bfb9-6777652fb84c/kube-rbac-proxy/0.log" Jun 03 16:34:07.971025 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:34:07.970955 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-755bbc489c-bptwf_1885968b-3bc3-437f-bfb9-6777652fb84c/prom-label-proxy/0.log" Jun 03 16:34:07.998208 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:34:07.998188 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-755bbc489c-bptwf_1885968b-3bc3-437f-bfb9-6777652fb84c/kube-rbac-proxy-rules/0.log" Jun 03 16:34:08.024047 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:34:08.024020 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-755bbc489c-bptwf_1885968b-3bc3-437f-bfb9-6777652fb84c/kube-rbac-proxy-metrics/0.log" Jun 03 16:34:09.188573 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:34:09.188513 2568 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-must-gather-clkt5/perf-node-gather-daemonset-v56c7"] Jun 03 16:34:09.192157 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:34:09.192140 2568 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-clkt5/perf-node-gather-daemonset-v56c7" Jun 03 16:34:09.197197 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:34:09.197177 2568 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-clkt5\"/\"openshift-service-ca.crt\"" Jun 03 16:34:09.197673 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:34:09.197652 2568 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-clkt5\"/\"default-dockercfg-zpvg7\"" Jun 03 16:34:09.199475 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:34:09.199456 2568 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-clkt5\"/\"kube-root-ca.crt\"" Jun 03 16:34:09.203598 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:34:09.203578 2568 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-clkt5/perf-node-gather-daemonset-v56c7"] Jun 03 16:34:09.363270 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:34:09.363231 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7t2m4\" (UniqueName: \"kubernetes.io/projected/dd7f7de9-e3d2-46bb-a165-13591c4c7de5-kube-api-access-7t2m4\") pod \"perf-node-gather-daemonset-v56c7\" (UID: \"dd7f7de9-e3d2-46bb-a165-13591c4c7de5\") " pod="openshift-must-gather-clkt5/perf-node-gather-daemonset-v56c7" Jun 03 16:34:09.363445 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:34:09.363362 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/dd7f7de9-e3d2-46bb-a165-13591c4c7de5-sys\") pod \"perf-node-gather-daemonset-v56c7\" (UID: \"dd7f7de9-e3d2-46bb-a165-13591c4c7de5\") " pod="openshift-must-gather-clkt5/perf-node-gather-daemonset-v56c7" Jun 03 16:34:09.363445 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:34:09.363391 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/dd7f7de9-e3d2-46bb-a165-13591c4c7de5-podres\") pod \"perf-node-gather-daemonset-v56c7\" (UID: \"dd7f7de9-e3d2-46bb-a165-13591c4c7de5\") " pod="openshift-must-gather-clkt5/perf-node-gather-daemonset-v56c7" Jun 03 16:34:09.363445 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:34:09.363417 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/dd7f7de9-e3d2-46bb-a165-13591c4c7de5-proc\") pod \"perf-node-gather-daemonset-v56c7\" (UID: \"dd7f7de9-e3d2-46bb-a165-13591c4c7de5\") " pod="openshift-must-gather-clkt5/perf-node-gather-daemonset-v56c7" Jun 03 16:34:09.363445 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:34:09.363442 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/dd7f7de9-e3d2-46bb-a165-13591c4c7de5-lib-modules\") pod \"perf-node-gather-daemonset-v56c7\" (UID: \"dd7f7de9-e3d2-46bb-a165-13591c4c7de5\") " pod="openshift-must-gather-clkt5/perf-node-gather-daemonset-v56c7" Jun 03 16:34:09.464346 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:34:09.464264 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/dd7f7de9-e3d2-46bb-a165-13591c4c7de5-sys\") pod \"perf-node-gather-daemonset-v56c7\" (UID: \"dd7f7de9-e3d2-46bb-a165-13591c4c7de5\") " pod="openshift-must-gather-clkt5/perf-node-gather-daemonset-v56c7" Jun 03 16:34:09.464346 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:34:09.464299 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/dd7f7de9-e3d2-46bb-a165-13591c4c7de5-podres\") pod \"perf-node-gather-daemonset-v56c7\" (UID: \"dd7f7de9-e3d2-46bb-a165-13591c4c7de5\") " pod="openshift-must-gather-clkt5/perf-node-gather-daemonset-v56c7" Jun 03 16:34:09.464346 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:34:09.464320 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/dd7f7de9-e3d2-46bb-a165-13591c4c7de5-proc\") pod \"perf-node-gather-daemonset-v56c7\" (UID: \"dd7f7de9-e3d2-46bb-a165-13591c4c7de5\") " pod="openshift-must-gather-clkt5/perf-node-gather-daemonset-v56c7" Jun 03 16:34:09.464633 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:34:09.464376 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/dd7f7de9-e3d2-46bb-a165-13591c4c7de5-sys\") pod \"perf-node-gather-daemonset-v56c7\" (UID: \"dd7f7de9-e3d2-46bb-a165-13591c4c7de5\") " pod="openshift-must-gather-clkt5/perf-node-gather-daemonset-v56c7" Jun 03 16:34:09.464633 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:34:09.464422 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/dd7f7de9-e3d2-46bb-a165-13591c4c7de5-lib-modules\") pod \"perf-node-gather-daemonset-v56c7\" (UID: \"dd7f7de9-e3d2-46bb-a165-13591c4c7de5\") " pod="openshift-must-gather-clkt5/perf-node-gather-daemonset-v56c7" Jun 03 16:34:09.464633 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:34:09.464442 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/dd7f7de9-e3d2-46bb-a165-13591c4c7de5-proc\") pod \"perf-node-gather-daemonset-v56c7\" (UID: \"dd7f7de9-e3d2-46bb-a165-13591c4c7de5\") " pod="openshift-must-gather-clkt5/perf-node-gather-daemonset-v56c7" Jun 03 16:34:09.464633 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:34:09.464450 2568 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-7t2m4\" (UniqueName: \"kubernetes.io/projected/dd7f7de9-e3d2-46bb-a165-13591c4c7de5-kube-api-access-7t2m4\") pod \"perf-node-gather-daemonset-v56c7\" (UID: \"dd7f7de9-e3d2-46bb-a165-13591c4c7de5\") " pod="openshift-must-gather-clkt5/perf-node-gather-daemonset-v56c7" Jun 03 16:34:09.464633 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:34:09.464451 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/dd7f7de9-e3d2-46bb-a165-13591c4c7de5-podres\") pod \"perf-node-gather-daemonset-v56c7\" (UID: \"dd7f7de9-e3d2-46bb-a165-13591c4c7de5\") " pod="openshift-must-gather-clkt5/perf-node-gather-daemonset-v56c7" Jun 03 16:34:09.464633 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:34:09.464565 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/dd7f7de9-e3d2-46bb-a165-13591c4c7de5-lib-modules\") pod \"perf-node-gather-daemonset-v56c7\" (UID: \"dd7f7de9-e3d2-46bb-a165-13591c4c7de5\") " pod="openshift-must-gather-clkt5/perf-node-gather-daemonset-v56c7" Jun 03 16:34:09.472081 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:34:09.472061 2568 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-clkt5\"/\"kube-root-ca.crt\"" Jun 03 16:34:09.483076 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:34:09.483058 2568 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-clkt5\"/\"openshift-service-ca.crt\"" Jun 03 16:34:09.493886 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:34:09.493861 2568 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7t2m4\" (UniqueName: \"kubernetes.io/projected/dd7f7de9-e3d2-46bb-a165-13591c4c7de5-kube-api-access-7t2m4\") pod \"perf-node-gather-daemonset-v56c7\" (UID: \"dd7f7de9-e3d2-46bb-a165-13591c4c7de5\") " pod="openshift-must-gather-clkt5/perf-node-gather-daemonset-v56c7" Jun 03 16:34:09.504796 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:34:09.504777 2568 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-clkt5\"/\"default-dockercfg-zpvg7\"" Jun 03 16:34:09.513512 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:34:09.513494 2568 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-clkt5/perf-node-gather-daemonset-v56c7" Jun 03 16:34:09.644623 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:34:09.644596 2568 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-clkt5/perf-node-gather-daemonset-v56c7"] Jun 03 16:34:09.646375 ip-10-0-128-88 kubenswrapper[2568]: W0603 16:34:09.646348 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-poddd7f7de9_e3d2_46bb_a165_13591c4c7de5.slice/crio-43634a965690bdf757fee4fa71c27b7126bb094af29853a431e5353907f3d67d WatchSource:0}: Error finding container 43634a965690bdf757fee4fa71c27b7126bb094af29853a431e5353907f3d67d: Status 404 returned error can't find the container with id 43634a965690bdf757fee4fa71c27b7126bb094af29853a431e5353907f3d67d Jun 03 16:34:09.648018 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:34:09.647997 2568 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jun 03 16:34:10.343577 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:34:10.343546 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-6c84757667-7mnxn_fd36c718-aaaa-4d15-a586-70d48dc664a8/console/0.log" Jun 03 16:34:10.482756 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:34:10.482721 2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-clkt5/perf-node-gather-daemonset-v56c7" event={"ID":"dd7f7de9-e3d2-46bb-a165-13591c4c7de5","Type":"ContainerStarted","Data":"43c6cbd0971dd2d1792cd294f9958cf6743b8f6bb7e41e21bc837df5c33d3295"} Jun 03 16:34:10.482756 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:34:10.482757 2568 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-clkt5/perf-node-gather-daemonset-v56c7" event={"ID":"dd7f7de9-e3d2-46bb-a165-13591c4c7de5","Type":"ContainerStarted","Data":"43634a965690bdf757fee4fa71c27b7126bb094af29853a431e5353907f3d67d"} Jun 03 16:34:10.482951 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:34:10.482788 2568 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-clkt5/perf-node-gather-daemonset-v56c7" Jun 03 16:34:10.499078 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:34:10.499036 2568 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-must-gather-clkt5/perf-node-gather-daemonset-v56c7" podStartSLOduration=1.499020254 podStartE2EDuration="1.499020254s" podCreationTimestamp="2026-06-03 16:34:09 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-06-03 16:34:10.498593472 +0000 UTC m=+1021.587020061" watchObservedRunningTime="2026-06-03 16:34:10.499020254 +0000 UTC m=+1021.587446844" Jun 03 16:34:11.850032 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:34:11.849996 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-lrqwv_0692610f-5dcf-4120-be04-37201b315bf6/dns/0.log" Jun 03 16:34:11.872743 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:34:11.872714 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-lrqwv_0692610f-5dcf-4120-be04-37201b315bf6/kube-rbac-proxy/0.log" Jun 03 16:34:11.897765 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:34:11.897738 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-2v2dn_b0a52397-ab84-4efe-a7e1-fb26e146dfbc/dns-node-resolver/0.log" Jun 03 16:34:12.537273 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:34:12.537241 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-6jcbf_2d19a6a4-13e1-407c-a488-189a3651f578/node-ca/0.log" Jun 03 16:34:13.468465 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:34:13.468436 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-686dffbf4bsjmtj_44aff344-e167-440b-b710-a464694c38af/istio-proxy/0.log" Jun 03 16:34:14.317607 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:34:14.317578 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-9bfn5_469cfe56-169a-4b98-b437-38f4ea41f481/serve-healthcheck-canary/0.log" Jun 03 16:34:14.825895 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:34:14.825864 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-4264d_6cb85c51-522f-4c5d-b982-8d4039519c70/kube-rbac-proxy/0.log" Jun 03 16:34:14.848924 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:34:14.848897 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-4264d_6cb85c51-522f-4c5d-b982-8d4039519c70/exporter/0.log" Jun 03 16:34:14.875131 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:34:14.875103 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-4264d_6cb85c51-522f-4c5d-b982-8d4039519c70/extractor/0.log" Jun 03 16:34:16.497266 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:34:16.497237 2568 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-clkt5/perf-node-gather-daemonset-v56c7" Jun 03 16:34:17.141453 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:34:17.141422 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6c656db94d-j7rhw_5b560183-6f5f-4f17-823e-7131b664fde3/manager/0.log" Jun 03 16:34:18.776933 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:34:18.776907 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-bd65fcf99-phvvj_8a83f4a5-c55d-4aae-b972-3fdf6daea302/manager/0.log" Jun 03 16:34:18.810970 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:34:18.810943 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-bfc7f696d-d7qj8_11ca5b7b-8c8d-41e6-9bf5-5025aafe9a96/openshift-lws-operator/0.log" Jun 03 16:34:23.175683 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:34:23.175643 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-96c6f749-klgt4_24cc0ece-f688-422f-a4cd-e2bd862e0ffd/migrator/0.log" Jun 03 16:34:23.196729 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:34:23.196698 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-96c6f749-klgt4_24cc0ece-f688-422f-a4cd-e2bd862e0ffd/graceful-termination/0.log" Jun 03 16:34:24.663777 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:34:24.663742 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-8s5gg_80818dcb-cac7-4ac4-a376-ea67605a5ff8/kube-multus/0.log" Jun 03 16:34:25.059938 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:34:25.059861 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-wvl4d_f601dafe-2b89-43a6-bb9e-2080b98aaa82/kube-multus-additional-cni-plugins/0.log" Jun 03 16:34:25.082749 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:34:25.082728 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-wvl4d_f601dafe-2b89-43a6-bb9e-2080b98aaa82/egress-router-binary-copy/0.log" Jun 03 16:34:25.114523 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:34:25.114502 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-wvl4d_f601dafe-2b89-43a6-bb9e-2080b98aaa82/cni-plugins/0.log" Jun 03 16:34:25.137676 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:34:25.137657 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-wvl4d_f601dafe-2b89-43a6-bb9e-2080b98aaa82/bond-cni-plugin/0.log" Jun 03 16:34:25.163348 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:34:25.163328 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-wvl4d_f601dafe-2b89-43a6-bb9e-2080b98aaa82/routeoverride-cni/0.log" Jun 03 16:34:25.185790 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:34:25.185772 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-wvl4d_f601dafe-2b89-43a6-bb9e-2080b98aaa82/whereabouts-cni-bincopy/0.log" Jun 03 16:34:25.208443 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:34:25.208419 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-wvl4d_f601dafe-2b89-43a6-bb9e-2080b98aaa82/whereabouts-cni/0.log" Jun 03 16:34:25.285776 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:34:25.285742 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-fhpnv_d93510f8-5732-4611-8aca-02573c7e296e/network-metrics-daemon/0.log" Jun 03 16:34:25.305716 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:34:25.305696 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-fhpnv_d93510f8-5732-4611-8aca-02573c7e296e/kube-rbac-proxy/0.log" Jun 03 16:34:26.519094 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:34:26.519051 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xz69h_e926d1e6-fe73-4a66-9488-1bd8d711ea79/ovn-controller/0.log" Jun 03 16:34:26.549123 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:34:26.549095 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xz69h_e926d1e6-fe73-4a66-9488-1bd8d711ea79/ovn-acl-logging/0.log" Jun 03 16:34:26.570263 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:34:26.570231 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xz69h_e926d1e6-fe73-4a66-9488-1bd8d711ea79/kube-rbac-proxy-node/0.log" Jun 03 16:34:26.593105 ip-10-0-128-88 kubenswrapper[2568]: I0603 16:34:26.593081 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xz69h_e926d1e6-fe73-4a66-9488-1bd8d711ea79/kube-rbac-proxy-ovn-metrics/0.log"